Governing AI Technologies in Weapon Systems from the Bottom Up: Practices to Sustain and Strengthen Human Control

Project Information

AutoPractices

Grant agreement ID: 101156237

DOI

10.3030/101156237

Project closed

EC signature date 15 May 2024

Start date 1 June 2024

End date 30 November 2025

Funded under

• European Research Council (ERC)

Total cost

No data

EU contribution

€ 150 000,00

Coordinated by

SYDDANSK UNIVERSITET
Denmark

Periodic Reporting for period 1 - AutoPractices (Governing AI Technologies in Weapon Systems from the Bottom Up: Practices to Sustain and Strengthen Human Control)

Reporting period: 2024-06-01 to 2025-11-30

Summary of the context and overall objectives of the project

In the practices of warfare, the use of AI technologies increasingly transforms targeting decision-making. Although often discussed in the context of autonomous weapon systems (AWS), AI technologies are also spreading in the military domain through their integration into various decision support systems. Using AI in the military always involves some human decision-making. But the quality of control that humans can exercise is compromised due to the demands they are placed under in terms of speed and interacting with multiple, networked systems.
The AutoPractices project starts from the assumption that the exercise of human agency in the development and deployment of AI systems should be sustained and strengthened. The need for exercising a sufficient level of human agency is a recognised governance principle across global and regional initiatives on AI in the military domain, such as the Group of Governmental Experts on Emerging Technologies in the area of Lethal Autonomous Weapons Systems (UN GGE on LAWS) or the Responsible AI in the Military Domain (REAIM) Summits. Such initiatives have set high-level legal and ethical principles on the human role in the use of force from the top-down. Yet, such principles do not spell out a path towards potential implementation.
This is an important point for two reasons: first, more detailed top-down provisions that would ease the practical implementation of human control as a principle cannot be expected soon. The governance debate has been moving forward slowly and has increasingly become embroiled in a charged geopolitical atmosphere. Second, AI technologies are already being used in targeting decision-making and have already shaped social norms, defined as understandings of appropriateness. This has been the main, thoroughly documented findings of the ERC-funded AutoNorms project. Such practical processes are therefore also already shaping what should count as a requisite quality of human control. These practical processes appear to currently lead towards accepting a compromised, diminished quality of human control.
This dual finding inspired the AutoPractices project to work towards sustaining a positive norm of human control/agency through a bottom-up process. Such a process entails adapting and reinforcing best practices of designing, training personnel for, and using systems integrating AI technologies in targeting decision-making.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The AutoPractices project has followed a four-step process: (1) idea generation, (2) research, (3) implementation, and (4) dissemination (see figure 1).
Stage 1: Idea generation. The AutoPractices project conducted extensive literature reviews to define its key concepts, namely the exercise of human agency and the lifecycle of AI systems. The exercise of human agency in the context of interacting with AI systems is defined as: The capacity to 1) understand and reasonably foresee a system’s functions and effects in a relevant context; and 2) to deliberate and decide upon suitable actions in a timely manner; and 3) to act in a way that can impact the use of the system. The AutoPractices project takes as a foundation the model of the AI lifecycle proposed by the IEEE Standards Association Research Group on Issues of Autonomy and AI for Defense Systems. This framework presents a granular way of thinking about the lifecycle with multiple points of human involvement and opportunities to exercise agency from development through testing, procurement, use and after use.
Stage 2: Research. The AutoPractices project mapped practices through engaging with diverse stakeholders by way of surveys, interviews, and workshops. A total of 49 stakeholders took part in the AutoPractices project. They represented five main groups: (1) political-ethical, (2) technical, (3) legal, (4) military, and (5) civil society experts. Based on an analysis of stakeholders’ survey and interview responses, the AutoPractices project produced a first output, the Map of Practices. This document brings together practices named by stakeholders and organises these into eight key themes. The Map of Practices served as the basis for discussion at the two AutoPractices stakeholder workshops (1 online, 1 in-person).
Stage 3: Implementation. Following the mapping of practices and in-depth discussion with stakeholders at the workshops, the AutoPractices project co-created the second and main output, the Best Practices Toolkit. The toolkit was produced iteratively in close collaboration with the project’s stakeholders and published in January 2026.
Stage 4: Dissemination. Dissemination of the toolkit continues in two phases. In the short term, stakeholders, as co-creators of the toolkit, raise awareness of how various practices shape emerging norms around human agency in the military domain. In the long term, actors who adjust their practices throughout the AI lifecycle help sustain and strengthen human agency.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The AutoPractices project produced a best practices toolkit to strengthen human agency in the military domain for policymakers, developers, and users of AI systems. This is a major advance from the state of the art as it is the first publication of this kind. Beyond listing numerous practices for stakeholders to perform across the whole lifecycle, there are four key take-aways for policymakers, developers, and users of AI-based systems in the military domain interested in strengthening the exercise of human agency: (1) establishing boundaries at the early stages of the AI lifecycle; (2) implementing feedback and communication mechanisms across the AI lifecycle; (3) prioritising education and training across the AI lifecycle; (4) adopting strategies and steps to implement the practices in this toolkit. Change association with the practices will be gradual: new practices cannot instantly replace old ones and will depend on existing policies that permit flexibility. Yet, the AutoPractices project underlines that momentum is building for a bottom-up, international, interdisciplinary social innovation process that can operate alongside – or independent of – any top-down governance. We are planning to continue the work of disseminating the AutoPractices toolkit through follow-on funding publications, including around capacity-building.

figure-1-autopractices-project.png