Governing AI Technologies in Weapon Systems from the Bottom Up: Practices to Sustain and Strengthen Human Control

Informacje na temat projektu

AutoPractices

Identyfikator umowy o grant: 101156237

DOI

10.3030/101156237

Projekt został zamknięty

Data podpisania przez KE 15 Maja 2024

Data rozpoczęcia 1 Czerwca 2024

Data zakończenia 30 Listopada 2025

Finansowanie w ramach

• European Research Council (ERC)

Koszt całkowity

Brak danych

Wkład UE

€ 150 000,00

Koordynowany przez

SYDDANSK UNIVERSITET
Denmark

Periodic Reporting for period 1 - AutoPractices (Governing AI Technologies in Weapon Systems from the Bottom Up: Practices to Sustain and Strengthen Human Control)

Okres sprawozdawczy: 2024-06-01 do 2025-11-30

Podsumowanie kontekstu i ogólnych celów projektu

In the practices of warfare, the use of AI technologies increasingly transforms targeting decision-making. Although often discussed in the context of autonomous weapon systems (AWS), AI technologies are also spreading in the military domain through their integration into various decision support systems. Using AI in the military always involves some human decision-making. But the quality of control that humans can exercise is compromised due to the demands they are placed under in terms of speed and interacting with multiple, networked systems.
The AutoPractices project starts from the assumption that the exercise of human agency in the development and deployment of AI systems should be sustained and strengthened. The need for exercising a sufficient level of human agency is a recognised governance principle across global and regional initiatives on AI in the military domain, such as the Group of Governmental Experts on Emerging Technologies in the area of Lethal Autonomous Weapons Systems (UN GGE on LAWS) or the Responsible AI in the Military Domain (REAIM) Summits. Such initiatives have set high-level legal and ethical principles on the human role in the use of force from the top-down. Yet, such principles do not spell out a path towards potential implementation.
This is an important point for two reasons: first, more detailed top-down provisions that would ease the practical implementation of human control as a principle cannot be expected soon. The governance debate has been moving forward slowly and has increasingly become embroiled in a charged geopolitical atmosphere. Second, AI technologies are already being used in targeting decision-making and have already shaped social norms, defined as understandings of appropriateness. This has been the main, thoroughly documented findings of the ERC-funded AutoNorms project. Such practical processes are therefore also already shaping what should count as a requisite quality of human control. These practical processes appear to currently lead towards accepting a compromised, diminished quality of human control.
This dual finding inspired the AutoPractices project to work towards sustaining a positive norm of human control/agency through a bottom-up process. Such a process entails adapting and reinforcing best practices of designing, training personnel for, and using systems integrating AI technologies in targeting decision-making.

Prace wykonane od początku projektu do końca okresu sprawozdawczego oraz najważniejsze dotychczasowe rezultaty

The AutoPractices project has followed a four-step process: (1) idea generation, (2) research, (3) implementation, and (4) dissemination (see figure 1).
Stage 1: Idea generation. The AutoPractices project conducted extensive literature reviews to define its key concepts, namely the exercise of human agency and the lifecycle of AI systems. The exercise of human agency in the context of interacting with AI systems is defined as: The capacity to 1) understand and reasonably foresee a system’s functions and effects in a relevant context; and 2) to deliberate and decide upon suitable actions in a timely manner; and 3) to act in a way that can impact the use of the system. The AutoPractices project takes as a foundation the model of the AI lifecycle proposed by the IEEE Standards Association Research Group on Issues of Autonomy and AI for Defense Systems. This framework presents a granular way of thinking about the lifecycle with multiple points of human involvement and opportunities to exercise agency from development through testing, procurement, use and after use.
Stage 2: Research. The AutoPractices project mapped practices through engaging with diverse stakeholders by way of surveys, interviews, and workshops. A total of 49 stakeholders took part in the AutoPractices project. They represented five main groups: (1) political-ethical, (2) technical, (3) legal, (4) military, and (5) civil society experts. Based on an analysis of stakeholders’ survey and interview responses, the AutoPractices project produced a first output, the Map of Practices. This document brings together practices named by stakeholders and organises these into eight key themes. The Map of Practices served as the basis for discussion at the two AutoPractices stakeholder workshops (1 online, 1 in-person).
Stage 3: Implementation. Following the mapping of practices and in-depth discussion with stakeholders at the workshops, the AutoPractices project co-created the second and main output, the Best Practices Toolkit. The toolkit was produced iteratively in close collaboration with the project’s stakeholders and published in January 2026.
Stage 4: Dissemination. Dissemination of the toolkit continues in two phases. In the short term, stakeholders, as co-creators of the toolkit, raise awareness of how various practices shape emerging norms around human agency in the military domain. In the long term, actors who adjust their practices throughout the AI lifecycle help sustain and strengthen human agency.

Innowacyjność oraz oczekiwany potencjalny wpływ (w tym dotychczasowe znaczenie społeczno-gospodarcze i szersze implikacje społeczne projektu)

The AutoPractices project produced a best practices toolkit to strengthen human agency in the military domain for policymakers, developers, and users of AI systems. This is a major advance from the state of the art as it is the first publication of this kind. Beyond listing numerous practices for stakeholders to perform across the whole lifecycle, there are four key take-aways for policymakers, developers, and users of AI-based systems in the military domain interested in strengthening the exercise of human agency: (1) establishing boundaries at the early stages of the AI lifecycle; (2) implementing feedback and communication mechanisms across the AI lifecycle; (3) prioritising education and training across the AI lifecycle; (4) adopting strategies and steps to implement the practices in this toolkit. Change association with the practices will be gradual: new practices cannot instantly replace old ones and will depend on existing policies that permit flexibility. Yet, the AutoPractices project underlines that momentum is building for a bottom-up, international, interdisciplinary social innovation process that can operate alongside – or independent of – any top-down governance. We are planning to continue the work of disseminating the AutoPractices toolkit through follow-on funding publications, including around capacity-building.

figure-1-autopractices-project.png