Symbolic logic framework for situational awareness in mixed autonomy

Project Information

SymAware

Grant agreement ID: 101070802

DOI

10.3030/101070802

Project closed

EC signature date 22 June 2022

Start date 1 October 2022

End date 31 March 2026

Funded under

• The European Innovation Council (EIC)

Total cost

€ 3 980 291,25

EU contribution

€ 3 980 291,00

Coordinated by

MAX-PLANCK-GESELLSCHAFT ZUR FORDERUNG DER WISSENSCHAFTEN EV
Germany

Periodic Reporting for period 1 - SymAware (Symbolic logic framework for situational awareness in mixed autonomy)

Reporting period: 2022-10-01 to 2023-09-30

Summary of the context and overall objectives of the project

The use of aerial and ground vehicles that can operate fully autonomously or with a high level of automation requiring human intervention is increasing. They need to work in in dynamically changing, outdoor environments for transportation, logistics, and mobility. In current uncertain real-world environments, safety and resilience are ensured by appropriate situational awareness, risk awareness, coordination, and decision making by humans. Examples are pilots recognizing adverse weather conditions and informing air traffic control and other pilots about the perceived risk, or car drivers recognising nearby playing kids and adapting their driving in an effort to avoid a potential accident. In increasingly autonomous operations, the situation awareness, risk awareness and experience of human operators that have played such vital roles can no longer be counted upon. Humans create mental models to understand hazards, outlining their components, potential changes over time, potential impacts, and control measures, which are the basis of human situational awareness (SA), involving the perception of environmental elements in time and space, comprehension of their significance, and anticipation of their near-future status. Achieving and maintaining SA is crucial for safety in human-operated road and air traffic control. However, modern robotic agents must make decisions that go beyond what traditional human-based SA models can encompass.

In this context, SymAware introduces a new cognitive architecture designed to align with the internal models of robotic agents and facilitate the safe collaboration of autonomous agents and humans. SymAware's primary objective is to provide a comprehensive framework for situational awareness in multi-agent systems, enabling agents to actively perceive risks in evolving environments and collaborate with robots and humans for complex, dynamic tasks. This framework encompasses various awareness dimensions, including knowledge, spatiotemporal, risk, and social aspects, using logic, symbolic computations, formal reasoning, and uncertainty quantification to achieve full situational awareness in multi-agent systems.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

We are providing a condensed overview of the tasks completed and key accomplishments achieved throughout this reporting period (RP). These accomplishments pertain to research and development efforts. The content is structured according to work packages, tasks, and the teams involved. Additionally, a details overview of the work performed and main achievements are available in the deliverable D8.2.

The primary objective of WP1 is to create a foundation for situational awareness in multi-agent systems (MASs). This involves developing a framework to represent awareness mathematically and a toolkit for building MASs with this awareness. It will also act as an integration hub, gathering input from other work packages. The logical characterization of awareness will be done using temporal logic. KTH in collaboration with other partners have provided different modeling techniques of MASs. Drawing inspiration from the existing framework of awareness (from Deliverable 5.1) within this project, we have already developed the first framework for awareness in MAS. In the first version of the proposed architecture, the situational awareness includes elements related to the system's situational awareness, encompassing aspects such as the system's state, intent, uncertainty, and risk. Furthermore, the framework in question combines various components, including knowledge, perception, communication, planning, control, the physical condition of the system, and interfaces for communication with human agents. MPI-SWS is leading the development of the tool which will implement the proposed architecture. The first version of the tool is already prepared and made public through GitLab.

WP2 aims to represent spatial and temporal awareness in multi-agent systems collaborating with humans, using formal verification languages. It also focuses on decentralized planning and decision-making to reduce risks. The approach developed by KTH involves decomposing global tasks into local tasks, considering communication networks during task decomposition, and in collaboration with UU, designing a model of human users for better human-agent understanding.

WP3 enhances risk awareness in MASs by quantifying and measuring risks for robots and humans. Different approaches are developed by TUE, including risk quantification based on simulation relations and invariant sets. UU is working on user-centered mechanisms for conveying risk during human-agent interactions. The task of predicting and covering potential environmental scenarios is also addressed by TUE.

WP4 focuses on knowledge-aware agents in dynamic environments, aiming to verify properties related to knowledge awareness and develop methodologies for agents with knowledge awareness goals. MPI-SWS is working on scenarios for both single and multi-agent systems, using symbolic computation methods for decision-making.

WP5 implements and validates the SymAware approach in two use cases: Unmanned Traffic Management (UTM) for drones and higher levels of autonomy in vehicles. NLR has developed an awareness-based operational concept for UAS operations, while TUE and SISW are working on autonomous driving use cases and scenario selection for testing.

WP6 establishes a framework for situational awareness in MASs to ensure trustworthy human-agent interaction. UU conducted a participatory design study on human interaction with autonomous vehicles, focusing on modulating autonomy levels and designing user interfaces. A follow-up study is planned to explore the impact of information sharing and user profiles on interaction and to examine EU transparency guidelines.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The activities of the consortium mentioned throughout the report D8.2 put SymAware on the right track for achieving all its milstones and delivering the proposed objectives in the rest of the project. These activities are expected to deliver the following innovations beyond the state-of-art:

1. Python framework with algorithms that incorporate awareness in the control of heterogeneous multi-agent systems.
2. Risk and task assignment for heterogeneous multi-agent systems.
3. A compositional framework for multi-agent control under spatiotemporal constraints.
4. Algorithms for improving safety and efficiency of autonomous systems using knowledge and intent awareness.
5. Accelerating safety assessment of autonomous systems using a new risk metric.
6. Guidelines for Ethical and Trustworthy Human-Agent Interaction (HAI) in aviation and automotive applications.