Periodic Reporting for period 1 - AI4REALNET (AI for REAL-world NETwork operation)
Periodo di rendicontazione: 2023-10-01 al 2024-09-30
AI4REALNET covers the perspective of AI-based solutions addressing critical systems (electricity, railway, and air traffic control) that can be simulated and traditionally operated by humans and where AI complements and augments human abilities. These networks operated by humans, often combining human expertise with control and supervision software and different levels of automation, will face challenges in handling increasing uncertainty (e.g. from weather, demand), combinatorial and sequential decisions to exploit network flexibility, and in human overseeing the increasing automation and intervene when required. In the AI4REALNET vision, high levels of human control and AI-based automation coexist with “optimal” balance. They are divided into a) full human control, b) co-learning between AI and humans, and c) trustworthy full AI-based control. It aims to achieve the following two strategic overarching goals: 1) to develop the next generation of decision-making methods powered by supervised and reinforcement learning, which aim at trustworthiness in AI-assisted human control with augmented cognition, hybrid human-AI co-learning, and autonomous AI, with the resilience, safety, and security of critical infrastructures as core requirements, and 2) to boost the development and validation of novel AI algorithms by the consortium and AI community via existing open-source AI-friendly digital environments capable of emulating realistic scenarios of physical systems operation and human decision-making, enabling a direct assessment of AI-based decision quality.
[WP1] During this period, WP1 focused on three tasks. Task 1.1 established the AI4REALNET conceptual framework using an interdisciplinary approach, integrating fields like psychology and cognitive engineering to study expert decision-making in complex, automated scenarios. This framework, structured into four layers, supports reliable and compliant decision-making through socio-technical, AI, and trustworthy AI perspectives. In Task 1.2 six use cases were detailed using IEC and ISO methodologies, with Key Performance Indicators (KPIs) capturing various dimensions, and ALTAI was applied to assess risks and ethical concerns. Task 1.3 dealt with extending the three digital platforms (Grid2Op, FLATLAND and BlueSky), enabling the testing of algorithms from WPs 2-3. The partners analysed the existing digital platforms jointly, identifying differences and commonalities from a technical perspective. The outcome of the analysis gives common ground for discussions on the development of the AI4REALNET system.
[WP2] State-of-the-art has been investigated, and the challenges and research directions have been identified in a project’s position paper. Key developments include:
- Inventorize the knowledge present in the three AI4REALNET domains and start of development of knowledge-assisted AI approaches
- Algorithm testing for distributed reinforcement learning (RL) in the Grid2OP environment and exploration of hierarchical RL approaches. Initial research on graph RL for power grid topology optimisation
- Advanced work in explainable AI, focusing on analysing agent behaviours. Development of methods to embed ethical dimensions into reinforcement learning alongside theoretical research on human factors for effective human-AI collaboration
[WP3] Developed requirements for human-AI interaction at generic and use case levels, specifying roles and tasks and abstracting interaction protocols into a unified representation. Research on uncertainty estimation methods began, focusing on their impact on decision-making [Task 3.1]. Collected RL training KPIs focused on conflicting objectives, reviewed multi-objective RL approaches, and explored methods to synthesise objectives, enabling agent training without immediate human data. Identified RL methods for human-AI co-learning, including behaviour cloning, inverse RL, and imitation learning, with synthetic data considered for limited real data use cases [Task 3.3]. Conceptualised workflows for human supervision in automated processes, designing a two-stage agent training plan using offline and online RL to align AI with human performance and build trust
[WP2] State-of-the-art has been investigated, and the challenges and research directions have been identified in a project’s position paper. Key developments include:
- Inventorize the knowledge present in the three AI4REALNET domains and start of development of knowledge-assisted AI approaches
- Algorithm testing for distributed reinforcement learning (RL) in the Grid2OP environment and exploration of hierarchical RL approaches. Initial research on graph RL for power grid topology optimisation
- Advanced work in explainable AI, focusing on analysing agent behaviours. Development of methods to embed ethical dimensions into reinforcement learning alongside theoretical research on human factors for effective human-AI collaboration
[WP3] Developed requirements for human-AI interaction at generic and use case levels, specifying roles and tasks and abstracting interaction protocols into a unified representation. Research on uncertainty estimation methods began, focusing on their impact on decision-making [Task 3.1]. Collected RL training KPIs focused on conflicting objectives, reviewed multi-objective RL approaches, and explored methods to synthesise objectives, enabling agent training without immediate human data. Identified RL methods for human-AI co-learning, including behaviour cloning, inverse RL, and imitation learning, with synthetic data considered for limited real data use cases [Task 3.3]. Conceptualised workflows for human supervision in automated processes, designing a two-stage agent training plan using offline and online RL to align AI with human performance and build trust
WP1 of AI4REALNET contributed with a) the establishment of a conceptual framework for AI-based decision-making in critical infrastructure networks, b) the development of system engineering views (operational, functional, and logical) to describe decision-making in critical infrastructure networks, c) an illustration of the interaction between human decision-making and AI-based recommendations to enhance the management of critical infrastructure networks, and d) an adaption of the ISO/IEC TR 24030 template and methodology to formally document the industry-driven UCs, and e) how to use the Assessment List for Trustworthy Artificial Intelligence (ALTAI) tool as a comprehensive self-assessment tool to capture non-functional requirements related to trustworthy AI in the design of the UCs. This work was achieved through a collaboration between technical domains, such as AI-based data science and simulations, and socio-technical domains focused on human sciences.
From WP2, there are two main contributions: a) a model to break the complexity originated by the course of dimensionality in large-scale reinforcement learning problems, which makes use of information-theoretic quantities such as mutual information to find the more effective way to split large state and action spaces effectively, and b) machine learning method for identifying and predicting power grid failures in deep reinforcement learning. These methods have the potential to be replicated in different contexts and domains and address challenges such as the scalability of reinforcement learning and explainability of AI agents’ behaviour.
Given that WP3 started in month M6, most of the work done is in preliminary phases or at a conceptual level, mainly proposing architectures for co-learning methods that focus on inverse learning, imitation learning and behaviour cloning, and defining human-AI interaction protocols and requirements.
From WP2, there are two main contributions: a) a model to break the complexity originated by the course of dimensionality in large-scale reinforcement learning problems, which makes use of information-theoretic quantities such as mutual information to find the more effective way to split large state and action spaces effectively, and b) machine learning method for identifying and predicting power grid failures in deep reinforcement learning. These methods have the potential to be replicated in different contexts and domains and address challenges such as the scalability of reinforcement learning and explainability of AI agents’ behaviour.
Given that WP3 started in month M6, most of the work done is in preliminary phases or at a conceptual level, mainly proposing architectures for co-learning methods that focus on inverse learning, imitation learning and behaviour cloning, and defining human-AI interaction protocols and requirements.