Periodic Reporting for period 2 - FOCETA (FOUNDATIONS FOR CONTINUOUS ENGINEERING OF TRUSTWORTHY AUTONOMY)
Période du rapport: 2021-10-01 au 2023-10-31
We developed a FOCETA methodology for designing correct and safe learning-enabled autonomous systems (LEAS), bridging the gap between the existing development and verification techniques for LEAS and their operation in the real world. The methodology includes two design flows. The first is for designing trustworthy learning-enabled components (LECs); the second is for integrating them with classically engineered components, resulting in correct and safe LEAS through iterative development and system operation cycles. The methodology allows updates of the LECs in response to emerging requirements from new scenarios, imperfect knowledge of the machine learning models and possible security threats. This is addressed within a continuous development and testing process, mixing software development and system operations in iterative cycles. In this process, formal specifications are used throughout the system life-cycle, both during development and operations. During the concept design phase, formal specifications improve the process of engineering requirements by making them precise. During the system implementation, specifications are used for verification and as oracles in the testing activities. Finally, formal specifications are monitored during system operation to detect violations of requirements and take corrective actions.
Industrial requirements:
We defined use case (UC), method and tool requirements, and detailed evaluation criteria were detailed for each UC. A new specification formalism was developed to match the requirements specific to LEAS. We defined a framework integrating workflows and tools to generate assurance cases for the UCs, using an off-the-shelf assurance framework. A specialized fragment of the assurance case patterns was developed to address the assurance of LECs and LEASs.
Modelling and simulation for autonomy:
We introduced a paradigm shift in modeling and simulation for LEAS. Our approach centers on component-based modeling and co-simulation, yielding the following major achievements: co-simulation architecture for LEAS with formal analysis capabilities, extended formal models that allow mixing model-based and data-driven components, ontology-based requirement formalization framework for natural language requirements of LEAS, specification mining from LEAS behaviors, incremental modeling for cyber-security, framework for virtual testing and scenario generation at scale, and trajectory prediction for LEAS.
Verification and Validation for Autonomy:
A comprehensive suite of methods was developed to rigorously verify Convolutional Neural Networks (CNNs), incorporating a coverage-guided testing framework, an equivalence checking framework, an XAI-enabled testing, an environment specification sampling, and a fault injection framework. Beyond CNNs, methodologies were extended to 3D object detectors, deep reinforcement learning, and decentralised control of multi-robot systems, with the formulation of reliability assessment and dedicated safety metrics.
Runtime Monitoring and Enforcement:
Theory and tools were developed to automatically generate runtime monitors and enforcers for LEAS. For perception systems, the developed tools DeepBox and LiMP can detect object-detection errors of DNN. For the control systems, the following runtime monitoring and assurance techniques/tools were developed or extended: RVCosim and RTAMT monitor learning-enabled systems w.r.t. formal specifications. TEMPEST computes shields to enforce safety specifications. DeepControl optimizes the interaction between different concurrent components of distributed systems. Most developed monitoring techniques and tools were integrated into FOCETA UCs to ensure correct system behavior during runtime.
Methods and tools applied to industrial cases demonstrations:
Methods and tools from previous WPs were implemented in FOCETA UCs based on defined requirements. UC1: Safe and Secure Intelligent Automated Valet Parking; UC2: Anaesthetic drug Target Control Infusion. In UC1, a simulation tool chain with vehicle, sensor, traffic models and a baseline system under test with two LECs implemented. For one of the validation scenarios, 5 requirement analysis and modelling technologies, 5 design-time testing technologies, and 5 runtime monitoring/enforcement technologies were integrated. UC1 highlights three continuous engineering workflows, establishing assurance patterns at both LEC and LEAS levels. UC2 developed a similar engineering workflow with a baseline simulation model integrating requirements, formal specifications mining, HiL/SiL laboratory testing technologies for NN control strategies, abnormal situations detection using runtime monitors, and iterative refinements.
Project management, Ethics requirements:
We coordinated, monitored and implemented the necessary measures for the achievement of FOCETA objectives on time, within the allocated budget and according to high quality standards. We also ensured compliance with ethics requirements set by the EC.
Dissemination, Replication and Exploitation:
We defined and implemented a tailored communication, dissemination and exploitation plan.
Overview of the results and their exploitation and dissemination:
Main KERs obtained:
• The FOCETA methodology, including the two design work flows, supported by open source FOCETA tools.
• A compositional simulation interconnect fabric for building digital twins of Cyber Physical Systems.
• SATORI framework for safe and trusted autonomous systems at runtime.
Results were disseminated via presentations at 77 conferences and other events, organisation of the joint OpenDR-FOCETA Summer School and Final Project Public Workshop and live demonstrations; publication of 58 Open Access papers; 6 teaching courses. We submitted the structured FOCETA certification strategy, contributed to the ISO PAS 8800 standard with FOCETA UC1 and submitted the the EU Innovation Radar Questionnaire.
We defined use case (UC), method and tool requirements, and detailed evaluation criteria were detailed for each UC. A new specification formalism was developed to match the requirements specific to LEAS. We defined a framework integrating workflows and tools to generate assurance cases for the UCs, using an off-the-shelf assurance framework. A specialized fragment of the assurance case patterns was developed to address the assurance of LECs and LEASs.
Modelling and simulation for autonomy:
We introduced a paradigm shift in modeling and simulation for LEAS. Our approach centers on component-based modeling and co-simulation, yielding the following major achievements: co-simulation architecture for LEAS with formal analysis capabilities, extended formal models that allow mixing model-based and data-driven components, ontology-based requirement formalization framework for natural language requirements of LEAS, specification mining from LEAS behaviors, incremental modeling for cyber-security, framework for virtual testing and scenario generation at scale, and trajectory prediction for LEAS.
Verification and Validation for Autonomy:
A comprehensive suite of methods was developed to rigorously verify Convolutional Neural Networks (CNNs), incorporating a coverage-guided testing framework, an equivalence checking framework, an XAI-enabled testing, an environment specification sampling, and a fault injection framework. Beyond CNNs, methodologies were extended to 3D object detectors, deep reinforcement learning, and decentralised control of multi-robot systems, with the formulation of reliability assessment and dedicated safety metrics.
Runtime Monitoring and Enforcement:
Theory and tools were developed to automatically generate runtime monitors and enforcers for LEAS. For perception systems, the developed tools DeepBox and LiMP can detect object-detection errors of DNN. For the control systems, the following runtime monitoring and assurance techniques/tools were developed or extended: RVCosim and RTAMT monitor learning-enabled systems w.r.t. formal specifications. TEMPEST computes shields to enforce safety specifications. DeepControl optimizes the interaction between different concurrent components of distributed systems. Most developed monitoring techniques and tools were integrated into FOCETA UCs to ensure correct system behavior during runtime.
Methods and tools applied to industrial cases demonstrations:
Methods and tools from previous WPs were implemented in FOCETA UCs based on defined requirements. UC1: Safe and Secure Intelligent Automated Valet Parking; UC2: Anaesthetic drug Target Control Infusion. In UC1, a simulation tool chain with vehicle, sensor, traffic models and a baseline system under test with two LECs implemented. For one of the validation scenarios, 5 requirement analysis and modelling technologies, 5 design-time testing technologies, and 5 runtime monitoring/enforcement technologies were integrated. UC1 highlights three continuous engineering workflows, establishing assurance patterns at both LEC and LEAS levels. UC2 developed a similar engineering workflow with a baseline simulation model integrating requirements, formal specifications mining, HiL/SiL laboratory testing technologies for NN control strategies, abnormal situations detection using runtime monitors, and iterative refinements.
Project management, Ethics requirements:
We coordinated, monitored and implemented the necessary measures for the achievement of FOCETA objectives on time, within the allocated budget and according to high quality standards. We also ensured compliance with ethics requirements set by the EC.
Dissemination, Replication and Exploitation:
We defined and implemented a tailored communication, dissemination and exploitation plan.
Overview of the results and their exploitation and dissemination:
Main KERs obtained:
• The FOCETA methodology, including the two design work flows, supported by open source FOCETA tools.
• A compositional simulation interconnect fabric for building digital twins of Cyber Physical Systems.
• SATORI framework for safe and trusted autonomous systems at runtime.
Results were disseminated via presentations at 77 conferences and other events, organisation of the joint OpenDR-FOCETA Summer School and Final Project Public Workshop and live demonstrations; publication of 58 Open Access papers; 6 teaching courses. We submitted the structured FOCETA certification strategy, contributed to the ISO PAS 8800 standard with FOCETA UC1 and submitted the the EU Innovation Radar Questionnaire.
• A specification language for LEAS. The integration of the tools related to LECs and LEAS in the ETB tool for building assurance cases.
• Continuous engineering flow for developing LEAS in design-time. Modular extendable co-simulation framework with formal analysis basis and support for data-driven and model-based paradigms. Ontology- and AI-based methods for requirement engineering.
• Testing and verification techniques for DL-based perception and control components of LEAS. Novel ML-based techniques for testing and verification of LEAS.
• Runtime assurance methods for perception and control systems. Integration of monitoring and enforcement technologies into LEAS.
• A continuous engineering platform integrating and validating FOCETA technologies within the automotive domain. FOCETA technologies validation on a HiL Test-bench platform for an autonomous infusion pump controller for Depth of Anaesthesia.
• Socio-economic impact and wider societal implications of the project: FOCETA significantly reduces the design and verification time for developing LEAS, thus providing a competitive advantage to the European industry in this domain. FOCETA provides useful insights in the integration of AI in LEAS and hence will improve the EU policies such as the AI Act. The safety and security-centric approach of FOCETA to the design of LEAS increases the technical trust in such systems and consequently improves the societal acceptance of these new technologies.
• Continuous engineering flow for developing LEAS in design-time. Modular extendable co-simulation framework with formal analysis basis and support for data-driven and model-based paradigms. Ontology- and AI-based methods for requirement engineering.
• Testing and verification techniques for DL-based perception and control components of LEAS. Novel ML-based techniques for testing and verification of LEAS.
• Runtime assurance methods for perception and control systems. Integration of monitoring and enforcement technologies into LEAS.
• A continuous engineering platform integrating and validating FOCETA technologies within the automotive domain. FOCETA technologies validation on a HiL Test-bench platform for an autonomous infusion pump controller for Depth of Anaesthesia.
• Socio-economic impact and wider societal implications of the project: FOCETA significantly reduces the design and verification time for developing LEAS, thus providing a competitive advantage to the European industry in this domain. FOCETA provides useful insights in the integration of AI in LEAS and hence will improve the EU policies such as the AI Act. The safety and security-centric approach of FOCETA to the design of LEAS increases the technical trust in such systems and consequently improves the societal acceptance of these new technologies.