Industrial requirements:
We defined use case (UC), method and tool requirements, and detailed evaluation criteria were detailed for each UC. A new specification formalism was developed to match the requirements specific to LEAS. We defined a framework integrating workflows and tools to generate assurance cases for the UCs, using an off-the-shelf assurance framework. A specialized fragment of the assurance case patterns was developed to address the assurance of LECs and LEASs.
Modelling and simulation for autonomy:
We introduced a paradigm shift in modeling and simulation for LEAS. Our approach centers on component-based modeling and co-simulation, yielding the following major achievements: co-simulation architecture for LEAS with formal analysis capabilities, extended formal models that allow mixing model-based and data-driven components, ontology-based requirement formalization framework for natural language requirements of LEAS, specification mining from LEAS behaviors, incremental modeling for cyber-security, framework for virtual testing and scenario generation at scale, and trajectory prediction for LEAS.
Verification and Validation for Autonomy:
A comprehensive suite of methods was developed to rigorously verify Convolutional Neural Networks (CNNs), incorporating a coverage-guided testing framework, an equivalence checking framework, an XAI-enabled testing, an environment specification sampling, and a fault injection framework. Beyond CNNs, methodologies were extended to 3D object detectors, deep reinforcement learning, and decentralised control of multi-robot systems, with the formulation of reliability assessment and dedicated safety metrics.
Runtime Monitoring and Enforcement:
Theory and tools were developed to automatically generate runtime monitors and enforcers for LEAS. For perception systems, the developed tools DeepBox and LiMP can detect object-detection errors of DNN. For the control systems, the following runtime monitoring and assurance techniques/tools were developed or extended: RVCosim and RTAMT monitor learning-enabled systems w.r.t. formal specifications. TEMPEST computes shields to enforce safety specifications. DeepControl optimizes the interaction between different concurrent components of distributed systems. Most developed monitoring techniques and tools were integrated into FOCETA UCs to ensure correct system behavior during runtime.
Methods and tools applied to industrial cases demonstrations:
Methods and tools from previous WPs were implemented in FOCETA UCs based on defined requirements. UC1: Safe and Secure Intelligent Automated Valet Parking; UC2: Anaesthetic drug Target Control Infusion. In UC1, a simulation tool chain with vehicle, sensor, traffic models and a baseline system under test with two LECs implemented. For one of the validation scenarios, 5 requirement analysis and modelling technologies, 5 design-time testing technologies, and 5 runtime monitoring/enforcement technologies were integrated. UC1 highlights three continuous engineering workflows, establishing assurance patterns at both LEC and LEAS levels. UC2 developed a similar engineering workflow with a baseline simulation model integrating requirements, formal specifications mining, HiL/SiL laboratory testing technologies for NN control strategies, abnormal situations detection using runtime monitors, and iterative refinements.
Project management, Ethics requirements:
We coordinated, monitored and implemented the necessary measures for the achievement of FOCETA objectives on time, within the allocated budget and according to high quality standards. We also ensured compliance with ethics requirements set by the EC.
Dissemination, Replication and Exploitation:
We defined and implemented a tailored communication, dissemination and exploitation plan.
Overview of the results and their exploitation and dissemination:
Main KERs obtained:
• The FOCETA methodology, including the two design work flows, supported by open source FOCETA tools.
• A compositional simulation interconnect fabric for building digital twins of Cyber Physical Systems.
• SATORI framework for safe and trusted autonomous systems at runtime.
Results were disseminated via presentations at 77 conferences and other events, organisation of the joint OpenDR-FOCETA Summer School and Final Project Public Workshop and live demonstrations; publication of 58 Open Access papers; 6 teaching courses. We submitted the structured FOCETA certification strategy, contributed to the ISO PAS 8800 standard with FOCETA UC1 and submitted the the EU Innovation Radar Questionnaire.
