SAFE AND EXPLAINABLE CRITICAL EMBEDDED SYSTEMS BASED ON AI

Informations projet

SAFEXPLAIN

N° de convention de subvention: 101069595

DOI

10.3030/101069595

Projet clôturé

Date de signature de la CE 13 Juin 2022

Date de début 1 Octobre 2022

Date de fin 30 Septembre 2025

Financé au titre de

• Digital, Industry and Space

Coût total

€ 3 891 875,00

Contribution de l’UE

€ 3 891 875,00

Coordonné par

BARCELONA SUPERCOMPUTING CENTER CENTRO NACIONAL DE SUPERCOMPUTACION
Spain

CORDIS fournit des liens vers les livrables publics et les publications des projets HORIZON.

Les liens vers les livrables et les publications des projets du 7e PC, ainsi que les liens vers certains types de résultats spécifiques tels que les jeux de données et les logiciels, sont récupérés dynamiquement sur OpenAIRE .

Livrables

Data Management Plan (1)

Data Management Plan

Document outlining how data will be managed during the project from internal and external point of view. The DMP will include a table specifying how the data will be exploited, shared for verification and reuse. Updates to this report in m12, m24 and m36

Documents, reports (22)

DL safety architectural patterns and platform

Provides a consolidated analysis of recurrent AI/DL-safety problems and the definition of a set of DL-safety patterns based on the standards and ongoing initiatives studied in T2.1/T2.2, and tailored to the safe DL execution platform.

Final exploitation report

Final update of exploitation plan taking into consideration the results of the evaluation process

Final proofs-of-concept, arguments, and DL components and libraries

This report will update D3.1 and D3.2 contents with their final state, and include hands-on recommendations for how to specify a DL component including the embedded DNN. Moreover, the report will elaborate on explainability, traceability and robustness for an MVP, and will include a handbook on how to develop DL components for CAIS.

Case study stubbing and early assessment of case study porting

Reports the specifications and the necessary stubbing made in each case study and an assessment on the impact it might have on the behaviour of the application. It also covers a status report on case study porting

Communication and Dissemination Plan

Description of the communication and dissemination strategy, the activities to be undertaken to achieve this, and KPIs to monitor progress. Dissemination results will be reported in the periodic and final reports.

Initial communication and dissemination report

Communication, dissemination and knowledge transfer activities until month 12 including the open communities and development environments where SAFEXPLAIN results will be disseminated.

Intermediate Exploitation Report

Joint and individual plans for exploitation updates

Requirements, Success Criteria and platforms

Report capturing the case study and FUSA standards requirements, the success criteria set for each milestone of the project, FUSA principles to consider in SAFEXPLAIN, and choices about the platform and the software stack

Exploitation Plan

Description of the exploitation strategy, the activities to be undertaken to achieve this, and KPIs to monitor progress. Exploitation results and an updated exploitation plan will be provided in the periodic and final reports.

Final Platform Technologies report

Report on the complete platform-related technologies, to be assessed by the case studies.

Case study porting and integration

Reports on the platform porting and SAFEXPLAIN software stack integration carried out incorporating all features from WP2, WP3, and WP4.

Gender and Equality Committee Report

Document capturing the gender aspects during the project lifetime with the involvement of all partners. Updates to this report in m24 and m36

Interim Plattform Technologies Report

This report captures the advancements done in T4.1-T4.4 by MS2. It will provide an assessment of the degree of completion in each of these tasks, and baseline technology where to integrate the case studies.

Project management plan and quality guidelines

Document that defines the main project procedures (contractual framework, project governance, project monitoring, internal communication, etc.) as well as the standards and quality procedures to be used throughout the project. This handbook will also include a section on Gender Equality guidelines for monitoring and management measures and cohesion activities with the PPP on AI, Data and Robotics and the CSA HORIZON CL4-2021-HUMAN-01-02.

Intermediate communication and dissemination report

Communication, dissemination and knowledge transfer activities until month 24 including the open communities and development environments where SAFEXPLAIN results will be disseminated.

Specificability, explainability, traceability, and robustness proof-of-concept and argumentation

Reports on (i) proposed AI specifications in correspondence with D2.1, (ii) Trustworthy and explainable AI architecture for dependable DL components and (iii) online safety supervision architecture. Also, it reports early specificability, explainability, traceability, and robustness arguments and requirements

Safety concept

This includes an update of D2.1 and D2.2, joined together in a safety concept prepared for certification authority review. The report includes the results and conclusions of the external safety assessment by certification authority(es).

Interim report on DL components and libraries

Reports on the progress done on the implementation of the DL components and the usage of the framework to train the neural networks. It also covers the report on the low-level optimization libraries for the selected accelerators

Safety guidelines

This report describes the consolidated safety guidelines as a result of WP2 activities and the main conclusions obtained from the reviews by certification authorities and certification experts. The guidelines include a set of technical solutions amenable to current safety standards and recommendations for future standard improvement and extension

Initial exploitation report

Updated analysis of the exploitation context, business opportunities and exploitable results identifications, based on the requirements gathered at year 1 (m12).

Safety lifecycle considerations

It establishes the initial selected safety techniques and restrictions to be addressed by WP3 and WP4 in: (i) DL-software specification, design and implementation (T2.1), and (ii) DL-software V&V (T2.2).

Final communication and dissemination report

Communication, dissemination and knowledge transfer activities until month 36 including the open communities and development environments where SAFEXPLAIN results will be disseminated.

Other (1)

Final DL components, libraries and the DL interface

Provides the final set of libraries, components and the developed platform-dependent optimizations updated in accordance with the feedback received from the case studies.

Publications

Conference proceedings (12)

Safety-Relevant AI-Based System Robustification with Neural Network Ensembles

Auteurs: Aldomà Coll, Adrià; Brando Guillaumes, Axel; Cazorla Almeida, Francisco Javier; Abella Ferrer, Jaume
Publié dans: 2024 IEEE 30th International Symposium on On-Line Testing and Robust System Design (IOLTS), 2024, ISBN 979-8-3503-7055-3
Éditeur: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/IOLTS60994.2024.10616095

AI-FSM: Towards Functional Safety Management for Artificial Intelligence-based Critical Systems

Auteurs: Javier Fernández, Irune Agirre, Jon Perez-Cerrolaza, Lorea Belategi, Ana Adell, Carlo Donzella, Jaume Abella
Publié dans: CARS@EDCC2024 Workshop - Critical Automotive applications: Robustness & Safety, 2024
Éditeur: EDCC

Leveraging image-based transformations to mitigate adversarial attacks in AI-based safety-critical systems

Auteurs: Marti Carlo, Axel Brando, Jaume Abella
Publié dans: 2025 IEEE 31th International Symposium on On-line Testing and Robust System Design (IOLTS), 2025, ISBN 979-8-3315-3334-2
Éditeur: Institute of Electrical and Electronics Engineers (IEEE)

Retrospective Uncertainties for Deep Models using Vine Copulas

Auteurs: Tagasovska, Natasa; Ozdemir, Firat; Brando Guillaumes, Axel
Publié dans: Proceedings of Machine Learning Research, 2023, ISSN 2640-3498
Éditeur: MLResearchPress
DOI: 10.48550/arxiv.2302.12606

Managing Sources of Uncertainty in Utilizing AI in Development and Deployment of Safety-Critical Autonomous Systems

Auteurs: Robert Lowe, Maria Ulan, Thanh Hai Bui, Ana Adell, Jokin Labaien, Axel Brando
Publié dans: 2025 IEEE 22nd International Conference on Software Architecture Companion (ICSA-C), 2025, ISSN 2768-4288
Éditeur: IEEE
DOI: 10.1109/ICSA-C65153.2025.00079

SAFEXPLAIN: Safe and Explainable Critical Embedded Systems Based on AI

Auteurs: Jaume Abella; Jon Perez; Cristofer Englund; Bahram Zonooz; Gabriele Giordana; Carlo Donzella; Francisco J. Cazorla; Enrico Mezzetti; Isabel Serra; Axel Brando; Irune Agirre; Fernando Eizaguirre; Thanh Hai Bui; Elahe Arani; Fahad Sarfraz; Ajay Balasubramaniam; Ahmed Badar; Ilaria Bloise; Lorenzo Feruglio; Ilaria Cinelli; Davide Brighenti; Davide Cunial
Publié dans: 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2023, ISSN 1558-1101
Éditeur: IEEE
DOI: 10.23919/date56975.2023.10137128

Detecting Low-Density Mixtures in High-Quantile Tails for pWCET Estimation

Auteurs: Blau Manau, Sergi Vilardell, Isabel Serra, Enrico Mezzetti, Jaume Abella, and Francisco J. Cazorla
Publié dans: 37th Euromicro Conference on Real-Time Systems (ECRTS 2025), Numéro 335, 2025
Éditeur: Schloss Dagstuhl – Leibniz-Zentrum für Informatik
DOI: 10.4230/LIPICS.ECRTS.2025.20

Probabilistic Timing Estimates in Scenarios Under Testing Constraints

Auteurs: Sergi Vilardell; Francesco Rossi; Gabriele Giordana; Isabel Serra; Enrico Mezzetti; Jaume Abella; Francisco J. Cazorla
Publié dans: Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing, 2025, ISBN 979-8-4007-0629-5
Éditeur: Association for Computing Machinery (ACM)

Semantic diverse DMR and TMR for high-integrity AI-based function efficiency

Auteurs: Marti Caro, Axel, Brando, Jaume Abella
Publié dans: ACM transactions on cyber-physical systems, Numéro 9, 2025, ISSN 2378-9638
Éditeur: Association for Computing Machinery (ACM)

Software-Only Semantic Diverse Redundancy for High-Integrity AI-Based Functionalities

Auteurs: Martí Caro, Axel Brando, Jaume Abella.
Publié dans: ERTS2024, 2024
Éditeur: ERTS

Differentiating Adversial Attacks from Natural Sensory Anomalies in Object Detection

Auteurs: Robert Lowe, Maria Ulan, Thanh Bui, Gabriele Giordana, Tobia Giani, Francesco Rossi
Publié dans: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases 2025, 2025
Éditeur: Springer Communications in Computer and Information Science
DOI: 10.5281/ZENODO.17037385

Standardizing the Probabilistic Sources of Uncertainty for the sake of Safety Deep Learning

Auteurs: Brando, Axel; Serra, Isabel; Mezzetti, Enrico; Cazorla Almeida, Francisco Javier; Abella Ferrer, Jaume
Publié dans: Proceedings of the Workshop on Artificial Intelligence Safety 2023 (SafeAI 2023) co-located with the Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023), 2023, ISSN 1613-0073
Éditeur: CEUR Workshop Proceedings