In more detail, SAFEXPLAIN has achieved its goals in the different fronts of the project: (A) safety lifecycle and safety architecture design, (B) solutions to provide meta-information along with DL software outcomes that allows quantifying uncertainty and detecting anomalies, (C) platform support and system services where to practically integrate the AI-based software architectures, and (D) case study integration and evaluation to assess SAFEXPLAIN technologies and concepts.
(A) A safety lifecycle describing how to set training and validation data for safety-relevant DL software, training processes, and inference processes during operation has been defined. Such safety lifecycle has already been assessed positively by relevant certification entities and domain experts. Also, safety architectures amenable for different degrees of integrity levels for DL software have been devised, and some of them explicitly applied to the project case studies. Such integrity levels relate to the cases where DL software provides complementary information related to the safety of the system, whether it intervenes in the safety management of the system, or whether it implements the safety functionality.
(B) Processes and solutions to assess the trustability of DL software have been identified, and applied to the challenges exposed in the case studies. Such solutions allow telling whether the system is being fed with data different to that used for training (e.g. system trained to identify people only, but a dog appears in the scene), whether the DL model used for raising predictions is capable of raising trustable predictions even if input data is similar to the one used for training, whether input data offers insufficient information to raise trustable outcomes (e.g. detecting and classifying overlapped objects), and whether any unforeseen anomaly occurred during the process.
(C) A suitable middleware has been deployed allowing to integrate DL-based safety-relevant applications onto the platform with appropriate levels of abstraction and providing the services needed by the applications. All services required by the case studies have been successfully implemented, as well as those features needed to properly control the application above. Related to the latter, the target platform of the project (NVIDIA Orin) has been carefully analyzed identifying how to master it, setting convenient configurations, and providing real-time guarantees to the DL-based applications to be run on top.
(D) Case studies have been integrated and assessed on top of the target platform, which includes the high-performance hardware platform, the middleware interfacing services for the application and control tasks, and the SAFEXPLAIN technologies needed for the execution of DL-based applications adhering to their safety requirements. Moreover, a generic fully open case study (aka core demo) has been also integrated and released openly to ease technology adoption across applications and domains.
Apart from all the technical advances achieved by the project, huge effort has been done to disseminate and communicate the achievements of the project in a wide variety of communities and audiences, spanning from technical specialists to industrial stakeholders and general audiences. Hand in hand with the dissemination efforts, exploitation efforts have allowed identifying the exploitable items of the project, defining exploitation paths for each one of them, and creating a dialogue between the relevant standardization bodies and the project. In particular, standardization bodies are working toward defining ways to enable the incorporation of DL software in safety-critical systems, and explicit communication with the project has provided those bodies with practical processes and examples that some of them have already incorporated to their syllabus. As part of the dialogue, SAFEXPLAIN partners have been exposed to the directions that those standards are taking so that project solutions already match those standards whenever they become final.
