Periodic Reporting for period 1 - PERIOD (Pursuing Efficient Reliability of Object Detection for automotive and aerospace applications)
Período documentado: 2020-10-16 hasta 2022-10-15
The complexity and sensitivity to perturbation of object detection frameworks is one of the most critical threads to the reliability of autonomous vehicles. The huge amount of hardware resources required to process frames in real time and the parallel structure of modern hardware accelerators for object detention frameworks exacerbate both the probability of a radiation-induced corruption and the impact of this corruption in the output correctness. The radiation-induced error rates in modern devices have been found to be hundreds to thousands of times higher than the limit imposed by the international reliability standard for autonomous vehicles. It is then of paramount importance to understand the fault mechanisms, to track the fault propagation in the computing architecture and the software to, ultimately, design efficient and effective hardening solutions.
Ensuring a high reliability of object detection has a strategic importance for the European society. Object detection is essential to implement self-driving cars and autonomous aerospace systems. Only a significant increase of the reliability of current (and future) object detection frameworks will allow the employment of autonomous vehicles in large scale. The main objective of PERIOD is actually to increase the safety of people driving, Europe security, and burst space exploration. PERIOD will give guidelines on how to produce reliable computing architectures and software to disclose autonomy for space applications, helping ESA to maintain a leading role in the international space market. Finally, Unmanned Aerial Vehicles (UAV) also require object detection. PERIOD will help the European union in developing more reliable frameworks for increasing the union security. As a result, PERIOD will strongly contribute to support crucial research and strategic sectors with consequent European excellence and competitiveness in the exponentially growing autonomous vehicles market, with the final aim of improving quality and safety of life in Europe.
Thus, the main objectives of PERIOD are:
1. Understand fault generation and propagation in current and future parallel, heterogeneous, and programmable computing architectures.
2. Identify the code portions or hardware resources whose corruption is responsible for erroneous detection, and formalize the distinction between tolerable and critical errors.
3. Develop and validate software and architectural hardware solutions to significantly reduce the error rate of current and next-generation object detection frameworks.
4. Increase the public's understanding on object detection reliability and disseminate the PERIOD results through frequent outreach activities dedicated to non-specialists (general public and high school students), with the final purpose of teaching reliability concepts in the autonomous vehicles era, encouraging the public interest in research careers, the training of skilled researchers, and the growth of new jobs and investments
Ensuring a high reliability of object detection has a strategic importance for the European society. Object detection is essential to implement self-driving cars and autonomous aerospace systems. Only a significant increase of the reliability of current (and future) object detection frameworks will allow the employment of autonomous vehicles in large scale. The main objective of PERIOD is actually to increase the safety of people driving, Europe security, and burst space exploration. PERIOD will give guidelines on how to produce reliable computing architectures and software to disclose autonomy for space applications, helping ESA to maintain a leading role in the international space market. Finally, Unmanned Aerial Vehicles (UAV) also require object detection. PERIOD will help the European union in developing more reliable frameworks for increasing the union security. As a result, PERIOD will strongly contribute to support crucial research and strategic sectors with consequent European excellence and competitiveness in the exponentially growing autonomous vehicles market, with the final aim of improving quality and safety of life in Europe.
Thus, the main objectives of PERIOD are:
1. Understand fault generation and propagation in current and future parallel, heterogeneous, and programmable computing architectures.
2. Identify the code portions or hardware resources whose corruption is responsible for erroneous detection, and formalize the distinction between tolerable and critical errors.
3. Develop and validate software and architectural hardware solutions to significantly reduce the error rate of current and next-generation object detection frameworks.
4. Increase the public's understanding on object detection reliability and disseminate the PERIOD results through frequent outreach activities dedicated to non-specialists (general public and high school students), with the final purpose of teaching reliability concepts in the autonomous vehicles era, encouraging the public interest in research careers, the training of skilled researchers, and the growth of new jobs and investments
The work performed during the project includes:
(1) A detailed analysis of the most advanced devices and algorithms for object detection, such as NVIDIA GPUs and accelerators, such as Google's TPUs. On the software side, the studied object detection codes are Convolution Neural Networks (CNNs), such as You Only Look Once, Faster R-CNN, BirdNet, ResNet, etc…
We have performed several radiation beam experiments, both remotely (at ChipIR, UK) and in presence (at ILL, France) that allowed to have a complete understanding of the main computing characteristics of the most advanced devices for CNN and object detection execution.
(2) We have evaluated the reliability of hardware and software for object detection. To have a realistic and detailed evaluation of faults generation and propagation we have combined accelerated neutron beam experiments with software/architecture fault injection. We have also identify the faults that are more critical for the object detection framework and should then be detected or corrected.
(3) We have designed efficient and effective hardening solutions at different levels of abstraction. We have considered and evaluate hardware protection on the most critical memory resources, we have proposed an updated architecture for using redundant GPU resources to have a fast error detection, and we have designed dedicated hardening solutions at software level, re-designing the neural network.
(4) Several important actions have been taken to disseminate PERIOD study and results. Given the pandemic issue, most of these events were online, not affecting the PERIOD budget.
A- I have participated to the researchers' night in September 2021, with a public interview and a "walk with the researcher". I have discussed with ten people the importance of object detection reliability while walking the streets of Torino.
B- I have given 8 lessons to high school pupils in Italy, discussing the importance and challenges in autonomous systems. The attendance and interaction were really exciting, with more than 160 students involved, overall.
C- I have been interviewed in the host institution web radio (hundreds of daily audience), I have recorded a podcast about the PERIOD project, I have talked about my research to international journalists.
D - I have organized a seminar at the host institution, attended by 50 researchers, and I have given two lessons to the host institution students about my research.
E - I was selected as the Marie Curie Fellow of the Week, with hundreds of interactions in the social media (facebook, linkedin, instagram, twitter)
F - We have published several papers in highly important journals and conferences (more details in the publication session). I gave 2 invited talks (100 of attendees), one at the SUREALIST workshop, in August, and one at the ART conference. I also organized SELSE 2021, as a General Chair, that was attended by more than 100 researchers and students worldwide.
(1) A detailed analysis of the most advanced devices and algorithms for object detection, such as NVIDIA GPUs and accelerators, such as Google's TPUs. On the software side, the studied object detection codes are Convolution Neural Networks (CNNs), such as You Only Look Once, Faster R-CNN, BirdNet, ResNet, etc…
We have performed several radiation beam experiments, both remotely (at ChipIR, UK) and in presence (at ILL, France) that allowed to have a complete understanding of the main computing characteristics of the most advanced devices for CNN and object detection execution.
(2) We have evaluated the reliability of hardware and software for object detection. To have a realistic and detailed evaluation of faults generation and propagation we have combined accelerated neutron beam experiments with software/architecture fault injection. We have also identify the faults that are more critical for the object detection framework and should then be detected or corrected.
(3) We have designed efficient and effective hardening solutions at different levels of abstraction. We have considered and evaluate hardware protection on the most critical memory resources, we have proposed an updated architecture for using redundant GPU resources to have a fast error detection, and we have designed dedicated hardening solutions at software level, re-designing the neural network.
(4) Several important actions have been taken to disseminate PERIOD study and results. Given the pandemic issue, most of these events were online, not affecting the PERIOD budget.
A- I have participated to the researchers' night in September 2021, with a public interview and a "walk with the researcher". I have discussed with ten people the importance of object detection reliability while walking the streets of Torino.
B- I have given 8 lessons to high school pupils in Italy, discussing the importance and challenges in autonomous systems. The attendance and interaction were really exciting, with more than 160 students involved, overall.
C- I have been interviewed in the host institution web radio (hundreds of daily audience), I have recorded a podcast about the PERIOD project, I have talked about my research to international journalists.
D - I have organized a seminar at the host institution, attended by 50 researchers, and I have given two lessons to the host institution students about my research.
E - I was selected as the Marie Curie Fellow of the Week, with hundreds of interactions in the social media (facebook, linkedin, instagram, twitter)
F - We have published several papers in highly important journals and conferences (more details in the publication session). I gave 2 invited talks (100 of attendees), one at the SUREALIST workshop, in August, and one at the ART conference. I also organized SELSE 2021, as a General Chair, that was attended by more than 100 researchers and students worldwide.
PERIOD significantly advances the knowledge on object detection reliability by considering realistic error models (provided by neutron beam experiments) and understanding the propagation of faults injected not only in memory elements, but also in computing resources. This is the first project that proposes such a wide spectrum reliability analysis. We consider and compare commercially available frameworks, different available computing architectures (parallel, heterogeneous, and programmable), and predict next-generation hardware and software systems reliability. Finally, the developed hardening strategies will address faults at different levels of abstractions and are intended to be extremely optimized and tailored specifically for object detection systems.
PERIOD results have a huge potential impact on both society, science, and economy. Ensuring reliable object detection frameworks reliability is the first, fundamental, step to allow the large-scale adoption of self-driving cars, clearly changing the transportation system and the economy of the automotive sector. Moreover, reliable autonomous vehicles are expected to burst deep space exploration.
PERIOD results have a huge potential impact on both society, science, and economy. Ensuring reliable object detection frameworks reliability is the first, fundamental, step to allow the large-scale adoption of self-driving cars, clearly changing the transportation system and the economy of the automotive sector. Moreover, reliable autonomous vehicles are expected to burst deep space exploration.