Periodic Reporting for period 2 - SMART-RCS (Disrupting the Restraint Control Systems (RCS) Market: Personalized Restraint Technology for Boosting Human Safety in Vehicles)
Reporting period: 2022-03-01 to 2023-02-28
Worldwide, over 1 million people die each year in road accidents and millions more suffer from injuries. Mandatory Passive Safety Systems (PSS) e.g. in case of a crash, trigger passive safety functions such as airbags and seat belts to reduce the number of fatalities and heavy injuries. However, these PSS follow a ”few-sizes-fit-all” approach (optimized to an “average person” – 175cm, 78kg and male) and thus today, especially for women, children, elderly and people deviating from the average, this leads to significantly higher risks. In addition, protection is tuned and optimized for an occupant in a standardized position without considering the occupant’s motion that may be induced by events occurring before the crash.
Our project disrupts the existing PSS market by introducing a novel highly personalized protection. For the first time, touchless 3D imaging sensors are used to derive precise and real-time information about each occupant to control the PSS mechanisms tailored to each individual occupant.
Summarized, the following objectives are tackled within the project in order to reach the aforementioned goals:
Objective A: to use emotion3D’s 3D human sensing methods and software and adapt it to the restraint control case
Objective B: to integrate this new software in Veoneer’s RCS
Objective C: to test and validate this system using profound testing expertise from AVL/HU & AVL/AT
Objective D: to contribute to testing & safety standards
Objective E: to pave the way to bring the new personalized RCS to the automotive market
During this project we are developing our system up to TRL8, standardize & prepare its components for the market needs and implement quality assurance processes required by the automotive industry. Also, a first pilot project with a car manufacturer shall be conducted to allow a short time-to-market across the EU and worldwide on the long run. Our impact targets: saving more than 11.000 lives and preventing over 600.000 injuries between 2025 and 2035.
Our project disrupts the existing PSS market by introducing a novel highly personalized protection. For the first time, touchless 3D imaging sensors are used to derive precise and real-time information about each occupant to control the PSS mechanisms tailored to each individual occupant.
Summarized, the following objectives are tackled within the project in order to reach the aforementioned goals:
Objective A: to use emotion3D’s 3D human sensing methods and software and adapt it to the restraint control case
Objective B: to integrate this new software in Veoneer’s RCS
Objective C: to test and validate this system using profound testing expertise from AVL/HU & AVL/AT
Objective D: to contribute to testing & safety standards
Objective E: to pave the way to bring the new personalized RCS to the automotive market
During this project we are developing our system up to TRL8, standardize & prepare its components for the market needs and implement quality assurance processes required by the automotive industry. Also, a first pilot project with a car manufacturer shall be conducted to allow a short time-to-market across the EU and worldwide on the long run. Our impact targets: saving more than 11.000 lives and preventing over 600.000 injuries between 2025 and 2035.
The performed work can be summarized as follows:
Objective A: to use emotion3D’s 3D human sensing methods and software and adapt it to the restraint control case:
emotion3D in close collaboration with the consortium members was working towards adapting existing 3D human sensing software. The software was adapted in such a way that Veoneer’s Time of Flight (ToF) camera could be used as an input for the algorithms and provides all the relevant output needed for controlling the RCS. After the first reporting period, a first version of the software was available. This version could already process the output generated by Veoneer’s ToF camera and does already provide meaningful output data. Veoneer's ToF camera was then further developed to obtain a prototype B, which has a smaller form factor and more precise measurements. At the end of the project, an optimized software in terms of accuracy and runtime is available, which also supports the optimized ToF.
Objective B: to integrate this new software in Veoneer’s RCS:
After adaption of emotion3D’s software framework, the software was ported onto Veoneer’s processing platform. First, prototype A was used and then the software was also ported on prototype B.
Objective C: to test and validate this system using profound testing expertise from AVL:
For setting up a testing and validation strategy, capturing massive amounts of data is needed. This data should be complete, precise, and reproducible, it should be representative in terms of test-persons, test-vehicles and test-scenarios, efficiently acquired and in compliance with GDPR. The objective was therefore – in collaboration with all consortium members – to define the relevant data to be acquired for the development, verification, and validation of the algorithms & systems. AVL has developed a central data acquisition system to acquire data from all sources and to ensure the synchronization of the same in all test environments. An additional, camera-based ground-truth system has been integrated in the data acquisition system. Furthermore, virtual test-scenarios were created to be executed on the dynamic Driver-in-the-Loop simulator at AVL and ensure reproducible and comparable data. A workflow was defined and is continuously improved, to efficiently execute the test scenarios in all environments. To validate and improve the increased safety of the Smart-RCS, a virtual crash-simulation was set up which will compare the injury severity of traditional restraint control systems vs. Smart RCS in various parameterizations.
Objective D: to contribute to testing & safety standards
On the one hand, testing procedures were developed that will represent a starting point for official testing protocols. On the other hand, regulators and safety organisations must be made aware of the safety benefits smart-RCS provides. Thus, we were engaging in discussions with several of such organisations. Also, we have compiled an advisory board consisting of top-level academia, Tier-1, OEM and regulatory representatives. At the end of the project, we had some recommendations and findings available, which we would also like to share with potential customers after the end of the project.
Objective E: to pave the way to bring the new personalized RCS to the automotive market
To date, we have set numerous general communication activities and engaged intensively with potential customers. The feedback from key stakeholders of the industry has been tremendously positive and there is strong interest from potential customers to start proof of concept development projects. We are on a good way to successfully bring Smart-RCS to the automotive market.
Exploitation:
The consortium developed a demonstrator that was shown to several OEMs over the course of this project. The demonstrator will also be used beyond the scope and after finishing the project.
Dissemination:
We published 2 publications at peer-reviewed conferences.
Objective A: to use emotion3D’s 3D human sensing methods and software and adapt it to the restraint control case:
emotion3D in close collaboration with the consortium members was working towards adapting existing 3D human sensing software. The software was adapted in such a way that Veoneer’s Time of Flight (ToF) camera could be used as an input for the algorithms and provides all the relevant output needed for controlling the RCS. After the first reporting period, a first version of the software was available. This version could already process the output generated by Veoneer’s ToF camera and does already provide meaningful output data. Veoneer's ToF camera was then further developed to obtain a prototype B, which has a smaller form factor and more precise measurements. At the end of the project, an optimized software in terms of accuracy and runtime is available, which also supports the optimized ToF.
Objective B: to integrate this new software in Veoneer’s RCS:
After adaption of emotion3D’s software framework, the software was ported onto Veoneer’s processing platform. First, prototype A was used and then the software was also ported on prototype B.
Objective C: to test and validate this system using profound testing expertise from AVL:
For setting up a testing and validation strategy, capturing massive amounts of data is needed. This data should be complete, precise, and reproducible, it should be representative in terms of test-persons, test-vehicles and test-scenarios, efficiently acquired and in compliance with GDPR. The objective was therefore – in collaboration with all consortium members – to define the relevant data to be acquired for the development, verification, and validation of the algorithms & systems. AVL has developed a central data acquisition system to acquire data from all sources and to ensure the synchronization of the same in all test environments. An additional, camera-based ground-truth system has been integrated in the data acquisition system. Furthermore, virtual test-scenarios were created to be executed on the dynamic Driver-in-the-Loop simulator at AVL and ensure reproducible and comparable data. A workflow was defined and is continuously improved, to efficiently execute the test scenarios in all environments. To validate and improve the increased safety of the Smart-RCS, a virtual crash-simulation was set up which will compare the injury severity of traditional restraint control systems vs. Smart RCS in various parameterizations.
Objective D: to contribute to testing & safety standards
On the one hand, testing procedures were developed that will represent a starting point for official testing protocols. On the other hand, regulators and safety organisations must be made aware of the safety benefits smart-RCS provides. Thus, we were engaging in discussions with several of such organisations. Also, we have compiled an advisory board consisting of top-level academia, Tier-1, OEM and regulatory representatives. At the end of the project, we had some recommendations and findings available, which we would also like to share with potential customers after the end of the project.
Objective E: to pave the way to bring the new personalized RCS to the automotive market
To date, we have set numerous general communication activities and engaged intensively with potential customers. The feedback from key stakeholders of the industry has been tremendously positive and there is strong interest from potential customers to start proof of concept development projects. We are on a good way to successfully bring Smart-RCS to the automotive market.
Exploitation:
The consortium developed a demonstrator that was shown to several OEMs over the course of this project. The demonstrator will also be used beyond the scope and after finishing the project.
Dissemination:
We published 2 publications at peer-reviewed conferences.
Progress beyond the state of the art and expected results:
Current restraint control systems’ capabilities of sensing, understanding and adapting to what happens inside the cabin are very limited. The world's first personalized restraint control system - SMART-RCS - will be able to take into account individual human body characteristics and situational factors to optimally engage the car’s passive safety systems in case of an inevitable crash or other critical situations (e.g. an emergency brake manoeuvre). How it works: Data from a 3D camera installed inside the vehicle cabin is processed by machine learning algorithms to obtain precise information about all occupants’ body physique, body position and pose, weight, sex and age as well as information about the seatbelt (e.g. applied correctly, twisted). This information is then used by an adaptive restraint control system to perfectly adjust every parameter of the car’s passive safety system to each occupant individually in real time. This includes the deployment of a multitude of airbags in every critical location with the best suitable level of force to protect each occupant in a personalized and situationally optimized way.
Potential impacts:
A contribution to road safety is a clear societal impact. Additionally, our personalized system is a major boost towards diversity & equality in automotive safety technology with wide implications throughout society as mobility is a basic human need, practically involving every human being.
Current restraint control systems’ capabilities of sensing, understanding and adapting to what happens inside the cabin are very limited. The world's first personalized restraint control system - SMART-RCS - will be able to take into account individual human body characteristics and situational factors to optimally engage the car’s passive safety systems in case of an inevitable crash or other critical situations (e.g. an emergency brake manoeuvre). How it works: Data from a 3D camera installed inside the vehicle cabin is processed by machine learning algorithms to obtain precise information about all occupants’ body physique, body position and pose, weight, sex and age as well as information about the seatbelt (e.g. applied correctly, twisted). This information is then used by an adaptive restraint control system to perfectly adjust every parameter of the car’s passive safety system to each occupant individually in real time. This includes the deployment of a multitude of airbags in every critical location with the best suitable level of force to protect each occupant in a personalized and situationally optimized way.
Potential impacts:
A contribution to road safety is a clear societal impact. Additionally, our personalized system is a major boost towards diversity & equality in automotive safety technology with wide implications throughout society as mobility is a basic human need, practically involving every human being.