Skip to main content

Robust and Reliable Environment Sensing and Situation Prediction for Advanced Driver Assistance Systems and Automated Driving

Periodic Reporting for period 3 - RobustSENSE (Robust and Reliable Environment Sensing and Situation Prediction for Advanced Driver Assistance Systems and Automated Driving)

Reporting period: 2017-08-01 to 2018-05-31

The vision of the RobustSENSE project partners is an autonomous vehicle capable of ensuring safe and comfortable travel for its occupants and other road users under all existing driving conditions.

Perceiving and understanding the environment and ultimately predicting the movement of objects in the vehicle’s surrounding are key factors to master automated driving. Systems on the market today are already capable of various ADAS functions under good environmental conditions. However, in adverse weather and lightning conditions the picture changes dramatically. In situations with increased risk of accidents as in harsh weather or bad lighting conditions, where ADAS systems can be particularly beneficial, human control is still needed. When system support would be particularly needed, current systems’ performance decreases drastically.

The next step to be taken is to increase the robustness of safety systems in all environmental conditions. RobustSENSE is aiming to enable systems to cope with real world requirements and introduces reliable, secure and trustable sensors and software by implementing self-diagnosis, adaptation and robustness.

During its lifetime the project focused on advanced methods for improved sensor technologies and sensor signal processing as well as innovative algorithms for sensor data fusion, scene understanding, behavioural planning, and trajectory planning. At the RobustSENSE Final Event in Ulm, Germany, on 16th of May 2018 the project showcased the potential of the developed technologies during driving demonstrations, a conference and exhibition. Further steps towards deployment of achievements are described in deliverable: D6.5 Exploitation Plan.
Coordinating the RobustSENSE project, the focus of WP1 was on supervising project activities and partner interactions. Project progress was regularly tracked: milestones, deliverables and resources. Besides the kick-off meeting, two General Assemblies, several physical PMT meetings, regular telcos and a Final Event took place. The "RobustSENSE Integration Manager" supervised the integration activities.

In WP2: The defined architecture of the RobustSENSE platform and its system specifications that were revised in P2 remained in P3. This included components for online operation and offline validation, the specified interfaces of the modules and the used metrics. Overall, work was finished in P2, but kept on supporting the consortium and presented the RobustSENSE architecture during the Final Event.

WP3: started with development of D3.1: system architecture of the environmental model. It was aligned with the overall architecture while considering the weaknesses of existing approaches. In P2 metrics for sensor performance monitoring were outlined (D3.4). Work on building up a prototype lidar sensor at 1550nm finished (D3.2). Data acquisition under adverse weather conditions. Further development of algorithms for lidar, radar and camera data processing to handle adverse weather conditions and to assess present performance and reliability. The development of high level fusion with uncertainty measurements was finalized.

In WP4: steps towards a coherent scene understanding, situation prediction, behavior planning and trajectory planning undertaken, including an algorithm performance assessment. The conceptual purpose of the modules was defined. Partners implemented the different parts for the coherent probabilistic modules and application scenarios were broadened. D4.2 included the description of recording and playback framework. Common interfaces among modules and a prototypical implementation of the modules achieved.

WP5: activities focused on the definition and development of the system performance assessment module. D5.1 collected data related to the definition of the system performance assessment module (SPAM) and use cases for testing. The assembly of sensor platforms continued, taking into account the RobustSENSE architecture. Parts of the sensor platform, incl. SPAM, were integrated in several test vehicles. Sensors were enhanced according to test outcomes and their performance validated.

The main task in WP6 was the development and execution of the overall dissemination and exploitation strategy. A corporate identity and a project were created. General information material developed. All partner supported in disseminating project results at conferences and events. Online channels were continuously updated. Exploitation activities included an online expert survey, an Exploitation Workshop and re-formulation of partner exploitation plans. Business plans for the main results of the project included in D6.5. During the RobustSENSE Final Event, all results were presented in a conference and exhibition. Achievements in the different architecture layers (WP3-WP5) demonstrated in demo vehicles.
RobustSENSE architecture, including interfaces between the different layers (e.g. Scene Understanding, Situation Prediction and Behaviour Planning) established.

Data acquisition: Concepts developed for specific radar and lidar sensors; Data acquisition in harsh weather taken into account.

Sensor data processing: Quality metrics defined; Overall architecture set; New lidar at 1550nm stable in tests: this low resolution sensor source can be used in foggy conditions, where the previous 905nm sensor stops working.

Information fusion: Quality metrics for output from data fusion to environment model defined; Fusion is able to detect ambiguity by redundancy.

Scene understanding: A concept and data collection for road condition monitoring using machine learning techniques developed; Road Understanding based on CNN concluded; Implementation of IDM-based behaviour models.

Situation prediction: Probability metrics for other traffic participants defined; Grid-based model-free learned prediction established; Trajectory generation using frenet frame for collision risk calculation using Minkowski sum; Performance assessment based on prediction improvement compared to basic model.

Behavioural planning: Measures and metrics defined; Definitions of metrics for radar and lidar sensor performance assessment worked out; Probabilistic Behaviour Planning based on Semantic State Space established.

Trajectory planning: Trajectory generation considering uncertainties of observed traffic; Multilane trajectory planning based on human behaviour and collision risk.

System Performance Monitoring: Development of sensor performance algorithms for the assessment of specific sensor performance. Generation of metrics as input of the overall SPAM of the RobustSENSE Architecture. By means of the SPAM outputs, the RobustSENSE platform is able to adapt online the safety gap. SPAM integrated in demonstrator vehicle.

Simulation framework: Simulation test bench developed to monitor the overall performance of the system. RobustSENSE platform with interfaces between layers implemented for vehicle on the testbed. Full adverse weather simulation possible.

Cooperative Driving: Complementing V2X data with using raw LiDAR data delivered from other vehicle with using pre-5G communication channel. Improved performance in turbulent snow and extended sensing range.

Most promising with regards to potential impact are:
1. RobustSENSE platform with self-diagnosis and adaptability
2. New lidar at 1550nm
In general, the initial impacts (as described in the DoA) remain.