Periodic Reporting for period 1 - ULTRACEPT (Ultra-layered perception with brain-inspired information processing for vehicle collision avoidance)
Reporting period: 2018-12-01 to 2020-11-30
This ULTRACEPT consortium proposes an innovative solution with brain-inspired multiple layered and multiple modalities information processing for trustworthy vehicle collision detection. Connecting multidisciplinary teams from different countries together via staff exchange and collaboration, it takes the advantages of low-cost spatial-temporal and parallel computing capacity of bio-inspired visual neural systems and multiple modalities data inputs in extracting potential collision cues at complex weather and lighting conditions.
To enhance collision detection by integrating multiple visual neural systems, researchers from the consortium proposed novel LGMD model and directional selective visual neural system models with separated on/off channels. On multiple visual neural system integration and coordination in real time systems, researchers have begun integrating multiple visual neural networks including e-LGMD1, LGMD2, and DS (directional sensitive neural network), into one autonomous robotic system for verification. This work was published in IEEE Access in 2020.
On the system for long distance hazard perception, the consortium proposed new small target movement detector (STMD) models. The proposed model can detect small targets only a few pixels in size. The STMD models are the first step for knowing objects are approaching in distance that may develop into hazards in seconds. The collaborators in Germany have proposed complex-valued neural networks for real-valued classification problems as a white-box model. The proposed models can select the most important features from the complete feature space through a self-organizing modeling process. They also proposed hybrid classification framework based on clustering. These proposed models and methods contributed to both hazard perception and text and road marking recognition.
In order to capture other modalities rather than normal colour vision data, researchers compared thermal sensors from major suppliers and identified the right type of thermal image sensor for data acquisition. An early stage researcher has been working on thermal image camera pre-processing algorithms to enhance the contrast of the thermal map. The early research on bio-inspired neural systems models for processing thermal images for collision detection has been completed. It demonstrated that the LGMD works well with temperature-map-based images sensors.
The researchers on secondments to partner universities and SME’s explored the other modality input – sounds - to enhance the safety aspect for driving. Their sound analysis for road condition recognition has been disseminated at an international conference.
In collaboration with consortium partners from universities in China and the EU, a road collision database has been created and published for open access in Github and will be maintained by the consortium to include more scenarios over time based on feedback from users.
As part of a new spin off from the research and development activities inspired by the research outcomes of this consortium, researchers from a German partner university have been focusing on how to implement collision avoidance in the robotic scenarios. They have presented a Human-Robot collaboration pipeline that generates efficient and collision-free robot trajectories based on the early motion trajectory and intended target predictions of human arm with optimization capability. The results show that the generated robot trajectory is safe and efficient to complete the whole task together with humans.
In summary, in this reporting period, despite the difficulties arising from the pandemic, the consortium has completed: 9 deliverables; organized 3 joint workshops and 1 training seminar; completed 139 researcher months secondments with another 36 months in progress; published 10 journal articles, 14 conference papers; achieved 3 milestones as planned.