Periodic Reporting for period 2 - TULIPP (Towards Ubiquitous Low-power Image Processing Platforms)
Période du rapport: 2017-08-01 au 2019-01-31
TULIPP (Towards Ubiquitous Low-power Image Processing Platforms), an EU initiative, funded by its Horizon 2020 programme, targeting the development of high-performance, energy-efficient embedded systems for the growing range of increasingly complex image processing applications, has delivered three Use Cases covering Medical X-Ray Imaging, automotive ADAS (Advanced Driver Assistance Systems) and UAVs (Unmanned Aircraft Vehicles). The new Use Cases, coupled with the TULIPP embedded computing reference platform, deliver outstanding results for embedded vision applications.
The Medical X-Ray Imaging Use Case combines an embedded computing board with a medical X-ray imaging sensor to eliminate the noise on images when radiation doses are reduced. The ADAS Use Case enables the implementation of pedestrian detection algorithms running in real-time on a small, energy efficient, embedded platform. The UAVs Use Case equips a UAV with real-time obstacle detection and avoidance capabilities based on a lightweight and low-cost stereo camera setup.
At first glance, Medical X-Ray Imaging, ADAS and UAVs would appear to have very little in common, but that’s only true when viewed from the perspective of the final application as they all have a requirement for high-performance image processing and they also suffer from the so-called SWaPC (Size, Weight and Power and Costs) computing constraints typical of embedded systems. TULIPP has addressed these challenges by taking a diverse range of application domains as the basis for defining a common reference processing platform comprising the hardware, the operating system and its programming environment that captures the commonality of real-time, high-performance image processing and vision applications.
TULIPP’s Medical X-Ray Imaging Use Case demonstrates advanced image enhancement algorithms for X-Ray images running at high frame rates. It focuses on improving the performance of X-Ray imaging Mobile C-Arms, which provide an internal view of a patient’s body in real-time during the course of an operation to deliver, increases surgeon efficiency and accuracy with minimal incision sizes, aids faster patient recovery and lowers nosocomial disease risks. Using TULIPP’s platform, which is the size of a smartphone, the Use Case demonstrates how radiation doses to which patients and staff are exposed, which are typically 30 times ambient radiation levels, can be reduced by 75% at the same time as maintaining the clarity of the real-time X-Ray images which would otherwise be rendered useless by the increases in the noise level on the images that a reduced radiation dose can cause.
ADAS adoption is dependent on the implementation of vision systems or on combinations of vision and radar and the algorithms must be capable of integration into a small, energy-efficient Electronic Control Unit (ECU). An ADAS algorithm should be able to process a video image stream with a frame size of 640x480 at a full 30Hz or at least at the half rate. The TULIPP ADAS Use Case demonstrates pedestrian recognition in real-time based on Viola & Jones algorithm. Using the TULIPP platform, the ADAS Use Case achieves a processing time per frame of 66ms, which means that the algorithm reaches the target of running on every second image when the camera runs at 30Hz.
TULIPP’s UAV Use Case demonstrates a real-time obstacle avoidance system for UAVs based on a stereo camera setup with cameras orientated in the direction of flight. Even though we talk about autonomous drones, most current systems are still remotely piloted by humans. The Use Case uses disparity maps, which are computed from the camera images, to locate obstacles in the flight path and to automatically steer the UAV around them. This is the necessary key towards totally autonomous drones.
The TULIPP Use Cases, coupled with TULIPP the development kit, comprising hardware platform, multi-core real-time operating system, development tool chain and guidelines, have demonstrated that the computational demands of complex image processing can be delivered in a diverse range of embedded applications within the context of challenging size, weight, power consumption and costs constraints.
At first glance, Medical X-Ray Imaging, ADAS and UAVs would appear to have very little in common, but that’s only true when viewed from the perspective of the final application as they all have a requirement for high-performance image processing and they also suffer from the so-called SWaPC (Size, Weight and Power and Costs) computing constraints typical of embedded systems. TULIPP has addressed these challenges by taking a diverse range of application domains as the basis for defining a common reference processing platform comprising the hardware, the operating system and its programming environment that captures the commonality of real-time, high-performance image processing and vision applications.
TULIPP’s Medical X-Ray Imaging Use Case demonstrates advanced image enhancement algorithms for X-Ray images running at high frame rates. It focuses on improving the performance of X-Ray imaging Mobile C-Arms, which provide an internal view of a patient’s body in real-time during the course of an operation to deliver, increases surgeon efficiency and accuracy with minimal incision sizes, aids faster patient recovery and lowers nosocomial disease risks. Using TULIPP’s platform, which is the size of a smartphone, the Use Case demonstrates how radiation doses to which patients and staff are exposed, which are typically 30 times ambient radiation levels, can be reduced by 75% at the same time as maintaining the clarity of the real-time X-Ray images which would otherwise be rendered useless by the increases in the noise level on the images that a reduced radiation dose can cause.
ADAS adoption is dependent on the implementation of vision systems or on combinations of vision and radar and the algorithms must be capable of integration into a small, energy-efficient Electronic Control Unit (ECU). An ADAS algorithm should be able to process a video image stream with a frame size of 640x480 at a full 30Hz or at least at the half rate. The TULIPP ADAS Use Case demonstrates pedestrian recognition in real-time based on Viola & Jones algorithm. Using the TULIPP platform, the ADAS Use Case achieves a processing time per frame of 66ms, which means that the algorithm reaches the target of running on every second image when the camera runs at 30Hz.
TULIPP’s UAV Use Case demonstrates a real-time obstacle avoidance system for UAVs based on a stereo camera setup with cameras orientated in the direction of flight. Even though we talk about autonomous drones, most current systems are still remotely piloted by humans. The Use Case uses disparity maps, which are computed from the camera images, to locate obstacles in the flight path and to automatically steer the UAV around them. This is the necessary key towards totally autonomous drones.
The TULIPP Use Cases, coupled with TULIPP the development kit, comprising hardware platform, multi-core real-time operating system, development tool chain and guidelines, have demonstrated that the computational demands of complex image processing can be delivered in a diverse range of embedded applications within the context of challenging size, weight, power consumption and costs constraints.
The project began work in 2016 and finished on January 2019 with a hands-on-tutorial to the ecosystem. TULIPP provided vision-based system designers with a reference platform that defines implementation rules and interfaces designed to tackle power consumption issues while delivering guaranteed, high performance computing power. TULIPP started an ecosystem of stakeholders that will last after then end of the project to provide its starter-kit and continuous improvements of the reference platform.
For more information on TULIPP, please visit: http://www.tulipp.eu. For further information on the TULIPP consortium members see:
Thales - www.thalesgroup.com
Efficient Innovation SAS - www.efficient-innovation.fr
Fraunhofer IOSB – www.iosb.fraunhofer.de
Hipperos – www.hipperos.com
Norges Teknisk-Naturvitenskapelige Universitet – www.ntnu.no
Ruhr-Universität Bochum – www.ruhr-uni-bochum.de
Sundance Multiprocessor Technology – www.sundance.com
Synective Labs – www.synective.se
For more information on TULIPP, please visit: http://www.tulipp.eu. For further information on the TULIPP consortium members see:
Thales - www.thalesgroup.com
Efficient Innovation SAS - www.efficient-innovation.fr
Fraunhofer IOSB – www.iosb.fraunhofer.de
Hipperos – www.hipperos.com
Norges Teknisk-Naturvitenskapelige Universitet – www.ntnu.no
Ruhr-Universität Bochum – www.ruhr-uni-bochum.de
Sundance Multiprocessor Technology – www.sundance.com
Synective Labs – www.synective.se