Designing machines that see, understand and interpret their environment An EU initiative has delivered a comprehensive reference platform for vision-based system designers. This will improve road safety, help create fully autonomous drones and increase surgical efficiency. Digital Economy © metamorworks, Shutterstock Just imagine a pedestrian glued to a mobile phone screen while crossing the street and not paying much attention to the red traffic light. A car approaches, its driver is perhaps feeling a bit drowsy because of lack of sleep and isn’t able to stop immediately. How can you avoid an accident in such a scene? From advanced driver assistance systems (ADASs) helping drivers navigate a vehicle, to search and rescue drones, to medical X-ray imaging, embedded vision technologies are increasingly used in a wide range of applications. These involve the integration of computer vision in machines that use algorithms to decode meaning from observing pixel patterns in images or video. To be able to correctly interpret their environment using complex visual inputs, embedded vision systems require lots of processing power. In addition to power consumption, designers of such embedded systems need to overcome other technical constraints such as cost, size, weight and acoustic noise. The EU-funded TULIPP project has addressed these challenges and developed a reference platform for vision-based system designers. The TULIPP solution will help “computer vision product designers to readily address the combined challenges of low power, low latency, high performance and real-time image processing design constraints,” as stated in a press release. Real-world cases The TULIPP reference platform consists of a full development kit and real-world use cases. The kit includes “an FPGA-based embedded, multicore computing board, parallel real-time operating system and development tool chain with guidelines,” according to the same press release. Field programmable gate array (FPGA) refers to an integrated circuit that can be programmed or reprogrammed to the required functionality or application after manufacturing. The use cases cover “medical X-ray imaging, automotive Advanced Driver Assistance Systems (ADAS) and Unmanned Aerial vehicles (UAVs).” The medical X-ray imaging application is aimed at improving surgical efficiency with the mobile C-arm use case. This device displays an internal view of a patient’s body in real time during the course of a surgical operation, allowing the doctor to make minimal incisions with more accuracy. This leads to faster recovery times and “lowers nosocomial disease risks and reduces by 75 % the radiation doses to which patients and staff are exposed,” the press release adds. Through its pedestrian detection application, the ADAS use case “achieves a processing time per frame of 66ms, which means that the algorithm reaches the target of running on every second image when the camera runs at 30Hz.” In the use case of UAV, TULIPP estimates depth images from a stereo camera setup that is orientated in the direction of flight. “Even though we talk about autonomous drones, most current systems are still remotely piloted by humans. The Use Case uses disparity maps, which are computed from the camera images, to locate obstacles in the flight path and to automatically steer the UAV around them. This is the necessary key towards fully autonomous drones.” The 3-year TULIPP (Towards Ubiquitous Low-power Image Processing Platforms) project ended in January 2019. It focused on developing high-performance, energy-efficient embedded systems for the growing variety of progressively complex image processing applications emerging across a wide range of industry sectors. For more information, please see: TULIPP project website Countries France
