A next generation high level sensing system to enable safe and high level automation of industrial vehicles.

Project Information

Tripleye 3D Perception System

Grant agreement ID: 190122948

DOI

10.3030/190122948

Project closed

EC signature date 3 February 2023

Start date 1 March 2023

End date 28 February 2026

Funded under

• The European Innovation Council (EIC)

Total cost

€ 3 569 375,00

EU contribution

€ 2 498 562,00

Coordinated by

TRIPLEYE GMBH
Germany

Periodic Reporting for period 1 - Tripleye 3D Perception System (A next generation high level sensing system to enable safe and high level automation of industrial vehicles.)

Reporting period: 2023-03-01 to 2024-02-29

Summary of the context and overall objectives of the project

The problem with today’s autonomous system architectures is that minimal changes in the camera alignment of less than 1 mm can lead to significant misalignments and invalidate measurements of the surrounding environment, leading to major safety concerns in autonomous vehicles. Tripleye has invented a revolutionary sensing system to solve this issue. Tripleye technology uses next generation camera only computer vision system that uses proprietary algorithms together with cameras to triangulate and capture high level 3d spatial components needed for advanced automation. Specifically, Tripleye offers a revolutionary solution for the safe and efficient operation of industrial vehicles. During the proposed EIC project, Tripleye looks to upgrade its software stacks, integrate the system into industrial machines in a real working environment for iterative testing.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In our project, we made significant advancements in the 3D Perception System, focusing on project management, motion and depth processing, and transitioning to embedded hardware.

Project Management: We implemented risk management and quality assurance frameworks, enhancing communication and financial oversight. Adjustments were made in response to continuous evaluations and external developments.
Motion and Depth Processing: Improved image processing techniques enhanced spatial recognition, with algorithms generating dense disparity maps for better depth perception.
Localization and Mapping: We refined motion detection using SLAM algorithms for precise vehicular movement tracking and improved localization with stereo camera technology.
Embedded Hardware: Analyzed embedded computing platforms for algoritms application optimization and adapted cameras to embedded boards.Algorithm Transition: Successfully transferred algorithms to embedded hardware, utilizing parallel processing for efficient performance.
Hardware Integration: Assembled sensing system with embedded computing and cameras. Combined different systems into one 360 FoV system, ensuring high-resolution, gap-free measurements through meticulous alignment.
Profiling and Optimization: Profiled the software stack to identify and optimize time-consuming processes. Updated algorithms with embedded hardware-oriented memory access, achieving real-time constraints.
Our project significantly advances 3D perception technology, optimizing motion and depth processing, and leveraging embedded hardware for improved image processing. This contributes to the technical and scientific fields associated with our objectives.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

In the realm of automation technology, three principal solutions vie for dominance, each hampered by distinct limitations that curtail their effectiveness and scalability. Firstly, model-based AI and camera systems utilize machine learning for environmental interpretation but struggle with certification due to their reliance on inferential measurements. Secondly, Lidar, Radar, and RGB camera suites face challenges with data management, complicating their integration and calibration within dynamic software ecosystems. Thirdly, stereoscopy offers direct measurements but is beleaguered by its inability to self-correct camera misalignments, necessitating frequent recalibrations that limit its utility in high-risk settings.
Emerging as a superior alternative, Tripleye 3D sensing system, leveraging off-the-shelf stereo fisheye cameras and innovative algorithms, excels in continuous recalibration, ensuring high accuracy in environmental visualization and obstacle avoidance. This breakthrough enables the system to potentially achieve functional safety certification, addressing the critical needs for enhanced operational productivity and safety.
Tripleye system markedly simplifies the installation and calibration process, dramatically reducing setup time to minutes, while its modular design facilitates scalability. It offers a cost-effective solution by merging the capabilities of Radar, Lidar, RGB, and Infrared cameras into a singular, advanced image processing system. This not only decreases installation and calibration time by 95% but also slashes costs by a factor of 3. Contrastingly, existing solutions such as AI-based systems and Lidar/Radar/RGB suites, while partially addressing automation challenges, fall short in high-risk scenarios due to their inherent limitations in data processing and sensor integration. Stereoscopy, despite its cost advantages, fails to meet the rigorous safety and operational standards required for high-level automation.
In summary, Tripleye system stands out for its ability to confront and mitigate prevalent issues in automation technology, offering significant improvements in safety, efficiency, and cost-effectiveness. This positions it as a transformative solution within the competitive landscape of automation technologies, promising to fulfill the increasingly complex demands of modern industries.