A next generation high level sensing system to enable safe and high level automation of industrial vehicles.

Informations projet

Tripleye 3D Perception System

N° de convention de subvention: 190122948

DOI

10.3030/190122948

Projet clôturé

Date de signature de la CE 3 Février 2023

Date de début 1 Mars 2023

Date de fin 28 Février 2026

Financé au titre de

• The European Innovation Council (EIC)

Coût total

€ 3 569 375,00

Contribution de l’UE

€ 2 498 562,00

Coordonné par

TRIPLEYE GMBH
Germany

Periodic Reporting for period 1 - Tripleye 3D Perception System (A next generation high level sensing system to enable safe and high level automation of industrial vehicles.)

Période du rapport: 2023-03-01 au 2024-02-29

Résumé du contexte et des objectifs généraux du projet

The problem with today’s autonomous system architectures is that minimal changes in the camera alignment of less than 1 mm can lead to significant misalignments and invalidate measurements of the surrounding environment, leading to major safety concerns in autonomous vehicles. Tripleye has invented a revolutionary sensing system to solve this issue. Tripleye technology uses next generation camera only computer vision system that uses proprietary algorithms together with cameras to triangulate and capture high level 3d spatial components needed for advanced automation. Specifically, Tripleye offers a revolutionary solution for the safe and efficient operation of industrial vehicles. During the proposed EIC project, Tripleye looks to upgrade its software stacks, integrate the system into industrial machines in a real working environment for iterative testing.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

In our project, we made significant advancements in the 3D Perception System, focusing on project management, motion and depth processing, and transitioning to embedded hardware.

Project Management: We implemented risk management and quality assurance frameworks, enhancing communication and financial oversight. Adjustments were made in response to continuous evaluations and external developments.
Motion and Depth Processing: Improved image processing techniques enhanced spatial recognition, with algorithms generating dense disparity maps for better depth perception.
Localization and Mapping: We refined motion detection using SLAM algorithms for precise vehicular movement tracking and improved localization with stereo camera technology.
Embedded Hardware: Analyzed embedded computing platforms for algoritms application optimization and adapted cameras to embedded boards.Algorithm Transition: Successfully transferred algorithms to embedded hardware, utilizing parallel processing for efficient performance.
Hardware Integration: Assembled sensing system with embedded computing and cameras. Combined different systems into one 360 FoV system, ensuring high-resolution, gap-free measurements through meticulous alignment.
Profiling and Optimization: Profiled the software stack to identify and optimize time-consuming processes. Updated algorithms with embedded hardware-oriented memory access, achieving real-time constraints.
Our project significantly advances 3D perception technology, optimizing motion and depth processing, and leveraging embedded hardware for improved image processing. This contributes to the technical and scientific fields associated with our objectives.

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

In the realm of automation technology, three principal solutions vie for dominance, each hampered by distinct limitations that curtail their effectiveness and scalability. Firstly, model-based AI and camera systems utilize machine learning for environmental interpretation but struggle with certification due to their reliance on inferential measurements. Secondly, Lidar, Radar, and RGB camera suites face challenges with data management, complicating their integration and calibration within dynamic software ecosystems. Thirdly, stereoscopy offers direct measurements but is beleaguered by its inability to self-correct camera misalignments, necessitating frequent recalibrations that limit its utility in high-risk settings.
Emerging as a superior alternative, Tripleye 3D sensing system, leveraging off-the-shelf stereo fisheye cameras and innovative algorithms, excels in continuous recalibration, ensuring high accuracy in environmental visualization and obstacle avoidance. This breakthrough enables the system to potentially achieve functional safety certification, addressing the critical needs for enhanced operational productivity and safety.
Tripleye system markedly simplifies the installation and calibration process, dramatically reducing setup time to minutes, while its modular design facilitates scalability. It offers a cost-effective solution by merging the capabilities of Radar, Lidar, RGB, and Infrared cameras into a singular, advanced image processing system. This not only decreases installation and calibration time by 95% but also slashes costs by a factor of 3. Contrastingly, existing solutions such as AI-based systems and Lidar/Radar/RGB suites, while partially addressing automation challenges, fall short in high-risk scenarios due to their inherent limitations in data processing and sensor integration. Stereoscopy, despite its cost advantages, fails to meet the rigorous safety and operational standards required for high-level automation.
In summary, Tripleye system stands out for its ability to confront and mitigate prevalent issues in automation technology, offering significant improvements in safety, efficiency, and cost-effectiveness. This positions it as a transformative solution within the competitive landscape of automation technologies, promising to fulfill the increasingly complex demands of modern industries.