Skip to main content
European Commission logo print header

EndoMapper: Real-time mapping from endoscopic video

Descrizione del progetto

Gli endoscopi medici potrebbero presto ottenere il via libera per la realtà aumentata e la navigazione autonoma

La mappatura delle regioni importanti è fondamentale per interagire con l’ambiente, dalle mappe satellitari dei ghiacciai alle mappe funzionali per i navigatori GPS. I veicoli autonomi funzionano anche abbinando i dati visivi a una mappa esistente che deve essere costantemente aggiornata ed ampliata. Ora, EndoMapper sta sviluppando un sistema di mappatura simile per supportare procedure endoscopiche quali la colonscopia, la biopsia tumorale o persino la distribuzione mirata di farmaci. Per raggiungere il primato mondiale, il team deve affrontare la sfida di modellare la non rigidità. A differenza di strade e vie secondarie, i tessuti e gli organi del corpo tendono a cambiare durante gli spostamenti dell’endoscopio. Sfruttando nuovi modelli matematici non rigidi e l’apprendimento automatico, l’endoscopia potrebbe presto entrare nel regno della realtà virtuale aumentata e della navigazione autonoma.

Obiettivo

Endoscopes traversing body cavities such as the colon are routine in medical practice. However, they lack any autonomy. An endoscope operating autonomously inside a living body would require, in real-time, the cartography of the regions where it is navigating, and its localization within the map. The goal of EndoMapper is to develop the fundamentals for real-time localization and mapping inside the human body, using only the video stream supplied by a standard monocular endoscope.

In the short term, will bring to endoscopy live augmented reality, for example, to show to the surgeon the exact location of a tumour that was detected in a tomography, or to provide navigation instructions to reach the exact location where to perform a biopsy. In the longer term, deformable intracorporeal mapping and localization will become the basis for novel medical procedures that could include robotized autonomous interaction with the live tissue in minimally invasive surgery or automated drug delivery with millimetre accuracy.

Our objective is to research the fundamentals of non-rigid geometry methods to achieve, for the first time, mapping from GI endoscopies. We will combine three approaches to minimize the risk. Firstly, we will build a fully handcrafted EndoMapper approach based on existing state-of-the-art rigid pipelines. Overcoming the non-rigidity challenge will be achieved by the new non-rigid mathematical models for perspective cameras and tubular topology. Secondly, we will explore how to improve using machine learning. We propose to work on new deep learning models to compute matches along endoscopy sequences to feed them to a VSLAM algorithm where the non-rigid geometry is still hard-coded. We finally plan to attempt a more radical end-to-end deep learning approach, that incorporates the mathematical models for non-rigid geometry as part of the training of data-driven learning algorithms.

Invito a presentare proposte

H2020-FETOPEN-2018-2020

Vedi altri progetti per questo bando

Bando secondario

H2020-FETOPEN-2018-2019-2020-01

Meccanismo di finanziamento

RIA - Research and Innovation action
æ

Coordinatore

UNIVERSIDAD DE ZARAGOZA
Contribution nette de l'UE
€ 1 439 125,00
Indirizzo
Calle pedro cerbuna 12
50009 Zaragoza
Spagna

Mostra sulla mappa

Regione
Noreste Aragón Zaragoza
Tipo di attività
Higher or Secondary Education Establishments
Collegamenti
Altri finanziamenti
€ 0,00

Partecipanti (3)