Project description DEENESFRITPL Medical endoscopes may soon get the green light for augmented reality and autonomous navigation Mapping of regions of importance is fundamental to interact with the environment, from satellite maps of glaciers to functional maps for GPS navigators. Autonomous vehicles also work by matching visual data to an existing map which needs to be constantly updated and extended. Now, EndoMapper is developing a similar mapping system to support endoscopic procedures like colonoscopy, tumour biopsy, or even targeted drug delivery. To accomplish their world-first, the team must tackle the challenge of modelling non-rigidity. Unlike roads and byways, the body's tissues and organs are prone to change as the endoscope moves. Taking advantage of new non-rigid mathematical models and machine learning, endoscopy may soon enter the realm of augmented virtual reality and autonomous navigation. Show the project objective Hide the project objective Objective Endoscopes traversing body cavities such as the colon are routine in medical practice. However, they lack any autonomy. An endoscope operating autonomously inside a living body would require, in real-time, the cartography of the regions where it is navigating, and its localization within the map. The goal of EndoMapper is to develop the fundamentals for real-time localization and mapping inside the human body, using only the video stream supplied by a standard monocular endoscope. In the short term, will bring to endoscopy live augmented reality, for example, to show to the surgeon the exact location of a tumour that was detected in a tomography, or to provide navigation instructions to reach the exact location where to perform a biopsy. In the longer term, deformable intracorporeal mapping and localization will become the basis for novel medical procedures that could include robotized autonomous interaction with the live tissue in minimally invasive surgery or automated drug delivery with millimetre accuracy. Our objective is to research the fundamentals of non-rigid geometry methods to achieve, for the first time, mapping from GI endoscopies. We will combine three approaches to minimize the risk. Firstly, we will build a fully handcrafted EndoMapper approach based on existing state-of-the-art rigid pipelines. Overcoming the non-rigidity challenge will be achieved by the new non-rigid mathematical models for perspective cameras and tubular topology. Secondly, we will explore how to improve using machine learning. We propose to work on new deep learning models to compute matches along endoscopy sequences to feed them to a VSLAM algorithm where the non-rigid geometry is still hard-coded. We finally plan to attempt a more radical end-to-end deep learning approach, that incorporates the mathematical models for non-rigid geometry as part of the training of data-driven learning algorithms. Fields of science natural sciencesearth and related environmental sciencesphysical geographycartographynatural sciencescomputer and information sciencesartificial intelligencemachine learningdeep learningnatural sciencesmathematicspure mathematicsgeometrynatural sciencescomputer and information sciencessoftwaresoftware applicationssimulation softwarenatural sciencesmathematicsapplied mathematicsmathematical model Keywords Visual SLAM Deformable Visual SLAM Machine Learning for Endoscopy Programme(s) H2020-EU.1.2. - EXCELLENT SCIENCE - Future and Emerging Technologies (FET) Main Programme H2020-EU.1.2.1. - FET Open Topic(s) FETOPEN-01-2018-2019-2020 - FET-Open Challenging Current Thinking Call for proposal H2020-FETOPEN-2018-2020 See other projects for this call Sub call H2020-FETOPEN-2018-2019-2020-01 Funding Scheme RIA - Research and Innovation action Coordinator UNIVERSIDAD DE ZARAGOZA Net EU contribution € 1 439 125,00 Address Calle pedro cerbuna 12 50009 Zaragoza Spain See on map Region Noreste Aragón Zaragoza Activity type Higher or Secondary Education Establishments Links Contact the organisation Opens in new window Website Opens in new window Participation in EU R&I programmes Opens in new window HORIZON collaboration network Opens in new window Other funding € 0,00 Participants (3) Sort alphabetically Sort by Net EU contribution Expand all Collapse all UNIVERSITE CLERMONT AUVERGNE France Net EU contribution € 784 950,00 Address 49 bd francois mitterrand 63000 Clermont ferrand See on map Region Auvergne-Rhône-Alpes Auvergne Puy-de-Dôme Activity type Higher or Secondary Education Establishments Links Contact the organisation Opens in new window Participation in EU R&I programmes Opens in new window HORIZON collaboration network Opens in new window Other funding € 0,00 ODIN MEDICAL LIMITED United Kingdom Net EU contribution € 320 652,50 Address 43-45 foley street W1W 7TS London See on map SME The organization defined itself as SME (small and medium-sized enterprise) at the time the Grant Agreement was signed. Yes Region London Inner London — West Westminster Activity type Private for-profit entities (excluding Higher or Secondary Education Establishments) Links Contact the organisation Opens in new window Participation in EU R&I programmes Opens in new window HORIZON collaboration network Opens in new window Other funding € 0,00 UNIVERSITY COLLEGE LONDON United Kingdom Net EU contribution € 1 148 750,00 Address Gower street WC1E 6BT London See on map Region London Inner London — West Camden and City of London Activity type Higher or Secondary Education Establishments Links Contact the organisation Opens in new window Website Opens in new window Participation in EU R&I programmes Opens in new window HORIZON collaboration network Opens in new window Other funding € 0,00
