EndoMapper: Real-time mapping from endoscopic video

Información del proyecto

EndoMapper

Identificador del acuerdo de subvención: 863146

DOI

10.3030/863146

Proyecto cerrado

Fecha de la firma de la CE 30 Agosto 2019

Fecha de inicio 1 Diciembre 2019

Fecha de finalización 30 Noviembre 2024

Financiado con arreglo a

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Coste total

€ 3 693 477,50

Aportación de la UE

€ 3 693 477,50

Coordinado por

UNIVERSIDAD DE ZARAGOZA
Spain

CORDIS proporciona enlaces a los documentos públicos y las publicaciones de los proyectos de los programas marco HORIZONTE.

Los enlaces a los documentos y las publicaciones de los proyectos del Séptimo Programa Marco, así como los enlaces a algunos tipos de resultados específicos, como conjuntos de datos y «software», se obtienen dinámicamente de OpenAIRE .

Resultado final

Documents, reports (12)

DefSLAM in tubular topology (Report)

Intermediate version where the tubular topology of T31 and the iterative nonlinear refinemens of T32 are incorporated The fundamentals and the algorithms are described in the form of articles

Learnt models for depth and deformation (Report)

Depth estimation model from monocular endoscopic images and initial versions of deformation estimation models report

Generative models for data association (Report)

Disentangled generative models for learning invariance, report.

Non-isometric NRSfM for the tubular topology (Report)

Articles for non-isometric NRSfM, taking point correspondences as inputs

(Nr)SfM for labeling matches in real endoscopies (Report)

(Nr)SfM-based learning of Siamese CNNs for Correspondence, report

Unsupervised data association in endoscopy (Report)

Selfsupervised FullyConvolutional Neural Networks based models for correspondence in endoscopies report

Labelling deformation in endoscopies (Report)

Simulated data generation and augmentation framework, report.

Isometric NRSfM for the tubular topology (Report)

Articles for isometric NRSfM, taking point correspondences as inputs.

Non-linear refinement for tubular topology (Report)

Articles for nonlinear refinement of the isometric the model taking the point correspondences and an initialization as inputs

Def SLAM (Report)

The different articles assembled in the report will describe the basis for doing Deformable Visual mapping.

EndoMapper: DL boosted DefSLAM in tubular topology (Report)

Final version of EndoMapper, report.

End-to-end deformable geometry estimation (Report)

Joint model for depth and deformation to facilitate integration within the EndoMapper VSLAM pipeline and end-2-end deformable geometry framework, report.

Demonstrators, pilots, prototypes (2)

EndoMapper challenges, typical use cases

Intermediate proof of concept to identify what are the main barriers to overcome during the research

EndoMapper, inspiring use cases

A final demonstrator intended to communicate the achievements and potential to a broad audience in the last stages of the project.

Other (1)

Website and Logo

Designing the logo. Create the EndoMapper website in the .eu domain

Publicaciones

Conference proceedings (19)

LightNeuS: Neural Surface Reconstruction in Endoscopy using Illumination Decline

Autores: Batlle, Víctor M.; Montiel, José M. M.; Fua, Pascal; Tardós, Juan D.
Publicado en: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, Edición 14229, 2023, Página(s) 502–512, ISBN 978-3-031-43999-5
Editor: Springer Nature
DOI: 10.1007/978-3-031-43999-5_48

<b>Ueg</b>Week 2020 Poster Presentations

Autores: Chadebecq F, Mountney P, Ahmad O.F, Kader R., Lovat L. Stoyanov D.
Publicado en: United European Gastroenterology Journal, Edición 8, 2023, Página(s) 144-887, ISSN 2050-6406
Editor: Wiley
DOI: 10.1177/2050640620927345

Semantic analysis of real endoscopies with unsupervised learned

Autores: Óscar Barbed, Cristina Oriol, Pablo Azagra Millán, Ana Cristina Murillo Arnal
Publicado en: Medical Imaging with Deep Learning 2022, 2022, ISSN 2640-3498
Editor: MIDL 2022

Reuse your features: unifying retrieval and feature-metric alignment

Autores: Javier Morlana; J.M.M. Montiel
Publicado en: 2023 IEEE International Conference on Robotics and Automation (ICRA 2023), Edición May 29 - June 2, 2023., 2024, Página(s) 6072-6079, ISBN 979-8-3503-2365-8
Editor: IEEE International Conference on Robotics and Automation
DOI: 10.1109/icra48891.2023.10160501

SD-DefSLAM: Semi-Direct Monocular SLAM for Deformable and Intracorporeal Scenes

Autores: Juan J. Gómez Rodríguez, José Lamarca, Javier Morlana, Juan D. Tardós, José M. M. Montiel
Publicado en: IEEE Inter. Conf. on Robotics and Automation, 2021, Página(s) 5170-5177, ISBN 978-1-7281-9078-5
Editor: IEEE
DOI: 10.1109/icra48506.2021.9561512

Normal Reconstruction from Specularity in the Endoscopic Setting

Autores: Karim Makki; Adrien Bartoli
Publicado en: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), 2023, ISBN 978-1-6654-7358-3
Editor: Institute of Electrical and Electronics Engineers
DOI: 10.1109/isbi53787.2023.10230672

LightDepth: Single-View Depth Self-Supervision from Illumination Decline

Autores: Javier Rodríguez-Puigvert, Víctor M. Batlle, J.M.M. Montiel, Ruben Martinez-Cantin, Pascal Fua, Juan D. Tardós, Javier Civera
Publicado en: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2024, Página(s) 21216-21226, ISBN 979-8-3503-0718-4
Editor: IEEE
DOI: 10.1109/iccv51070.2023.01945

Tracking monocular camera pose and deformation for SLAM inside the human body

Autores: Juan J. Gómez Rodríguez; J.M.M. Montiel; Juan D. Tardós
Publicado en: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, ISBN 978-1-6654-7927-1
Editor: Institute of Electrical and Electronics Engineers
DOI: 10.1109/iros47612.2022.9981203

Predicting Image Relationships with Image Box Embeddings during Endoscopy

Autores: Anita Rau, Lourdes Agapito, and Danail Stoyanov
Publicado en: Conference on New Technologies for Computer and Robot Assisted Surgery 2022, 2022
Editor: Predicting Image Relationships with Image Box Embeddings during Endoscopy

Generalized Product-of-Experts for Learning Multimodal Representations in Noisy Environments

Autores: Abhinav Joshi; Naman Gupta; Jinang Shah; Binod Bhattarai; Ashutosh Modi; Danail Stoyanov
Publicado en: 2022 International Conference on Multimodal Interaction, 2022, Página(s) 83-93, ISBN 978-1-4503-9390-4
Editor: Association for Computing Machinery
DOI: 10.1145/3536221.3556596

Efficient tool segmentation for endoscopic videos in the wild

Autores: Clara Tomasini, Ínigo Alonso, Luis Riazuelo, Ana C Murillo
Publicado en: Medical Imaging with Deep Learning 2022, 2022
Editor: MIDL 2022

ColonMapper: topological mapping and localization for colonoscopy

Autores: Javier Morlana, Juan D. Tardós, J. M. M. Montiel
Publicado en: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, Página(s) 6329-6336, ISBN 979-8-3503-8457-4
Editor: IEEE
DOI: 10.1109/icra57147.2024.10610426

The Drunkard’s Odometry: Estimating Camera Motion in Deforming Scenes

Autores: David Recasens Lafuente, Martin R. Oswald, Marc Pollefeys, Javier Civera
Publicado en: Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Edición 2023, 2023
Editor: Neural Information Processing Systems 36 (NeurIPS 2023)

Cálculo de flujo óptico denso en colonoscopia mediante aprendizaje no supervisado

Autores: Gonzalo, Ivan; Morlana, Javier; Martínez Montiel, José María
Publicado en: XLIV Jornadas de Automática., 2023, Página(s) 861-866, ISBN 978-84-9749-860-9
Editor: Universidade da Coruña. Servizo de Publicacións.
DOI: 10.17979/spudc.9788497498609.861

Using Specularities to Boost Non-Rigid Structure-from-Motion

Autores: Agniva Sengupta, Karim Makki, Adrien Bartoli
Publicado en: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, Página(s) 2652-2659, ISBN 979-8-3503-8457-4
Editor: IEEE
DOI: 10.1109/icra57147.2024.10610803

Self-supervised Visual Place Recognition for Colonoscopy Sequences

Autores: Javier Morlana, Pablo Azagra, Javier Civera and José M.M. Montiel
Publicado en: Medical Imaging with Deep Learning (MIDL 2021), 2021
Editor: MIDL

Photometric single-view dense 3D reconstruction in endoscopy

Autores: Víctor M. Batlle; J.M.M. Montiel; Juan D. Tardós
Publicado en: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Edición October 23-27, 2022, 2022, Página(s) 4904-4910, ISBN 978-1-6654-7927-1
Editor: IEEE
DOI: 10.1109/iros47612.2022.9981742

Colonoscopic 3D Reconstruction by Tubular Non-Rigid Structure-from-Motion

Autores: Agniva Sengupta and Adrien Bartoli
Publicado en: International Conference on Information Processing in Computer-Assisted Interventions, 2021
Editor: The 12th International Conference on Information Processing in Computer-Assisted Interventions (IPCAI 2021)

SuperPoint features in endoscopy

Autores: Barbed, O. L.; Chadebecq, F.; Morlana, J.; Martínez-Montiel, J. M.; Murillo, A. C.
Publicado en: Imaging Systems for GI Endoscopy, and Graphs in Biomedical Image Analysis, Edición 13754, 2022, Página(s) 45–55, ISBN 978-3-031-21083-9
Editor: Springer, Cham
DOI: 10.1007/978-3-031-21083-9_5

Thesis and dissertations (6)

Estimación de Escala Absoluta para Structure from Motion en Endoscopio con Fuente de Luz Cercana.

Autores: Fernandes Araujo, Anyiel; Martínez Montiel, José María
Publicado en: reponame:Zaguán. Repositorio Digital de la Universidad de Zaragoza, Edición TAZ-TFG-2022-2042, 2022
Editor: Universidad de Zaragoza, EINA

Estimación de flujo óptico en sistemas con luz y cámara solidarios para endoscopia monocular.

Autores: Royo Meneses, Diego; Martínez Montiel, José María
Publicado en: Repositorio Zaguan (Universidad de Zaragoza), Edición TAZ-TFM-2022-034, 2022
Editor: Universidad de Zaragoza

Extracción de características en imágenes de procedimientos médicos con técnicas de deep learning (Feature extraction on medical images with Deep Learning)

Autores: Oscar Leon Barbed
Publicado en: 2020
Editor: University of Zaragoza

Monocular slam for deformable scenarios

Autores: J. Lamarca (supervised by J. M. M. Montiel)
Publicado en: 2021
Editor: University of Zaragoza

Estimación numérica de las propiedades diferenciales del campo de deformación en SLAM deformable con FlowNet2

Autores: Royo Meneses, Diego; Martínez Montiel, José María; Lamarca Peiro, José
Publicado en: Repositorio Zaguan (Universidad de Zaragoza), Edición TAZ-TFG-2020-2342, 2020
Editor: Universidad de Zaragoza

Estimación de profundidad con redes neuronales profundas en vídeos de endoscopias (Depth estimation with deep neural networks in endoscopy videos)

Autores: David Recasens
Publicado en: 2020
Editor: University of Zaragoza

Peer reviewed articles (15)

ToTem NRSfM: Object-Wise Non-rigid Structure-from-Motion with a Topological Template

Autores: Agniva Sengupta, Adrien Bartoli
Publicado en: International Journal of Computer Vision, Edición 132, 2024, Página(s) 2135-2176, ISSN 0920-5691
Editor: Kluwer Academic Publishers
DOI: 10.1007/s11263-023-01923-6

Elliptical specularity detection in endoscopy with application to normal reconstruction

Autores: Karim Makki, Kilian Chandelon, Adrien Bartoli
Publicado en: International Journal of Computer Assisted Radiology and Surgery, Edición 18, 2023, Página(s) 1323-1328, ISSN 1861-6429
Editor: Springer Science and Business Media LLC
DOI: 10.1007/s11548-023-02904-3

Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation

Autores: Yueming Jin, Yang Yu, Cheng Chen, Zixu Zhao, Pheng-Ann Heng, Danail Stoyanov
Publicado en: IEEE Transactions on Medical Imaging, 2022, ISSN 0278-0062
Editor: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tmi.2022.3177077

FedDP: Dual Personalization in Federated Medical Image Segmentation

Autores: Jiacheng Wang, Yueming Jin, Danail Stoyanov, Liansheng Wang
Publicado en: IEEE Transactions on Medical Imaging, Edición 43, 2024, Página(s) 297-308, ISSN 0278-0062
Editor: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tmi.2023.3299206

SimCol3D — 3D reconstruction during colonoscopy challenge

Autores: Anita Rau, Sophia Bano, Yueming Jin, Pablo Azagra, Javier Morlana, Rawen Kader, Edward Sanderson, Bogdan J. Matuszewski, Jae Young Lee, Dong-Jae Lee, Erez Posner, Netanel Frank, Varshini Elangovan, Sista Raviteja, Zhengwen Li, Jiquan Liu, Seenivasan Lalithkumar, Mobarakol Islam, Hongliang Ren, Laurence B. Lovat, José M.M. Montiel, Danail Stoyanov
Publicado en: Medical Image Analysis, Edición 96, 2024, Página(s) 103195, ISSN 1361-8415
Editor: Elsevier BV
DOI: 10.1016/j.media.2024.103195

Bimodal Camera Pose Prediction for Endoscopy

Autores: Anita Rau, Binod Bhattarai, Lourdes Agapito, Danail Stoyanov
Publicado en: IEEE Transactions on Medical Robotics and Bionics, Edición 5, 2023, Página(s) 978-989, ISSN 2576-3202
Editor: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/tmrb.2023.3320267

NR-SLAM: Non-Rigid Monocular SLAM

Autores: Juan J. Gómez Rodríguez, José M.M. Montiel, Juan D. Tardós
Publicado en: IEEE Transactions on Robotics, Edición Early Access, 2024, Página(s) 1-13, ISSN 1552-3098
Editor: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2024.3422004

DefSLAM: Tracking and Mapping of Deforming Scenes From Monocular Sequences

Autores: Jose Lamarca, Shaifali Parashar, Adrien Bartoli, J. M. M. Montiel
Publicado en: IEEE Transactions on Robotics, 2020, Página(s) 1-13, ISSN 1552-3098
Editor: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2020.3020739

The shading isophotes: Model and methods for Lambertian planes and a point light

Autores: Damien Mariyanayagam, Adrien Bartoli
Publicado en: Computer Vision and Image Understanding, Edición 248, 2024, Página(s) 104135, ISSN 1077-3142
Editor: Academic Press
DOI: 10.1016/j.cviu.2024.104135

Endo-Depth-and-Motion: Reconstruction and tracking in endoscopic videos using depth networks and photometric constraints

Autores: David Recasens, José Lamarca, José M. Fácil, J.M.M. Montiel, and Javier Civera
Publicado en: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2021, Página(s) 7225-7232, ISSN 2377-3766
Editor: IEEE
DOI: 10.1109/lra.2021.3095528

Endomapper dataset of complete calibrated endoscopy procedures

Autores: Pablo Azagra; Carlos Sostres; Ángel Ferrández; Luis Riazuelo; Clara Tomasini; O. León Barbed; Javier Morlana; David Recasens; Víctor M. Batlle; Juan J. Gómez-Rodríguez; Richard Elvira; Julia López; Cristina Oriol; Javier Civera; Juan D. Tardós; Ana C. Murillo; Angel Lanas; José M. M. Montiel
Publicado en: Scientific Data, Edición 10:671(2023), 2023, Página(s) 1-16, ISSN 2052-4463
Editor: Springer Nature
DOI: 10.1038/s41597-023-02564-7

A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its Effect on Image Correspondence

Autores: Rema Daher, Francisco Vasconcelos, Danail Stoyanov
Publicado en: Medical Image Analysis, Edición 90, 2024, Página(s) 102994, ISSN 1361-8415
Editor: Elsevier BV
DOI: 10.1016/j.media.2023.102994

Direct and Sparse Deformable Tracking

Autores: Jose Lamarca; Juan J. Gomez Rodriguez; Juan D. Tardos; J.M.M. Montiel
Publicado en: IEEE ROBOTICS AND AUTOMATION LETTERS, Edición 7(4), 2022, Página(s) 11450-11457, ISSN 2377-3766
Editor: Institute of Electrical and electronics engineers
DOI: 10.1109/lra.2022.3201253

Histogram of Oriented Gradients meet deep learning: A novel multi-task deep network for 2D surgical image semantic segmentation

Autores: Binod Bhattarai, Ronast Subedi, Rebati Raman Gaire, Eduard Vazquez, Danail Stoyanov
Publicado en: Medical Image Analysis, Edición 85, 2024, Página(s) 102747, ISSN 1361-8415
Editor: Elsevier BV
DOI: 10.1016/j.media.2023.102747

Colonoscopic 3D reconstruction by tubular non-rigid structure-from-motion

Autores: Agniva Sengupta and Adrien Bartoli
Publicado en: International Journal of Computer Assisted Radiology and Surgery, 2021, Página(s) 1237–1241, ISSN 1861-6429
Editor: International Journal of Computer Assisted Radiology and Surgery
DOI: 10.1007/s11548-021-02409-x

Other (3)

Reconstrucción 3D a escala real a partir de imágenes monoculares de endoscopio

Autores: Víctor Martínez Batlle (supervised by Juan Domingo Tardós Solano)
Publicado en: 2021
Editor: University of Zaragoza

EndoMapper dataset of complete calibrated endoscopy procedures

Autores: Pablo Azagra, Carlos Sostres, Ángel Ferrandez, Luis Riazuelo, Clara Tomasini, Oscar León Barbed, Javier Morlana, David Recasens, Victor M. Batlle, Juan J. Gómez-Rodríguez, Richard Elvira, Julia López, Cristina Oriol, Javier Civera, Juan D. Tardós, Ana Cristina Murillo, Angel Lanas, José M.M. Montiel
Publicado en: 2021
Editor: arxiv
DOI: 10.7303/syn26707219

CudaSIFT-SLAM: multiple-map visual SLAM for full procedure mapping in real human endoscopy

Autores: Richard Elvira, Juan D. Tardós, José M.M. Montiel
Publicado en: Arxiv, Edición -, 2024
Editor: Arxiv
DOI: 10.48550/arxiv.2405.16932

Book chapters (5)

On the Uncertain Single-View Depths in Colonoscopies

Autores: Javier Rodriguez-Puigvert, David Recasens, Javier Civera, Ruben Martinez-Cantin
Publicado en: Lecture Notes in Computer Science, Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, 2024, Página(s) 130-140, ISBN 978-3-031-16437-8
Editor: Springer Nature Switzerland
DOI: 10.1007/978-3-031-16437-8_13

Tracking Adaptation to Improve SuperPoint for 3D Reconstruction in Endoscopy

Autores: O. León Barbed, José M. M. Montiel, Pascal Fua, Ana C. Murillo
Publicado en: Lecture Notes in Computer Science, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, 2024, Página(s) 583-593
Editor: Springer Nature Switzerland
DOI: 10.1007/978-3-031-43907-0_56

Sim2Real in Endoscopy Segmentation with a Novel Structure Aware Image Translation

Autores: Clara Tomasini, Luis Riazuelo, Ana C. Murillo
Publicado en: Lecture Notes in Computer Science, Simulation and Synthesis in Medical Imaging, Edición LNCS,volume 15187, 2024, Página(s) 89-101, ISBN 978-3-031-73281-2
Editor: Springer Nature Switzerland
DOI: 10.1007/978-3-031-73281-2_9

Topological SLAM in Colonoscopies Leveraging Deep Features and Topological Priors

Autores: Javier Morlana, Juan D. Tardós, José M. M. Montiel
Publicado en: Lecture Notes in Computer Science, Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, 2024, Página(s) 733-743, ISBN 978-3-031-72120-5
Editor: Springer Nature Switzerland
DOI: 10.1007/978-3-031-72120-5_68

A Client-Server Deep Federated Learning for Cross-Domain Surgical Image Segmentation

Autores: Ronast Subedi, Rebati Raman Gaire, Sharib Ali, Anh Nguyen, Danail Stoyanov, Binod Bhattarai
Publicado en: Lecture Notes in Computer Science, Data Engineering in Medical Imaging, Edición LNCS,volume 14314, 2023, Página(s) 21-33, ISBN 978-3-031-44992-5
Editor: Springer Nature Switzerland
DOI: 10.1007/978-3-031-44992-5_3

Derechos de propiedad intelectual

Other (8)

SD-DefSLAM

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/SD-DefSLAM
Fecha: 2022-04-05
Solicitante(s): UNIVERSIDAD DE ZARAGOZA

DefSLAM

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/DefSLAM
Fecha: 2020-07-06
Solicitante(s): UNIVERSIDAD DE ZARAGOZA

DefSLAM

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/DefSLAM
Fecha: 2020-07-06
Solicitante(s): UNIVERSITE CLERMONT AUVERGNE

Endo-Depth-And-Motion

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/Endo-Depth-and-Motion
Fecha: 2021-03-04
Solicitante(s): UNIVERSIDAD DE ZARAGOZA

Endo-Depth-And-Motion

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/Endo-Depth-and-Motion
Fecha: 2021-03-04
Solicitante(s): UNIVERSIDAD DE ZARAGOZA

DefSLAM

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/DefSLAM
Fecha: 2020-07-06
Solicitante(s): UNIVERSITE CLERMONT AUVERGNE

DefSLAM

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/DefSLAM
Fecha: 2020-07-06
Solicitante(s): UNIVERSIDAD DE ZARAGOZA

SD-DefSLAM

Número de solicitud/publicación: N/A https://github.com/UZ-SLAMLab/SD-DefSLAM
Fecha: 2022-04-05
Solicitante(s): UNIVERSIDAD DE ZARAGOZA