EndoMapper: Real-time mapping from endoscopic video

Informazioni relative al progetto

EndoMapper

ID dell’accordo di sovvenzione: 863146

DOI

10.3030/863146

Progetto chiuso

Data della firma CE 30 Agosto 2019

Data di avvio 1 Dicembre 2019

Data di completamento 30 Novembre 2024

Finanziato da

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Costo totale

€ 3 693 477,50

Contributo UE

€ 3 693 477,50

Coordinato da

UNIVERSIDAD DE ZARAGOZA
Spain

CORDIS fornisce collegamenti ai risultati finali pubblici e alle pubblicazioni dei progetti ORIZZONTE.

I link ai risultati e alle pubblicazioni dei progetti del 7° PQ, così come i link ad alcuni tipi di risultati specifici come dataset e software, sono recuperati dinamicamente da .OpenAIRE .

Risultati finali

Documents, reports (12)

DefSLAM in tubular topology (Report)

Intermediate version where the tubular topology of T31 and the iterative nonlinear refinemens of T32 are incorporated The fundamentals and the algorithms are described in the form of articles

Learnt models for depth and deformation (Report)

Depth estimation model from monocular endoscopic images and initial versions of deformation estimation models report

Generative models for data association (Report)

Disentangled generative models for learning invariance, report.

Non-isometric NRSfM for the tubular topology (Report)

Articles for non-isometric NRSfM, taking point correspondences as inputs

(Nr)SfM for labeling matches in real endoscopies (Report)

(Nr)SfM-based learning of Siamese CNNs for Correspondence, report

Unsupervised data association in endoscopy (Report)

Selfsupervised FullyConvolutional Neural Networks based models for correspondence in endoscopies report

Labelling deformation in endoscopies (Report)

Simulated data generation and augmentation framework, report.

Isometric NRSfM for the tubular topology (Report)

Articles for isometric NRSfM, taking point correspondences as inputs.

Non-linear refinement for tubular topology (Report)

Articles for nonlinear refinement of the isometric the model taking the point correspondences and an initialization as inputs

Def SLAM (Report)

The different articles assembled in the report will describe the basis for doing Deformable Visual mapping.

EndoMapper: DL boosted DefSLAM in tubular topology (Report)

Final version of EndoMapper, report.

End-to-end deformable geometry estimation (Report)

Joint model for depth and deformation to facilitate integration within the EndoMapper VSLAM pipeline and end-2-end deformable geometry framework, report.

Demonstrators, pilots, prototypes (2)

EndoMapper challenges, typical use cases

Intermediate proof of concept to identify what are the main barriers to overcome during the research

EndoMapper, inspiring use cases

A final demonstrator intended to communicate the achievements and potential to a broad audience in the last stages of the project.

Other (1)

Website and Logo

Designing the logo. Create the EndoMapper website in the .eu domain

Pubblicazioni

Conference proceedings (19)

LightNeuS: Neural Surface Reconstruction in Endoscopy using Illumination Decline

Autori: Batlle, Víctor M.; Montiel, José M. M.; Fua, Pascal; Tardós, Juan D.
Pubblicato in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, Numero 14229, 2023, Pagina/e 502–512, ISBN 978-3-031-43999-5
Editore: Springer Nature
DOI: 10.1007/978-3-031-43999-5_48

<b>Ueg</b>Week 2020 Poster Presentations

Autori: Chadebecq F, Mountney P, Ahmad O.F, Kader R., Lovat L. Stoyanov D.
Pubblicato in: United European Gastroenterology Journal, Numero 8, 2023, Pagina/e 144-887, ISSN 2050-6406
Editore: Wiley
DOI: 10.1177/2050640620927345

Semantic analysis of real endoscopies with unsupervised learned

Autori: Óscar Barbed, Cristina Oriol, Pablo Azagra Millán, Ana Cristina Murillo Arnal
Pubblicato in: Medical Imaging with Deep Learning 2022, 2022, ISSN 2640-3498
Editore: MIDL 2022

Reuse your features: unifying retrieval and feature-metric alignment

Autori: Javier Morlana; J.M.M. Montiel
Pubblicato in: 2023 IEEE International Conference on Robotics and Automation (ICRA 2023), Numero May 29 - June 2, 2023., 2024, Pagina/e 6072-6079, ISBN 979-8-3503-2365-8
Editore: IEEE International Conference on Robotics and Automation
DOI: 10.1109/icra48891.2023.10160501

SD-DefSLAM: Semi-Direct Monocular SLAM for Deformable and Intracorporeal Scenes

Autori: Juan J. Gómez Rodríguez, José Lamarca, Javier Morlana, Juan D. Tardós, José M. M. Montiel
Pubblicato in: IEEE Inter. Conf. on Robotics and Automation, 2021, Pagina/e 5170-5177, ISBN 978-1-7281-9078-5
Editore: IEEE
DOI: 10.1109/icra48506.2021.9561512

Normal Reconstruction from Specularity in the Endoscopic Setting

Autori: Karim Makki; Adrien Bartoli
Pubblicato in: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), 2023, ISBN 978-1-6654-7358-3
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/isbi53787.2023.10230672

LightDepth: Single-View Depth Self-Supervision from Illumination Decline

Autori: Javier Rodríguez-Puigvert, Víctor M. Batlle, J.M.M. Montiel, Ruben Martinez-Cantin, Pascal Fua, Juan D. Tardós, Javier Civera
Pubblicato in: 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2024, Pagina/e 21216-21226, ISBN 979-8-3503-0718-4
Editore: IEEE
DOI: 10.1109/iccv51070.2023.01945

Tracking monocular camera pose and deformation for SLAM inside the human body

Autori: Juan J. Gómez Rodríguez; J.M.M. Montiel; Juan D. Tardós
Pubblicato in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, ISBN 978-1-6654-7927-1
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/iros47612.2022.9981203

Predicting Image Relationships with Image Box Embeddings during Endoscopy

Autori: Anita Rau, Lourdes Agapito, and Danail Stoyanov
Pubblicato in: Conference on New Technologies for Computer and Robot Assisted Surgery 2022, 2022
Editore: Predicting Image Relationships with Image Box Embeddings during Endoscopy

Generalized Product-of-Experts for Learning Multimodal Representations in Noisy Environments

Autori: Abhinav Joshi; Naman Gupta; Jinang Shah; Binod Bhattarai; Ashutosh Modi; Danail Stoyanov
Pubblicato in: 2022 International Conference on Multimodal Interaction, 2022, Pagina/e 83-93, ISBN 978-1-4503-9390-4
Editore: Association for Computing Machinery
DOI: 10.1145/3536221.3556596

Efficient tool segmentation for endoscopic videos in the wild

Autori: Clara Tomasini, Ínigo Alonso, Luis Riazuelo, Ana C Murillo
Pubblicato in: Medical Imaging with Deep Learning 2022, 2022
Editore: MIDL 2022

ColonMapper: topological mapping and localization for colonoscopy

Autori: Javier Morlana, Juan D. Tardós, J. M. M. Montiel
Pubblicato in: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, Pagina/e 6329-6336, ISBN 979-8-3503-8457-4
Editore: IEEE
DOI: 10.1109/icra57147.2024.10610426

The Drunkard’s Odometry: Estimating Camera Motion in Deforming Scenes

Autori: David Recasens Lafuente, Martin R. Oswald, Marc Pollefeys, Javier Civera
Pubblicato in: Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Numero 2023, 2023
Editore: Neural Information Processing Systems 36 (NeurIPS 2023)

Cálculo de flujo óptico denso en colonoscopia mediante aprendizaje no supervisado

Autori: Gonzalo, Ivan; Morlana, Javier; Martínez Montiel, José María
Pubblicato in: XLIV Jornadas de Automática., 2023, Pagina/e 861-866, ISBN 978-84-9749-860-9
Editore: Universidade da Coruña. Servizo de Publicacións.
DOI: 10.17979/spudc.9788497498609.861

Using Specularities to Boost Non-Rigid Structure-from-Motion

Autori: Agniva Sengupta, Karim Makki, Adrien Bartoli
Pubblicato in: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, Pagina/e 2652-2659, ISBN 979-8-3503-8457-4
Editore: IEEE
DOI: 10.1109/icra57147.2024.10610803

Self-supervised Visual Place Recognition for Colonoscopy Sequences

Autori: Javier Morlana, Pablo Azagra, Javier Civera and José M.M. Montiel
Pubblicato in: Medical Imaging with Deep Learning (MIDL 2021), 2021
Editore: MIDL

Photometric single-view dense 3D reconstruction in endoscopy

Autori: Víctor M. Batlle; J.M.M. Montiel; Juan D. Tardós
Pubblicato in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Numero October 23-27, 2022, 2022, Pagina/e 4904-4910, ISBN 978-1-6654-7927-1
Editore: IEEE
DOI: 10.1109/iros47612.2022.9981742

Colonoscopic 3D Reconstruction by Tubular Non-Rigid Structure-from-Motion

Autori: Agniva Sengupta and Adrien Bartoli
Pubblicato in: International Conference on Information Processing in Computer-Assisted Interventions, 2021
Editore: The 12th International Conference on Information Processing in Computer-Assisted Interventions (IPCAI 2021)

SuperPoint features in endoscopy

Autori: Barbed, O. L.; Chadebecq, F.; Morlana, J.; Martínez-Montiel, J. M.; Murillo, A. C.
Pubblicato in: Imaging Systems for GI Endoscopy, and Graphs in Biomedical Image Analysis, Numero 13754, 2022, Pagina/e 45–55, ISBN 978-3-031-21083-9
Editore: Springer, Cham
DOI: 10.1007/978-3-031-21083-9_5

Thesis and dissertations (6)

Estimación de Escala Absoluta para Structure from Motion en Endoscopio con Fuente de Luz Cercana.

Autori: Fernandes Araujo, Anyiel; Martínez Montiel, José María
Pubblicato in: reponame:Zaguán. Repositorio Digital de la Universidad de Zaragoza, Numero TAZ-TFG-2022-2042, 2022
Editore: Universidad de Zaragoza, EINA

Estimación de flujo óptico en sistemas con luz y cámara solidarios para endoscopia monocular.

Autori: Royo Meneses, Diego; Martínez Montiel, José María
Pubblicato in: Repositorio Zaguan (Universidad de Zaragoza), Numero TAZ-TFM-2022-034, 2022
Editore: Universidad de Zaragoza

Extracción de características en imágenes de procedimientos médicos con técnicas de deep learning (Feature extraction on medical images with Deep Learning)

Autori: Oscar Leon Barbed
Pubblicato in: 2020
Editore: University of Zaragoza

Monocular slam for deformable scenarios

Autori: J. Lamarca (supervised by J. M. M. Montiel)
Pubblicato in: 2021
Editore: University of Zaragoza

Estimación numérica de las propiedades diferenciales del campo de deformación en SLAM deformable con FlowNet2

Autori: Royo Meneses, Diego; Martínez Montiel, José María; Lamarca Peiro, José
Pubblicato in: Repositorio Zaguan (Universidad de Zaragoza), Numero TAZ-TFG-2020-2342, 2020
Editore: Universidad de Zaragoza

Estimación de profundidad con redes neuronales profundas en vídeos de endoscopias (Depth estimation with deep neural networks in endoscopy videos)

Autori: David Recasens
Pubblicato in: 2020
Editore: University of Zaragoza

Peer reviewed articles (15)

ToTem NRSfM: Object-Wise Non-rigid Structure-from-Motion with a Topological Template

Autori: Agniva Sengupta, Adrien Bartoli
Pubblicato in: International Journal of Computer Vision, Numero 132, 2024, Pagina/e 2135-2176, ISSN 0920-5691
Editore: Kluwer Academic Publishers
DOI: 10.1007/s11263-023-01923-6

Elliptical specularity detection in endoscopy with application to normal reconstruction

Autori: Karim Makki, Kilian Chandelon, Adrien Bartoli
Pubblicato in: International Journal of Computer Assisted Radiology and Surgery, Numero 18, 2023, Pagina/e 1323-1328, ISSN 1861-6429
Editore: Springer Science and Business Media LLC
DOI: 10.1007/s11548-023-02904-3

Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation

Autori: Yueming Jin, Yang Yu, Cheng Chen, Zixu Zhao, Pheng-Ann Heng, Danail Stoyanov
Pubblicato in: IEEE Transactions on Medical Imaging, 2022, ISSN 0278-0062
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tmi.2022.3177077

FedDP: Dual Personalization in Federated Medical Image Segmentation

Autori: Jiacheng Wang, Yueming Jin, Danail Stoyanov, Liansheng Wang
Pubblicato in: IEEE Transactions on Medical Imaging, Numero 43, 2024, Pagina/e 297-308, ISSN 0278-0062
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tmi.2023.3299206

SimCol3D — 3D reconstruction during colonoscopy challenge

Autori: Anita Rau, Sophia Bano, Yueming Jin, Pablo Azagra, Javier Morlana, Rawen Kader, Edward Sanderson, Bogdan J. Matuszewski, Jae Young Lee, Dong-Jae Lee, Erez Posner, Netanel Frank, Varshini Elangovan, Sista Raviteja, Zhengwen Li, Jiquan Liu, Seenivasan Lalithkumar, Mobarakol Islam, Hongliang Ren, Laurence B. Lovat, José M.M. Montiel, Danail Stoyanov
Pubblicato in: Medical Image Analysis, Numero 96, 2024, Pagina/e 103195, ISSN 1361-8415
Editore: Elsevier BV
DOI: 10.1016/j.media.2024.103195

Bimodal Camera Pose Prediction for Endoscopy

Autori: Anita Rau, Binod Bhattarai, Lourdes Agapito, Danail Stoyanov
Pubblicato in: IEEE Transactions on Medical Robotics and Bionics, Numero 5, 2023, Pagina/e 978-989, ISSN 2576-3202
Editore: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/tmrb.2023.3320267

NR-SLAM: Non-Rigid Monocular SLAM

Autori: Juan J. Gómez Rodríguez, José M.M. Montiel, Juan D. Tardós
Pubblicato in: IEEE Transactions on Robotics, Numero Early Access, 2024, Pagina/e 1-13, ISSN 1552-3098
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2024.3422004

DefSLAM: Tracking and Mapping of Deforming Scenes From Monocular Sequences

Autori: Jose Lamarca, Shaifali Parashar, Adrien Bartoli, J. M. M. Montiel
Pubblicato in: IEEE Transactions on Robotics, 2020, Pagina/e 1-13, ISSN 1552-3098
Editore: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2020.3020739

The shading isophotes: Model and methods for Lambertian planes and a point light

Autori: Damien Mariyanayagam, Adrien Bartoli
Pubblicato in: Computer Vision and Image Understanding, Numero 248, 2024, Pagina/e 104135, ISSN 1077-3142
Editore: Academic Press
DOI: 10.1016/j.cviu.2024.104135

Endo-Depth-and-Motion: Reconstruction and tracking in endoscopic videos using depth networks and photometric constraints

Autori: David Recasens, José Lamarca, José M. Fácil, J.M.M. Montiel, and Javier Civera
Pubblicato in: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2021, Pagina/e 7225-7232, ISSN 2377-3766
Editore: IEEE
DOI: 10.1109/lra.2021.3095528

Endomapper dataset of complete calibrated endoscopy procedures

Autori: Pablo Azagra; Carlos Sostres; Ángel Ferrández; Luis Riazuelo; Clara Tomasini; O. León Barbed; Javier Morlana; David Recasens; Víctor M. Batlle; Juan J. Gómez-Rodríguez; Richard Elvira; Julia López; Cristina Oriol; Javier Civera; Juan D. Tardós; Ana C. Murillo; Angel Lanas; José M. M. Montiel
Pubblicato in: Scientific Data, Numero 10:671(2023), 2023, Pagina/e 1-16, ISSN 2052-4463
Editore: Springer Nature
DOI: 10.1038/s41597-023-02564-7

A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its Effect on Image Correspondence

Autori: Rema Daher, Francisco Vasconcelos, Danail Stoyanov
Pubblicato in: Medical Image Analysis, Numero 90, 2024, Pagina/e 102994, ISSN 1361-8415
Editore: Elsevier BV
DOI: 10.1016/j.media.2023.102994

Direct and Sparse Deformable Tracking

Autori: Jose Lamarca; Juan J. Gomez Rodriguez; Juan D. Tardos; J.M.M. Montiel
Pubblicato in: IEEE ROBOTICS AND AUTOMATION LETTERS, Numero 7(4), 2022, Pagina/e 11450-11457, ISSN 2377-3766
Editore: Institute of Electrical and electronics engineers
DOI: 10.1109/lra.2022.3201253

Histogram of Oriented Gradients meet deep learning: A novel multi-task deep network for 2D surgical image semantic segmentation

Autori: Binod Bhattarai, Ronast Subedi, Rebati Raman Gaire, Eduard Vazquez, Danail Stoyanov
Pubblicato in: Medical Image Analysis, Numero 85, 2024, Pagina/e 102747, ISSN 1361-8415
Editore: Elsevier BV
DOI: 10.1016/j.media.2023.102747

Colonoscopic 3D reconstruction by tubular non-rigid structure-from-motion

Autori: Agniva Sengupta and Adrien Bartoli
Pubblicato in: International Journal of Computer Assisted Radiology and Surgery, 2021, Pagina/e 1237–1241, ISSN 1861-6429
Editore: International Journal of Computer Assisted Radiology and Surgery
DOI: 10.1007/s11548-021-02409-x

Other (3)

Reconstrucción 3D a escala real a partir de imágenes monoculares de endoscopio

Autori: Víctor Martínez Batlle (supervised by Juan Domingo Tardós Solano)
Pubblicato in: 2021
Editore: University of Zaragoza

EndoMapper dataset of complete calibrated endoscopy procedures

Autori: Pablo Azagra, Carlos Sostres, Ángel Ferrandez, Luis Riazuelo, Clara Tomasini, Oscar León Barbed, Javier Morlana, David Recasens, Victor M. Batlle, Juan J. Gómez-Rodríguez, Richard Elvira, Julia López, Cristina Oriol, Javier Civera, Juan D. Tardós, Ana Cristina Murillo, Angel Lanas, José M.M. Montiel
Pubblicato in: 2021
Editore: arxiv
DOI: 10.7303/syn26707219

CudaSIFT-SLAM: multiple-map visual SLAM for full procedure mapping in real human endoscopy

Autori: Richard Elvira, Juan D. Tardós, José M.M. Montiel
Pubblicato in: Arxiv, Numero -, 2024
Editore: Arxiv
DOI: 10.48550/arxiv.2405.16932

Book chapters (5)

On the Uncertain Single-View Depths in Colonoscopies

Autori: Javier Rodriguez-Puigvert, David Recasens, Javier Civera, Ruben Martinez-Cantin
Pubblicato in: Lecture Notes in Computer Science, Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, 2024, Pagina/e 130-140, ISBN 978-3-031-16437-8
Editore: Springer Nature Switzerland
DOI: 10.1007/978-3-031-16437-8_13

Tracking Adaptation to Improve SuperPoint for 3D Reconstruction in Endoscopy

Autori: O. León Barbed, José M. M. Montiel, Pascal Fua, Ana C. Murillo
Pubblicato in: Lecture Notes in Computer Science, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, 2024, Pagina/e 583-593
Editore: Springer Nature Switzerland
DOI: 10.1007/978-3-031-43907-0_56

Sim2Real in Endoscopy Segmentation with a Novel Structure Aware Image Translation

Autori: Clara Tomasini, Luis Riazuelo, Ana C. Murillo
Pubblicato in: Lecture Notes in Computer Science, Simulation and Synthesis in Medical Imaging, Numero LNCS,volume 15187, 2024, Pagina/e 89-101, ISBN 978-3-031-73281-2
Editore: Springer Nature Switzerland
DOI: 10.1007/978-3-031-73281-2_9

Topological SLAM in Colonoscopies Leveraging Deep Features and Topological Priors

Autori: Javier Morlana, Juan D. Tardós, José M. M. Montiel
Pubblicato in: Lecture Notes in Computer Science, Medical Image Computing and Computer Assisted Intervention – MICCAI 2024, 2024, Pagina/e 733-743, ISBN 978-3-031-72120-5
Editore: Springer Nature Switzerland
DOI: 10.1007/978-3-031-72120-5_68

A Client-Server Deep Federated Learning for Cross-Domain Surgical Image Segmentation

Autori: Ronast Subedi, Rebati Raman Gaire, Sharib Ali, Anh Nguyen, Danail Stoyanov, Binod Bhattarai
Pubblicato in: Lecture Notes in Computer Science, Data Engineering in Medical Imaging, Numero LNCS,volume 14314, 2023, Pagina/e 21-33, ISBN 978-3-031-44992-5
Editore: Springer Nature Switzerland
DOI: 10.1007/978-3-031-44992-5_3

Diritti di proprietà intellettuale

Other (8)

SD-DefSLAM

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/SD-DefSLAM
Data: 2022-04-05
Candidato/i: UNIVERSIDAD DE ZARAGOZA

DefSLAM

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/DefSLAM
Data: 2020-07-06
Candidato/i: UNIVERSIDAD DE ZARAGOZA

DefSLAM

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/DefSLAM
Data: 2020-07-06
Candidato/i: UNIVERSITE CLERMONT AUVERGNE

Endo-Depth-And-Motion

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/Endo-Depth-and-Motion
Data: 2021-03-04
Candidato/i: UNIVERSIDAD DE ZARAGOZA

Endo-Depth-And-Motion

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/Endo-Depth-and-Motion
Data: 2021-03-04
Candidato/i: UNIVERSIDAD DE ZARAGOZA

DefSLAM

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/DefSLAM
Data: 2020-07-06
Candidato/i: UNIVERSITE CLERMONT AUVERGNE

DefSLAM

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/DefSLAM
Data: 2020-07-06
Candidato/i: UNIVERSIDAD DE ZARAGOZA

SD-DefSLAM

Numero candidatura/pubblicazione: N/A https://github.com/UZ-SLAMLab/SD-DefSLAM
Data: 2022-04-05
Candidato/i: UNIVERSIDAD DE ZARAGOZA