Intelligent Robotic Endoscopes for Improved Healthcare Services

Project Information

IRE

Grant agreement ID: 101135082

DOI

10.3030/101135082

EC signature date 23 November 2023

Start date 1 March 2024

End date 29 February 2028

Funded under

• Digital, Industry and Space

Total cost

€ 6 189 085,00

EU contribution

€ 6 189 082,00

Coordinated by

KOBENHAVNS UNIVERSITET
Denmark

Periodic Reporting for period 1 - IRE (Intelligent Robotic Endoscopes for Improved Healthcare Services)

Reporting period: 2024-03-01 to 2025-08-31

Summary of the context and overall objectives of the project

The challenge with colonoscopy is that detecting cancer depends on the skills of the clinician operating the endoscope.
IRE aims to empower endoscopy technology by inventing next-step autonomy robotic functionalities that create a step change in undertaking non-repetitive colonoscopy cancer screening procedures in realistic laboratory settings. The five main objectives can be summarised as development of:

1) Soft robotic endoscopes - which will tackle research challenges that will bring autonomous endoscopes closer. A prototype colonoscope with AI-controlled tip stiffness for fine interaction, and a prototype colonoscope will be developed.

2) Robotic test and train phantoms - a physical phantom of the colon based on imaging data of a large population of patients combined with mechanical estimation of tissue wall properties. KPI: 1)performance Indicators is an inert sensorised phantom of the colon for training and measurable evaluation of colonoscope devices and navigation algorithms 2) an active sensorised phantom of the colon for measurable evaluation of colonoscopy devices and navigation algorithms.

3) AI-enabled digital endoscope twins - are developed to avoid the tedious manual tasks of processing segmentation results to produce simulation meshes and applying the methodology to images from a large population. KPI: 1)software pipeline for automated meshing of abdomen anatomy, a differentiable software library for the simulation of endoscope interaction with soft cavities. 2) a prototype virtual-reality training simulator of colonoscopy interventions.

4) AI-powered navigation development - aims to address high-level autonomy, aiming at autonomous navigation by designing a computational model of endoscopy assessment from runtime image input. KPI: 1) a software library for endoscopy guidance using learning-based intervention assessment. 2) a prototype human-robot interaction system for operator assistance in colonoscopy navigation.

5) OPEN/FAIR models and data - Progress on clinically oriented robotics requires representative datasets, as well as clinically validated protocols for testing and evaluation. The development of models, datasets, and protocols that meet these needs will enable progress of the project. By releasing these repositories IRE will facilitate further open research in the field.

The consortium gathers the expertise and key players to make the project a success including within anthropological studies of human-robot interface to investigate and solve the challenges in adapting technology in clinics and the patient’s reception of the technology.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Activities and achievements:

• Developed 435 high-quality, simulation-ready colon meshes, with artifacts removed from the rectum region — enabling precise and realistic modeling for surgical simulation.

• Developed A fully integrated endoscope colon simulator, combining anatomical models with real endoscopic data.

• Developed A novel formulation of forward and differentiable rigid-body dynamics using Lie-algebra rotation derivatives — enabling simulations 4.3× faster than conventional methods while improving optimization for endoscope tip navigation.

• Developed the first theoretical framework for optimal sensor positioning, a foundation for improving robotic perception and accuracy.

• Developed Initial experiments with mechanochromic materials, advancing the tactile sensing capabilities of soft robotic systems.

• Developed intelligent soft robotic platforms for colonoscopy, integrating sensing for improved safety and efficacy.

• Developed prototypes of reinforcement learning environments, supporting adaptive and data-driven robotic control, which have been validated.

• Developed a modular and configurable software interface for streamlined data import and management.

• Developed and worked on a proof-of-concept reward design and navigation policy learning, which will establish a key basis for autonomous robotic guidance.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The aim is to create significant impact in three specific areas:
1) European Technology leadership - by tackling the needs of AI powered intelligent soft robotics:
a) methods to increase the level of autonomy for handling complex interactions with dynamically changing environments with complex geometry. b) Methods for efficient design and manufacturing of soft robots.
Smart soft robotic endoscopes and advanced phantoms are key enabling technologies in the ground-breaking spectrum where increased level of autonomy will result in faster and safer treatments in health care.

2) Improved cancer treatment: With a rapidly increasing demand in EU for colonoscopies for cancer location/treatment, and the fact that human operators inherently have a varied skill level, we see a market for intelligent robotic endoscopes. Intelligent robotic endoscopes will provide faster, safer colonoscopies, which in turn gives an increased ability to detect early stage cancer.

3) Improved medical training: Training clinical personal for treatments with endoscopes is challenging and time consuming. Current phantoms are too simple, and accessibility to train on animals/corpses or real humans is limited. There is a demand for new phantom technology and alternative models, to allow medical training centres to upscale on educating clinicians, within the entire clinical competence lifespan from initial introductory training to continuous clinical feedback.

To ensure that the project will create impact on the above we create synergies between disciplines to develop methods to increase the level of anatomy and design phantoms of the colon and robotics with cooperation between robotic engineers, MedTech Professionals and clinicians, which will be tested continuously by clinicians and engineers and MedTech professional in cooperation at internal meeting and workshops. Within the first 18 months internal workshops and work meetings have been carried out between CAMES and University of Tübingen and CAMES and UTWENTE. Furthermore CAMES initiated an internal workshop with all partners at INRIA.

The cooperation has led to development of phantoms soft robotics, digital and physical phantoms of the colon for clinical training and training of robots.
Results so far (first 18 months) include A training simulator to learn how to annotate poplyps during the retraction of the colonoscope - ready for exploitation in a mature market.