Community Research and Development Information Service - CORDIS

H2020

EurEyeCase Report Summary

Project ID: 645331
Funded under: H2020-EU.2.1.1.5.

Periodic Reporting for period 1 - EurEyeCase (Use Case for European Robotics in Ophthalmologic Micro−Surgery)

Reporting period: 2015-01-01 to 2015-12-31

Summary of the context and overall objectives of the project

Epiretinal membrane (ERM) and retinal vein occlusion (RVO) are common pathologic conditions (prevalence of 12% for people aged over 70 for ERM, and current estimate of 16 million suffering for RVO worldwide) with high complication rates that greatly reduce quality of life of the affected people. ERM can be treated by peeling the epiretinal membrane and RVO by cannulating the occluded microvessel(s).

Currently epiretinal membrane peeling is considered to be a high-risk procedure. The surgery is only performed by expert vitreoretinal surgeons in a limited number of centres worldwide. The procedure is performed manually and is known to be error-prone. In contrast, EurEyeCase will enable reliable epiretinal membrane peeling by means of robot-assistance. This will be achieved by ensuring peeling forces stay below 7.5mN and instrument motion does not exceed 80 μm/s, thus avoiding retinal tear. An overall peeling success rate of 95% is targeted.

Retinal vein cannulation and subsequent injection of thrombolytic agent is currently not put into practice due to the extreme requirements for positioning precision. In addition, a 10 minutes flush of a thrombolytic agent is required to achieve lysis of the thrombus. Manually, it is possible to flush during 20 seconds at the most. Currently, only the symptoms secondary to retinal vein occlusions are treated, without solving the actual problem. EurEyeCase will demonstrate the feasibility of retinal vein cannulation by means of robot-assistance. It will do so by cannulating small retinal veins at the retinal periphery with diameters as low as 50μm. EurEyeCase will target an overall cannulation and injection success rate of 95% in a clinically relevant setup. The safety and intended use of robot-assisted retinal vein cannulation will be demonstrated in order to prepare a first-in-vivo study at the end of the project on five patients with acute central retinal vein occlusion.

To achieve these objectives, robot-assistance control schemes for micro-positioning and micro-force manipulation surgical tasks are developed, starting by the design of a set of innovative miniature sensorised instruments for vitreoretinal surgery (VR surgery) that help improve peeling and cannulation tasks. These smart instruments include proximity, contact and force sensing in an extreme compact package. Their use provide new insights in the mechanisms taking place during membrane peeling and vein cannulation. In parallel, a robust online 3D reconstruction of retina (10Hz), based on stereoscopic images incorporating information from OCT, pose-, contact- and force-sensing is develop, together with robot technology and instruments, allowing conducting clinically relevant research on complex vitreoretinal surgical techniques such as epiretinal membrane peeling and vessel cannulation. Currently, such research is difficult or impossible due to procedural complexity and limitations in surgical skills. The mechanisms that are investigated thanks to EurEyeCase technology are amongst others research towards optimal peeling and cannulation strategies.

Finally, exploitation objectives are also part of EurEyeCase. A selection of as much as 13 possible technological outcomes, including novel instruments, dedicated guidance software, and patent-pending robot-assistive instruments are identified. It is expected that all these components will experience a substantial advancement w.r.t. the respective TRL level throughout the project. EurEyeCase also identifies and creates business opportunities putting European surgical robotics on the map, notably by means of detailed FTO-analyses, expansion of the current patent portfolio, targeted market studies and product refinement in close collaboration with surgeons.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Results on ‘WP1 – Clinical validation and user evaluation’:
The overall aim of this WP is to provide needed medical background knowledge and guidance to the engineers in order to steer developments towards clinically relevant technology. A detailed surgical workflow analysis has been conducted during this first year of the project. It was firstly only based on the manual protocols before being updated to include robotic assistance steps. Such analysis has then been extensively used as a basis to elaborate a 3-year experimental plan, to specify requirements for test-beds, to define the EurEyeCase setup technical specifications and to determine the appropriate software architecture. Multiple test-beds have been developed: some testbeds focus on realism with perfused ex-vivo pig eyes mock-up while others are completely artificial (mechanical components) and allow peeling and cannulation of respectively polymer membrane and vessels. These test-beds have been evaluated by EurEyeCase clinical partners. Feedback from these tests will be incorporated accordingly in further versions. Two major milestones have also been reached before the end of this first year with the organisation of the first EurEyeCase experimental campaign that aimed at evaluating the feasibility of retinal vein cannulation and membrane peeling with both co-manipulation and tele-manipulation setups. Clinicians were again involved for getting feedback and concluded that both setups are capable to perform both procedures. Apart from feedback on feasibility, they also gave additional inputs regarding the design of experimental trials and the strategic choices to follow during the second year of the project to improve the performances of EurEyeCase system.

Results on ‘WP2 – Sensor and robotic platform integration into advanced microsurgical operating suite’:
This WP develops sensors and integrates all hardware components into an advanced operating suite. For the first year of the project, only the integration of the robots in the surgical suite, the integration of the OCT-stereo-vision system with the microscope and the development of the sensors are foreseen. Operating environment has been reproduced around the robots (use of surgical microscope, operating table, light fibers, displaying screens) and allows to mimic all steps required when performing a robotically-assisted procedure in a surgical environment, i.e. calibration of the robot, positioning at the entry point with possible variability of patient head size, stabilization of the robot near the patient head, deploying safety procedures, checking the compatibility with the microscope, etc. In parallel, a camera-based imaging system has been installed on the microscope to acquire stereoscopic images of the eye. An iOCT Camera has also been integrated into the surgical microscope and a switching unit has been developed to switch between different modes (iOCT camera and fiber-probe sensor).
Finally, multiple complementary approaches have been proposed to develop adequate sensors, leading to very promising preliminary results that will be further exploited in year 2 to go towards integrated sensors, coupled with more advanced control laws. Regarding the current states, a fiber probe integrated in the OCT-Camera base device has been developed for proximity sensing, a force-sensing cannulation needle has been designed to assess interaction forces between the tool and the tissue and contact sensing based on electrical impedance is currently under investigation.

Results on ‘WP3 – Environment modelling and control’:
Multiple imaging sensors are part of the EurEyeCase robotic platform, the main ones being an OCT device and a stereo-vision system, offering multiple arrays of data to be analyzed. In this work package, the first year of the project has seen efforts towards the development of segmentation and annotation algorithms aiming to recognize major anatomical structures.
Regarding the stereo-vision system, an accurate retinal blood vessel segmentation in 2D RGB fundus images has been performed first using state-of-the-art computer vision algorithms for object detection and machine learning. A new approach, out-performing literature approaches in terms of detection accuracy for the task has been further developed, also exhibiting promising processing speed indicating a possible real-time use intra-operatively. Due to the lack of data acquired with the EurEyeCase robotic setup, training data used for the developed method have been found online, but the data-driven aspect of the algorithms enables them to also fit with data acquired with the project setup.
For the OCT system, algorithmic developments have been focusing on retinal layers identification by leveraging B-scans and C-scans data. A pre-processing step has been developed in order to properly deal with geometrical rescaling and artifacts compensation. For scan data ensured to be correct, a segmentation process has been developed, mainly relying on mathematical filtering and morphology. Preliminary results have also been obtained for retinal vessel identification.
Finally, calibration and 3D reconstruction topics for the stereo-vision system have been briefly addressed with the creation of a chess-board image and dedicated protocol for images with a strong magnification factor.

Results on ‘WP4 – System integration and operation schemes’:
To create a complete robotic setup made of different kind of sensors and in order to better assist the surgeon operating, each component must comply with the overall design and not interfere with other components. Based on the workflow analysis performed as part of WP1 and clinical feedbacks, the first achievement of this WP has been the identification of specifications for the overall system and each sub-system. Main specifications have been revolving around the physical interfacing of such subsystem; but also the virtual interfacing with data type, transfer protocol and frequency.
Secondly, a central Finite State Machine has been designed to ensure that all devices and system sub-components interact properly with each other. The results of the workflow analysis (WP1) have been used to identify the states of the FSM, and the implementation has been performed using ROS/OROCOS and LUA programming libraries. In order to fuse together the central FSM and the many different sensor data processes, a draft of the overall software architecture has been prepared with detailed communication framework and in a component-based setup. The software will also incorporate data recording, logging, and system health components to ensure proper behavior and avoid adverse events from software failure. Two versions of the software are being planned: a first one with limited and extremely safe functionalities for use during clinical trials; and a second one with all sensor data processing possible meant for research only, and making use of already existing third-party programming libraries.
Finally, some aspects for the creation of new Human-Machine interfaces have also been explored, such as acoustic feedbacks or a belt with unconventional haptic feedbacks.

Results on ‘WP5 - Technology transfer, exploitation and dissemination’:
The applicable medical regulations and standardizations have been analyzed in an early stage of the project. A documentation structure has been defined and provided to the partners for use in course of the entire project as well as for product development phase after the EurEyeCase project. In parallel, a detailed FTO analysis has been performed, supporting the development of innovative and inventive hardware components (WP2) and software functionalities (WP3). A first market study has also been performed. Focus has initially been on robot assistance for eye surgery at large.
Regarding the exploitation, business development activities including industrial and academic collaborations outside the consortium have been undertaken, supporting and enabling the development of exploitation opportunities, both with the prototype robotic operation suite and the final system.
Finally, dissemination activities have included a fair number of publications and presentations to the international community; a major dissemination event was the CRAS 2015 workshop that was co-ordinated by KUL and where TUE received a best paper award.

Results on ‘WP6 – Management’:
No less than 11 physical meetings have been held within the consortium, which allowed improving the communication and to exchange ideas more efficiently on the various research and development topics. In addition to these meetings, bi-weekly skype meetings have been organized since November to report on the progress of the technical partners and raise (and solve) possible issues on all kinds of topics. Meeting minutes have been circulated after this meeting. Regarding the financial management, each entity has been asked to report on its costs and efforts to the project coordinator every 6 months.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The main objective of EurEyeCase is to accomplish pre-clinical validation of robot-assisted membrane peeling and retinal vein cannulation and to prepare for a first in-vivo study of robot-assisted retinal vein cannulation.

However, all technological components that will be progressed to this end have their own value. A mixture of dedicated equipment (training models for VR, cannulation needles, contact-, OCT- and force-sensor integrated instruments), software components (3D retina reconstruction and OCT-guidance) and systems (teleoperation and comanipulation systems) have been identified as potentially relevant.

After the first year of EurEyeCase project, the teleoperation and co-manipulation setups as well as a number of test-beds are available within the consortium for experimenting and training. These setups have been further developed in close collaboration with clinicians. While stand-alone systems in the beginning, integration of these systems in an OR-like environment has been achieved. Progress was made to improve their usability and to reduce the setup time. A start was made of the risk analysis of both setups. In parallel progress was made with respect to instrument development. Cannulation needles with force-sensor capabilities have been designed, manufactured, tested and interfaced. Experiments with force sensing on cannulation of ex-vivo eyes showed encouraging results as the force sensing effectively indicated the motion of cannulation. This technology is believed to help preventing double puncture. An OCT fiber probe sensor has been designed and some first experiments with distance based feedback have been obtained. A switching device was built to allow switching between different OCT modalities (OCT camera versus OCT fiber). Efforts were made to reduce the communication time (and delay) between the OCT camera output and the overall robot control system so as to allow use of this sensing modality in a feedback control loop. Progress was made to acquire and process images from stereo- and oct-camera to enhance procedure safety. The goal will be to estimate relative distance between the instrument and the membranes and vessels through these modalities. A first application could entail using rough distance sensing to automatically adjust co-manipulation gains or scaling factors in the case of teleoperation. A significant effort was spent to develop a reliable software architecture that could support development and testing of the two operation modalities. We expect that this effort will pay off in the 2nd and 3rd year of the project.

Given the already advanced state of technology, EurEyeCase offers a unique opportunity to progress the general state-of-the-art in surgical robotics by deploying context-aware robotic assistance, notably via the OCT and stereo-imaging systems. Preliminary experiments have been conducted to include sensory information and intra-operative models to improve the safety and performance of the surgical procedure. Take-up of this level of assistance into commercial products seems currently feasible and thus, is expected in the foreseeable future.

Related information

Record Number: 190239 / Last updated on: 2016-11-14
Follow us on: RSS Facebook Twitter YouTube Managed by the EU Publications Office Top