Periodic Reporting for period 3 - EurEyeCase (Use Case for European Robotics in Ophthalmologic Micro−Surgery)
Reporting period: 2017-01-01 to 2018-02-28
Epiretinal membrane and retinal vein occlusion are common pathologic conditions with high complication rates that greatly reduce quality of life of the affected people. They can be treated by peeling the epiretinal membrane and cannulating the occluded microvessel(s) but are currently high-risk procedures. To improve their outcomes, robot-assistance control schemes for micro-positioning and micro-force manipulation surgical tasks are developed, starting by the design of a set of innovative miniature sensorised instruments. These smart instruments include proximity, contact and force sensing in an extreme compact package. In parallel, a robust online 3D reconstruction of retina, based on stereoscopic images incorporating information from OCT, pose-, contact- and force-sensing is developed together with robot technology and instruments, allowing conducting clinically relevant research on complex vitreoretinal surgical techniques. Currently, such research is difficult or impossible due to procedural complexity and limitations in surgical skills. The mechanisms that are investigated thanks to EurEyeCase technology are amongst others research towards optimal peeling and cannulation strategies. The safety and intended use of robot-assisted procedures will be demonstrated in order to prepare a first-in-vivo study at the end of the project.
EurEyeCase also identified and created business opportunities putting European surgical robotics on the map, notably by means of detailed FTO-analyses, expansion of the current patent portfolio, targeted market studies and product refinement in close collaboration with surgeons.
EurEyeCase also identified and created business opportunities putting European surgical robotics on the map, notably by means of detailed FTO-analyses, expansion of the current patent portfolio, targeted market studies and product refinement in close collaboration with surgeons.
WP1
WP1 extracted the medical background knowledge and guidance to steer engineering developments towards clinically relevant outcome. Developments on test-beds have continued. The different test-beds have been evaluated by EurEyeCase clinical partners throughout. Multiple data-gathering experiments were set up to record data. Clinicians were involved providing feedback on the benefit of employing sensor-integrated instruments during the interventions. Considerable time has been spent on refining the strategy to move to in-vivo experiments by the end of the project.
WP2
WP2 developed sensors and integrated all hardware components into an advanced operating suite (including camera-based imaging systems to acquire stereoscopic images, iOCT camera integrated into the surgical microscope, a switching unit to switch between modalities). Different variations of a head immobilization systems have been prototyped and tested. Different concepts to improve the OCT retina imaging and S/N ratio were explored. The most reliable distance feedback is found to come from the OCT Ascan, allowing clear identification of the retina surface. An effort was done to speed up these measurements. Force sensor-integrated experiments have been further developed. Remote feedback by the OCT Ascan and force-sensor have been used as a safety loop for robotic control.
WP3
The algorithms used pre-operatively were optimized and integrated in EurEyeCase setup, working now real-time. An accurate retinal blood vessel segmentation has been performed with the stereo-vision system, showing large improvements (accuracy, execution time) compared to competing methods. Pre-operative modelling aspects were finalized, with the construction of the detailed textured 3D model of the retina within the stereo 3D imaging. For the OCT system, algorithms focused on retinal layers identification by leveraging B-scans and C-scans data. A pre-processing step has been developed to properly deal with geometrical rescaling and artifacts. Preliminary results have also been obtained for retinal vessel identification.
WP4
EurEyeCase system setup v1 - focuses on the targeted clinical trials, requires documented software and hardware solutions adequate to high TRL levels. We have further detailed the v1 architecture, its communication interfaces, its graphical interface(s), and business logic. Regular integration meetings have been held to develop and test together the system components. In parallel to v1, the v2 setup forms a system for the next generation, where advanced assistive features such as OCT A/B/C-scan labelling, stereo image processing and segmentation of instruments were further developed.
WP5
The results of the FTO analysis have been monitored and updated. Business development activities including industrial and academic collaborations have been undertaken. Importantly, several collaboration activities reaching outside have been done. Thhese interactions confirmed the unmet need and market opportunities in the field of vitreoretinal surgery. Finally, dissemination activities have included a fair number of publications and presentations to the international community.
WP6
In addition to bi-weekly meeting organized between software developers, physical meetings were held to integrate further all EurEyeCase components in a unique platform, successfully evaluated by the clinicians. Weekly skype meetings have also been conducted to report on the progress, raise (and solve) possible issues on all kinds of topics and discuss the strategical orientation for Y3, notably on the choice of relevant technologies to conduct human clinical trials.
WP1 extracted the medical background knowledge and guidance to steer engineering developments towards clinically relevant outcome. Developments on test-beds have continued. The different test-beds have been evaluated by EurEyeCase clinical partners throughout. Multiple data-gathering experiments were set up to record data. Clinicians were involved providing feedback on the benefit of employing sensor-integrated instruments during the interventions. Considerable time has been spent on refining the strategy to move to in-vivo experiments by the end of the project.
WP2
WP2 developed sensors and integrated all hardware components into an advanced operating suite (including camera-based imaging systems to acquire stereoscopic images, iOCT camera integrated into the surgical microscope, a switching unit to switch between modalities). Different variations of a head immobilization systems have been prototyped and tested. Different concepts to improve the OCT retina imaging and S/N ratio were explored. The most reliable distance feedback is found to come from the OCT Ascan, allowing clear identification of the retina surface. An effort was done to speed up these measurements. Force sensor-integrated experiments have been further developed. Remote feedback by the OCT Ascan and force-sensor have been used as a safety loop for robotic control.
WP3
The algorithms used pre-operatively were optimized and integrated in EurEyeCase setup, working now real-time. An accurate retinal blood vessel segmentation has been performed with the stereo-vision system, showing large improvements (accuracy, execution time) compared to competing methods. Pre-operative modelling aspects were finalized, with the construction of the detailed textured 3D model of the retina within the stereo 3D imaging. For the OCT system, algorithms focused on retinal layers identification by leveraging B-scans and C-scans data. A pre-processing step has been developed to properly deal with geometrical rescaling and artifacts. Preliminary results have also been obtained for retinal vessel identification.
WP4
EurEyeCase system setup v1 - focuses on the targeted clinical trials, requires documented software and hardware solutions adequate to high TRL levels. We have further detailed the v1 architecture, its communication interfaces, its graphical interface(s), and business logic. Regular integration meetings have been held to develop and test together the system components. In parallel to v1, the v2 setup forms a system for the next generation, where advanced assistive features such as OCT A/B/C-scan labelling, stereo image processing and segmentation of instruments were further developed.
WP5
The results of the FTO analysis have been monitored and updated. Business development activities including industrial and academic collaborations have been undertaken. Importantly, several collaboration activities reaching outside have been done. Thhese interactions confirmed the unmet need and market opportunities in the field of vitreoretinal surgery. Finally, dissemination activities have included a fair number of publications and presentations to the international community.
WP6
In addition to bi-weekly meeting organized between software developers, physical meetings were held to integrate further all EurEyeCase components in a unique platform, successfully evaluated by the clinicians. Weekly skype meetings have also been conducted to report on the progress, raise (and solve) possible issues on all kinds of topics and discuss the strategical orientation for Y3, notably on the choice of relevant technologies to conduct human clinical trials.
The main objective of EurEyeCase was to accomplish pre-clinical validation of robot-assisted membrane peeling and retinal vein cannulation and to prepare for a first in-vivo study of robot-assisted procedures. All technological components that were progressed to this end have their own value. A mixture of dedicated equipment, software components and systems have been identified as potentially relevant and have been further developed in close collaboration with clinicians. Integration of stand-alone systems in an OR-like environment has been achieved, with progress to improve their usability and to reduce the setup time. In parallel cannulation needles with force-sensor capabilities have been designed, manufactured, tested and interfaced to effectively indicated the instant of cannulation; OCT-fiber probes have been designed and experimented with distance based feedback strategies (visual, auditory and haptic). Progress was made to acquire and process images from stereo- and OCT-camera to enhance procedure safety. Vessel segmentation algorithms allow detecting the vessel structures. A significant effort has been spent to develop a reliable software architecture that would support development and testing of the two operation modalities in a clinical setting.
Given the already advanced state of technology, EurEyeCase offered a unique opportunity to progress the general state-of-the-art in surgical robotics by deploying context-aware robotic assistance, notably via force, OCT and stereo-imaging measurement systems. The consortium has prepared the developments to the level that they can be tested in a clinical setting. The performance of advanced algorithms have been validated in an in-vivo animal setting. After that a world-first in-human feasibility experiment was conducted. Five patients underwent a vitreoretinal intervention in which OCT-fiber probes were used and feedback from this probes was generated. Experiments with visual, auditory and haptic feedback were conducted. This means that a significant step was made towards inclusion of intelligence into a surgical robotic commercial product.
Given the already advanced state of technology, EurEyeCase offered a unique opportunity to progress the general state-of-the-art in surgical robotics by deploying context-aware robotic assistance, notably via force, OCT and stereo-imaging measurement systems. The consortium has prepared the developments to the level that they can be tested in a clinical setting. The performance of advanced algorithms have been validated in an in-vivo animal setting. After that a world-first in-human feasibility experiment was conducted. Five patients underwent a vitreoretinal intervention in which OCT-fiber probes were used and feedback from this probes was generated. Experiments with visual, auditory and haptic feedback were conducted. This means that a significant step was made towards inclusion of intelligence into a surgical robotic commercial product.