Skip to main content

Computational Biophotonics for Endoscopic Cancer Diagnosis and Therapy

Periodic Reporting for period 3 - COMBIOSCOPY (Computational Biophotonics for Endoscopic Cancer Diagnosis and Therapy)

Reporting period: 2018-07-01 to 2019-12-31

Medical Background

Both, cancer diagnosis and tumor therapy are frequently performed using interventional procedures. In this context, replacement of traditional open surgery procedures with minimally-invasive interventions for tumor diagnosis, staging and therapy represents one of the most important challenges in modern healthcare. Minimally-invasive procedures provide numerous advantages in contrast to open surgery, including reduced surgical trauma, less pain medication, earlier convalescence, better cosmetic results, shorter hospitalization terms and lower costs. Furthermore, they are often the only promising treatment option for patients that are not eligible for surgery, due to old age or poor overall medical condition for example. Conventional medical imaging equipment in minimally-invasive procedures (e.g. endoscopes, laparoscopes), however, often offer poor tissue differentiation (e.g. healthy vs (pre)malignant or perfused vs not perfused), which results in inadequate treatment, long procedure times and high rates of complication. Hence, interventional imaging can be regarded as one of the key issues that should be addressed in cancer therapy.


State of the Art

A modality suitable for interventional procedures should provide real-time discrimination of local tissue with a high contrast-to-noise-ratio (CNR) and spatio-temporal context for global orientation and instrument guidance. It should ideally be radiation-free to prevent the patient and staff from being exposed to harmful ionizing radiation and facilitate integration into the clinical workflow, in addition to featuring a compact design at a low cost for a wide range of applicability and acceptance. Unfortunately, none of the imaging modalities widely used at a clinical level meets all of these requirements.
Two relatively young and so far separate fields of research, referred to as biophotonics and computer-assisted interventions (CAI), have addressed the challenge of interventional imaging by two very different approaches. Biophotonics techniques, which analyze the interaction between light and tissue, have led to major progress with respect to local classification using functional information but major issues, such as tissue motion, long acquisition times and difficulties in data sampling, display and interpretation still have to be addressed. CAI techniques have focused on structural information and have brought immense progress with respect to instrument guidance via multi-modal image fusion and object tracking, especially in open surgical procedures on rigid anatomy. However, sensitivity in tumor detection as well precision in tumor localization in the presence of tissue deformation must still be improved in order to fully exploit the potential of CAI. Furthermore, while different clinical and preclinical image modalities may provide different cues with respect to tissue classification (image contrast, shape, texture, function etc.), both the clinical and the technical state-of-the-art have focused on an isolated analysis of specific image modalities and failed to evaluate the complementary information available from images acquired before and during procedures.

Approach

The goal of the COMBIOSCOPY project is to develop new concepts for interventional imaging that (1) provide real-time discrimination of local tissue with a high contrast-to-noise-ratio (2) are radiation-free to prevent the patient and staff from being exposed to harmful ionizing radiation and (3) feature a compact design at a low cost for a wide range of applicability and acceptance. This shall be achieved by the systematic integration and mutual enhancement of biophotonics and computer assisted intervention techniques. Core of the project are novel machine learning - based algorithms that convert the high-dimensional multispectral data acquired into intuitive information that can be used by physicians for real-time clinical decision making.
Current status of individual work-packages

The achievements of the first half of the project correspond to five of the seven work packages (WPs) that are part of the COMBIOSCOPY grant; the first set of WPs have been devoted to the development of basic methods and techniques (WP1-5) that can potentially be applied for a wide range of applications, while the second set targets specific clinical challenges (WP6-7).

I. Hardware development (COMBIOSCOPY WP1)

Due to their potential to reconstruct the molecular tissue composition beneath the visible surface with high spatio-temporal resolution, spectral imaging techniques are applied within the scope of this project. While multispectral optical imaging is a passive technique that yields 2D reflection images but requires no contact with the tissue and no additional illumination unit, multispectral photoacoustic imaging has the advantage of providing tomographic images at a depth range of several centimeters. The following components were developed for this project:

Simulation framework for camera selection

Tissue reflectances captured by a multispectral camera encode physiological properties including oxygenation and blood volume fraction. Optimal camera properties such as filter responses depend crucially on the application, and choosing a suitable camera for a research project and/or a clinical problem (commercially) is not straightforward. To address this issue, we have proposed a generic framework for quantitative and application-specific performance assessment of cameras and optical subsystem without the need for any physical optical system setup [13]. Based on user input on camera characteristics and properties of the target domain, the framework quantifies the performance of the given camera configuration with large amounts of Monte-Carlo generated data and a user-defined performance metric. The advantage offered by being able to test the desired configuration without the need for purchasing expensive components may save system designers money, time and energy.

Multispectral endoscopes

In collaboration with our partners at Imperial College London we developed a first multispectral laparoscope based on fast filter wheel technology [17]. The filters were chosen using the band selection method developed in WP2 [18]. Based on insights gained from the first prototype and to address challenges resulting from sequential measurements we subsequently developed a second prototype based on the xiSpec (XIMEA GmbH) snapshot camera. This setup allows us to record 16 multispectral bands simultaneously. Due to its standard C mount connector the camera can be attached to off-the-shelf laparoscopes and thus easily integrated into clinical workflows.
A similar configuration was used to design a flexible catheter (eg. Polyscope from PolyDiagnost GmbH) that can be used in colonoscopy. The polyscope comes equipped with a long fiber optic that is inserted into a dedicated channel. PolyDiagnost GmbH also manufactured a c-mount adapter that allowed fiber optic to be connected the xiSpec (XIMEA GmbH) snapshot camera. Experiments for data collection on patients during colonoscopy were performed. After three trials it was concluded that the imaging system setup was not adequate for acquiring diagnostic images for analysis. For this reason, it was decided to shift the focus of the project to open surgery instead.
We have also tested an alternative method for multispectral data acquisition based on a combination of sparse hyperspectral and dense red-green-blue data. This approach holds promise for being able to augment the standard white light view but informed by detailed spectral data from known locations on the tissue surface [8]. This system used a small fibre optic probe that is compatible with a flexible endoscope and has the additional advantage of allowing tissue surface profilometry data to be acquired through the structured lighting technique [9].

Multispectral photoacoustic system
The goal of the COMBIOSCOPY project is to develop new concepts for interventional imaging that (1) provide fast discrimination of local tissue with a high contrast-to-noise-ratio (2) are radiation-free to prevent the patient and staff from being exposed to harmful ionizing radiation and (3) feature a compact design at a low cost for a wide range of applicability and acceptance. This shall be achieved by the systematic integration and mutual enhancement of biophotonics and computer assisted intervention techniques. Core of the project are novel machine learning - based algorithms that convert high-dimensional multispectral data acquired with multispectral optical and photoacoustic imaging techniques into intuitive information that can be used by physicians for real-time clinical decision making.

Achievements so far:

Replacing traditional open surgery with minimally-invasive techniques for complicated interventions such as partial tumor resection or anastomosis one of the most important challenges in modern healthcare. In these and many other procedures, characterization of the tissue perfusion and oxygenation remains challenging by means of visual inspection. Conventional laparoscopes are limited by “imitating” the human eye; multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia the lack of powerful digital image processing and bulky, nonstandard equipment prevents realizing the full potential of multispectral imaging (MSI) to date.

In the scope of the COMBIOSCOPY project, we developed the first multispectral laparoscopic imaging setup featuring (1) a compact and lightweight laparoscope built from commercially available parts which is straightforward to assemble and (2) the possibility to complement the conventional surgical view on the patient (RGB video images) with relevant morphological and functional information at an imaging rate of 30Hz

By combining the first video-rate capable multispectral sensor with advanced image processing techniques, our approach pioneers fluent perfusion monitoring and tissue discrimination with multispectral imaging.

Similar methodology has been developed to pioneer interventional multispectral photoacoustic imaging. Our concepts allow for fast estimation of important tissue parameters such as oxygenation at imaging depth of several centimeters.

Awards (Selection)
2017: Berlin-Brandenburg Academy Prize 2017 (Lena Maier-Hein)
http://www.bbaw.de/presse/pressemitteilungen/pressemitteilungen-2017/pm_12_maier-hein

2017: Best Pitch at Science Sparks Startups (Sebastian Wirkert, Anant Vemuri, Lena Maier-Hein)
http://www.technologiepark-heidelberg.de/aktuelles/news/news-details/article/ideen-entzuenden-auf-dem-science-sparks-start-ups-symposium/

2017: conhIT Nachwuchspreis (Janek Gröhl)
https://digitalhealthjobs.de/campus-pioniere-innovationen-digitale-gesundheit/

2016: Emil Salzer Prize (Lena Maier-Hein)
https://www.dkfz.de/en/presse/pressemitteilungen/2016/dkfz-pm-16-52a-Using-sound-and-light-for-navigating-inside-the-body.php

2015: Thomas-Gessmann Prize in Robotics and Automation (Justin Iszatt)
https://www.hs-heilbronn.de/10737476/jahresbericht-hhn-2015.pdf

2015: Thomas-Gessmann Prize in Medical Informatics (Janek Gröhl)
https://www.hs-heilbronn.de/10737476/jahresbericht-hhn-2015.pdf
1. Machine learning - based real-time quantification of tissue oxygenation in laparoscopic surgery.