Final Report Summary - SUMMER (Software for the Use of Multi-Modality images in External Radiotherapy)
Challenges and objectives
The ultimate goal for radiotherapy (RT) is to reach the highest probability of cure with the least side effects and morbidity. De-spite significant progress in imaging technologies over the last decades, some difficulties remain to identify tumour functional heterogeneity and to sufficiently control local tumour growth. There is a need to characterize tumours in terms of morphology, movement and molecular profile before, during and after treatments. RT is now more dependent on multimodal medical imaging (CT, MRI for morphology, PET with several tracers, MRS, DMRI, fMRI for molecular profile) than it has ever been - and that dependence is only going to become greater.
The scientific objectives of SUMMER Consortium is to agree on and develop mutual software that radiation oncologists, physicians, physicists and radiologists can share to study, annotate and contour images in RT, based on the following anticipated results:
• Creation of fast and intuitive tools for RT planning and evaluation using biological information (multimodal registration and fusion, segmentation of biological target volume, transparent and adaptive interface)
• Proposition of possible standards for image types used in external RT workflow (cross-disciplinary discussions and analyses)
• Contribution to clinical efforts and evidence based medicine (better targeting tumor tissue, increasing safety for OARs, platform shared among involved medical practitioners, suitable for patient treatment and follow-up)
Potential impact and use
There is a raised attention for the need of better knowledge about why one patient survives and the other does not after equal treatment and the need for better treatment strategies exploiting these insights. “SUMMER” intends to develop solutions which target this demand. The focus is not only on the single patient, but (still highly unusual) to support scientific research concerning large populations of patients. “SUMMER” will deliver technologies which can be used by many researchers attempting to transfer cancer from a deadly disease to a chronic disease. Furthermore, “SUMMER” will contribute to standardization of imaging modalities in use, clinical workflow and visualization in the context of external radiotherapy planning – standardization was also mentioned at the conference as being an essential element for success.
Besides scientific software design, the program delivers to a group of young researchers, individual training programs to develop other (cancer) diagnostics, planning, treatment and research equipment. They will also be trained to collaborate cross disciplinary, which is essential in their further careers, as cancer research is gradually shifting from academic solo career striving to collaborative effort between clinicians, academic researchers and industrials.
During the last decade several research groups have emerged in Europe focusing on different parts of the topics covered by “SUMMER”. These groups have to train their own recruits and very often lack a tight connection to the clinical users of the technology. In parallel, several companies in Europe are developing products and services related to image-guided therapy. Their need for well-trained human resources is high, and improved training of young researchers is likely to reduce cost and accelerate innovation for the relevant businesses. “SUMMER” network will implement a transfer of knowledge program that will seek to remedy for this.
The current situation in Europe makes it likely that the demand for well-trained researchers in this field will increase, as well as software solutions. This is motivated by the following:
• Europe has an aging population, which is related to higher incidence of cancer, and in particular to certain types of cancer, e.g. prostate cancer. The added value of multi-imaging radiotherapy planning is very high. This will increase the market for the industries involved and the amount of research efforts in the universities;
• The need and demand of multi-imaging in radiotherapy will continue to increase – as the merit of the techniques disseminates in both scientific and public domain;
• There is a rapid progress in computer and imaging technology that acts as an enabler for new methods – this trend is most likely to continue and act as a catalyst for industrial and academic R&D in this field.
Work performed and main results achieved
SUMMER Consortium chose lung and brain tumours as targeted locations and they will be used as proof of concept for evaluating the prototype. ICR Toulouse and UKL Freiburg radiation oncologists created acquisition and treatment protocols, which describe the general treatment of both lung and brain tumour patients, the features of the datasets and their use in the treatment process. Clinical partners (ICR Toulouse, UKL Freiburg and FSL Rome) gathered datasets to evaluate technical performance. In parallel, evaluation protocols have been designed for testing the various functionalities integrated in the SUMMER prototype, in terms of ergonomics, clinical usability and performance.
TU Delft conducted the analysis of current radiotherapy context. Observations of the practice of clinical experts were conducted at ICR Toulouse and UKL Freiburg in order to understand the current workflow, analyse and evaluate of the existing radiotherapy software. This led to identify some pitfalls and to gather insights for the design of new SUMMER interface. Thanks to a joint effort of AQUILAB and TU Delft, that was also the occasion to write down functional requirements of the complete SUMMER prototype, which specifies what intended software should do.
Following the same strategy, VRVis generated a list of multi-modal visualisation requirements on general point of view. Then, they switched to use-case-based scenarios to extend these requirements and achieve breakthrough results. The first scenario is related to visualization of time dependent PET-CT data based on UKL expertise, including fusion of 4D PET and CT, 4D contours, dose planning and respiratory tumour motion in lungs. The second scenario is related to MR spectroscopy data based on ICR expertise. We achieved comprehensive tool for fusing information of MRS with the information derived from other multi-modal images and several intra- and inter-patient studies.
Regarding work-packages related to segmentation and registration topics, young Fellows achieved good results in their research work. A state-of-the-art solution for rigid and deformable registration with a substantial improvement in reliability and clinical usefulness has been delivered by MUW Vienna. Based on rigid registration software, 2D and 3D feature descriptors are assessed for automatic quantitative registration error. In the same time, novel cost functions for deformable registration using the Demons algorithm has been developed and was evaluated on CT / CBCT co-registration. Several tools have been developed and tested for multimodality image processing. AQUILAB developed a tool for interactive segmentation of organ at risk in thoracic region based on graph-cut methodology, and a tool for automatic segmentation of subcortical region in brain based on deep learning and image textural features. Three different algorithms have been developed by ICR Toulouse to improve signal processing and metabolite quantification pipeline for MRSI data. A particular attention is also put to refine the usability and efficiency of the interface design by a joint work of AQUILAB, ICR Toulouse and TU Delft. Various segmentation algorithms for the use of PET images produced with different tracers have been analysed and integrated by UKL Freiburg. In the FSL Rome unit, an MR protocol has been designed to use functional MRI and diffusion tensor imaging sequences for extracting relevant information, like functional connectivity networks or tumour spread along white matter tracts.
The integration work led to a single prototype demonstrator regrouping all of these research results. AQUILAB coordinated the definition of technical requirements (including e.g. implementation platform, data structure and format, machine specifications) in order to facilitate the work and the cohesion between the partners.
Among the 10 early stage researchers participating to SUMMER, the Consortium still target 9 PhDs with 6 “secured”. 17 training events have been organized and SUMMER Consortium achieved a total 74 scientific publication including 22 joint publications. 12 additional publications have been submitted and are pending some reviews.
The ultimate goal for radiotherapy (RT) is to reach the highest probability of cure with the least side effects and morbidity. De-spite significant progress in imaging technologies over the last decades, some difficulties remain to identify tumour functional heterogeneity and to sufficiently control local tumour growth. There is a need to characterize tumours in terms of morphology, movement and molecular profile before, during and after treatments. RT is now more dependent on multimodal medical imaging (CT, MRI for morphology, PET with several tracers, MRS, DMRI, fMRI for molecular profile) than it has ever been - and that dependence is only going to become greater.
The scientific objectives of SUMMER Consortium is to agree on and develop mutual software that radiation oncologists, physicians, physicists and radiologists can share to study, annotate and contour images in RT, based on the following anticipated results:
• Creation of fast and intuitive tools for RT planning and evaluation using biological information (multimodal registration and fusion, segmentation of biological target volume, transparent and adaptive interface)
• Proposition of possible standards for image types used in external RT workflow (cross-disciplinary discussions and analyses)
• Contribution to clinical efforts and evidence based medicine (better targeting tumor tissue, increasing safety for OARs, platform shared among involved medical practitioners, suitable for patient treatment and follow-up)
Potential impact and use
There is a raised attention for the need of better knowledge about why one patient survives and the other does not after equal treatment and the need for better treatment strategies exploiting these insights. “SUMMER” intends to develop solutions which target this demand. The focus is not only on the single patient, but (still highly unusual) to support scientific research concerning large populations of patients. “SUMMER” will deliver technologies which can be used by many researchers attempting to transfer cancer from a deadly disease to a chronic disease. Furthermore, “SUMMER” will contribute to standardization of imaging modalities in use, clinical workflow and visualization in the context of external radiotherapy planning – standardization was also mentioned at the conference as being an essential element for success.
Besides scientific software design, the program delivers to a group of young researchers, individual training programs to develop other (cancer) diagnostics, planning, treatment and research equipment. They will also be trained to collaborate cross disciplinary, which is essential in their further careers, as cancer research is gradually shifting from academic solo career striving to collaborative effort between clinicians, academic researchers and industrials.
During the last decade several research groups have emerged in Europe focusing on different parts of the topics covered by “SUMMER”. These groups have to train their own recruits and very often lack a tight connection to the clinical users of the technology. In parallel, several companies in Europe are developing products and services related to image-guided therapy. Their need for well-trained human resources is high, and improved training of young researchers is likely to reduce cost and accelerate innovation for the relevant businesses. “SUMMER” network will implement a transfer of knowledge program that will seek to remedy for this.
The current situation in Europe makes it likely that the demand for well-trained researchers in this field will increase, as well as software solutions. This is motivated by the following:
• Europe has an aging population, which is related to higher incidence of cancer, and in particular to certain types of cancer, e.g. prostate cancer. The added value of multi-imaging radiotherapy planning is very high. This will increase the market for the industries involved and the amount of research efforts in the universities;
• The need and demand of multi-imaging in radiotherapy will continue to increase – as the merit of the techniques disseminates in both scientific and public domain;
• There is a rapid progress in computer and imaging technology that acts as an enabler for new methods – this trend is most likely to continue and act as a catalyst for industrial and academic R&D in this field.
Work performed and main results achieved
SUMMER Consortium chose lung and brain tumours as targeted locations and they will be used as proof of concept for evaluating the prototype. ICR Toulouse and UKL Freiburg radiation oncologists created acquisition and treatment protocols, which describe the general treatment of both lung and brain tumour patients, the features of the datasets and their use in the treatment process. Clinical partners (ICR Toulouse, UKL Freiburg and FSL Rome) gathered datasets to evaluate technical performance. In parallel, evaluation protocols have been designed for testing the various functionalities integrated in the SUMMER prototype, in terms of ergonomics, clinical usability and performance.
TU Delft conducted the analysis of current radiotherapy context. Observations of the practice of clinical experts were conducted at ICR Toulouse and UKL Freiburg in order to understand the current workflow, analyse and evaluate of the existing radiotherapy software. This led to identify some pitfalls and to gather insights for the design of new SUMMER interface. Thanks to a joint effort of AQUILAB and TU Delft, that was also the occasion to write down functional requirements of the complete SUMMER prototype, which specifies what intended software should do.
Following the same strategy, VRVis generated a list of multi-modal visualisation requirements on general point of view. Then, they switched to use-case-based scenarios to extend these requirements and achieve breakthrough results. The first scenario is related to visualization of time dependent PET-CT data based on UKL expertise, including fusion of 4D PET and CT, 4D contours, dose planning and respiratory tumour motion in lungs. The second scenario is related to MR spectroscopy data based on ICR expertise. We achieved comprehensive tool for fusing information of MRS with the information derived from other multi-modal images and several intra- and inter-patient studies.
Regarding work-packages related to segmentation and registration topics, young Fellows achieved good results in their research work. A state-of-the-art solution for rigid and deformable registration with a substantial improvement in reliability and clinical usefulness has been delivered by MUW Vienna. Based on rigid registration software, 2D and 3D feature descriptors are assessed for automatic quantitative registration error. In the same time, novel cost functions for deformable registration using the Demons algorithm has been developed and was evaluated on CT / CBCT co-registration. Several tools have been developed and tested for multimodality image processing. AQUILAB developed a tool for interactive segmentation of organ at risk in thoracic region based on graph-cut methodology, and a tool for automatic segmentation of subcortical region in brain based on deep learning and image textural features. Three different algorithms have been developed by ICR Toulouse to improve signal processing and metabolite quantification pipeline for MRSI data. A particular attention is also put to refine the usability and efficiency of the interface design by a joint work of AQUILAB, ICR Toulouse and TU Delft. Various segmentation algorithms for the use of PET images produced with different tracers have been analysed and integrated by UKL Freiburg. In the FSL Rome unit, an MR protocol has been designed to use functional MRI and diffusion tensor imaging sequences for extracting relevant information, like functional connectivity networks or tumour spread along white matter tracts.
The integration work led to a single prototype demonstrator regrouping all of these research results. AQUILAB coordinated the definition of technical requirements (including e.g. implementation platform, data structure and format, machine specifications) in order to facilitate the work and the cohesion between the partners.
Among the 10 early stage researchers participating to SUMMER, the Consortium still target 9 PhDs with 6 “secured”. 17 training events have been organized and SUMMER Consortium achieved a total 74 scientific publication including 22 joint publications. 12 additional publications have been submitted and are pending some reviews.