Periodic Reporting for period 2 - OpenGTN (Open Ground Truth Training Network : Magnetic resonance image simulationfor training and validation of image analysis algorithms)
Período documentado: 2020-01-01 hasta 2021-12-31
Problem:
Magnetic Resonance (MR) imaging is the major medical imaging modality for brain and spine anatomy and pathology. A clear trend can be observed from visual image interpretation to computer-assisted diagnosis by quantification of disease-specific biomarkers, derived from the MR images. The major components in image quantification applications are tissue and organ segmentation and tissue/disease classification. Manual segmentation is too tedious and cumbersome for daily clinical practice and would lead to large inter-user variability. Much research is therefore performed on automatic segmentation techniques, and especially in the past decade the machine learning technique of deep learning is increasingly used. Training, validation and benchmarking of these techniques is currently impeded by the lack of large MR image databases with exact reference segmentations (ground truth).
Objective:
The openGTN research followed an innovative approach to overcome the current barriers for wide uptake in clinical practice of automatic MR image segmentation. By combining mathematical organ models with physical and biological tissue properties and image simulation and synthesis methods, a substantial public image databases has been established providing ample MR images with ground truth (exact) segmentations, by which fast and accurate optimization and validation of image segmentation algorithms can be enabled.
Importance for society:
The large MRI databases significantly contributes to faster and less costly development and validation of MR image segmentation techniques, thus facilitating their faster acceptance in daily clinical practice. With these techniques, the diagnosis and treatment of patients can be performed faster and more accurate, eventually leading to better disease diagnosis and treatment selection and outcome.
Conclusions:
The project has successfully realized its major goals:
- methods were developed for simulating (based on MR physics) and synthesizing (based on deep learning) very realistic MR image data of the brain, spine and heart, with ample anatomical variation, with and without pathology
- a large simulated/synthesized public MR image database was made available with ground truth references for the design, optimization, validation & benchmarking of image segmentation methods
- improved, fast and accurate MR image segmentation methods were developed, that are less sensitive to variation in the image appearance (such as intensity variations, noise, scanner origin)
Magnetic Resonance (MR) imaging is the major medical imaging modality for brain and spine anatomy and pathology. A clear trend can be observed from visual image interpretation to computer-assisted diagnosis by quantification of disease-specific biomarkers, derived from the MR images. The major components in image quantification applications are tissue and organ segmentation and tissue/disease classification. Manual segmentation is too tedious and cumbersome for daily clinical practice and would lead to large inter-user variability. Much research is therefore performed on automatic segmentation techniques, and especially in the past decade the machine learning technique of deep learning is increasingly used. Training, validation and benchmarking of these techniques is currently impeded by the lack of large MR image databases with exact reference segmentations (ground truth).
Objective:
The openGTN research followed an innovative approach to overcome the current barriers for wide uptake in clinical practice of automatic MR image segmentation. By combining mathematical organ models with physical and biological tissue properties and image simulation and synthesis methods, a substantial public image databases has been established providing ample MR images with ground truth (exact) segmentations, by which fast and accurate optimization and validation of image segmentation algorithms can be enabled.
Importance for society:
The large MRI databases significantly contributes to faster and less costly development and validation of MR image segmentation techniques, thus facilitating their faster acceptance in daily clinical practice. With these techniques, the diagnosis and treatment of patients can be performed faster and more accurate, eventually leading to better disease diagnosis and treatment selection and outcome.
Conclusions:
The project has successfully realized its major goals:
- methods were developed for simulating (based on MR physics) and synthesizing (based on deep learning) very realistic MR image data of the brain, spine and heart, with ample anatomical variation, with and without pathology
- a large simulated/synthesized public MR image database was made available with ground truth references for the design, optimization, validation & benchmarking of image segmentation methods
- improved, fast and accurate MR image segmentation methods were developed, that are less sensitive to variation in the image appearance (such as intensity variations, noise, scanner origin)
Overview of all work done and results achieved:
- MR image simulation software was selected and made operable (JEMRIS and MRXCAT)
- MR image synthesis software was developed on the basis of generative adversarial neural networks (SPADE-GAN)
- anatomical models of the brain, spine and heart were developed, incl. ample realistic variations, image artefacts (noise) and pathology
- deep learning image segmentation experiments were performed on MRI data from various sources, to investigate the effect of variation in image appearance
- concepts were developed and tested for image-appearance insensitive segmentation
- a large database of simulated and synthesized MRI data for brain, spine and heart was made publicly available (via the website opengtn.eu)
- results were reported at multiple scientific conferences and in scientific journal papers
- results were communicated via various other media (project website, LinkedIn, Twitter, Facebook)
Exploitation of results:
- the generated large database of MRI has been used by beneficiaries Philips and TU/e to augment the training of deep-learning neural-network based image segmentation; trained networks performed significantly better after the augmentation
- the methodology and software that were developed to synthesize MRI data have been exploited by Philips for research on federated learning, i,e, synthesizing image data in the hospital instead of on the basis of data retrieved from the hospital; such data can then be used to optimize image analysis algorithms, tailored to the hospital's image appearance; it also reduces the effort needed to acquire human data for research
Dissemination of results:
- a few scientific journal papers have already been submitted
- a significant number of scientific articles is in preparation and under review at scientific journals
- results have been presented orally or as posters at the major conferences in the field of MRI (ESMRMB, ISMSM) and image analysis (MICCAI)
- all publications are listed on the website opengtn.eu and can be downloaded from there
- MR image simulation software was selected and made operable (JEMRIS and MRXCAT)
- MR image synthesis software was developed on the basis of generative adversarial neural networks (SPADE-GAN)
- anatomical models of the brain, spine and heart were developed, incl. ample realistic variations, image artefacts (noise) and pathology
- deep learning image segmentation experiments were performed on MRI data from various sources, to investigate the effect of variation in image appearance
- concepts were developed and tested for image-appearance insensitive segmentation
- a large database of simulated and synthesized MRI data for brain, spine and heart was made publicly available (via the website opengtn.eu)
- results were reported at multiple scientific conferences and in scientific journal papers
- results were communicated via various other media (project website, LinkedIn, Twitter, Facebook)
Exploitation of results:
- the generated large database of MRI has been used by beneficiaries Philips and TU/e to augment the training of deep-learning neural-network based image segmentation; trained networks performed significantly better after the augmentation
- the methodology and software that were developed to synthesize MRI data have been exploited by Philips for research on federated learning, i,e, synthesizing image data in the hospital instead of on the basis of data retrieved from the hospital; such data can then be used to optimize image analysis algorithms, tailored to the hospital's image appearance; it also reduces the effort needed to acquire human data for research
Dissemination of results:
- a few scientific journal papers have already been submitted
- a significant number of scientific articles is in preparation and under review at scientific journals
- results have been presented orally or as posters at the major conferences in the field of MRI (ESMRMB, ISMSM) and image analysis (MICCAI)
- all publications are listed on the website opengtn.eu and can be downloaded from there
Final results:
The project has successfully realized its major goals:
- methods were developed for simulating (based on MR physics) and synthesizing (based on deep learning) very realistic MR image data of the brain, spine and heart, with ample anatomical variation, with and without pathology
- a large simulated/synthesized public MR image database was made available with ground truth references for the design, optimization, validation & benchmarking of image segmentation methods
- improved, fast and accurate MR image segmentation methods were developed, that are less sensitive to variation in the image appearance (intensity variations, noise, scanner origin)
Socia-economic impact:
Optimization of medical image analysis such as image segmentation requires large amounts of annotated human image data, especially when deep-learning neural networks are applied.
Acquiring these data from hospitals is very time consuming and costly. The same holds for scanning human volunteers.
The openGTN project has shown that potentially up to 80% of human image data can be replaced by simulated/synthesized data.
Alternatively, algorithm performance can be improved by augmenting human data with ample simulated/synthesized data.
Overall this will enable faster and less costly algorithm design, optimization and validation.
The simulated/synthesized data is publicly available, so that the complete research community can benefit.
Wider societal implications:
The use of the openGTN public simulated/synthesized MR image data has the potential:
- to speed up and lower the cost of medical image analysis development
- to improve the quality of developed algorithms/products
which eventually will be beneficial for medical industry, clinicians, patients and their relatives, and cost of healthcare.
The project has successfully realized its major goals:
- methods were developed for simulating (based on MR physics) and synthesizing (based on deep learning) very realistic MR image data of the brain, spine and heart, with ample anatomical variation, with and without pathology
- a large simulated/synthesized public MR image database was made available with ground truth references for the design, optimization, validation & benchmarking of image segmentation methods
- improved, fast and accurate MR image segmentation methods were developed, that are less sensitive to variation in the image appearance (intensity variations, noise, scanner origin)
Socia-economic impact:
Optimization of medical image analysis such as image segmentation requires large amounts of annotated human image data, especially when deep-learning neural networks are applied.
Acquiring these data from hospitals is very time consuming and costly. The same holds for scanning human volunteers.
The openGTN project has shown that potentially up to 80% of human image data can be replaced by simulated/synthesized data.
Alternatively, algorithm performance can be improved by augmenting human data with ample simulated/synthesized data.
Overall this will enable faster and less costly algorithm design, optimization and validation.
The simulated/synthesized data is publicly available, so that the complete research community can benefit.
Wider societal implications:
The use of the openGTN public simulated/synthesized MR image data has the potential:
- to speed up and lower the cost of medical image analysis development
- to improve the quality of developed algorithms/products
which eventually will be beneficial for medical industry, clinicians, patients and their relatives, and cost of healthcare.