Periodic Reporting for period 2 - Deep4MI (Deep Learning for Medical Imaging: Learning Clinically Useful Information from Images)
Periodo di rendicontazione: 2022-07-01 al 2023-12-31
Medical imaging has revolutionized medicine and healthcare like no other recent technology, and is now an integral part of diagnosis, treatment planning, treatment delivery and follow-up. It provides an unparalleled ability to image anatomy and function with high spatial (and temporal) resolution. Its success has led to a dramatic increase in the number of medical imaging examinations. Despite this success, medical imaging is often stressful for patients, requires patient cooperation and is difficult in the presence of motion (e.g. patient motion or breathing motion). Furthermore, even more than 100 years after the discovery of X-rays, the interpretation of medical images relies almost exclusively on human experts. All of the above mean that there is a strong need for increased automation and quantification in order to reduce costs, increase efficiency and patient-friendliness, and provide higher diagnostic and prognostic accuracy for clinical decision making.
At the same time, machine learning and deep learning techniques have made significant advances and have started to make a large impact in many real-world applications. The aim of this proposal is to exploit these advances to address the above challenges and to achieve a paradigm shift in the way information is extracted from medical images for diagnostics, therapy and follow-up. The project aims to do this by developing a transformative and synergistic approach to medical imaging in which acquisition, reconstruction, analysis and interpretation will be tightly coupled, with bidirectional feedback between the different stages, in order to optimize the overall objective of the imaging pipeline: Extracting clinically useful and actionable information. To achieve this step change, the project aims to develop novel deep learning approaches for image acquisition, reconstruction, analysis and interpretation that can be trained in an end-to-end fashion, allowing fast and more efficient imaging.
At the same time, machine learning and deep learning techniques have made significant advances and have started to make a large impact in many real-world applications. The aim of this proposal is to exploit these advances to address the above challenges and to achieve a paradigm shift in the way information is extracted from medical images for diagnostics, therapy and follow-up. The project aims to do this by developing a transformative and synergistic approach to medical imaging in which acquisition, reconstruction, analysis and interpretation will be tightly coupled, with bidirectional feedback between the different stages, in order to optimize the overall objective of the imaging pipeline: Extracting clinically useful and actionable information. To achieve this step change, the project aims to develop novel deep learning approaches for image acquisition, reconstruction, analysis and interpretation that can be trained in an end-to-end fashion, allowing fast and more efficient imaging.
One of the goals of the projects is to improve how computers generate and understand medical images. In this respect, the project has develop new ways for computers to improve the quality of medical images of the heart, especially when there motion (e.g. breathing or heart movement). As this project focuses on enhancing the quality and understanding of medical images, we have made several key advances. Our focus has been specifically of Magentic Resonance Images (MRI) of the heart and brain.
Our key advances include the following:
1. We have developed on advanced techniques to improve the speed of acquisition of MRI of heart. For this, we have focused on novel neural network methods that can deal with motion of the beating heart but also motion due to breathing of the patient. This makes MRI more comfortable as the patient has to spend less time in the scanner and does not have to hold their breath for long persions of time.
2. Another major advance includes development of novel AI methods for reconstructing images and geometric models from new born infants. Conventionally, the recontstruction of geometric models of the developing brain is very challenging due to motion of the infants and the limited resoltion of the ianges. However, the developed methods overcome this limitation by exploiting prior knowledge, in this case the age of the infants.
3. Aditionally, the project has developed robust deep learning models that address the so-called domain shift problem. This problem occurs when AI models are trained with data from one source (also called the source domain) and then applied to data from another source (also called the target domain). In clinical scenarios, the AI models often do not generalize well to the unseen domains. To tackle this, two different approaches have been developed. In the first approach we ue a causality-inspired data augmentation approach to generate additional training data. In the second approach investigates an alternative approach to single-source domain generalization: Here, we have developed an adversarial data augmentation approach.
Our key advances include the following:
1. We have developed on advanced techniques to improve the speed of acquisition of MRI of heart. For this, we have focused on novel neural network methods that can deal with motion of the beating heart but also motion due to breathing of the patient. This makes MRI more comfortable as the patient has to spend less time in the scanner and does not have to hold their breath for long persions of time.
2. Another major advance includes development of novel AI methods for reconstructing images and geometric models from new born infants. Conventionally, the recontstruction of geometric models of the developing brain is very challenging due to motion of the infants and the limited resoltion of the ianges. However, the developed methods overcome this limitation by exploiting prior knowledge, in this case the age of the infants.
3. Aditionally, the project has developed robust deep learning models that address the so-called domain shift problem. This problem occurs when AI models are trained with data from one source (also called the source domain) and then applied to data from another source (also called the target domain). In clinical scenarios, the AI models often do not generalize well to the unseen domains. To tackle this, two different approaches have been developed. In the first approach we ue a causality-inspired data augmentation approach to generate additional training data. In the second approach investigates an alternative approach to single-source domain generalization: Here, we have developed an adversarial data augmentation approach.
The project has already made several advances in the area of generation, analysis and interpretation of medical images (see above) and demonstrated the clinical usefulness in several applications. The remaining part of the project will dive deeper into integrated approaches for the generation, analysis and interpretation of medical images. This way we can ensure that the information extracted from medical images is as useful as possible for doctors.