Periodic Reporting for period 1 - JUNO (Joint Volumetric Reconstruction and Automated Analysis of the Fetal Heart from Cardiovascular Magnetic Resonance Images)
Okres sprawozdawczy: 2015-11-16 do 2017-11-15
The JUNO project was aimed at improving the present capabilities of cardiovascular magnetic resonance (CMR) by tackling its limitations with an image processing approach. The specific field of application was fetal CMR, but due to a lack of sufficient data of this type the project focused on the development of methods to improve aspects of CMR imaging in the adult. More specifically, the problem being addressed was the lack of automated techniques to evaluate and improve the quality of short-axis CMR image stacks, which are the reference images for structural and functional assessment of the heart. Quality control on these images is usually performed visually by the radiologist, necessarily leading to operator-dependent evaluations. Moreover, in the last decade several initiatives have been launched both within the EU and worldwide for the acquisition of large-scale open-access databases of CMR images (e.g. the UK Biobank, now ongoing, which will in the end include full CMR scans for 100’000 subjects). While for these large databases visual quality assessment and manual adjustments are unfeasible, failure to correctly identify corrupted scans might affect the results of subsequent automated analyses with undesirable effects. Accordingly, JUNO was dedicated to the development of techniques to allow fully-automated quality assessment and correction of CMR image stacks. The specific issues tackled were the following ones: 1) inter-slice motion detection & correction, 2) heart coverage estimation, 3) cardiac contrast estimation.
Within JUNO, the MSCA Fellow developed and implemented the following three techniques:
1) a learning-based technique for fully-automated detection and correction of inter-slice motion in adult CMR image stacks. This technique allows the detection of CMR image stacks that have undergone motion (because of differences in the breath-holding positions held by the subject during the acquisition), and is capable of realign the stacks either exploiting learned 3D models of the cardiac shape or other acquired images (long-axis CMR images) used as reference (see Fig.1). This technique was tested on large subsets of two different datasets, the UK Digital Heart Project and the UK Biobank. The obtained results show that the approach is highly effective in detecting and compensating for inter-slice motion, and could thus be adopted in the clinical practice. Scientific dissemination was performed with two oral presentations at international conferences (CinC 2016 and ISMRM 2017) together with two accompanying short papers;
2) a learning-based technique for the fully-automated estimation of the heart coverage for CMR image stacks. This technique allows the identification of CMR image stacks that do not cover the whole left ventricular region (see Fig.2). The technique was tested on a large subset of the UK Biobank containing 3000 CMR stacks. The obtained results show the robustness and the accuracy of the proposed technique, which could be exploited in the clinical routine. Scientific dissemination was performed with a presentation at an international conference (FIMH 2017) together with an accompanying short paper;
3) a learning-based technique for the fully-automated estimation of the cardiac contrast for CMR image stacks. This technique allows the identification of CMR image stacks that do not offer an optimal image contrast between the cardiac structures of the left ventricle (see Fig.3). The technique was tested on a subset of the UK Biobank containing 100 CMR stacks. The results show that the proposed approach estimates cardiac contrast with a high degree of accuracy.
Notably, these three techniques have been implemented using the same machine learning approach (i.e. hybrid random forests), and consequently can be considered different steps of a comprehensive quality control pipeline for CMR image stacks. This pipeline is now described in detail in a journal paper currently in submission.
In addition, the MSCA Fellow collaborated to the development and implementation of the following two techniques:
4) a semi-supervised learning approach for network-based segmentation of CMR image stacks. This technique achieves cardiac segmentation with an accuracy comparable to that of human interpreters (see Fig.3). Scientific dissemination was performed with a presentation at an international conference (MICCAI 2017) with an accompanying short paper and with the submission of a journal paper (now undergoing revision);
5) a technique for learning low-dimensional representations of cardiac remodelling from super-resolved CMR image stacks based on a convolutional variational autoencoder. These representations can be then used for classification tasks. This approach yielded a very high accuracy in the discrimination among healthy, hypertrophic and dilated cardiomyopathy subjects, and shows promise for unsupervised classification of pathologies associated with ventricular remodelling. Scientific dissemination was performed with the submission of an abstract to an upcoming conference.
1) a learning-based technique for fully-automated detection and correction of inter-slice motion in adult CMR image stacks. This technique allows the detection of CMR image stacks that have undergone motion (because of differences in the breath-holding positions held by the subject during the acquisition), and is capable of realign the stacks either exploiting learned 3D models of the cardiac shape or other acquired images (long-axis CMR images) used as reference (see Fig.1). This technique was tested on large subsets of two different datasets, the UK Digital Heart Project and the UK Biobank. The obtained results show that the approach is highly effective in detecting and compensating for inter-slice motion, and could thus be adopted in the clinical practice. Scientific dissemination was performed with two oral presentations at international conferences (CinC 2016 and ISMRM 2017) together with two accompanying short papers;
2) a learning-based technique for the fully-automated estimation of the heart coverage for CMR image stacks. This technique allows the identification of CMR image stacks that do not cover the whole left ventricular region (see Fig.2). The technique was tested on a large subset of the UK Biobank containing 3000 CMR stacks. The obtained results show the robustness and the accuracy of the proposed technique, which could be exploited in the clinical routine. Scientific dissemination was performed with a presentation at an international conference (FIMH 2017) together with an accompanying short paper;
3) a learning-based technique for the fully-automated estimation of the cardiac contrast for CMR image stacks. This technique allows the identification of CMR image stacks that do not offer an optimal image contrast between the cardiac structures of the left ventricle (see Fig.3). The technique was tested on a subset of the UK Biobank containing 100 CMR stacks. The results show that the proposed approach estimates cardiac contrast with a high degree of accuracy.
Notably, these three techniques have been implemented using the same machine learning approach (i.e. hybrid random forests), and consequently can be considered different steps of a comprehensive quality control pipeline for CMR image stacks. This pipeline is now described in detail in a journal paper currently in submission.
In addition, the MSCA Fellow collaborated to the development and implementation of the following two techniques:
4) a semi-supervised learning approach for network-based segmentation of CMR image stacks. This technique achieves cardiac segmentation with an accuracy comparable to that of human interpreters (see Fig.3). Scientific dissemination was performed with a presentation at an international conference (MICCAI 2017) with an accompanying short paper and with the submission of a journal paper (now undergoing revision);
5) a technique for learning low-dimensional representations of cardiac remodelling from super-resolved CMR image stacks based on a convolutional variational autoencoder. These representations can be then used for classification tasks. This approach yielded a very high accuracy in the discrimination among healthy, hypertrophic and dilated cardiomyopathy subjects, and shows promise for unsupervised classification of pathologies associated with ventricular remodelling. Scientific dissemination was performed with the submission of an abstract to an upcoming conference.
To the best of the knowledge available to the MSCA Fellow, no comprehensive pipeline for quality assessment and motion correction of CMR images has been presented in the literature. The techniques developed during the implementation of JUNO offer a potential solution to achieve fully-automated, fast and robust quality control for the CMR images, which is a necessary step before the application of functional analyses in order to ensure the reliability of the obtained results. This is particularly important for the many studies using large-scale open-access databases of CMR images that have been built in the past years. Moreover, JUNO contributed to the development of techniques for the analysis of CMR images, achieving top accuracies in the tasks of image segmentation and classification for disease identification. As a whole, the techniques developed within JUNO offer some of the most advanced tools for CMR image analysis, and could be thus exploited to increase the amount of clinically-relevant information extracted using this imaging modality.