Periodic Reporting for period 2 - icovid (AI-based chest CT analysis enabling rapid COVID diagnosis and prognosis)
Okres sprawozdawczy: 2021-09-01 do 2023-02-28
The impact COVID-19 has on health care systems and society as a whole is enormous. To date (April 2023), over 600 million COVID-19 cases have resulted in nearly 6.8 million deaths globally, with over 2 million of these occurring in Europe. When icovid was set up in 2020, no vaccine was available yet and clinicians focussed on early identification of SARS-CoV-2 coronavirus infections and optimizing the care path of COVID-19 positive cases. In line with this, the icovid project focused on developing, validating and deploying icolung, a CE-marked AI-based medical device software, to support the diagnosis and prognosis of patients suspected of COVID-19. By making use of lung Computed Tomography (CT) scans together with clinical information, icolung has proven to help guide radiologists, pulmonologists and intensive care unit (ICU) physicians in their patient management action by providing insights learned from a large population of treated COVID-19 patients.
Key achievements during the second (and final) reporting period of the project are:
● Method improvement
o Improved the diagnostic performance for the automated diagnosis of CT images of COVID-19 and confounding lung pathologies.
o Developed multifactorial models to improve diagnosis and prognosis of at-risk patients. These models combine risk factors from multiple types of data (e.g. demographic, laboratory results, imaging) using advanced AI-enabled analysis, ensuring maximal value is extracted from available patient information.
o Improved lesion segmentation based on nnU-Net architecture, obtaining 0.63 Dice and 0.56 normalized volume difference, which leads to improved severity scoring. Our approach ranked 24th out of > 220 participants on the COVID-19-20 segmentation challenge.
o Novel methods for interactive segmentation, including a multi-scale online likelihood network (MONet) for scribbles-based AI-assisted segmentation. Our approach achieved a 5.86% higher Dice while achieving 24.67% less perceived NASA-TLX workload score than the current state-of-the-art method.
o Improved interpretable classification approach, based on EfficientNet and a Bootstrap-Your-Own-Latent (BYOL) pretraining scheme. Our final model obtained 0.81 Area Under the Curve (AUC) on a challenging independent test set.
o A novel approach for unsupervised domain adaptation from non- to contrast-enhanced CT, based on a combination of self-supervision and contrastive, unpaired image-to-image translation.
o A ‘Digital Imaging and Communications in Medicine’ (DICOM) based prognosis prediction model, based on image data, age and gender. Our model obtained an AUC of 0.77 on an independent test set, and ranked 6th out of > 120 participants in the STOIC 2021 challenge.
● Demonstrating value
o The consortium was able to collect data on well over 10 000 patients, however the CT scans of only 8852 patients in the final multi-site dataset were matching the technical needs of icolung. Of these, 2296 were COVID-19 positive.
o We demonstrated that icolung enables a risk-based stratification of patients with acute COVID-19 infection using chest CT images. icolung predicted both ICU admission risk and in-hospital death.
o The health economic assessment showed that icolung is cost-effective in reducing the risk of transmission, with a low prevalence of COVID-19 infections when using a EUR 20000 as a willingness-to-pay threshold. Concerning the hospitalization cost, icolung is cost-effective at a higher value of COVID-19 prevalence and risk of hospitalization.
o The performed reader study showed that with the support of icolung, radiographers were able to increase sensitivity of diagnosing COVID-19 to a level close to consultant performance. The algorithm also improved the sensitivity of all reader groups at identifying normal scans. The radiologists also indicated a good usability of the software.
● Large scale deployment
o icolung has been and will continue to be offered to clinicians free of charge.
o icolung has been adopted by over 180 clinical centres in 27 countries, resulting in over 58 079 patient scans analysed by icolung in a clinical routine setting.
o Released and installed 5 new versions of the icolung software (v5-9).
o Icolung CE marking (up to v7, except v6) for the assessment of lung involvement was achieved. CE marking is still pending for diagnostic support functionality and for the advanced multifactorial prognostic models to enable these models in clinical practice.
o The Comunicare Analytics platform now provides access to all prognostic models developed within icovid, particularly the three prognostic models (MUM). Specifications for the integration of the Application Programming Interface (API) in icolung have been prepared, delivered and implemented.
o Clinical implementation guidance materials (both on a technical/operational level and an educational level) were developed and distributed amongst the icolung users. A clinical guidance program is also in place, helping users understand and make the best use of icolung.
Based on this activity and outputs, we believe it’s fair to state that icovid completed all major scientific goals and has had tremendous impact for patients and clinicians. All publications and software code has been made freely accessible online, and we will continue to offer the icolung software available for free to clinicians to use and are now also investigating the re-use of the software code in different disease areas.
● Method improvement
o Improved the diagnostic performance for the automated diagnosis of CT images of COVID-19 and confounding lung pathologies.
o Developed multifactorial models to improve diagnosis and prognosis of at-risk patients. These models combine risk factors from multiple types of data (e.g. demographic, laboratory results, imaging) using advanced AI-enabled analysis, ensuring maximal value is extracted from available patient information.
o Improved lesion segmentation based on nnU-Net architecture, obtaining 0.63 Dice and 0.56 normalized volume difference, which leads to improved severity scoring. Our approach ranked 24th out of > 220 participants on the COVID-19-20 segmentation challenge.
o Novel methods for interactive segmentation, including a multi-scale online likelihood network (MONet) for scribbles-based AI-assisted segmentation. Our approach achieved a 5.86% higher Dice while achieving 24.67% less perceived NASA-TLX workload score than the current state-of-the-art method.
o Improved interpretable classification approach, based on EfficientNet and a Bootstrap-Your-Own-Latent (BYOL) pretraining scheme. Our final model obtained 0.81 Area Under the Curve (AUC) on a challenging independent test set.
o A novel approach for unsupervised domain adaptation from non- to contrast-enhanced CT, based on a combination of self-supervision and contrastive, unpaired image-to-image translation.
o A ‘Digital Imaging and Communications in Medicine’ (DICOM) based prognosis prediction model, based on image data, age and gender. Our model obtained an AUC of 0.77 on an independent test set, and ranked 6th out of > 120 participants in the STOIC 2021 challenge.
● Demonstrating value
o The consortium was able to collect data on well over 10 000 patients, however the CT scans of only 8852 patients in the final multi-site dataset were matching the technical needs of icolung. Of these, 2296 were COVID-19 positive.
o We demonstrated that icolung enables a risk-based stratification of patients with acute COVID-19 infection using chest CT images. icolung predicted both ICU admission risk and in-hospital death.
o The health economic assessment showed that icolung is cost-effective in reducing the risk of transmission, with a low prevalence of COVID-19 infections when using a EUR 20000 as a willingness-to-pay threshold. Concerning the hospitalization cost, icolung is cost-effective at a higher value of COVID-19 prevalence and risk of hospitalization.
o The performed reader study showed that with the support of icolung, radiographers were able to increase sensitivity of diagnosing COVID-19 to a level close to consultant performance. The algorithm also improved the sensitivity of all reader groups at identifying normal scans. The radiologists also indicated a good usability of the software.
● Large scale deployment
o icolung has been and will continue to be offered to clinicians free of charge.
o icolung has been adopted by over 180 clinical centres in 27 countries, resulting in over 58 079 patient scans analysed by icolung in a clinical routine setting.
o Released and installed 5 new versions of the icolung software (v5-9).
o Icolung CE marking (up to v7, except v6) for the assessment of lung involvement was achieved. CE marking is still pending for diagnostic support functionality and for the advanced multifactorial prognostic models to enable these models in clinical practice.
o The Comunicare Analytics platform now provides access to all prognostic models developed within icovid, particularly the three prognostic models (MUM). Specifications for the integration of the Application Programming Interface (API) in icolung have been prepared, delivered and implemented.
o Clinical implementation guidance materials (both on a technical/operational level and an educational level) were developed and distributed amongst the icolung users. A clinical guidance program is also in place, helping users understand and make the best use of icolung.
Based on this activity and outputs, we believe it’s fair to state that icovid completed all major scientific goals and has had tremendous impact for patients and clinicians. All publications and software code has been made freely accessible online, and we will continue to offer the icolung software available for free to clinicians to use and are now also investigating the re-use of the software code in different disease areas.
We are proud to say that so far, we have helped clinicians with the management and diagnosis of more than 58 079 COVID-19 patients from over 180 clinical centres. This was possible due to very fast deployment of icolung (Q2 of 2020) and as icolung, while being cloud-based, was fully automated and integrated into the clinical workflow. In addition, being cloud-based enabled us to automatically release new versions of icolung into the clinical workflow without any additional installation requirements. These new versions included several new features and optimisations of our models. In parallel we’ve built a real-world data set of 8852 patients of which 2296 are COVID-19 positive. This dataset will be made available for research beyond the scope of the project. The performed reader study shows that with the support of icolung, radiographers were able to increase sensitivity of diagnosing COVID-19 to a level close to consultant performance. The algorithm also improved the sensitivity of all reader groups at identifying normal scans. The health economic assessment showed that icolung is cost-effective in reducing the risk of transmission, with a low prevalence of COVID-19 infections when using a EUR 20000 as a willingness-to-pay threshold.