COMputational Models FOR patienT stratification in urologic cancers – Creating robust and trustworthy multimodal AI for health care

Project Information

COMFORT

Grant agreement ID: 101079894

DOI

10.3030/101079894

EC signature date 17 March 2023

Start date 1 April 2023

End date 31 March 2027

Funded under

• Health

Total cost

€ 5 906 596,25

EU contribution

€ 5 906 596,25

Coordinated by

KLINIKUM DER TECHNISCHEN UNIVERSITÄT MÜNCHEN (TUM KLINIKUM)
Germany

Periodic Reporting for period 1 - COMFORT (COMputational Models FOR patienT stratification in urologic cancers – Creating robust and trustworthy multimodal AI for health care)

Reporting period: 2023-04-01 to 2024-09-30

Summary of the context and overall objectives of the project

Cancer remains one of the leading causes of death worldwide, with prostate cancer and kidney cancer affecting millions of Europeans each year. Early and accurate diagnosis is crucial for improving patient outcomes, but current clinical methods often struggle to fully utilize the vast amounts of complex patient data available.
The COMFORT project aims to develop advanced artificial intelligence (AI) models that can analyze multiple types of medical data simultaneously - including medical imaging and clinical notes - to assist doctors in detecting and diagnosing prostate and kidney cancers earlier and more accurately. By combining information from different sources, these "multimodal" AI models have the potential to recognize patterns and make predictions that may not be apparent to human clinicians alone.
A key focus of COMFORT is ensuring these AI tools are trustworthy and can be confidently adopted in clinical practice. The project is developing methods to make the AI decision-making process more transparent and explainable. It also aims to identify and mitigate potential biases to ensure the models work fairly for all patient groups.
To validate the real-world impact, COMFORT will conduct a large multinational clinical study testing the AI models across multiple hospitals. This will provide crucial evidence on how the technology performs in actual clinical settings and its potential to improve patient care. If successful, COMFORT's AI models could enable more personalized and precise cancer diagnosis and prognosis. This may help doctors tailor treatments more effectively, potentially improving survival rates while reducing unnecessary procedures. The project's innovations may also accelerate the broader adoption of trustworthy AI in healthcare, supporting Europe's digital transformation of health systems.
By bringing together clinical, technical, and ethical expertise from across Europe, COMFORT aims to advance the state-of-the-art in medical AI and translate cutting-edge research into practical tools that can make a real difference in cancer care. The project's open science approach will also ensure its findings and resources benefit the wider research community and society.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The COMFORT project has made significant progress in developing advanced AI models for prostate and kidney cancer diagnosis. Our team has successfully created and annotated large datasets of medical imaging and clinical data, including over 8,310 prostate MRIs and 1,802 MRI series for kidney cancer. We have developed the MRSegmentator, an open-source tool capable of segmenting 40 different anatomical structures across multiple MRI sequences, which is already being used by researchers worldwide (500 downloads per month).
In natural language processing, we have made strides in automating the structuring of radiology reports using large language models. We have also developed innovative techniques for cross-modality transfer learning between CT and MRI, potentially reducing the need for extensive manual annotations.
Our team has furthermore conducted a large-scale multinational survey on patient attitudes towards AI in healthcare, gathering responses from 13,806 patients across 43 countries. This study provides valuable insights into public perceptions and concerns regarding AI in medical settings.
Based on bilateral interactions with project stakeholders, including AI engineers, software developers and healthcare professionals, a non-exhaustive list of initial functional and non-functional requirements was developed. Based on this list the initial reference architecture for the COMFORT platform was documented and delivered. This documentation will serve as the main reference point for the delivery of the prototypes and the integration of the multimodal AI models. Subsequently, these prototypes will serve as the main experimentation site for the clinical prospective study.
Further, we have conducted thorough literature reviews to identify the trustworthiness of multimodal architecture in the medical domain and applied the causal inference technique to explore fairness in text data. The large-scale survey motivates the design of AI trustworthiness model.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

COMFORT's multimodal AI approach represents a significant advance over existing single-modality models. By integrating information from medical imaging, clinical notes, and other data sources, our models have the potential to provide more comprehensive and accurate diagnoses than current methods. Our work on explainable AI and bias mitigation is pushing the boundaries of trustworthy AI in healthcare. We are developing techniques to make AI decision-making processes more transparent and understandable to clinicians, which is crucial for building trust and ensuring responsible AI deployment in medical settings.
To ensure successful uptake of these innovations, further research is needed to refine the models and validate their performance in diverse clinical settings. Demonstration projects in real-world healthcare environments will be crucial to prove the technology's value and ease of integration into existing workflows. Support for commercialization and navigating regulatory frameworks will be essential to bring these tools to market and make them widely available to healthcare providers.

comfort-logo-final.jpg