Periodic Reporting for period 2 - CANCER-RADIOMICS (Deep Learning for Automated Quantification of Radiographic Tumor Phenotypes)
Periodo di rendicontazione: 2022-02-01 al 2023-07-31
What is the problem being addressed?
One important reason for the slow progress in the fight against cancer, is the fact that cancer is a “moving target”. It is constantly evolving and diversifying, changing its phenotype, its genomic composition, and through metastatic spread, even its location. This is particularly important for cancer patients treated with immunotherapy. Despite the remarkable success of these novel therapies, the clinical benefit remains limited to a subset. As immunotherapy is expensive and could bring unnecessary toxicity there is a direct need to identify beneficial patients, but this remains difficult in the clinic today. Artificial Intelligence-based biomarkers could provide this information by analyzing standard-of-care medical imaging data as it can depict a full picture of the entire tumour burden, providing information of each cancer lesion within a single non-invasive examination.
Why is it important for society?
Immunotherapy is regarded as one of the major breakthroughs in cancer treatment and despite its remarkable success, the clinical benefit remains limited to only a subset of patients. As immunotherapy treatment is expensive and could bring unnecessary toxicity, there is a strong need to identify patients that will likely benefit prior to therapy, thereby enhancing cancer care and reducing treatment cost. Different biomarkers have been investigated with variable success, such as levels of PD-L1, infiltrating lymphocytes, and frequency of genetic mutations. As these biomarkers by definition only represent a small sample, imaging can depict a full picture of the entire tumour burden, providing information of each single cancer lesion within a single non-invasive examination.
What are the overall objectives?
This ERC proposal is investigating the potential of AI-imaging biomarkers to help physicians select patients who will likely benefit from immunotherapy. This research proposal consists of three complementary objectives that will be investigated parallel: I: development of baseline AI-imaging biomarkers for cancer response to test the hypothesis that deep-learning imaging biomarkers can improve immunotherapy response prediction before the start of treatment, II) Serial Immunotherapy Response Prediction to test the hypothesis that deep-learning imaging biomarkers integrating serial pre- and post-treatment imaging can improve tumour response predictions, and III) test the hypothesis that imaging biomarkers may characterize underlying biological factors, including somatic mutations, and investigate if these can improve response predictions.
One important reason for the slow progress in the fight against cancer, is the fact that cancer is a “moving target”. It is constantly evolving and diversifying, changing its phenotype, its genomic composition, and through metastatic spread, even its location. This is particularly important for cancer patients treated with immunotherapy. Despite the remarkable success of these novel therapies, the clinical benefit remains limited to a subset. As immunotherapy is expensive and could bring unnecessary toxicity there is a direct need to identify beneficial patients, but this remains difficult in the clinic today. Artificial Intelligence-based biomarkers could provide this information by analyzing standard-of-care medical imaging data as it can depict a full picture of the entire tumour burden, providing information of each cancer lesion within a single non-invasive examination.
Why is it important for society?
Immunotherapy is regarded as one of the major breakthroughs in cancer treatment and despite its remarkable success, the clinical benefit remains limited to only a subset of patients. As immunotherapy treatment is expensive and could bring unnecessary toxicity, there is a strong need to identify patients that will likely benefit prior to therapy, thereby enhancing cancer care and reducing treatment cost. Different biomarkers have been investigated with variable success, such as levels of PD-L1, infiltrating lymphocytes, and frequency of genetic mutations. As these biomarkers by definition only represent a small sample, imaging can depict a full picture of the entire tumour burden, providing information of each single cancer lesion within a single non-invasive examination.
What are the overall objectives?
This ERC proposal is investigating the potential of AI-imaging biomarkers to help physicians select patients who will likely benefit from immunotherapy. This research proposal consists of three complementary objectives that will be investigated parallel: I: development of baseline AI-imaging biomarkers for cancer response to test the hypothesis that deep-learning imaging biomarkers can improve immunotherapy response prediction before the start of treatment, II) Serial Immunotherapy Response Prediction to test the hypothesis that deep-learning imaging biomarkers integrating serial pre- and post-treatment imaging can improve tumour response predictions, and III) test the hypothesis that imaging biomarkers may characterize underlying biological factors, including somatic mutations, and investigate if these can improve response predictions.
In the first 2.5 years, our work has been divided along the three main work packages. In specific, we are actively building baseline imaging biomarkers for immunotherapy response (WP1), developing serial immunotherapy response biomarkers using pre- and post-treatment imaging (WP2), and testing if imaging biomarkers may characterise underlying biological factors, and investigate if these can improve response predictions (WP3). This research is conducted at two institutions: Maastricht University (host) in the Netherlands and Harvard University in Boston. Personnel was successfully onboarded in Maastricht University and integrated within both institutions and all team members actively participate in bi-weekly Zoom meetings to ensure direct and frequent communication.
Data collection, curation, and annotation of large datasets of cancer patients treated with immunotherapies, in combination with surgery, radiation, and/or targeted therapies, in standard-of-care and clinical trials, is currently ongoing. Clinical experts are identifying the right datasets, curating the right clinical data using 3D Slicer, localising the right imaging scans, as well as curating the imaging data to assure sufficient imaging quality. After these steps, clinical experts perform the annotation of the imaging data by identifying and annotating all visible cancer lesions. Target lesions are defined as any visible tumour lesions (primary or metastases) on baseline and serial imaging. In a second step, we define the response kinetics of these lesions (Task 3 of WP1 and 2). For example, lesions that disappear on the follow-up scan are flagged as complete response. Furthermore, sub-cohorts will have detailed mutational profiling of primaries and/or metastases have been also identified, and a large subset have been approved and annotated already. Lastly, we also identified and got access to clinical trial data that are collected in a prospective and standardised manor.
For work package 1 and 2, we need to convert CT images into suitable data structures for the development of AI applications. For this we are developing standardised preprocessing pipelines, able to load CT images, identify lesions, and normalise the input CT cube around the tumour lesion. We are utilising novel self-supervised learning (SSL) approaches that became the state of the art in deep learning recently, to address these issues. SSL learns from unlabeled data and is based on an artificial neural network. We are adopting a very similar strategy to develop a network able to quantify the spectrum of tumour lesions on a CT. For this we are using a dataset of 32,735 tumour lesions in the bone, abdomen, mediastinum, liver, lung, kidney, and pelvis of 4,427 unique patients. Using this dataset, we also use data augmentation to yield a less sensitive network to interpolation, resizing, and imaging artifacts. Initial experiments demonstrated strong potential with high performance in predicting treatment response of lung cancer lesions, and detailed experiments are ongoing to draw firm conclusions. In work package 3, we are investigating the potential of using imaging biomarkers as a proxy for measuring biological characteristics, as defined through the genomic analysis, among others.
Data collection, curation, and annotation of large datasets of cancer patients treated with immunotherapies, in combination with surgery, radiation, and/or targeted therapies, in standard-of-care and clinical trials, is currently ongoing. Clinical experts are identifying the right datasets, curating the right clinical data using 3D Slicer, localising the right imaging scans, as well as curating the imaging data to assure sufficient imaging quality. After these steps, clinical experts perform the annotation of the imaging data by identifying and annotating all visible cancer lesions. Target lesions are defined as any visible tumour lesions (primary or metastases) on baseline and serial imaging. In a second step, we define the response kinetics of these lesions (Task 3 of WP1 and 2). For example, lesions that disappear on the follow-up scan are flagged as complete response. Furthermore, sub-cohorts will have detailed mutational profiling of primaries and/or metastases have been also identified, and a large subset have been approved and annotated already. Lastly, we also identified and got access to clinical trial data that are collected in a prospective and standardised manor.
For work package 1 and 2, we need to convert CT images into suitable data structures for the development of AI applications. For this we are developing standardised preprocessing pipelines, able to load CT images, identify lesions, and normalise the input CT cube around the tumour lesion. We are utilising novel self-supervised learning (SSL) approaches that became the state of the art in deep learning recently, to address these issues. SSL learns from unlabeled data and is based on an artificial neural network. We are adopting a very similar strategy to develop a network able to quantify the spectrum of tumour lesions on a CT. For this we are using a dataset of 32,735 tumour lesions in the bone, abdomen, mediastinum, liver, lung, kidney, and pelvis of 4,427 unique patients. Using this dataset, we also use data augmentation to yield a less sensitive network to interpolation, resizing, and imaging artifacts. Initial experiments demonstrated strong potential with high performance in predicting treatment response of lung cancer lesions, and detailed experiments are ongoing to draw firm conclusions. In work package 3, we are investigating the potential of using imaging biomarkers as a proxy for measuring biological characteristics, as defined through the genomic analysis, among others.
Deep learning innovations: As these methods were invested very recently and after submission of this proposal, we didn’t include it in the approach. However, as one of the main difficulties is generating enough training data for development of these imaging-AI biomarkers, novel methods such as self supervised learning (SSL) addresses this can be crucially important as it allows us to develop models that are of higher performance. Indeed, our first results show strong performance. Also, other deep learning methods such as contractive learning will be evaluated.
Novel patient level biomarkers: Our team has also developed novel imaging biomarkers quantifying the general health of patients. For example, we developed a novel deep learning algorithm to quantify body composition measures on the CT images. These have been demonstrated to significantly predict outcomes of cancer patients treated with immunotherapy. Especially the change of body composition during treatment is very predictive of outcomes. For this ERC project, we will combine these novel measures together with the tumour biomarkers to further improve long term outcome predictions including overall survival (OS) and progression free survival (PFS).
Novel patient level biomarkers: Our team has also developed novel imaging biomarkers quantifying the general health of patients. For example, we developed a novel deep learning algorithm to quantify body composition measures on the CT images. These have been demonstrated to significantly predict outcomes of cancer patients treated with immunotherapy. Especially the change of body composition during treatment is very predictive of outcomes. For this ERC project, we will combine these novel measures together with the tumour biomarkers to further improve long term outcome predictions including overall survival (OS) and progression free survival (PFS).