Periodic Reporting for period 1 - AIS-CaP (Interpretable Artificial Intelligence across Scales for Next-Generation Cancer Prognostics)
Berichtszeitraum: 2022-04-01 bis 2024-09-30
Imagine a world where computers help doctors diagnose and treat cancer more accurately than ever before. This is the promise of computational pathology, a field that uses powerful computers to analyze images of tissue samples. These images, called whole-slide images (WSIs), are like Google Maps for cells, showing every detail of the tissue.
In recent years, artificial intelligence (AI) and machine learning (ML) have supercharged this field. Computers can now do things like spotting tumors and measuring their size, often better than human experts. But the real game-changer is on the horizon: using AI not just to mimic what doctors do, but to uncover new insights that could transform cancer treatment.
Here’s the catch: current methods are like teaching a computer to recognize cats in photos by showing it lots of cat pictures. It works well for simple tasks, but cancer is complex. For example, determining how aggressive a cancer is (cancer grading) involves looking at the shape and structure of cells, and right now, computers can’t do this any better than doctors.
Why? Because humans design the rules for grading cancer, and computers are just following those rules.
The next big step is to teach computers to see the bigger picture, literally. Instead of looking at small parts of a tissue sample, they could analyze the entire image, picking up patterns and details that humans might miss. This could lead to discovering new signs of cancer that could predict a patient’s outcome or response to treatment, which is crucial in the era of personalized medicine.
To make this leap, we need better algorithms that can handle the massive, high-resolution images produced by modern scanners. One promising approach is called weakly-supervised learning, where the computer learns from the overall pattern in the tissue, not just the labeled parts.
But there’s a problem: current methods focus on small areas and miss the forest for the trees. They also struggle with explaining their decisions, which is a big deal when those decisions affect patient care.
The solution? A new algorithm that looks at the whole forest, not just the trees. This approach could help us find common threads across different types of cancer and make sense of the complex information in tissue samples. If successful, it could revolutionize how we understand and treat cancer, making care more personalized and effective.
In recent years, artificial intelligence (AI) and machine learning (ML) have supercharged this field. Computers can now do things like spotting tumors and measuring their size, often better than human experts. But the real game-changer is on the horizon: using AI not just to mimic what doctors do, but to uncover new insights that could transform cancer treatment.
Here’s the catch: current methods are like teaching a computer to recognize cats in photos by showing it lots of cat pictures. It works well for simple tasks, but cancer is complex. For example, determining how aggressive a cancer is (cancer grading) involves looking at the shape and structure of cells, and right now, computers can’t do this any better than doctors.
Why? Because humans design the rules for grading cancer, and computers are just following those rules.
The next big step is to teach computers to see the bigger picture, literally. Instead of looking at small parts of a tissue sample, they could analyze the entire image, picking up patterns and details that humans might miss. This could lead to discovering new signs of cancer that could predict a patient’s outcome or response to treatment, which is crucial in the era of personalized medicine.
To make this leap, we need better algorithms that can handle the massive, high-resolution images produced by modern scanners. One promising approach is called weakly-supervised learning, where the computer learns from the overall pattern in the tissue, not just the labeled parts.
But there’s a problem: current methods focus on small areas and miss the forest for the trees. They also struggle with explaining their decisions, which is a big deal when those decisions affect patient care.
The solution? A new algorithm that looks at the whole forest, not just the trees. This approach could help us find common threads across different types of cancer and make sense of the complex information in tissue samples. If successful, it could revolutionize how we understand and treat cancer, making care more personalized and effective.
In the project, we focus on several objectives, which are:
1. Data collection, unification and management
2. Prognostic models incorporating local detail and global context
3. Development of multi-task models for pan-cancer feature discovery
4. Explainability for prognostic models
5. Validation of outcome prediction
The first objective focuses on collection of vast, pan-cancer datasets of 1000s of patients with both digizited histopathology data and outcome data. The steps to collect these data involve developing automated data preprocesisgn tools, such as for quality control, tissue identification and anonymization. Furthermore, after completion of the project all data will be made publicly available.
In objective 2, we will further develop and refine our weakly supervised learning method, Streaming Stochastic Gradient Descent, to be easier to use and apply and more computationally efficient and flexible. This will allow other researchers in the oncology field to use it. The software will also be made open-source. Last, we will provide initial proof that this method is capable of predicting patient outcomes for individual cancer types, such as prostate cancer.
Objective 3 extends this approach to pan-cancer models, AI systems that can incorporate information across different diseases and identify common and distinct prognostic features. By enforcing the models to learn specific, human understandable concepts, these features can later be used to drive, for example, research into drugs targeting specific disease properties. Last, these pan-cancer features could be leveraged as prognostic biomarkers in rare cancer, where not enough data is available to train complex AI systems.
Objective 4 aims to leverage recent developments in large language models (LLMs) to push the explainability of AI systems to the next level, and make it more similar to human communication. Specifically, by combining vision and language, we aim to have AI systems be able to articulate, in writing, their decisions. In this objective we will aim to assess the quality of these 'captions' and determine the risk of hallucinations, thus enabling us to prove the usefulness of these approaches.
Last, Objective 5 validates the full pipeline across three different cancerous entities, specifically prostate, breast, and pancreatic cancer. By prospectively collection new data from existing European initiatives, such as BigPicture and PANCAIM, we will be able to perform this validation at an unprecedented scale, with thousands of patients across many different European countries.
1. Data collection, unification and management
2. Prognostic models incorporating local detail and global context
3. Development of multi-task models for pan-cancer feature discovery
4. Explainability for prognostic models
5. Validation of outcome prediction
The first objective focuses on collection of vast, pan-cancer datasets of 1000s of patients with both digizited histopathology data and outcome data. The steps to collect these data involve developing automated data preprocesisgn tools, such as for quality control, tissue identification and anonymization. Furthermore, after completion of the project all data will be made publicly available.
In objective 2, we will further develop and refine our weakly supervised learning method, Streaming Stochastic Gradient Descent, to be easier to use and apply and more computationally efficient and flexible. This will allow other researchers in the oncology field to use it. The software will also be made open-source. Last, we will provide initial proof that this method is capable of predicting patient outcomes for individual cancer types, such as prostate cancer.
Objective 3 extends this approach to pan-cancer models, AI systems that can incorporate information across different diseases and identify common and distinct prognostic features. By enforcing the models to learn specific, human understandable concepts, these features can later be used to drive, for example, research into drugs targeting specific disease properties. Last, these pan-cancer features could be leveraged as prognostic biomarkers in rare cancer, where not enough data is available to train complex AI systems.
Objective 4 aims to leverage recent developments in large language models (LLMs) to push the explainability of AI systems to the next level, and make it more similar to human communication. Specifically, by combining vision and language, we aim to have AI systems be able to articulate, in writing, their decisions. In this objective we will aim to assess the quality of these 'captions' and determine the risk of hallucinations, thus enabling us to prove the usefulness of these approaches.
Last, Objective 5 validates the full pipeline across three different cancerous entities, specifically prostate, breast, and pancreatic cancer. By prospectively collection new data from existing European initiatives, such as BigPicture and PANCAIM, we will be able to perform this validation at an unprecedented scale, with thousands of patients across many different European countries.
The project aims to have a significant amount of results that reach beyond state-of-the-art in terms of AI methodology, cancer biomarker discovery, and clinical application. First, in terms of AI methodology, most existing method that provide weakly-supervised capabilities in computational pathology cannot be optimized end-to-end for a specific tasks. Our unique streaming approach mitigates this issue and allows for end-to-end learning for any tasks in computational pathology, whether tumor detection, mutation prediction, or assessment of progression-free survival. By improving the existing method and making it publicly available, we provide these possibilities not just to computational pathology researchers but to any discipline that works with large data samples.
Second, we aim to discover new morphological biomarkers that related to patient outcome. These can be architectural features of cancer growth at the cellular level but might also encompass the role of the cancer-micro-environment, for example, the interaction with the immune system or the connective tissue. These discoveries might impact other fields depending on the type of identified featureser fields. For example, if immune system involvement seems to be an important biomarker for patient outcome, this might also be a relevant feature to investigate in the context of immunotherapy. Additionally, we might be able to discover features associated with specific genetic mutations, such as BRCA1 or BRCA2, which might allow earlier and better screening of breast tissue of patients with these genetic predispositions.
Last, current clinical applications of AI aim to repeat the same task that a doctor, be it a pathologist, radiologist, or oncologist. Due to the pressure on our healthcare systems and the expected shortage of doctors, this is a research direction of research. However, this will never allow AI algorithms to get better than doctors, as they are presented as the gold standard. In this project, we specifically aim to uncover the additional prognostic information in digital pathology data that is inaccessible to human experts, either because it is too time-consuming to extract manually or because our visual system can simply not assess that information accurately. As such, we aim to deliver the next generation of AI systems that go beyond human capabilities for a variety of different cancer types.
Second, we aim to discover new morphological biomarkers that related to patient outcome. These can be architectural features of cancer growth at the cellular level but might also encompass the role of the cancer-micro-environment, for example, the interaction with the immune system or the connective tissue. These discoveries might impact other fields depending on the type of identified featureser fields. For example, if immune system involvement seems to be an important biomarker for patient outcome, this might also be a relevant feature to investigate in the context of immunotherapy. Additionally, we might be able to discover features associated with specific genetic mutations, such as BRCA1 or BRCA2, which might allow earlier and better screening of breast tissue of patients with these genetic predispositions.
Last, current clinical applications of AI aim to repeat the same task that a doctor, be it a pathologist, radiologist, or oncologist. Due to the pressure on our healthcare systems and the expected shortage of doctors, this is a research direction of research. However, this will never allow AI algorithms to get better than doctors, as they are presented as the gold standard. In this project, we specifically aim to uncover the additional prognostic information in digital pathology data that is inaccessible to human experts, either because it is too time-consuming to extract manually or because our visual system can simply not assess that information accurately. As such, we aim to deliver the next generation of AI systems that go beyond human capabilities for a variety of different cancer types.