Periodic Reporting for period 1 - MICS (Multiphoton imaging with computational specificity)
Reporting period: 2023-06-01 to 2024-06-30
Autoimmune diseases, particularly inflammatory bowel diseases (IBD), are a growing global health challenge. These conditions involve complex immune system dysfunctions that damage healthy tissues, leading to chronic inflammation. Diagnosing and understanding these diseases often requires invasive biopsies and labor-intensive processes, which delay timely diagnosis and research. Traditional methods use chemical stains to highlight specific tissue structures, but these processes are slow, costly, and unsuitable for real-time, in vivo analysis.
The MICS project – Multiphoton Imaging with Computational Specificity is designed to overcome these limitations by combining high-resolution label-free multiphoton microscopy (MPM) with the power of artificial intelligence (AI). Specifically, the project uses deep neural networks to enhance the specificity of MPM images, enabling them to function like traditional stained images while retaining the advantages of being label-free and non-invasive.
Deep neural networks are advanced computational models inspired by how the human brain processes information. These networks consist of multiple layers of interconnected "neurons" that transform input data, such as images, into meaningful outputs, like classifications or predictions. To train a neural network on image data, it is shown thousands of labeled examples, such as images of healthy tissue and inflamed tissue. The network learns patterns by adjusting its internal connections through a process called training, which uses algorithms to minimize errors between its predictions and the known annotations. Over time, the network becomes highly skilled at recognizing features in unseen images. In many AI applications, the limiting factor is to actually obtain these large data sets of high-quality and well-trusted annotations.
By combining previously trained deep neural networks with label-free multiphoton imaging, MICS will be able to directly image samples from many autoimmune diseases without any labor-intensive processing of biopsies and then to digitally augment the readout. This approach offers a faster, non-invasive alternative to conventional histological analysis and has a great potential to advance research into autoimmune diseases and pave the way for new diagnostic tools.
The MICS project – Multiphoton Imaging with Computational Specificity is designed to overcome these limitations by combining high-resolution label-free multiphoton microscopy (MPM) with the power of artificial intelligence (AI). Specifically, the project uses deep neural networks to enhance the specificity of MPM images, enabling them to function like traditional stained images while retaining the advantages of being label-free and non-invasive.
Deep neural networks are advanced computational models inspired by how the human brain processes information. These networks consist of multiple layers of interconnected "neurons" that transform input data, such as images, into meaningful outputs, like classifications or predictions. To train a neural network on image data, it is shown thousands of labeled examples, such as images of healthy tissue and inflamed tissue. The network learns patterns by adjusting its internal connections through a process called training, which uses algorithms to minimize errors between its predictions and the known annotations. Over time, the network becomes highly skilled at recognizing features in unseen images. In many AI applications, the limiting factor is to actually obtain these large data sets of high-quality and well-trusted annotations.
By combining previously trained deep neural networks with label-free multiphoton imaging, MICS will be able to directly image samples from many autoimmune diseases without any labor-intensive processing of biopsies and then to digitally augment the readout. This approach offers a faster, non-invasive alternative to conventional histological analysis and has a great potential to advance research into autoimmune diseases and pave the way for new diagnostic tools.
The MICS project focuses on advancing a cutting-edge AI technique called Digital Staining (DS) to enhance the capabilities of label-free multiphoton microscopy (MPM). Digital Staining is an emerging concept in computational microscopy where images from one domain, such as label-free images, are transformed into another domain, like those of specific biomolecular stains. This approach allows researchers to extract the specificity of traditional staining without physically staining the samples.
One of the first major achievements of MICS was the publication of a comprehensive review article on Digital Staining in the high-impact journal PhotoniX (Kreiss, Lucas, et al., 2023). This review highlighted the transformative potential of DS and discussed its advantages in addressing key limitations of traditional and label-free imaging methods.
Digital Staining pairs images from label-free modalities, such as MPM autofluorescence, with chemically stained reference images from the same sample. This bypasses two major obstacles:
1. For training, DS eliminates the need for labor-intensive manual annotations because the ground truth is directly derived from the paired imaging procedure.
2. After deployment, trained DS models can digitally infer specific staining without requiring the tedious and time-consuming processes of sample preparation, sectioning, and staining.
The PhotoniX review also underscored a critical gap: while computational specificity for common optical modalities like phase contrast or single-photon autofluorescence is well-developed, applications for advanced nonlinear optical imaging techniques like MPM remain underexplored.
Building on these findings, MICS is now developing Digital Staining methods specifically tailored to MPM autofluorescence for immune cells. Preliminary results have been presented at the SPIE Photonics West Conference and Duke AI Day 2024, where they garnered positive feedback from the research community.
In parallel, MICS is utilizing an annotated database of 3D MPM images from inflammatory bowel disease (IBD) tissues to train a deep neural network for automated disease analysis. This network aims to predict the presence of IBD, identify the type of inflammation, and assess disease severity.
Through these achievements, MICS is pioneering AI-driven solutions to overcome key limitations in label-free imaging, with the potential to revolutionize how we study and diagnose autoimmune diseases.
One of the first major achievements of MICS was the publication of a comprehensive review article on Digital Staining in the high-impact journal PhotoniX (Kreiss, Lucas, et al., 2023). This review highlighted the transformative potential of DS and discussed its advantages in addressing key limitations of traditional and label-free imaging methods.
Digital Staining pairs images from label-free modalities, such as MPM autofluorescence, with chemically stained reference images from the same sample. This bypasses two major obstacles:
1. For training, DS eliminates the need for labor-intensive manual annotations because the ground truth is directly derived from the paired imaging procedure.
2. After deployment, trained DS models can digitally infer specific staining without requiring the tedious and time-consuming processes of sample preparation, sectioning, and staining.
The PhotoniX review also underscored a critical gap: while computational specificity for common optical modalities like phase contrast or single-photon autofluorescence is well-developed, applications for advanced nonlinear optical imaging techniques like MPM remain underexplored.
Building on these findings, MICS is now developing Digital Staining methods specifically tailored to MPM autofluorescence for immune cells. Preliminary results have been presented at the SPIE Photonics West Conference and Duke AI Day 2024, where they garnered positive feedback from the research community.
In parallel, MICS is utilizing an annotated database of 3D MPM images from inflammatory bowel disease (IBD) tissues to train a deep neural network for automated disease analysis. This network aims to predict the presence of IBD, identify the type of inflammation, and assess disease severity.
Through these achievements, MICS is pioneering AI-driven solutions to overcome key limitations in label-free imaging, with the potential to revolutionize how we study and diagnose autoimmune diseases.
Our initial experiments used a fluorescently-labeled antibody staining to derive high-quality annotations for two different types of immune cells. A deep neural network was then able to classify these cells with around 90% accuracy, based solely on their label-free, two-photon autofluorescence. This elegant use of antibody staining to obtain ground truth annotations for Digital Staining (DS) follows best scientific practices. Moreover, its application to label-free multiphoton imaging of immune cells represents a technical innovation that goes beyond the state of the art. Based on the successfully implementation of this robust AI framework for Digital Staining, MICS is now ready for the next stage – to generate more data from various different immune cells and to train a much larger and more powerful digital staining model. This framework incorporates advanced neural network architectures, perturbation experiments to confirm specificity, and validation against independent datasets. Similarly, a parallel AI framework was developed for tissue classification, utilizing an annotated database of 3D MPM images to predict inflammatory bowel disease (IBD) presence, inflammatory pathways, and disease severity. Both frameworks are poised to advance precision diagnostics and provide new insights into immune-mediated tissue remodeling.
To maximize its future impact, the MICS project aims to extend its applications to human IBD biopsies, leveraging the capabilities of Digital Staining and tissue classification for clinical diagnostics. Expanding the approach to in vivo endomicroscopy would enable real-time disease monitoring, offering a powerful tool for precision medicine. Fully clinical applications will require further validation studies, regulatory alignment, and partnerships with healthcare providers.
Future grant proposals will explore these directions, aiming to bring MICS innovations closer to clinical translation. The project's concepts and early successes have already generated significant interest from collaborators in vascular biology, pathology, and other related fields. This enthusiasm underscores the broad potential of MICS to revolutionize label-free imaging across diverse biomedical disciplines.
To maximize its future impact, the MICS project aims to extend its applications to human IBD biopsies, leveraging the capabilities of Digital Staining and tissue classification for clinical diagnostics. Expanding the approach to in vivo endomicroscopy would enable real-time disease monitoring, offering a powerful tool for precision medicine. Fully clinical applications will require further validation studies, regulatory alignment, and partnerships with healthcare providers.
Future grant proposals will explore these directions, aiming to bring MICS innovations closer to clinical translation. The project's concepts and early successes have already generated significant interest from collaborators in vascular biology, pathology, and other related fields. This enthusiasm underscores the broad potential of MICS to revolutionize label-free imaging across diverse biomedical disciplines.