Periodic Reporting for period 3 - REVEAL (Neuronal microscopy for cell behavioural examination and manipulation)
Berichtszeitraum: 2024-01-01 bis 2025-06-30
Hepatocellular carcinoma (HCC) is the most common liver cancer, accounting for 90% of all liver cancers, and the second most frequent cause of cancer-related deaths worldwide. One of the main hallmarks of HCC is biological heterogeneity at the level of single cell morphologies and genetic fingerprints. These variations are often too subtle to detect or are extremely rare events, highlighting the need for new biophotonics tools capable to decipher the biomechanism at the origin of these modifications. In the REVEAL project, we coupled live cell microscopy and omics analysis to obtain evidence of the dynamic processes at the origin of HCC. By introducing the concept of neuronal microscopy, we built a novel technique where hardware and software analysis are seamlessly integrated, and uses computational neural networks to determine cell trajectories, i.e. the evolution of cell phenotypes. The software relies heavily on the use of neural networks for image formation, cell analysis, cell fate prediction and decision making to: i) image and analyse in quasi real-time, cell behaviours of thousands of cells in parallel, ii) identify variations in cell-state indicative of disease origin, iii) pinpoint the location of the cell of interest, and iv) pick-up cells for omics analysis as a final step. The Consortium was a strong partnership of leading European research centers as well as companies, universities and medical centres. The overall is made of 7 partners coming from 4 European countries, CEA-Leti (project coordinator), ENS-Lyon, Iprasense, Policlinico Milano, University of Munich, Sartorius, and Politechnika Warszawska.
Our work was structured on four important objectives: 1) develop a neuronal microscope, (2) develop a neuronal cell picking microscope, (3) discover biological signatures at the origin of liver cancer disease, and (4) develop a 3D neuronal microscope.
(1) In the REVEAL project we have contributed to three important evolutionary leaps:
a- computational microscopy, i.e. optics is replaced by algorithms
b- neuronal microscopy, i.e. algorithms are replaced by neural networks
c- an all-neural framework, i.e. physical and biological models can be built upon neural networks, from sample reconstruction to the of study of cell-cell interactions within a large population.
One important achievement of the project is the development of the ‘silico visual cortex’, the interconnection of several neural networks used for image formation and analysis. This software was coupled with a dedicated imaging platform, allowing for time-lapse imaging of cell cultures with four different imaging modalities. Overall, we have demonstrated that several functionalities are now at reach with convolutional neural networks, i.e. image formation, cell analysis, tracking, classification, and prediction. Importantly the framework of deep learning allows fast computation and continuous performance improvements. The ability to conduct live imaging and analysis in quasi real-time (>10.000 cells in few tens of seconds), let us envision the microscope able of decision that will come next.
(2) A concept of a new instrument, called a neuronal cell picking microscope, has been designed and validated with the first proofs-of-concept. It combines a bi-modal microscopy setup with an automated cell picking system, and has been integrated with the silico visual cortex for detection of phenotypic heterogeneity. The system is still on validation to assess its picking performances.
(3) Our stated objective in REVEAL is to validate the capacity of neuronal microscopy to detect and identify the biological heterogeneity that take root in patients at early stages of dysmetabolism in the liver. Under the aegis of this project, we characterized many different types of patient derived cells that grow on a 2D matrix in laboratory culture as well as patient derived normal, or tumoral 3D organoids (mini livers) that reflect either the health or the cancerous states. Moreover, we have developed state-of-the-art murine 2D and 3D models that reflect nearly every stage of liver disease- from early stage metabolic disorders to late stage cancer. These laboratory models have been used to develop both the 2D and the 3D neuronal microscope. We are actively interrogating the molecular signatures of these samples with the aim to provide a state-of-the-art omics signature to the images acquired by the neuronal microscope.
(4) We successfully develop and metrologically validate a 'smart' 3D intensity-only diffraction microscope, specifically designed for large-scale bio-studies of multi-scattering samples. The imaging platform features two imaging arms: a guiding arm for pre-selection of 3D objects and a high-resolution imaging for real time 3D imaging. The microscope has been integrated with a reconstruction software consisting of a multi-slice light scattering model for 3D reconstruction of living multicellular objects. The microscope has been tested and validated on objects of increasing complexity, meaning phase-calibrated phantoms and liver organoids. The platform has been integrated with neural network-based segmentation algorithms for automatic analysis of healthy and tumoral liver organoids.
(1) In the REVEAL project we have contributed to three important evolutionary leaps:
a- computational microscopy, i.e. optics is replaced by algorithms
b- neuronal microscopy, i.e. algorithms are replaced by neural networks
c- an all-neural framework, i.e. physical and biological models can be built upon neural networks, from sample reconstruction to the of study of cell-cell interactions within a large population.
One important achievement of the project is the development of the ‘silico visual cortex’, the interconnection of several neural networks used for image formation and analysis. This software was coupled with a dedicated imaging platform, allowing for time-lapse imaging of cell cultures with four different imaging modalities. Overall, we have demonstrated that several functionalities are now at reach with convolutional neural networks, i.e. image formation, cell analysis, tracking, classification, and prediction. Importantly the framework of deep learning allows fast computation and continuous performance improvements. The ability to conduct live imaging and analysis in quasi real-time (>10.000 cells in few tens of seconds), let us envision the microscope able of decision that will come next.
(2) A concept of a new instrument, called a neuronal cell picking microscope, has been designed and validated with the first proofs-of-concept. It combines a bi-modal microscopy setup with an automated cell picking system, and has been integrated with the silico visual cortex for detection of phenotypic heterogeneity. The system is still on validation to assess its picking performances.
(3) Our stated objective in REVEAL is to validate the capacity of neuronal microscopy to detect and identify the biological heterogeneity that take root in patients at early stages of dysmetabolism in the liver. Under the aegis of this project, we characterized many different types of patient derived cells that grow on a 2D matrix in laboratory culture as well as patient derived normal, or tumoral 3D organoids (mini livers) that reflect either the health or the cancerous states. Moreover, we have developed state-of-the-art murine 2D and 3D models that reflect nearly every stage of liver disease- from early stage metabolic disorders to late stage cancer. These laboratory models have been used to develop both the 2D and the 3D neuronal microscope. We are actively interrogating the molecular signatures of these samples with the aim to provide a state-of-the-art omics signature to the images acquired by the neuronal microscope.
(4) We successfully develop and metrologically validate a 'smart' 3D intensity-only diffraction microscope, specifically designed for large-scale bio-studies of multi-scattering samples. The imaging platform features two imaging arms: a guiding arm for pre-selection of 3D objects and a high-resolution imaging for real time 3D imaging. The microscope has been integrated with a reconstruction software consisting of a multi-slice light scattering model for 3D reconstruction of living multicellular objects. The microscope has been tested and validated on objects of increasing complexity, meaning phase-calibrated phantoms and liver organoids. The platform has been integrated with neural network-based segmentation algorithms for automatic analysis of healthy and tumoral liver organoids.
One important achievement in REVEAL is the development of the ‘silico visual cortex’, the interconnection of several neural networks used for image formation, cell analysis and cell classification. We developed He2Cl, an innovative approach for characterizing cellular heterogeneity by analyzing single-cell morpho-dynamic behaviours captured through lens-free microscopy. This method encompasses a comprehensive computational pipeline - from image processing to the identification of cellular sub-phenotypes—centered around a novel unsupervised clustering algorithm specifically designed to classify heterogeneous cell populations.
This software has been successfully integrated in a bi-modal imaging platform allowing for time-lapse imaging of cell cultures and automated cell picking. Following the identification of cells of interest by lensfree imaging, the picking module receives their coordinates and collect these cells automatically. The picking module consists of small glass capillaries (20-220 µm in diameter) coupled to a fluidic system enabling a gentle cell transfer for downstream omics analysis.
Politechnika Warszawska has pioneered the methodology to design and print 3D microstructures as novel targets for phase imaging. Importantly, the team conceived a new pipeline to obtain phase-calibrated objects that perfectly mimic the real structure of biological 2D cells and 3D organoids. In REVEAL, we actively used these phantoms for characterization of the phase systems and algorithms developed in the context of 2D and 3D microscopy, and the samples are now under validation in many different microscopy laboratories. The approach is the state-of-the-art for metrological analysis in the field of quantitative phase imaging for biomedical applications.
The REVEAL consortium created the first 3D neuronal microscope, by coupling a large surface scanning 3D microscope with neuronal algorithms for 3D reconstruction and analysis. The system has been tested in operative environment with different liver organoid models and primary tissue samples of patients undergoing cholecystectomy or hepatic resection for hepatocellular carcinoma both in intra- and extra-tumoral tissues.
This software has been successfully integrated in a bi-modal imaging platform allowing for time-lapse imaging of cell cultures and automated cell picking. Following the identification of cells of interest by lensfree imaging, the picking module receives their coordinates and collect these cells automatically. The picking module consists of small glass capillaries (20-220 µm in diameter) coupled to a fluidic system enabling a gentle cell transfer for downstream omics analysis.
Politechnika Warszawska has pioneered the methodology to design and print 3D microstructures as novel targets for phase imaging. Importantly, the team conceived a new pipeline to obtain phase-calibrated objects that perfectly mimic the real structure of biological 2D cells and 3D organoids. In REVEAL, we actively used these phantoms for characterization of the phase systems and algorithms developed in the context of 2D and 3D microscopy, and the samples are now under validation in many different microscopy laboratories. The approach is the state-of-the-art for metrological analysis in the field of quantitative phase imaging for biomedical applications.
The REVEAL consortium created the first 3D neuronal microscope, by coupling a large surface scanning 3D microscope with neuronal algorithms for 3D reconstruction and analysis. The system has been tested in operative environment with different liver organoid models and primary tissue samples of patients undergoing cholecystectomy or hepatic resection for hepatocellular carcinoma both in intra- and extra-tumoral tissues.