Periodic Reporting for period 4 - 5D-NanoTrack (Five-Dimensional Localization Microscopy for Sub-Cellular Dynamics)
Periodo di rendicontazione: 2023-05-01 al 2024-04-30
Observing processes inside live cells is important for understanding a of basic biology, as well as diseases, and subsequently for the design of drugs and treatments. Fluorescence microscopy is a useful and widespread technique that enables the visualization of cellular structures by fluorescent labeling. Super-resolution techniques have revolutionized cellular imaging by microscope design combined with the development of corresponding image processing algorithms. Still, current capabilities to observe dynamic processes as well as nanoscale structures inside the cells are limited, due to inherent trade-offs between capturing information on the 5 fundamental dimensions of microscopic imaging – spatial (x,y,z), temporal (time) and spectral (color).
Point-Spread-Function (PSF) engineering is a computational-imaging approach used to enable 3D and/or multi-color imaging, by using specially designed optical elements, namely, phase masks, that add new degrees of freedom to microscopy and can improve spatial and temporal resolution, as well as enable volumetric imaging. The full potential of this method, however, has not yet been reached in live-cell imaging, due to significant bottlenecks in optical engineering and signal processing, which are the focus in this project.
Our first two objectives in this project were aimed at revolutionizing both the microscope hardware by designing novel, cost-effective optical setups, and image processing software by deep learning approaches. The third objective focused on optimizing live cell fluorescent labeling and demonstrating the efficacy of our method using challenging biological applications such as real-time visualization of the critical process of repairing breaks in DNA in live cells.
Point-Spread-Function (PSF) engineering is a computational-imaging approach used to enable 3D and/or multi-color imaging, by using specially designed optical elements, namely, phase masks, that add new degrees of freedom to microscopy and can improve spatial and temporal resolution, as well as enable volumetric imaging. The full potential of this method, however, has not yet been reached in live-cell imaging, due to significant bottlenecks in optical engineering and signal processing, which are the focus in this project.
Our first two objectives in this project were aimed at revolutionizing both the microscope hardware by designing novel, cost-effective optical setups, and image processing software by deep learning approaches. The third objective focused on optimizing live cell fluorescent labeling and demonstrating the efficacy of our method using challenging biological applications such as real-time visualization of the critical process of repairing breaks in DNA in live cells.
Optical element design for 3D multicolor microscopy
A phase mask is an optical element that provides either 3D information (x,y,z) or spectral information (color) of the sample in a 2D monochromatic (grayscale) image, enabling faster data acquisition than volumetric scanning or imaging one color at a time. We designed and implemented phase masks in four different optical setups. In the first setup we applied a lens in an existing flow-imaging microscope for 3D visualization of cellular processes in thousands of cells in just a few minutes (Figure 1; Nature Nanotechnology, 2020). The second setup included combination of spectral splitting and 3D phase masks for simultaneous visualization of three colors or more using a single, monochromatic camera (Nano Letters, 2021). For the third setup we designed and manufactured a novel cost-effective 3D multi-color phase mask by new fabrication procedures developed in our lab (Nature Communications, 2021; Light: Science & Applications, 2023). Finally, we modified a high throughput microscope with compact PSF engineering, which allowed us to enhance the imaging depth-of-field and, combined with deep learning, recover 3D information using single snapshots (Figure 2; Nature Communications, 2024).
Computational system design for 3D imaging of complex samples
Image post processing is a key component in improving imaging. Specifically, the full potential of deep learning to improve microscopy is starting to unravel in recent years. We used deep learning to process the complicated PSF-engineered images, to decode spectral information from a grayscale camera, and to capture cell dynamics with localization microscopy. Moreover, we demonstrated the usage of deep learning to design optimized optical components for different goals. Our main methods, including publicly available algorithmic implementations include:
1. DeepSTORM3D: an algorithm for fast image processing and optical element design, improving the accuracy of nanoscale point emitter localization (Figure 3; Nature Methods, 2020).
2. Multicolor Microscopy and PSF Engineering by Deep Learning: deep learning methods to differentiate colors in grayscale images and design optimized optical elements (Optics Express, 2019).
3. VIPR: An efficient algorithm to calibrate optical systems quickly, improving image accuracy (Optics Express, 2020).
4. Diffractive Optical System Design: A differentiable propagation model for optimizing diffractive optical systems (Optics Express, 2022).
5. Large-FOV 3D Localization Microscopy: A fast and accurate PSF generator and localizer for large fields of view (Science Advances, 2024; Optics Continuum, 2023).
6. DBlink: A deep-learning-based method for dynamic localization microscopy in super resolution, capturing long-term dependencies in SMLM data (Nature Methods, 2023).
Tracking 3D chromatin dynamics in live cells
1. Quantifying double strand break and repair dynamics in yeast cells: The structure and dynamics of DNA determine the destination of a cell and play a key role in various cell processes and functions. Observing DNA inside the nucleus is important for understanding diseases and natural processes, and subsequently for the design of drugs and treatments. However, such observation is extremely challenging, due to the compact sizes involved and dense Chromatin organization. We apply PSF engineering to observe DNA dynamics during double strand break and repair in the mating type switching process in yeast. First, we developed methods to follow and quantify mating type switching occurrence in real-time, at the single live cell level. We then followed DNA dynamics of two relevant loci, and observed two stages in DNA dynamics that lead to repair by gene conversion. This subtle dynamic is observable thanks to the PSF engineering methods we developed during this project (Figure 4).
2. Quantifying cell cycle dependent chromatin dynamics during interphase: The study of cell cycle progression and regulation is important to understanding of fundamental biophysics, aging, and disease mechanisms. Local chromatin movements are generally considered to be constrained and relatively consistent during all interphase stages. We challenge this claim and detected varying cell-cycle-dependent motion constraint levels imposed on the chromatin in healthy and cancerous cells using PSF engineering microscopy (Figure 5; iScience, 2022).
A phase mask is an optical element that provides either 3D information (x,y,z) or spectral information (color) of the sample in a 2D monochromatic (grayscale) image, enabling faster data acquisition than volumetric scanning or imaging one color at a time. We designed and implemented phase masks in four different optical setups. In the first setup we applied a lens in an existing flow-imaging microscope for 3D visualization of cellular processes in thousands of cells in just a few minutes (Figure 1; Nature Nanotechnology, 2020). The second setup included combination of spectral splitting and 3D phase masks for simultaneous visualization of three colors or more using a single, monochromatic camera (Nano Letters, 2021). For the third setup we designed and manufactured a novel cost-effective 3D multi-color phase mask by new fabrication procedures developed in our lab (Nature Communications, 2021; Light: Science & Applications, 2023). Finally, we modified a high throughput microscope with compact PSF engineering, which allowed us to enhance the imaging depth-of-field and, combined with deep learning, recover 3D information using single snapshots (Figure 2; Nature Communications, 2024).
Computational system design for 3D imaging of complex samples
Image post processing is a key component in improving imaging. Specifically, the full potential of deep learning to improve microscopy is starting to unravel in recent years. We used deep learning to process the complicated PSF-engineered images, to decode spectral information from a grayscale camera, and to capture cell dynamics with localization microscopy. Moreover, we demonstrated the usage of deep learning to design optimized optical components for different goals. Our main methods, including publicly available algorithmic implementations include:
1. DeepSTORM3D: an algorithm for fast image processing and optical element design, improving the accuracy of nanoscale point emitter localization (Figure 3; Nature Methods, 2020).
2. Multicolor Microscopy and PSF Engineering by Deep Learning: deep learning methods to differentiate colors in grayscale images and design optimized optical elements (Optics Express, 2019).
3. VIPR: An efficient algorithm to calibrate optical systems quickly, improving image accuracy (Optics Express, 2020).
4. Diffractive Optical System Design: A differentiable propagation model for optimizing diffractive optical systems (Optics Express, 2022).
5. Large-FOV 3D Localization Microscopy: A fast and accurate PSF generator and localizer for large fields of view (Science Advances, 2024; Optics Continuum, 2023).
6. DBlink: A deep-learning-based method for dynamic localization microscopy in super resolution, capturing long-term dependencies in SMLM data (Nature Methods, 2023).
Tracking 3D chromatin dynamics in live cells
1. Quantifying double strand break and repair dynamics in yeast cells: The structure and dynamics of DNA determine the destination of a cell and play a key role in various cell processes and functions. Observing DNA inside the nucleus is important for understanding diseases and natural processes, and subsequently for the design of drugs and treatments. However, such observation is extremely challenging, due to the compact sizes involved and dense Chromatin organization. We apply PSF engineering to observe DNA dynamics during double strand break and repair in the mating type switching process in yeast. First, we developed methods to follow and quantify mating type switching occurrence in real-time, at the single live cell level. We then followed DNA dynamics of two relevant loci, and observed two stages in DNA dynamics that lead to repair by gene conversion. This subtle dynamic is observable thanks to the PSF engineering methods we developed during this project (Figure 4).
2. Quantifying cell cycle dependent chromatin dynamics during interphase: The study of cell cycle progression and regulation is important to understanding of fundamental biophysics, aging, and disease mechanisms. Local chromatin movements are generally considered to be constrained and relatively consistent during all interphase stages. We challenge this claim and detected varying cell-cycle-dependent motion constraint levels imposed on the chromatin in healthy and cancerous cells using PSF engineering microscopy (Figure 5; iScience, 2022).
We further implemented our methods in other research areas and beyond the nanoscale:
1. Optical genome mapping (OGM): Efficient tapping into genomic information from a single microscopic image of an intact DNA molecule is an outstanding challenge and its solution will open new frontiers in molecular diagnostics. We develop three different methods for efficient DNA labeling design and image analysis that will advance OGM for precision medicine (Bioinformatics, 2023; Bioinformatics Advances, 2024; Bioinformatics, 2023).
2. Surface profiling: Characterizing the shape of a surface is challenging for temporally changing structures, which requires high-speed acquisition, and for capturing geometries with large axial steps. We leverage PSF engineering for scan-free, dynamic, micro-surface profiling (Science Advances, 2020).
3. Monocular kilometer-scale passive ranging by PSF engineering: We present a simple and small long-range, telescope-based passive ranging monocular for distance estimation (beyond 1.7 km) (Optics Express, 2022).
1. Optical genome mapping (OGM): Efficient tapping into genomic information from a single microscopic image of an intact DNA molecule is an outstanding challenge and its solution will open new frontiers in molecular diagnostics. We develop three different methods for efficient DNA labeling design and image analysis that will advance OGM for precision medicine (Bioinformatics, 2023; Bioinformatics Advances, 2024; Bioinformatics, 2023).
2. Surface profiling: Characterizing the shape of a surface is challenging for temporally changing structures, which requires high-speed acquisition, and for capturing geometries with large axial steps. We leverage PSF engineering for scan-free, dynamic, micro-surface profiling (Science Advances, 2020).
3. Monocular kilometer-scale passive ranging by PSF engineering: We present a simple and small long-range, telescope-based passive ranging monocular for distance estimation (beyond 1.7 km) (Optics Express, 2022).