Periodic Reporting for period 1 - MAFIn (Module for Aberration Correction and Fast Volumetric Imaging in a Light Sheet Fluorescence Microscope)
Période du rapport: 2019-09-01 au 2020-11-30
Conventional imaging systems are capable of observing in 2D with relative high resolution (and even super resolution) with speeds that can vary from a few frames per second (fps) (confocal microscopy) to hundreds of fps (eg. LSFM). However, any interesting and fully useful dynamic biological process occurs in 3D (volumes).
Light Sheet Fluorescence Microscopy (LSFM) is a powerful technique for three-dimensional live imaging of biological samples that permits imaging large specimens over long periods of time. Typically, LSFM involves illuminating the sample with a beam, shaped into a thin sheet (the light sheet), in such a way that only a plane with thickness equivalent to that of the light sheet is excited. The light sheet is usually fixed and placed at the focal plane of the detection objective. The fluorescence light emitted from the fluorophores in this plane is collected by a perpendicular objective and registered by a camera as the sample is displaced through the light sheet, producing a three-dimensional image of the sample. Still, the need to physically move the sample entails that typical LSFM system are not fast enough to capture some of the fastest biological processes, such as neuronal calcium waves.
Wavefront Coding (WFC) is a technique that extends the depth of field (DoF) of the microscope by means of a phase mask, permitting the placement of the light sheet at different axial positions while still producing well-focused images. This way, it is possible to move the light-sheet through the sample (instead of the other way around), allowing recording speeds of tens of volumes per second.
However, WFC introduces a significant distortion on the observed image, which has to be corrected after acquisition by means of a deconvolution process. Traditionally, deconvolutions are made after the experiment has been completed, impeding the monitoring of the sample during it.
The aim of the MAFIn project is to develop a universally-compatible module capable of fast volumetric imaging under high resolution conditions, based on the combination of a LSFM with WFC techniques. In addition, the MAFIn module is also capable of correcting the optical aberrations suffered by the image by employing an Adaptive Optics (AO) system, which also provides the WFC capabilities of the system.
For this, a user friendly, real-time 3D deconvolution algorithm has been developed in order to allow monitoring of the sample during experiments. This algorithm is to be accompanied by an easy-to-use Graphical User Interface to control it along with the WFC and AO capabilities of the module.
Light Sheet Fluorescence Microscopy (LSFM) is a powerful technique for three-dimensional live imaging of biological samples that permits imaging large specimens over long periods of time. Typically, LSFM involves illuminating the sample with a beam, shaped into a thin sheet (the light sheet), in such a way that only a plane with thickness equivalent to that of the light sheet is excited. The light sheet is usually fixed and placed at the focal plane of the detection objective. The fluorescence light emitted from the fluorophores in this plane is collected by a perpendicular objective and registered by a camera as the sample is displaced through the light sheet, producing a three-dimensional image of the sample. Still, the need to physically move the sample entails that typical LSFM system are not fast enough to capture some of the fastest biological processes, such as neuronal calcium waves.
Wavefront Coding (WFC) is a technique that extends the depth of field (DoF) of the microscope by means of a phase mask, permitting the placement of the light sheet at different axial positions while still producing well-focused images. This way, it is possible to move the light-sheet through the sample (instead of the other way around), allowing recording speeds of tens of volumes per second.
However, WFC introduces a significant distortion on the observed image, which has to be corrected after acquisition by means of a deconvolution process. Traditionally, deconvolutions are made after the experiment has been completed, impeding the monitoring of the sample during it.
The aim of the MAFIn project is to develop a universally-compatible module capable of fast volumetric imaging under high resolution conditions, based on the combination of a LSFM with WFC techniques. In addition, the MAFIn module is also capable of correcting the optical aberrations suffered by the image by employing an Adaptive Optics (AO) system, which also provides the WFC capabilities of the system.
For this, a user friendly, real-time 3D deconvolution algorithm has been developed in order to allow monitoring of the sample during experiments. This algorithm is to be accompanied by an easy-to-use Graphical User Interface to control it along with the WFC and AO capabilities of the module.
We have developed a 45x45 cm2 module featuring an AO system and a high-speed imaging camera. The MAFIn module is capable of aberration correction as well as extend the depth of field (DOF) by using wavefront coding (WFC) techniques. The module connects optically to both the microscope and the camera by ISO 38mm ports, a standard used by many commercial LSFM microscopes, and only a trigger signal is required to electronically connect it to a host microscope. Its size and the use of standard connections makes MAFIn readily compatible with most axially scanned LSFM systems.
Since the WFC technique introduces severe image distortion, we have developed a real-time 3D deconvolution algorithm to correct it and make possible the monitoring of the sample during experiments. Our software makes use of two different GPU computing frameworks to optimize deconvolution speed, CUDA and OpenCL. Additionally, two deconvolution strategies, Wiener’s and Least Squares deconvolution with Tikhonov regularization, have been implemented and characterized. Our algorithm can deconvolve an acquire 512x512 pixel image in less than 1 millisecond. This allows it to present fully deconvolved volumetric images to the user at a screen refresh rate of 60Hz.
Additionally, a user-friendly GUI has been developed to present in 3D the deconvolved volumetric images. This GUI also allows controlling the deconvolution algorithm, adaptive optics, and WFC functionalities by non-expert users. Both the deconvolution algorithm and GUI have been developed in Python 2.7 using open libraries and should be compatible with any modern computer running Windows or Linux operative systems.
Module compatibility has been tested in several of our LSFM microscopes. Using a 10x objective, MAFIn extended the depth of field (DoF) of the microscope by a factor of 24, resulting in a useful imaging volume of 1300x1300x190 µm3. Fluorescent beads could be identified and precisely located within this volume in real-time using our deconvolution algorithm. We have demonstrated the ability of the system to track the trajectory fluorescent microspheres suspended in water after agitation at up to 77 Vol/s.
A key entity in the commercialization of MAFIn is M Squared, a manufacturer and supplier of optical components and systems including LSFM (the M Squared Life Aurora Imaging System). M Squared has shown interest in commercializing ICFO’s background technology in LSFM, as well as relevant technology developed in subsequent projects like MAFIn.
The MAFIn technology readiness level will ideally be raised to TRL8/9 by 2023, through collaborative research and development between ICFO and M Squared Life to develop a standardized design suitable for volume production.
Since the WFC technique introduces severe image distortion, we have developed a real-time 3D deconvolution algorithm to correct it and make possible the monitoring of the sample during experiments. Our software makes use of two different GPU computing frameworks to optimize deconvolution speed, CUDA and OpenCL. Additionally, two deconvolution strategies, Wiener’s and Least Squares deconvolution with Tikhonov regularization, have been implemented and characterized. Our algorithm can deconvolve an acquire 512x512 pixel image in less than 1 millisecond. This allows it to present fully deconvolved volumetric images to the user at a screen refresh rate of 60Hz.
Additionally, a user-friendly GUI has been developed to present in 3D the deconvolved volumetric images. This GUI also allows controlling the deconvolution algorithm, adaptive optics, and WFC functionalities by non-expert users. Both the deconvolution algorithm and GUI have been developed in Python 2.7 using open libraries and should be compatible with any modern computer running Windows or Linux operative systems.
Module compatibility has been tested in several of our LSFM microscopes. Using a 10x objective, MAFIn extended the depth of field (DoF) of the microscope by a factor of 24, resulting in a useful imaging volume of 1300x1300x190 µm3. Fluorescent beads could be identified and precisely located within this volume in real-time using our deconvolution algorithm. We have demonstrated the ability of the system to track the trajectory fluorescent microspheres suspended in water after agitation at up to 77 Vol/s.
A key entity in the commercialization of MAFIn is M Squared, a manufacturer and supplier of optical components and systems including LSFM (the M Squared Life Aurora Imaging System). M Squared has shown interest in commercializing ICFO’s background technology in LSFM, as well as relevant technology developed in subsequent projects like MAFIn.
The MAFIn technology readiness level will ideally be raised to TRL8/9 by 2023, through collaborative research and development between ICFO and M Squared Life to develop a standardized design suitable for volume production.
MAFIn produced a first-of-its-kind WFC module prototype. The module’s ability to correct aberration errors and perform extended DOF by WFC allows users to gain faster imaging speeds from their LSFM microscopes. Such task has been previously developed in our lab. However, our previous demonstrator was at TRL2 and it required a specialized operator to run the system. Know how on advanced microscopy, adaptive optics, programming and control of instrumentation and sample preparation were required for a successful use of our system. In addition, the deconvolution algorithm was previously requiring several hours of processing to get the final data. Currently, our module contains a specialized software that interfaces the microscope with the AO instrumentation in addition to a friendly GUI that allows a quick characterization and real time deconvolution. This, besides minimizing systematic errors, allows focusing our efforts on the sample viability and in general on the biological experiment.
The module developed in MAFIn has a wide range of potential use cases, making it attractive to a broad end user base in neuroscience, in particular, and in biomedical research in general. One example of target application is fast functional imaging in 3D such as calcium imaging in the brain or in any other sample in which neuronal communication is important (eg. retinas). Furthermore, LSFM in general is applicable to a wider range of fields such as development biology, cell faith during embryo development, oncology, plant research etc and in general, to any kind of biomedical research that could benefit from high resolution, fast 3D imaging.
The current commercialization plans for MAFIn will allow the broader scientific community to have access to imaging speeds greater than those currently provided only by systems with costs upwards of 400k€ by a fraction of that price.
The module developed in MAFIn has a wide range of potential use cases, making it attractive to a broad end user base in neuroscience, in particular, and in biomedical research in general. One example of target application is fast functional imaging in 3D such as calcium imaging in the brain or in any other sample in which neuronal communication is important (eg. retinas). Furthermore, LSFM in general is applicable to a wider range of fields such as development biology, cell faith during embryo development, oncology, plant research etc and in general, to any kind of biomedical research that could benefit from high resolution, fast 3D imaging.
The current commercialization plans for MAFIn will allow the broader scientific community to have access to imaging speeds greater than those currently provided only by systems with costs upwards of 400k€ by a fraction of that price.