Periodic Reporting for period 1 - AdaptiveSTED (Real-time automatic aberration correction for easy high-resolution imaging in complex specimens, by STED and other point-scanning microscopy techniques)
Berichtszeitraum: 2019-01-01 bis 2020-12-31
Optical aberrations limit the practical working distance and resolution in optical microscopes. They originate mainly from refractive index variations within the sample object and their effect typically increases as a function of imaging depth. Their effect is most evident in super-resolution systems, due to the high numerical aperture objectives they require. Optical aberrations can be corrected with adaptive optical elements -- and to an extent, with post-processing methods, such as image deconvolution.The current state-of-the-art adaptive optics systems can be divided into two categories: (1.) direct wavefront measurement based and (2.) sensorless. In (1.) typically a Shack-Hartmann (SH) wavefront sensor is used to directly quantify the phase variations caused by the optical aberrations that can then be corrected with a deformable mirror (DM) or a spatial light modulator (SLM). In (2.) no wavefront sensor is used, but instead one iteratively applies different aberrations on the DM/SLM and tries to find settings that maximise the image quality, based on some metric, such as image brightness. The problem with (1.) is that a separate wavefront sensor is required, and the SH sensor typically requires multi-photon excitation, as otherwise out-of-focus background noise will preclude the wavefront measurement. The problem with (2.) is that iterative optimization takes a lot of time, and it is also detrimental to the sample due to the large number of images required.
The aim of the AdaptiveSTED project was to develop a third (3.) kind of an aberration correction scheme for laser-scanning optical microscopy (LSM) that would not require a separate wavefront sensor, nor the iterative calibration. The idea was born out of the realization that when one replaces the typical single-element detector in an LSM with a detector array, it is possible to collect information of the microscope’s effective point-spread-function at every single sampling position -- directly from the recorded fluorescence signal. A second important benefit of using an array detector, it is also possible to improve the microscope’s optical resolution and signal-to-noise ratio (SNR) by image scanning microscopy (ISM) pixel reassignment. A second important aim of the AdaptiveSTED project was to develop an adaptive image reconstruction method for ISM, to ensure optimal image quality even in challenging imaging conditions.
The aim of the AdaptiveSTED project was to develop a third (3.) kind of an aberration correction scheme for laser-scanning optical microscopy (LSM) that would not require a separate wavefront sensor, nor the iterative calibration. The idea was born out of the realization that when one replaces the typical single-element detector in an LSM with a detector array, it is possible to collect information of the microscope’s effective point-spread-function at every single sampling position -- directly from the recorded fluorescence signal. A second important benefit of using an array detector, it is also possible to improve the microscope’s optical resolution and signal-to-noise ratio (SNR) by image scanning microscopy (ISM) pixel reassignment. A second important aim of the AdaptiveSTED project was to develop an adaptive image reconstruction method for ISM, to ensure optimal image quality even in challenging imaging conditions.
In WP1 highly optimized algorithms for image-scanning microscopy (ISM) image reconstruction algorithms were developed, which enable optimal image quality even in highly demanding imaging conditions. It was also shown that the information extracted from the ISM reconstruction can be used to directly estimate optical aberrations, without a separate wavefront sensor. For the purpose of studying the effects of optical aberrations to the array detector recordings in detail, a new Fingerprint optical simulation software was written. The new image reconstruction algorithm was shown to guarantee exquisite super-resolution images, deep inside complex biological samples. In WP2 the new laser-scanning microscope system was developed, in which a deformable mirror was integrated into an single photon avalanche diode (SPAD) array-based ISM system with visible and two-photon excitation. A completely new microscope data acquisition software was written. The software takes care of all aspects of data acquisition, ISM image reconstruction, adaptive optics control etc. -- all working in real time, to enable the feedback mechanism necessary to control the adaptive optics with information obtained from the ISM image reconstruction. Also, a deep learning model was implemented with Tensorflow that enables the extraction quantitative aberration information (Zernike amplitudes) from the SPAD array recordings.
The array detector makes it possible to sense optical aberrations directly from the recorded fluorescence signal. This has the potential of significantly simplifying the way optical microscopes for deep imaging are built -- as no separate wavefront sensor is required. Also, multi-photon excitation is no longer necessary, as the array-detector benefits from optical sectioning. Also, the new automatic and blind image reconstruction methods make it possible to generate high quality images, and to an extent correct aberrations even in microscope systems without adaptive optical elements. The project also opened a new research line at the MMS group, which will lead to several future inventions (as well as launch new research careers through PhDs).