Periodic Reporting for period 2 - MCMQCT (Multi-functional Computational Microscopy for Quantitative Cell Tracking)
Reporting period: 2017-08-01 to 2018-07-31
While these exciting technological advances were the enablers of progress in many fields, the demand for high quality imaging data is ever increasing, influenced by our own perception of modern image quality. However, state of the art optical microscopes are expensive, require a skillful human resource to operate and maintain them, which might create bottlenecks in clinics and research centers, where this resource is shared by numerous laboratories, which ultimately slows down diagnosis and discovery.
In this project, we are developing novel microscopes and computational tools to address the issues of throughput, cost and labor required for biomedical imaging, for a new generation of optical microscopes with the ultimate goal of making biomedical imaging more accessible to clinics, research and recreation.
2) Deep learning based holographic image reconstruction: Phase recovery is a classical inverse imaging problem with wide range of applications for materials, life sciences and fundamental physics, using diverse radiation sources and detectors in regimes such as X-ray, electron beams and visible light. Numerical inverse methods applied to address the phase retrieval problem has led to numerous scientific discoveries. We have demonstrated a deep convolutional neural network for holographic image recovery and phase retrieval, using only a single hologram. We show that the results are comparable to results that are previously obtained with 2-4 measurements, and reconstruction speed >4 times faster than previously reported. These results provide an exciting new approach for phase recovery and inverse imaging problems.
3) Data driven microscope image enhancement: We have demonstrated a significant enhancement of imaging throughput, extended depth-of-field and spatial resolution of an optical microscope using a deep convolutional neural network. The performance enhancement doesn’t require any additional hardware or special design to the microscope. The results were demonstrated for clinically relevant tissues. The technique is widely applicable to other microscopic modalities and inverse imaging problems.
4) High-fidelity image reconstruction from noisy and distorted images: Standard deconvolution microscopy techniques are as good as the numerical approximation of the image formation model. However, spatial aberrations, spectral aberrations, noise, and even sample preparations issues, might be significantly different from image to image and effect the imaging process in a manner that makes the process of numerically modelling the image formation an intractable problem, even between different fields-of-view of the same sample, taken with the same imaging system. We have demonstrated a deep network to learn the statistical transformations between the mobile and optimized benchtop microscope images, which represent two different ends of the spectrum in terms of image quality. This result represents a new generation of inverse imaging techniques, which Instead of trying to come up with a non-tractable forward model for image degradation, learns how to predict the benchtop microscope image that is most statistically likely to correspond to the degraded input image.
5) Enhancing microscope images, using an algorithmic framework: Using deep learning tools, we have been able to decode meaningful images from novel optical instruments that substantially deviate from standard linear-shift-invariant optical components. For example, we have created a “shadow casting” structure, that can allow us to acquire 3D information in spectral regions that lenses and optical components are challenging to fabricate, such as X-ray. We have also demonstrated meaningful imaging through multi-mode under white light illumination, which presents a leapfrog in comparison to previously reported imaged objects through a multi-mode fiber in terms of their information capacity. This project has a potential to be used for thin medical devices and industrial inspection purposes.