Periodic Reporting for period 1 - CyberScoPy (Smart, Event-Based Microscopy for Cell Biology)
Período documentado: 2023-10-01 hasta 2025-03-31
Microscopy-based imaging is an invaluable tool for modern quantitative cell biology. Commercial microscopes can acquire pre-programmed timelapse sequences, called Multi-Dimensional Acquisition (MDA), to observe the behaviors of cells at different scales and in all their complexity. Yet, such acquisitions are passive and cannot adapt in real time to optimize imaging parameters, detect specific (rare) cellular events or even zoom on cellular processes of interest when they occur. Indeed, software development to achieve unsupervised automation is lagging, which limits in several ways our ability to identify and image complex biological events in real time and at the proper 4D scales. We are at a moment in which smart systems and artificial intelligence are being used everywhere, including in laboratories to improve the functioning of many (scientific) devices. This is however not the case for microscopy workflows which are outdated. A key practical limitation of current workflows is the fact that the images are only analyzed at the end of the experiment, which sequentially separates the workflows of image acquisition and image analysis and prevent the user to adjust imaging modalities in real time to detect and zoom in on interesting cell features and dynamics. The ability to employ real-time image analysis to inform, optimize and adjust the settings of ongoing image acquisitions would be a game changer for cell biologists studying complex, dynamic cellular processes by automating processes that would traditionally be carried out manually thus reducing error and saving time. Here, we want to develop a user-friendly software enabling cell biologists to upgrade their existing microscopy system into smart, event-based microscopy.
During the Cybersco.Py project we performed several technological improvement/ development contributing significant progress to the CyberSco.Py software ambition. First, we transform our first proof of principle into a more mature open source software, with numerous techical improvement and documentation to ensure it can be further used by the community through our public Github account. A key point of the proposal was to allow Cybersco.Py to pilot more devices. We worked on integrating microfluidic controler (pressure control, pumps), optogenetic devices (DMD from Andor), additional cameras (e.g. Quantitative Phase Imaging) and several basic probes (temperature, light, vibration,etc...) to register the conditions in which the experiments are done. Altogether, this increases the possibility of CyberSco.Py to generate trigger based actions and, hence, complex experiments for the benefit of cell biologists. This is still a work in progress, and one need to add more devices natively, even though it is possible to connect to the large library of devices that are already recognized by MicroManager. We also develop a novel, smart auto focus system that is very powerfull to analyze complex biological samples, by focusing only on the sample of interest. Taken together, the outcome of the action is a novel version of the open source Cybersco.Py software which is more reliable and can drive more microscopy equipments.
The Cybersco.py project is functionning in a lab environment (TRL4) and still need software development to reach higher TRL. This can be done (and the team will continue working on such development) but, to be efficient and scalable, it should be done in collaboration with microscopy system and microscopy devices companies. In addition, we have started to include AI agent (i.e. LLMs) into our software to even further facilitate the interaction between the user and the microscope. This is a very promising avenue of research which we will further explore in the next version of CyberSco.Py.