Periodic Reporting for period 1 - 4D-BioSTEM (4D scanning transmission electron microscopy for structural biology)
Período documentado: 2024-01-01 hasta 2025-06-30
The 4D-BioSTEM project aims to develop 4D-STEM technology for the investigation of biological samples. 4D-STEM is an electron microscopy method that in the materials sciences has allowed structural investigations at superior resolution than other electron microscopy methods, and at the same time, 4D-STEM gives access to additional information about the sample, such as the identity of specific atoms.
Here, we will adapt the 4D-STEM method to be also applicable to life sciences samples, such as single proteins, filamentous proteins, or sections of cells and tissue. The challenge hereby is that biological samples are highly sensitive to the electron beam and rapidly deteriorate under the electron illumination. The 4D-STEM method therefore has to be tuned to also perform well under low electron dose conditions, so that the fragile proteins and biological tissue sections can also be imaged. The merit of the method, once implemented, would be superior resolution at improved contrast, so that also smaller proteins and finer details in the cells could be resolved.
Here, we will adapt the 4D-STEM method to be also applicable to life sciences samples, such as single proteins, filamentous proteins, or sections of cells and tissue. The challenge hereby is that biological samples are highly sensitive to the electron beam and rapidly deteriorate under the electron illumination. The 4D-STEM method therefore has to be tuned to also perform well under low electron dose conditions, so that the fragile proteins and biological tissue sections can also be imaged. The merit of the method, once implemented, would be superior resolution at improved contrast, so that also smaller proteins and finer details in the cells could be resolved.
-Differential phase contrast for biological specimens
The acquisition and processing of differential phase contrast (DPC) and full 4D STEM has been implemented at current cryo-STEM hardware platforms by dedicated software developments, e.g. Velox. The implementation of routine 4D-cryoSTEM data acquisition for biological single-particle samples is currently under way. Following our earlier studies on tobacco mosaic virus (TMV) (Lazic et al. 2022), we imaged a series of other biological specimens: 70S bacterial ribosomes (2 MDa), apoferritin (500 kDa) and hemoglobin (60 kDa) using integrated DPC (manuscript in preparation). While TMV (> GDa) can be routinely resolved at near-atomic resolution, the smaller ribosome is now resolved at ~8.0 resolution while apoferritin and hemoglobin 3D image reconstructions fail due to lack of sufficient contrast. The Müller-Caspary group (LMU) has further focussed on the principal characterization of DPC based on segmented ring detectors, the spatial frequency dependent contrast transfer has been studied in both simulations of vitrified proteins. Two common manuscripts with Sachse (FZJ) and Müller-Caspary (LMU) are currently in preparation to demonstrate state-of-the-art capabilities of DPC when applied to biological specimens.
-Ptychography for biological specimens
The Stahlberg group (EPFL) has focused on the implementation of low-dose 4D-STEM for proteins and tissue sections. An aberration-corrected STEM was equipped with a TVIPS scan generator, allowing to control the electron beam at highest precision, so that the focussed beam can be stepped over the specimen at very high speed in a random, pre-defined pattern. At the same time, the instrument was equipped with a DECTRIS Arina camera, allowing to record images of 192x192 pixels at 30,000 frames per second. A setup based on SerialEM allowed the semi-automated collection of 4D-STEM datasets of different proteins. The recorded, very large 4D-STEM datasets were then subjected to image reconstruction, using the py4Dstem software that is publicly available under the GPL from a group in Berkeley, USA. This has resulted in first medium-high reconstructions from 4D-STEM, published in a common publication between the Sachse, Müller-Caspary and Stahlberg groups under Küçükoglu et al., Nature Communications (2024).
Regular common measurement campaigns at the cryo-microscopes in Lausanne and Jülich have been conducted together with the LMU partners to establish two main goals. First, a full cryo-STEM workflow was established which means the automated procedure from 4D-cryo-STEM data acquisition of millions of frames from autonomously selected regions on grids with vitrified proteins under individual, optimised foci, the ptychographic reconstructions including probe retrieval and multiple 2D projections of single particles separately, particle picking and 3D reconstruction using Relion or CryoSPARC, and to refine single projections to iteratively improve the 3D model.
-Optimizing image reconstructions of low-dose
In order to fully make use of the rich 4D-cryoSTEM data, several aspects have been scrutinized by Müller-Caspary (LMU) in detail. First, for low-dose phase retrieval, the Wirtinger Flow algorithm has been introduced to 4D STEM, and a benchmarking study of the attainable resolution in dependence of the electron dose and the type of the loss function has been published (Micron 185, 103688, 2024). Second, a considerable effort has been spent to retrieve the scanning probe under low-dose conditions. The benefits have been worked out comprehensively in experiment and simulations for direct ptychography methods (SSB, WDD), and for iterative single- and inverse multislice ptychography, which we published in Applied Physics Letters 126, 081602, 2025. Finally, contemporary cryo-STEM optics is often optimised for conventional TEM by changing the optical distances such that the illuminating probe edge is free of Fresnel fringes in parallel illumination mode. These challenges could be solved by incorporating the optical characteristics of cryo-(S)TEM hardware into our ptychographic algorithms (ePIE, Wirtinger Flow), making cryo-STEM ptychography available on the widespread cryo-(S)TEM microscopes equipped with the fringe-free setup. A common publication within the consortium is currently in preparation.
The acquisition and processing of differential phase contrast (DPC) and full 4D STEM has been implemented at current cryo-STEM hardware platforms by dedicated software developments, e.g. Velox. The implementation of routine 4D-cryoSTEM data acquisition for biological single-particle samples is currently under way. Following our earlier studies on tobacco mosaic virus (TMV) (Lazic et al. 2022), we imaged a series of other biological specimens: 70S bacterial ribosomes (2 MDa), apoferritin (500 kDa) and hemoglobin (60 kDa) using integrated DPC (manuscript in preparation). While TMV (> GDa) can be routinely resolved at near-atomic resolution, the smaller ribosome is now resolved at ~8.0 resolution while apoferritin and hemoglobin 3D image reconstructions fail due to lack of sufficient contrast. The Müller-Caspary group (LMU) has further focussed on the principal characterization of DPC based on segmented ring detectors, the spatial frequency dependent contrast transfer has been studied in both simulations of vitrified proteins. Two common manuscripts with Sachse (FZJ) and Müller-Caspary (LMU) are currently in preparation to demonstrate state-of-the-art capabilities of DPC when applied to biological specimens.
-Ptychography for biological specimens
The Stahlberg group (EPFL) has focused on the implementation of low-dose 4D-STEM for proteins and tissue sections. An aberration-corrected STEM was equipped with a TVIPS scan generator, allowing to control the electron beam at highest precision, so that the focussed beam can be stepped over the specimen at very high speed in a random, pre-defined pattern. At the same time, the instrument was equipped with a DECTRIS Arina camera, allowing to record images of 192x192 pixels at 30,000 frames per second. A setup based on SerialEM allowed the semi-automated collection of 4D-STEM datasets of different proteins. The recorded, very large 4D-STEM datasets were then subjected to image reconstruction, using the py4Dstem software that is publicly available under the GPL from a group in Berkeley, USA. This has resulted in first medium-high reconstructions from 4D-STEM, published in a common publication between the Sachse, Müller-Caspary and Stahlberg groups under Küçükoglu et al., Nature Communications (2024).
Regular common measurement campaigns at the cryo-microscopes in Lausanne and Jülich have been conducted together with the LMU partners to establish two main goals. First, a full cryo-STEM workflow was established which means the automated procedure from 4D-cryo-STEM data acquisition of millions of frames from autonomously selected regions on grids with vitrified proteins under individual, optimised foci, the ptychographic reconstructions including probe retrieval and multiple 2D projections of single particles separately, particle picking and 3D reconstruction using Relion or CryoSPARC, and to refine single projections to iteratively improve the 3D model.
-Optimizing image reconstructions of low-dose
In order to fully make use of the rich 4D-cryoSTEM data, several aspects have been scrutinized by Müller-Caspary (LMU) in detail. First, for low-dose phase retrieval, the Wirtinger Flow algorithm has been introduced to 4D STEM, and a benchmarking study of the attainable resolution in dependence of the electron dose and the type of the loss function has been published (Micron 185, 103688, 2024). Second, a considerable effort has been spent to retrieve the scanning probe under low-dose conditions. The benefits have been worked out comprehensively in experiment and simulations for direct ptychography methods (SSB, WDD), and for iterative single- and inverse multislice ptychography, which we published in Applied Physics Letters 126, 081602, 2025. Finally, contemporary cryo-STEM optics is often optimised for conventional TEM by changing the optical distances such that the illuminating probe edge is free of Fresnel fringes in parallel illumination mode. These challenges could be solved by incorporating the optical characteristics of cryo-(S)TEM hardware into our ptychographic algorithms (ePIE, Wirtinger Flow), making cryo-STEM ptychography available on the widespread cryo-(S)TEM microscopes equipped with the fringe-free setup. A common publication within the consortium is currently in preparation.
Common publication
The common publication by the three partners led by Küçükoglu/Stahlberg et al., Nature Communications (2024) (https://doi.org/10.1038/s41467-024-52403-5(se abrirá en una nueva ventana)) has received a lot of attention, so far been cited 41 times (as of August 2025), indicating a rather high impact in the field.
- Towards a common software for 4D-BioSTEM
Software development towards a common image processing environment, in which novel algorithms developed and contributed by the Müller-Caspary, Sachse and Stahlberg groups can be integrated, has started. This new software system is called “4d”. The disseminated software will include modes of automated data acquisition as well as image reconstructions. The 4d software package will boost visibility of the 4D-STEM method and allow our 4D-BioSTEM teamwork to merge all algorithmic developments into a unified GUI system.
- Pilot installation of Timepix4 at LMU
The Timepix4 technology has been installed at LMU Munich for use in 4D-BioSTEM as a detector beyond state-of-the-art. Being one of the first and priorised labs, software for automated data acquisition, data handling and event-based data processing is being developed with first versions running successfully since 06/2025. In this respect, new collaborations with the LMU Physics Dept. (Prof. A. Urban, Prof. E. Cortes, PD Q. Akkerman) have been established to broadcast and apply the phase retrieval methodologies developed in 4D-BioSTEM to different applications in Physical Sciences, which has already led to a newly granted bilateral DFG proposal (Müller-Caspary/Urban), and a further one under consideration at the DFG.
The common publication by the three partners led by Küçükoglu/Stahlberg et al., Nature Communications (2024) (https://doi.org/10.1038/s41467-024-52403-5(se abrirá en una nueva ventana)) has received a lot of attention, so far been cited 41 times (as of August 2025), indicating a rather high impact in the field.
- Towards a common software for 4D-BioSTEM
Software development towards a common image processing environment, in which novel algorithms developed and contributed by the Müller-Caspary, Sachse and Stahlberg groups can be integrated, has started. This new software system is called “4d”. The disseminated software will include modes of automated data acquisition as well as image reconstructions. The 4d software package will boost visibility of the 4D-STEM method and allow our 4D-BioSTEM teamwork to merge all algorithmic developments into a unified GUI system.
- Pilot installation of Timepix4 at LMU
The Timepix4 technology has been installed at LMU Munich for use in 4D-BioSTEM as a detector beyond state-of-the-art. Being one of the first and priorised labs, software for automated data acquisition, data handling and event-based data processing is being developed with first versions running successfully since 06/2025. In this respect, new collaborations with the LMU Physics Dept. (Prof. A. Urban, Prof. E. Cortes, PD Q. Akkerman) have been established to broadcast and apply the phase retrieval methodologies developed in 4D-BioSTEM to different applications in Physical Sciences, which has already led to a newly granted bilateral DFG proposal (Müller-Caspary/Urban), and a further one under consideration at the DFG.