Periodic Reporting for period 1 - CryoET-CryoCloud (Streamlining structural biology: Developing a high-throughput cloud-based cryo-electron tomography platform)
Okres sprawozdawczy: 2023-07-01 do 2024-12-31
Cryo-electron tomography (cryo-ET) is a cutting-edge imaging technique that allows scientists to observe the intricate details of life at molecular resolutions. This method provides unprecedented views of biological structures in their native environment, enabling researchers to uncover the complex dynamics and interactions of proteins at the cellular and subcellular levels.
Despite its revolutionary potential, the technique is still being developed and has been underutilized in the broader scientific community due to its complexity and the specialized knowledge required to process and analyze the data. The primary challenge with cryo-ET, next to the sample preparation, is the complexity of its data analysis process. Cryo-ET generates vast amounts of data that are both computationally intensive and intricate to handle. Moreover, the existing tools for analyzing cryo-ET data are often fragmented and not user-friendly, requiring extensive computational expertise and resources. This fragmentation hinders the efficiency and accessibility of cryo-ET, limiting its use to a small number of specialized laboratories equipped with the necessary computational infrastructure and technical know-how.
To address these challenges, our project focused on developing a streamlined, cloud-based workflow for cryo-ET data analysis together with the company CryoCloud. Our goal was to create a comprehensive and user-friendly workflow that simplifies the analysis process, making it more accessible and less intimidating for researchers across various scientific disciplines.
The newly developed workflow integrates all necessary tools into a single, cohesive cloud-based platform. This integration allows for seamless processing of cryo-ET data, from initial data preprocessing to subtomogram averaging the determiation of a 3D protein structure. By leveraging cloud computing, we ensure that the computational load is managed efficiently, enabling users to process large datasets without the need for specialized local hardware.
Despite its revolutionary potential, the technique is still being developed and has been underutilized in the broader scientific community due to its complexity and the specialized knowledge required to process and analyze the data. The primary challenge with cryo-ET, next to the sample preparation, is the complexity of its data analysis process. Cryo-ET generates vast amounts of data that are both computationally intensive and intricate to handle. Moreover, the existing tools for analyzing cryo-ET data are often fragmented and not user-friendly, requiring extensive computational expertise and resources. This fragmentation hinders the efficiency and accessibility of cryo-ET, limiting its use to a small number of specialized laboratories equipped with the necessary computational infrastructure and technical know-how.
To address these challenges, our project focused on developing a streamlined, cloud-based workflow for cryo-ET data analysis together with the company CryoCloud. Our goal was to create a comprehensive and user-friendly workflow that simplifies the analysis process, making it more accessible and less intimidating for researchers across various scientific disciplines.
The newly developed workflow integrates all necessary tools into a single, cohesive cloud-based platform. This integration allows for seamless processing of cryo-ET data, from initial data preprocessing to subtomogram averaging the determiation of a 3D protein structure. By leveraging cloud computing, we ensure that the computational load is managed efficiently, enabling users to process large datasets without the need for specialized local hardware.
During the 18 months of the project the following work was performed:
* Development of an End-to-End Workflow: The core activity was the creation of a fully integrated workflow within the CryoCloud application. This workflow facilitates every step of the cryo-ET data analysis process, from tilt series alignment to subtomogram averaging. By automating these processes, the platform significantly reduces the manual effort and expertise required, making it accessible to a broader range of researchers.
* Implementation of Scalable Cloud Infrastructure: We engineered the backend to leverage cloud computing resources. The scalability from using cloud resources is crucial for handling the large data volumes generated by cryo-ET studies, allowing for efficient data processing without the need for local high-performance computing infrastructure.
* User Interface Development: A significant focus was placed on developing a user-friendly interface accessible via a web browser. The interface was designed to be intuitive, allowing users to easily navigate through complex data analysis processes. This design choice was instrumental in making advanced cryo-ET analysis techniques more accessible to non-specialist users.
* Benchmarking and collection of user feedback: we tested the integrated workflow by running the end-to-end analysis on raw dataset, and also obtained user feedback on the prorotype.
* Cost analysis: Using the benchmark results, we determined the compute costs for data analysis.
Outcomes of the actions:
• We implemented an end-to-end solution for cryoET data analysis into the CryoCloud app, which is easily accessible via a web-browser, uses scalable cloud infrastructure in the back, has a user-friendly interface, and can be easily accessed by users globally within a few minutes.
• The collected feedback from users during and after the implementation helped guide the development and identify areas of improvement.
• The cost analysis for tilt series alignment and tomogram reconstructions highlights that the developed solution is extremely cost-efficient and could provide significant value for day to day use, while costs for the classification and refinement should ideally be improved.
* Development of an End-to-End Workflow: The core activity was the creation of a fully integrated workflow within the CryoCloud application. This workflow facilitates every step of the cryo-ET data analysis process, from tilt series alignment to subtomogram averaging. By automating these processes, the platform significantly reduces the manual effort and expertise required, making it accessible to a broader range of researchers.
* Implementation of Scalable Cloud Infrastructure: We engineered the backend to leverage cloud computing resources. The scalability from using cloud resources is crucial for handling the large data volumes generated by cryo-ET studies, allowing for efficient data processing without the need for local high-performance computing infrastructure.
* User Interface Development: A significant focus was placed on developing a user-friendly interface accessible via a web browser. The interface was designed to be intuitive, allowing users to easily navigate through complex data analysis processes. This design choice was instrumental in making advanced cryo-ET analysis techniques more accessible to non-specialist users.
* Benchmarking and collection of user feedback: we tested the integrated workflow by running the end-to-end analysis on raw dataset, and also obtained user feedback on the prorotype.
* Cost analysis: Using the benchmark results, we determined the compute costs for data analysis.
Outcomes of the actions:
• We implemented an end-to-end solution for cryoET data analysis into the CryoCloud app, which is easily accessible via a web-browser, uses scalable cloud infrastructure in the back, has a user-friendly interface, and can be easily accessed by users globally within a few minutes.
• The collected feedback from users during and after the implementation helped guide the development and identify areas of improvement.
• The cost analysis for tilt series alignment and tomogram reconstructions highlights that the developed solution is extremely cost-efficient and could provide significant value for day to day use, while costs for the classification and refinement should ideally be improved.
The development of this cloud-based workflow represents a significant advancements for the application of cryo-ET. As a fully integrated, cloud-based solution, it provides automation features that remove many of the barriers traditionally faced by researchers. This platform makes molecular imaging more accessible and also has the potential to significantly enhance research efficiency in both academia and industry. Ultimately, our project contributes to the democratization of high-resolution molecular imaging, fostering new discoveries and innovations in various fields such as cell biology, neurology, and virology.
While the prototype has proven effective, it requires further refinement to transition into a fully operational production tool. Currently, it lacks robust capabilities for data manipulation and advanced visualization, which are critical for detailed scientific analysis and interpretation.
CryoCloud is actively exploring opportunities to further develop and commercialize the workflow. Recognizing the need for additional features and enhancements, the company is seeking partnerships and funding to expand the platform’s capabilities and reach.
In conclusion, the ERC-funded project successfully addressed a critical bottleneck in cryo-ET utilization, paving the way for broader adoption and transformative impacts across the scientific spectrum.
Overview of the results:
• We implemented an end-to-end solution for cryoET data analysis into the CryoCloud app, which is easily accessible via a web-browser, uses scalable cloud infrastructure in the back, has a user-friendly interface, and can be easily accessed by users globally within a few minutes.
• The implemented solution is greatly facilitating many steps and makes cryo-ET more accessible, as it allows users to store and analyze cryo-ET data without any existing IT infrastructure, while the case study and published tutorial provide a starting point for unexperienced users.
• However, the current solution is still a prototype and needs more development to be used in production. The main limitations is the lack of data quality assessment, data visualization and data filtering tools.
• The CryoCloud team is currently seeking more funding to continue the development and commercialization of the developed prototype.
While the prototype has proven effective, it requires further refinement to transition into a fully operational production tool. Currently, it lacks robust capabilities for data manipulation and advanced visualization, which are critical for detailed scientific analysis and interpretation.
CryoCloud is actively exploring opportunities to further develop and commercialize the workflow. Recognizing the need for additional features and enhancements, the company is seeking partnerships and funding to expand the platform’s capabilities and reach.
In conclusion, the ERC-funded project successfully addressed a critical bottleneck in cryo-ET utilization, paving the way for broader adoption and transformative impacts across the scientific spectrum.
Overview of the results:
• We implemented an end-to-end solution for cryoET data analysis into the CryoCloud app, which is easily accessible via a web-browser, uses scalable cloud infrastructure in the back, has a user-friendly interface, and can be easily accessed by users globally within a few minutes.
• The implemented solution is greatly facilitating many steps and makes cryo-ET more accessible, as it allows users to store and analyze cryo-ET data without any existing IT infrastructure, while the case study and published tutorial provide a starting point for unexperienced users.
• However, the current solution is still a prototype and needs more development to be used in production. The main limitations is the lack of data quality assessment, data visualization and data filtering tools.
• The CryoCloud team is currently seeking more funding to continue the development and commercialization of the developed prototype.