The ExaMode consortium is composed of seven partners that collaborated to reach the project objectives. The academic partners University of Applied Sciences Western Switzerland (HES-SO Valais, Switzerland, coordinator), the University of Padova (Italy) and the Radboud University Medical Center (Netherlands) strongly collaborated to develop methods and tools for dealing with data heterogeneity and to extract multimodal knowledge from reports and digital pathology images. The industrial partners Tooploox (Poland) and SIRMA AI (Bulgaria) focused on new products that match the requirements of hospitals and histopathology departments. SURF (Netherlands), provided High-Performance Computing (HPC) resources to the consortium and actively collaborated on the development of the research tools and product prototypes. Clinical partners (Azienda Sanitaria Provinciale Di Catania and Radboud University Medical Center) provided data and annotations in accordance with ethics requirements, clinical guidance to research and tools validation.
One year after the beginning of ExaMode, the Covid-19 pandemic created a global challenge that affected the consortium and, particularly, the hospitals involved in the project that were reorganized in order to fight the pandemic. Nevertheless, ExaMode obtained impressive results, including the completion of all 47 deliverables and 10 milestones, the publication of over 120 scientific papers that obtained over 1500 citations by the project end in mid 2023 and the release of 25 open source software libraries that empower the application of machine learning to multimodal biomedical data.
Among the results obtained in the context of ExaMode, the following are particularly noteworthy.
First, data that are a fundamental resource for machine learning. During ExaMode, the consortium acquired tens of thousands of high resolution microscopy images and related anonymized reports from clinical practice, as well as data from other publicly available sources (including for instance the scientific biomedical literature).
Second, several software resources were produced to extract knowledge from heterogeneous clinical data. Libraries were mostly made open source (
https://www.examode.eu/software/(s’ouvre dans une nouvelle fenêtre)) and allow important applications such as dealing with data variability, data compression, segmentation and classification of digital pathology images, the extraction of concepts from text, the manual annotation and exploration of medical text, the visualization and exploration of biomedical concepts and ontologies, data transfer between centers, weakly supervised learning, multimodal learning and multimodal ontologies.
Finally, several product prototypes were developed by the companies involved in ExaMode and tested in clinical settings. SIRMA AI developed Histographer and SNOMEDICO: respectively a platform to support pathologists in making more informed decisions (exploiting the similarity to other cases in their clinical practice or in scientific literature) and a REST API service for structuring diagnosis information based on SNOMED CT ontology. Tooploox developed three prototypes (VIRTUM P1, P2 and P3), allowing to manage and annotate histopathological slides to detect tissue abnormalities and to aid researchers by making the PubMed Central knowledge-base interconnected with visual context references.
