Periodic Reporting for period 2 - eCREAM (enabling Clinical Research in Emergency and Acute care Medicine through automated data extraction)
Période du rapport: 2024-03-01 au 2025-08-31
High-quality emergency departments (EDs) are essential for optimal patient care. To improve their clinical and organisational performance, EDs must be assessed using validated criteria, which requires reliable data collection. However, the fast-paced emergency setting, large patient volume, and chronic staff shortages make dedicated data gathering impractical.
The only viable solution is automatic extraction of data from electronic health records (EHRs). While some EHR information is structured (e.g. lab results), much of the relevant content is found in free text notes written by clinicians and nurses, making extraction complex.
To address this, the eCREAM project was launched, involving partners from seven European countries and more than 30 EDs. Its main goal is to develop artificial intelligence–based natural language processing (NLP) tools capable of extracting clinical information from different sources to create robust databases. eCREAM will also design a new EHR prioritising accurate, trustworthy data collection without increasing staff workload.
Two use cases will test its real-world application: assessing EDs’ propensity to hospitalise patients, and creating real-time dashboards to inform clinicians, citizens, and policymakers about ED status, supporting better decision-making.
The project’s impact is expected to be substantial. By sharing unique data, eCREAM will contribute to the European Health Data Space, support researchers with structured datasets, and advance NLP methods for the medical domain. Its dashboards will empower users: patients may choose less crowded EDs, reducing waiting times and overcrowding; emergency services can better coordinate transfers, alleviating pressure on high-volume centres.
Beyond the project’s official end, eCREAM aims to expand its network and include new patient groups. Most importantly, it will provide sustainable tools for long-term emergency medicine research by enabling automatic extraction of EHR data through its tool and developing a new EHR. These innovations will overcome long-standing barriers in the field, ultimately improving both research capacity and the quality of emergency care.
The only viable solution is automatic extraction of data from electronic health records (EHRs). While some EHR information is structured (e.g. lab results), much of the relevant content is found in free text notes written by clinicians and nurses, making extraction complex.
To address this, the eCREAM project was launched, involving partners from seven European countries and more than 30 EDs. Its main goal is to develop artificial intelligence–based natural language processing (NLP) tools capable of extracting clinical information from different sources to create robust databases. eCREAM will also design a new EHR prioritising accurate, trustworthy data collection without increasing staff workload.
Two use cases will test its real-world application: assessing EDs’ propensity to hospitalise patients, and creating real-time dashboards to inform clinicians, citizens, and policymakers about ED status, supporting better decision-making.
The project’s impact is expected to be substantial. By sharing unique data, eCREAM will contribute to the European Health Data Space, support researchers with structured datasets, and advance NLP methods for the medical domain. Its dashboards will empower users: patients may choose less crowded EDs, reducing waiting times and overcrowding; emergency services can better coordinate transfers, alleviating pressure on high-volume centres.
Beyond the project’s official end, eCREAM aims to expand its network and include new patient groups. Most importantly, it will provide sustainable tools for long-term emergency medicine research by enabling automatic extraction of EHR data through its tool and developing a new EHR. These innovations will overcome long-standing barriers in the field, ultimately improving both research capacity and the quality of emergency care.
With regard to the development of the eCREAM language model, substantial progress was made during the second reporting period. A set of evaluation tasks was designed to assess the performance of existing models and identify the most promising candidates for further development within the project. These tasks were also used to measure the impact of additional pre-training, which was carried out using medical texts.
A wide range of LLMs of different sizes were evaluated. The models most relevant to eCREAM’s objectives were selected, and overall performance was generally good. Pre-training did not lead to significant improvements, however, so a new evaluation cycle focusing on larger models, for which pre-training is likely unnecessary given their already high baseline performance, was recently begun.
The development of the evaluation tasks also led to the creation of a public leaderboard (hosted on the Hugging Face platform), where models are ranked according to overall performance as well as task-specific scores.
In parallel, the NLP-DeVal protocol for the development and validation of eCREAM’s LLM was finalised and received ethics committee approval. Data from the first ED have already been obtained, facilitated by the eCREAM’s IT platform for processing medical record data, which has been installed at the first site to test its functionality and effectiveness and is now being deployed in the EDs of the partner countries.
Regarding the development of the new EHR for EDs, work progressed with the design of additional modules and the start of the software development phase. A collaboration was also initiated with a company specialising in medical record certification, ensuring compliance with current European legislation from the outset. Specific efforts were made to adapt the triage system used in Greece, paving the way for a field trial in that country within the year.
The protocols for projects related to use cases 1 and 2 have been finalised and are being approved by the ethics committees. Data from the promoting centre have already been transferred to the central study server, and annotation of medical records has begun. These annotations will provide the gold standard required for fine-tuning and final evaluation of the project’s language model.
Finally, the Translation Manager, designed to support localisation of all project IT products into the consortium’s languages, has been completed and tested, with excellent results.
A wide range of LLMs of different sizes were evaluated. The models most relevant to eCREAM’s objectives were selected, and overall performance was generally good. Pre-training did not lead to significant improvements, however, so a new evaluation cycle focusing on larger models, for which pre-training is likely unnecessary given their already high baseline performance, was recently begun.
The development of the evaluation tasks also led to the creation of a public leaderboard (hosted on the Hugging Face platform), where models are ranked according to overall performance as well as task-specific scores.
In parallel, the NLP-DeVal protocol for the development and validation of eCREAM’s LLM was finalised and received ethics committee approval. Data from the first ED have already been obtained, facilitated by the eCREAM’s IT platform for processing medical record data, which has been installed at the first site to test its functionality and effectiveness and is now being deployed in the EDs of the partner countries.
Regarding the development of the new EHR for EDs, work progressed with the design of additional modules and the start of the software development phase. A collaboration was also initiated with a company specialising in medical record certification, ensuring compliance with current European legislation from the outset. Specific efforts were made to adapt the triage system used in Greece, paving the way for a field trial in that country within the year.
The protocols for projects related to use cases 1 and 2 have been finalised and are being approved by the ethics committees. Data from the promoting centre have already been transferred to the central study server, and annotation of medical records has begun. These annotations will provide the gold standard required for fine-tuning and final evaluation of the project’s language model.
Finally, the Translation Manager, designed to support localisation of all project IT products into the consortium’s languages, has been completed and tested, with excellent results.
The first potential impact concerns the eCREAM project’s main goal and how it fits into the European Commission’s Electronic Health Record exchange Format (EEHRxF) initiative. The EEHRxF is currently being developed to empower patients to access their health data from any healthcare professional and from any European country. An important secondary goal of the EEHRxF is the use of data for research, which is eCREAM’s main objective. Specifically, eCREAM aims to develop a system to exploit EHRs to enable research in the field of emergency medicine. By developing robust NLP tools and ensuring data interoperability and accessibility, in line with the European EEHRxF initiative, eCREAM aims to revolutionize clinical research in emergency medicine, providing insights to improve patient care, and operational efficiency in emergency settings.
Another potential impact involves eCREAM’s first use case, which will address the issue of hospitalization rate. Permitting EDs to compare their data with those of others will promote the auditing of care and management practices, enabling healthcare professionals to make more informed decisions, reducing inappropriate hospital admissions and discharges, and, ultimately, improving patient outcomes.
Another potential impact involves eCREAM’s first use case, which will address the issue of hospitalization rate. Permitting EDs to compare their data with those of others will promote the auditing of care and management practices, enabling healthcare professionals to make more informed decisions, reducing inappropriate hospital admissions and discharges, and, ultimately, improving patient outcomes.