Periodic Reporting for period 1 - EmergAI (Enhancing emergency department safety, efficacy and cost-effectiveness by artificial intelligence)
Berichtszeitraum: 2022-10-01 bis 2025-03-31
Background
Emergency care costs are increasing in developed societies, both in rates of emergency department (ED) visits per person and in costs per visit, and are growing faster than other areas of healthcare spending. With limited and unstructured data, ED staff make quick decisions about probabilities for multiple diagnoses and risks. Both underestimation and overestimation of these probabilities lead to increased costs and patient harm. Hence, there is desperate need for clinical decision-support systems in the ED.
Aim
To develop a clinical decision support system for emergency medicine doctors, using sensor data, health records data and patient-reported data, validated in a randomized clinical trial, in order to improve the safety, efficacy and cost-effectiveness of emergency care.
Objectives
We will: Develop machine learning (ML)-powered diagnosis and risk prediction algorithms for common and dangerous conditions based on age, sex, presenting complaints, previous diagnoses, ECGs, and vital parameters; develop and validate a patient-centred technical platform for collecting, storing and sharing patient-reported data and three-dimensional symptom drawings; develop ML-powered diagnosis and risk prediction algorithms for common and dangerous conditions based on patient-reported data and symptom drawings; conduct a large-scale prospective ED data collection for internal and external validation of ML models using a common format for online applications and for further data collection; develop a Bayesian network-powered ED-based clinical decision support system that generates probabilities for diagnoses and 30-day mortality risks and suggestions for the most valuable next step, from data in multiple formats, with visual representation of probabilities, risks and uncertainties and Bayes factors for potential next steps; and conduct a randomized clinical trial investigating the usefulness, effectiveness and safety of the new decision support system.
Importance
The potential clinical gains by this research program are unequivocal. Helping emergency doctors to more efficient collection, documentation and processing of patient-data, and more efficient decision making, has the potential to both decrease unnecessary work-up of very-low-risk patients (thereby cutting costs, false positives, and patient harms), and ensure doctors don’t miss dangerous diagnoses. This will lead to savings for healthcare providers, and patients’ lives saved. The potential scientific gains are much more far-reaching. Capturing an entirely new variety of patient-generated data, and using it to predict disease, is a massive developmental step for e-health globally. This will produce vast opportunities for new research around mining the entire multidimensional space of patient-generated symptom data in order to predict disease events; and I propose calling it symptomics. In the context of emergency department patients, machine learning-guided symptomics has potential for pattern recognition and decision support in a chaotic work environment where it is badly needed today. Machine learning holds great potential for healthcare, but the field needs optimal cases to investigate its utility. I believe pattern recognition in chaotic and time-sensitive clinical environments, such as in the present study, will provide such an optimal case.
Emergency care costs are increasing in developed societies, both in rates of emergency department (ED) visits per person and in costs per visit, and are growing faster than other areas of healthcare spending. With limited and unstructured data, ED staff make quick decisions about probabilities for multiple diagnoses and risks. Both underestimation and overestimation of these probabilities lead to increased costs and patient harm. Hence, there is desperate need for clinical decision-support systems in the ED.
Aim
To develop a clinical decision support system for emergency medicine doctors, using sensor data, health records data and patient-reported data, validated in a randomized clinical trial, in order to improve the safety, efficacy and cost-effectiveness of emergency care.
Objectives
We will: Develop machine learning (ML)-powered diagnosis and risk prediction algorithms for common and dangerous conditions based on age, sex, presenting complaints, previous diagnoses, ECGs, and vital parameters; develop and validate a patient-centred technical platform for collecting, storing and sharing patient-reported data and three-dimensional symptom drawings; develop ML-powered diagnosis and risk prediction algorithms for common and dangerous conditions based on patient-reported data and symptom drawings; conduct a large-scale prospective ED data collection for internal and external validation of ML models using a common format for online applications and for further data collection; develop a Bayesian network-powered ED-based clinical decision support system that generates probabilities for diagnoses and 30-day mortality risks and suggestions for the most valuable next step, from data in multiple formats, with visual representation of probabilities, risks and uncertainties and Bayes factors for potential next steps; and conduct a randomized clinical trial investigating the usefulness, effectiveness and safety of the new decision support system.
Importance
The potential clinical gains by this research program are unequivocal. Helping emergency doctors to more efficient collection, documentation and processing of patient-data, and more efficient decision making, has the potential to both decrease unnecessary work-up of very-low-risk patients (thereby cutting costs, false positives, and patient harms), and ensure doctors don’t miss dangerous diagnoses. This will lead to savings for healthcare providers, and patients’ lives saved. The potential scientific gains are much more far-reaching. Capturing an entirely new variety of patient-generated data, and using it to predict disease, is a massive developmental step for e-health globally. This will produce vast opportunities for new research around mining the entire multidimensional space of patient-generated symptom data in order to predict disease events; and I propose calling it symptomics. In the context of emergency department patients, machine learning-guided symptomics has potential for pattern recognition and decision support in a chaotic work environment where it is badly needed today. Machine learning holds great potential for healthcare, but the field needs optimal cases to investigate its utility. I believe pattern recognition in chaotic and time-sensitive clinical environments, such as in the present study, will provide such an optimal case.
We have compiled a dataset containing the Electronic Health Records (EHR) of 2,167,278 patients – representing 7,801,285 individual visits at the emergency departments (ED) of 12 hospitals around Sweden. The EHR contains information about the visit, such as the demographics (e.g. age, sex), presenting complaints, vital parameters, diagnoses, medications, blood tests, patient registries, death registries, cancer registries, immigration registries, life support, ECGs, CT-scan results, scintigraphies, and echocardiographies. We also compiled a dataset based on the governmental socioeconomic registries SCB LISA and HUSH to determine what role socioeconomic factors play in the accuracy of our models.
We have harmonised all presenting complaints onto the SNOMED-CT ontology. The initially more than 6000 presenting complaints could be harmonised to circa 600. We have also harmonised blood tests onto the same ontology. Diagnoses and drugs were already harmonised onto the ICD and ATC ontologies.
Using the 600,000 ECGs in the database, we have trained and evaluated a convolutional neural network with residual blocks to predict acute myocardial infarction, including both infarctions with an ECG pattern that can be readily detected by humans as well as ECGs without a pattern known to humans. We have also trained probabilistic models for use with ECGs, predicting continuous values of potassium.
For all patients in the emergency room, we have used their age, sex, and presenting complaint to predict probabilities of death and probabilities of the most common diagnoses within 30 days in every patient for every visit, using Bayesian logistic regression.
We have prospectively collected self-reported data and rich EHR data on all patients in the ED of Uppsala University Hospital.
We have harmonised all presenting complaints onto the SNOMED-CT ontology. The initially more than 6000 presenting complaints could be harmonised to circa 600. We have also harmonised blood tests onto the same ontology. Diagnoses and drugs were already harmonised onto the ICD and ATC ontologies.
Using the 600,000 ECGs in the database, we have trained and evaluated a convolutional neural network with residual blocks to predict acute myocardial infarction, including both infarctions with an ECG pattern that can be readily detected by humans as well as ECGs without a pattern known to humans. We have also trained probabilistic models for use with ECGs, predicting continuous values of potassium.
For all patients in the emergency room, we have used their age, sex, and presenting complaint to predict probabilities of death and probabilities of the most common diagnoses within 30 days in every patient for every visit, using Bayesian logistic regression.
We have prospectively collected self-reported data and rich EHR data on all patients in the ED of Uppsala University Hospital.
For the decision support model development, we have created a set of predictive tools based on Bayesian statistics which predicts the risk of mortality of a patient within 30 days based on sex, age and presenting complaint (and their interactions).
Our work presenting a novel method that allows for fast and memory-friendly training of LLM ensembles is a breakthrough of unknown magnitude. We show that the resulting ensembles can detect hallucinations and are a viable approach in practice as only one GPU is needed for training and inference. LLMs are otherwise expensive to train and run; they need large amounts of computations and memory, preventing the use of ensemble methods in practice.
The digital systems developed or refined in the present project have moved boundaries significantly in terms of facilitating digital participation and self-reporting in clinical studies. Both the Symptoms and the minforskning.se systems are now used in other studies; with gains including remote enrollment in a randomised clinical drug trial, and self-administered enrollment in other clinical studies.
The clearest achievement in the project so far has been the development of ML models for ECG interpretation. Our model using a convolutional neural network with residual blocks reached very good performance in classifying acute myocardial infarctions using ECGs. The performance of the model (a C-index of 0.99 for large myocardial infarctions) is very much higher than that of human cardiologists (typically a C-index of less than 0.75 for such infarctions). We extended that work investigating other types of ML models, and some work with probabilistic models reached a moderate performance in predicting potassium levels.
Our work presenting a novel method that allows for fast and memory-friendly training of LLM ensembles is a breakthrough of unknown magnitude. We show that the resulting ensembles can detect hallucinations and are a viable approach in practice as only one GPU is needed for training and inference. LLMs are otherwise expensive to train and run; they need large amounts of computations and memory, preventing the use of ensemble methods in practice.
The digital systems developed or refined in the present project have moved boundaries significantly in terms of facilitating digital participation and self-reporting in clinical studies. Both the Symptoms and the minforskning.se systems are now used in other studies; with gains including remote enrollment in a randomised clinical drug trial, and self-administered enrollment in other clinical studies.
The clearest achievement in the project so far has been the development of ML models for ECG interpretation. Our model using a convolutional neural network with residual blocks reached very good performance in classifying acute myocardial infarctions using ECGs. The performance of the model (a C-index of 0.99 for large myocardial infarctions) is very much higher than that of human cardiologists (typically a C-index of less than 0.75 for such infarctions). We extended that work investigating other types of ML models, and some work with probabilistic models reached a moderate performance in predicting potassium levels.