In medicine, many heads are better than one
To err is human. But in medicine, errors cost lives, and many of these errors result from an incorrect diagnosis. However, what if we could increase diagnostic accuracy by combining the diagnoses of many physicians? An international research team supported by the EU-funded HACID project has now developed a fully automated procedure that uses knowledge engineering techniques to take full advantage of collective intelligence (CI) in medicine and healthcare.
One versus the collective
The team tested their solution on more than 1 300 medical cases provided by a medical crowdsourcing platform called The Human Diagnosis Project. Each case was independently diagnosed by 10 diagnosticians. Comparing the single diagnoses with their collective solution, the team found that their method substantially boosted diagnostic accuracy. The single diagnosticians achieved 46 % accuracy. However, pooling the decisions of 10 diagnosticians increased it to 76 %. “Our results show the life-saving potential of tapping into the collective intelligence of the global medical community to reduce diagnostic errors and increase patient safety,” report the researchers in their paper published in the journal ‘PNAS’. The greatest challenge when combining independent diagnoses in open-ended medical diagnostics is identifying which diagnoses point to the same medical concept. This involves mundane problems such as different spellings, the use of capitalisation and typos. It also involves more complicated issues such as whether or not two reported diagnoses are equivalent. To identify exact medical concepts from free-text diagnoses, the team’s solution relies on a combination of semantic knowledge graphs, natural language processing, and the world’s most comprehensive, multilingual clinical healthcare terminology – the SNOMED Clinical Terms ontology – for standardisation. “This work presents a fully automated pipeline—spanning from the aggregation of diagnoses to the evaluation of the results obtained via CI—that can harness the power of independent medical experts in the medical domain at large. This thus vastly extends the application of CI in medical diagnostics beyond simple binary or multiclass classification or numeric estimation tasks,” the study authors report. “Our results show that aggregation of independent responses from multiple users leads to substantial improvements in diagnostic accuracy across aggregation rules, medical specialties, chief complaints, and tenure levels of users.”
Removing the human from the loop
The solution eliminates all manual, human intervention. Study co-author Dr Vito Trianni of HACID project coordinator National Research Council, Italy, states in a ‘EurekAlert!’ news release: “A key contribution of our work is that, while the human-provided diagnoses maintain their primacy, our aggregation and evaluation procedures are fully automated, avoiding possible biases in the generation of the final diagnosis and allowing the process to be more time- and cost-efficient.” Importantly, because the solution is fully automated, “it can operate in an actual, real-time clinical setting, where the ground truth is unknown at the time of judgments,” the paper reports. The researchers are currently collaborating within the HACID (Hybrid Human Artificial Collective Intelligence in Open-Ended Decision Making) project to bring their application one step closer to market. For more information, please see: HACID project website
Keywords
HACID, medical, diagnosis, misdiagnosis, diagnostic, accuracy, collective intelligence, medicine, diagnostician