Taking smart diagnostic support tools to the next level
According to the Society to Improve Diagnosis in Medicine, in the United States alone diagnostic errors lead to the death or injury of 40 000 to 80 000 patients annually. Misdiagnosis has been estimated to occur 10 % to 15 % of the time and is a problem across medical specialties. “The reason is usually the experience of medical professionals, time pressures or biases and what we call ‘anchoring’, where clinicians jump to conclusions too early,” explains Eitan Ron, coordinator of the EU-funded Kahun project. To address this, Kahun has built AI-driven diagnostic support tools: “based on first-of-its-kind technology that mimics the clinical reasoning of trained physicians,” says Ron.
Patient clinical assessment chatbot and innovative app
The initial app was released in early 2021 and around 7 000 clinicians have already used its free version for tens of thousands of cases. Kahun Medical, the project host, has also reached an agreement with the ‘New England Journal of Medicine’ to input knowledge to its medical simulation tool, Healer. “To help win acceptance from the medical community, we also offered a patient clinical assessment chatbot, based on the same technology, which collects information from patients prior to consultations,” notes Ron. This chatbot has already been integrated into the pre-visit, virtual clinical assessment of the American-based HelixVM acute-care telemedicine service. It is also being used by Healthbridge, a cloud-based practice management solution, based in South Africa, which serves 3 000 clinics.
Clinical reasoning engine: powered by AI, mapped by medical professionals
The project’s clinical reasoning engine relies on a structured knowledge graph, based on medical literature annotated and mapped by medical professionals. “Currently, the map holds more than 30 million insights into the relationships between diseases, findings, complications, lab results, risk factors, etc.,” adds Ron. The initial clinician app, available for both Apple and Android, lets clinicians input patient details such as age and sex, along with clinical details such as symptoms and test results. This is then fed into a dedicated interface. The AI engine, developed by the team, then uses the knowledge graph to generate a differential diagnosis and suggestions for further tests, with the original sources used by the algorithm available. For the patient chatbot, Kahun’s clinical assessment is completed by patients through a web-based chatbot. The AI uses the knowledge graph to generate a clinical interview. Each answer determines the most suitable next question and allows the system to continually update its working differential diagnosis. The interview ends when the engine has gathered enough information. Afterwards, a summary including history of present illness, review of systems, suggested differential diagnosis and next steps is added to the relevant patient’s health records.
Accuracy of differential diagnosis passes testing with flying colours
Kahun’s diagnostic performance was tested against clinical cases extracted from the United States Medical Licensing Examination Step 2 Clinical Skills. The cases were entered into Kahun, and the accuracy of its differential diagnosis evaluated. Each diagnostic performance was measured for diagnostic sensitivity and for case-specific success rates. The study included 91 clinical cases, with 78 different primary complaints and a mean of 38 findings for each case. In total, 272 diagnoses were expected. Of these, Kahun suggested 231 within the top 20 of its diagnoses, 209 within the top 10, and 168 within the top 5. “Overall, our tool comprehensively managed a wide range of clinical findings and was shown to be diagnostically accurate,” concludes Ron. In another study to evaluate the data-gathering function of eight chatbot symptom-checkers using 28 clinical cases, the Kahun patient chatbot was found to be the most efficient, with the highest recall rate of pertinent case findings.
Keywords
Kahun, diagnosis, clinical, reasoning engine, AI, patient, chatbot, algorithm, telemedicine
