Periodic Reporting for period 1 - VALIDATE (V A L I D A T E - Validation of a Trustworthy AI-based Clinical Decision Support System for Improving Patient Outcome in Acute Stroke Treatment)
Período documentado: 2022-05-01 hasta 2023-10-31
Based on previously developed models and an existing prototype of a clinical decision support system (patent pending), we set out in this project to further develop, test, and validate this clinical decision support for the treatment stratification of acute stroke patients to improve patient outcome. Machine learning (ML)-enabled Artificial intelligence (AI) methods are increasingly adopted in the medical field. Implementing ML-based CDSSs have the potential to be go beyond the current clinical state-of-the-art as AI excels at finding complex and non-linear relationships across a multitude of prognostic variables. AI also has the promise to combine different modalities, such as imaging and clinical values, leading to powerful stratification tools accounting for a multitude of patient sub-populations. Our consortium combines excellence in technical and medical machine learning development with the clinical expertise of three leading stroke hospital partners. Additionally, our consortium benefits from the special expertise in the development of trustworthy AI, software design, and the translation of AI models to the clinical setting with focus on the regulatory process. By leveraging the available medical data and exploiting technological opportunities in the field of AI, and developing and validating trustworthy AI solutions to be implemented in the clinical workflow we are seeking to surpass the clinical state-of-the-art by making a significant and sustainable impact on the treatment of acute stroke that will improve patient survival, outcome and quality of life. The results of our work will serve as a pathway for future projects and we will make our experiences public in the form of standard operating procedures (SOPs) in the areas of development, testing, validation, and regulatory processes.
The following project objectives were defined:
Objective 1: Create a trustworthy clinical AI framework
Objective 2: Transition an existing AI prognostic model from TRL 3 through to validation at TRL 6
Objective 3: Create a demonstrator system that integrates a prognostic model for stroke outcome with a use-case specific UX/UI concept in order to ensure maximized usability, performance and safety in the clinical setting.
Objective 4: Clinically validate a demonstrator system in the clinical setting by conducting a multi-centre prospective study
Objective 5: To develop a plan for regulatory acceptability of the validated prognostic tool.
Objective 6: To provide a framework for patient communication and integration in order to ensure a) maximized involvement of patients and patient organisations in the study planning and development and b) outcome reporting.
Objective 7: To systematically assess, structure and synthesize the processes and outcomes of the project in order to establish SOPs for the integration of AI in healthcare.
Objective 8: Evaluate cost-effectiveness; identify and pursue commercial exploitation of scientific results by using generated knowledge and intellectual property.
The following project objectives were defined:
Objective 1: Create a trustworthy clinical AI framework
Objective 2: Transition an existing AI prognostic model from TRL 3 through to validation at TRL 6
Objective 3: Create a demonstrator system that integrates a prognostic model for stroke outcome with a use-case specific UX/UI concept in order to ensure maximized usability, performance and safety in the clinical setting.
Objective 4: Clinically validate a demonstrator system in the clinical setting by conducting a multi-centre prospective study
Objective 5: To develop a plan for regulatory acceptability of the validated prognostic tool.
Objective 6: To provide a framework for patient communication and integration in order to ensure a) maximized involvement of patients and patient organisations in the study planning and development and b) outcome reporting.
Objective 7: To systematically assess, structure and synthesize the processes and outcomes of the project in order to establish SOPs for the integration of AI in healthcare.
Objective 8: Evaluate cost-effectiveness; identify and pursue commercial exploitation of scientific results by using generated knowledge and intellectual property.
Over the first 18 months, the VALIDATE project has concluded a series of z-inspection workshops to define the ethical requirements for a process of generating trustworthy artificial intelligence. These were mapped to the requirements of the EU trustworthy AI guidelines.
The technological requirements for the VALIDATE clinical decision support tool were defined and together with the ethical requirements mappes to the several cycles of AI development to guide the developments, validation and preparation of the AI tools for operation in three pilot trials. The models were refined under application of the CRISP-ML(Q) standard and methodology (Cross-Industry Standard Process for the development of Machine Learning applications with Quality assurance methodology). After data harmonization, the retrospective training data from the three pilot sites enabled adaptions of the AI models.
The technical requirements for the VALIDATE app (a clinical decision support tool for actute ischemic stroke treatment) and its architecture were defined using the IBM Unified Method Framework. The patient immediate care pathway was defined to identify pain-points and a value-tree for users and beneficiaries of the tool. In compliane with ethics requirements, the tool was developed such that it works on local laptops or hospital servers and without internet connection.
A 'study-initiation package' was concluded, including the technology trial registration under study-ID NCT05622539 in clinicantrials.gov and in submitting the study protocol to the three local ethics boards. The NORA patient app has been adapted such that it enables patient follow-up on their PROMs (Patient Reported Outcome Measures) and Quality of Life measures. Ist was also translated from Spanish to English and German.
The retrospective technology trials in three clinical centers (Val d'Hebron Hospital in Barcelona, Spain; Universitätsklinik Heidelberg in Heidelberg, Germany and Hadassah Medical Center in Jerusalem, Israel) were conducted to enable model training on retrospective data. The prospective technology trials in the same three centers were prepared.
The framework for an economic modelling with a budget-impact analysis was planned and will be concluded soon. The treatment alternatives of "acute ischemic stroke" treatment with and without a VALIDATE decision support tool were drafted to identify instances and measures where the budget-impact analysis can model the economic and potentially socio-economic impacts if the VALIDATE solutions were scaled up after project end. Initial exploitable assets per beneficiary and the key expoitable project results were identified in a dedicated consortium workshop and surveys.
A series of patient information videos on stroke was developed by NORA and patient representatives via the beneficiary SAFE were involved in the process of tailoring the NORA app to the patients needs in VALIDATE.
The technological requirements for the VALIDATE clinical decision support tool were defined and together with the ethical requirements mappes to the several cycles of AI development to guide the developments, validation and preparation of the AI tools for operation in three pilot trials. The models were refined under application of the CRISP-ML(Q) standard and methodology (Cross-Industry Standard Process for the development of Machine Learning applications with Quality assurance methodology). After data harmonization, the retrospective training data from the three pilot sites enabled adaptions of the AI models.
The technical requirements for the VALIDATE app (a clinical decision support tool for actute ischemic stroke treatment) and its architecture were defined using the IBM Unified Method Framework. The patient immediate care pathway was defined to identify pain-points and a value-tree for users and beneficiaries of the tool. In compliane with ethics requirements, the tool was developed such that it works on local laptops or hospital servers and without internet connection.
A 'study-initiation package' was concluded, including the technology trial registration under study-ID NCT05622539 in clinicantrials.gov and in submitting the study protocol to the three local ethics boards. The NORA patient app has been adapted such that it enables patient follow-up on their PROMs (Patient Reported Outcome Measures) and Quality of Life measures. Ist was also translated from Spanish to English and German.
The retrospective technology trials in three clinical centers (Val d'Hebron Hospital in Barcelona, Spain; Universitätsklinik Heidelberg in Heidelberg, Germany and Hadassah Medical Center in Jerusalem, Israel) were conducted to enable model training on retrospective data. The prospective technology trials in the same three centers were prepared.
The framework for an economic modelling with a budget-impact analysis was planned and will be concluded soon. The treatment alternatives of "acute ischemic stroke" treatment with and without a VALIDATE decision support tool were drafted to identify instances and measures where the budget-impact analysis can model the economic and potentially socio-economic impacts if the VALIDATE solutions were scaled up after project end. Initial exploitable assets per beneficiary and the key expoitable project results were identified in a dedicated consortium workshop and surveys.
A series of patient information videos on stroke was developed by NORA and patient representatives via the beneficiary SAFE were involved in the process of tailoring the NORA app to the patients needs in VALIDATE.
VALIDATE extends beyond the current state-of-the-art on three levels in
1) Improving patient outcome by advancing clinical treatment,
2) expanding the limits of AI development and application methods,
and 3) developing, testing and integrating AI-based clinical solutions for future up-scaling.
1) Improving patient outcome by advancing clinical treatment,
2) expanding the limits of AI development and application methods,
and 3) developing, testing and integrating AI-based clinical solutions for future up-scaling.