During the second period (months 19–36), VALIDATE strengthened its Trustworthy AI activities and advanced the toolchain from development into multicentre validation. The project completed a qualitative stakeholder study to evaluate operationalisation of the Trustworthy AI framework, conducting eight semi-structured interviews with international experts from academia, clinical practice, research ethics and AI governance. Additionally, the consortium completed the second audit cycle of the Trustworthy AI framework, moving the framework to a GitHub wiki and updating requirement tracking using a simplified traffic-light status system and improved traceability to development phases and related artefacts.
The VALIDATE AI models were further refined, building on Iteration 2, and a final model was integrated into the VALIDATE tool for prospective validation. A multi-step training approach was applied, updating a base dataset with VALIDATE site data, and an agreed validation scheme was implemented. The neural-network model was prepared for the prospective study, including fine-tuning across sites, development of a calibration methodology for realistic probabilities across mRS outcomes, and accompanying explainable AI analyses.
Retrospective validation was further strengthened through clinically relevant sub-group analyses and benchmarking against established prognostic tools, alongside robustness testing under complete, partially missing and partially imputed data conditions. The modelling framework was also extended to support additional model families including tree-based approaches and a lifecycle monitoring concept was developed to detect dataset/model shift and potential model staleness during prospective use. Work also progressed on FHIR/JSON-aligned interfaces to support future interoperability and integration pathways. The validation set was extended with >5,000 additional patients from five additional centres (France, Japan, USA) plus a national registry (The Netherlands), and preliminary results were presented at MICCAI and the European Stroke Organisation Conference.
The demonstrator was finalised as an iOS mobile app with a backend that includes the ML model and was delivered via TestFlight for a closed user group. For the validation study, the backend was deployed to laptops and delivered with managed smartphones running the app. Research assistants were trained and supported, intended use workshops were conducted to support MDR-aligned product definition, and usability/experience surveys including SUS were set up and initiated with research assistants and physicians. The prospective phase received ethics approvals at all three sites (UKH 20Dec2023; HMC 4Apr2024; VHIR 25Jun2024), and the prospective observational multicentre shadowing study began with real-time data captured via the app and predictions generated.
Work for the socioeconomic impact assessment was also advanced, with additional groundwork for the cost-effectiveness analysis, budget-impact modelling, clinical guideline uptake, regulatory pathways, and business sustainability, undertaken. These activities further clarified the value proposition for future stakeholders and supported translation of the validated AI risk scoring tool into subsequent trials.
