Periodic Reporting for period 1 - ASSESS-DHT (ASSESS-DHT: DEVELOPMENT AND HARMONISATION OF METHODOLOGIES FOR ASSESSING DIGITAL HEALTH TECHNOLOGIES IN EUROPE)
Reporting period: 2024-01-01 to 2025-06-30
The overarching objective of ASSESS DHT is to consolidate and streamline existing methods and tools for approval of innovative DHT and develop a new fit-for-purpose generic health technology assessment (HTA) framework of Digital Health Technologies (DHTs), harmonised for Europe and aligned with key regulations like the European Health Data Space (EHDS).
This is achieved as follows:
(1) investigate and incorporate assessment criteria on privacy, cybersecurity, data storage and handling, interoperability, usability, etc.
(2) incorporate a phased adoption path through this framework, utilise methods and experiences of the German DiGA, French PECAN models, include industry experience, exemplified for digital therapeutic DHT (digiceuticals)
(3) deep dive framework with criteria and methodology suitable for iterative assessment of dynamic learning/improving DHT, including their optimal timings
(4) develop a holistic assessment layer on top of the generic framework for evidencing and assessing the value of digitally transformed care pathways underpinned by DHT
(5) engage with key stakeholders to ensure uptake of relevant views and inputs for developing and later validating the generic framework and other components in the project
(6) establish an open access European repository for collecting and sharing evaluation methods, studies, results.
The projects pathway to impact is in four pillars
Pillar 1: A consolidation and preparation pillar that collates and synthesises the existing assessment frameworks and tools for HTA of DHTs, state-of-the-art findings from initiatives, projects, relevant standards and regulatory inputs.
Pillar 2: Toolkit development is the innovation powerhouse of the project, constructing a new assessment framework with pathways for special case DHT and fast track, published for consultation and for road-testing by our case studies and updated with feedback, to be externally published as an overarching manual for DHT assessment accompanied by guidance and checklist tools.
Pillar 3: A testing and validation pillar that will undertake deep dive road-testing of our initial framework for HTA of DHTs with chosen software developers to exemplify DHT challenge areas of interest and importance across Europe.
Pillar 4: An uptake and sustainability pillar that engages with external stakeholders, initially as requirements holders to focus attention on important challenges, later to review proposed framework, case study experience, and finally endorse/ promote the new framework across Europe.
This is achieved as follows:
(1) investigate and incorporate assessment criteria on privacy, cybersecurity, data storage and handling, interoperability, usability, etc.
(2) incorporate a phased adoption path through this framework, utilise methods and experiences of the German DiGA, French PECAN models, include industry experience, exemplified for digital therapeutic DHT (digiceuticals)
(3) deep dive framework with criteria and methodology suitable for iterative assessment of dynamic learning/improving DHT, including their optimal timings
(4) develop a holistic assessment layer on top of the generic framework for evidencing and assessing the value of digitally transformed care pathways underpinned by DHT
(5) engage with key stakeholders to ensure uptake of relevant views and inputs for developing and later validating the generic framework and other components in the project
(6) establish an open access European repository for collecting and sharing evaluation methods, studies, results.
The projects pathway to impact is in four pillars
Pillar 1: A consolidation and preparation pillar that collates and synthesises the existing assessment frameworks and tools for HTA of DHTs, state-of-the-art findings from initiatives, projects, relevant standards and regulatory inputs.
Pillar 2: Toolkit development is the innovation powerhouse of the project, constructing a new assessment framework with pathways for special case DHT and fast track, published for consultation and for road-testing by our case studies and updated with feedback, to be externally published as an overarching manual for DHT assessment accompanied by guidance and checklist tools.
Pillar 3: A testing and validation pillar that will undertake deep dive road-testing of our initial framework for HTA of DHTs with chosen software developers to exemplify DHT challenge areas of interest and importance across Europe.
Pillar 4: An uptake and sustainability pillar that engages with external stakeholders, initially as requirements holders to focus attention on important challenges, later to review proposed framework, case study experience, and finally endorse/ promote the new framework across Europe.
Main achievements in the first 18 months are
(a) the first version of Taxonomy of DHTs for Assessment Purposes, enabling us to further develop assessment requirements for assessing DHT. Based on different methodologies (survey HTA agencies, scoping review), this taxonomy presents a conceptual model for the landscape of DHT.
(b) A landscape and gap analysis with an overview of existing assessment frameworks for DHTs. Based on different methodologies (scoping review and survey HTA agencies) existing frameworks and methods for the assessment of DHT were analysed.
(c) ASSESS DHT provides a framework for ensuring data protection and regulatory compliance in digital health technologies. It aligns with GDPR, the AI Act, and MDR, establishing criteria for health data protection in DHTs assessments.
(d) A further achievement is where the project addresses the complexity of data quality in the life cycle, focusing on how DHTs conduct and govern assessments. It provides evidence-based guidance to enhance transparency, mitigate biases, and ensure robust, trustworthy DHTs, particularly in AI-driven applications.
(e) A comprehensive checklist to support developers and HTA agencies in assessing the cybersecurity of digital health devices was developed. Its utility and effectiveness will be evaluated through case studies in the project, with refinements based on user feedback.
(f) The project developed a guideline for external validation of Artificial Intelligence-based Clinical Decision Support Systems (AI-CDSS) using local real-world data (RWD) at the pre-deployment, post-regulatory approval stage (VALID-AI - VAlidation of Local Indicators in the pre-Deployment phase of AI-CDSS).
(g) An interim version of the ASSESS-DHT manual collates work across the Work Packages and provides guidance and tools on DHT challenge areas to support piloting activities. The manual provides guidance on conducting assessments of diverse DHTs at different stages of their life cycle. Areas included are traditionally assessed in HTA such as safety, effectiveness and cost-effectiveness and specialist topic considerations such as cybersecurity, AI, data protection and data quality.
(h) Specifications of the repository were defined that serves as a central hub for HTA methodologies and evidence, supporting DHT developers, policymakers, and HTA bodies in making evidence-based decisions while ensuring compliance with evolving regulatory frameworks.
(a) the first version of Taxonomy of DHTs for Assessment Purposes, enabling us to further develop assessment requirements for assessing DHT. Based on different methodologies (survey HTA agencies, scoping review), this taxonomy presents a conceptual model for the landscape of DHT.
(b) A landscape and gap analysis with an overview of existing assessment frameworks for DHTs. Based on different methodologies (scoping review and survey HTA agencies) existing frameworks and methods for the assessment of DHT were analysed.
(c) ASSESS DHT provides a framework for ensuring data protection and regulatory compliance in digital health technologies. It aligns with GDPR, the AI Act, and MDR, establishing criteria for health data protection in DHTs assessments.
(d) A further achievement is where the project addresses the complexity of data quality in the life cycle, focusing on how DHTs conduct and govern assessments. It provides evidence-based guidance to enhance transparency, mitigate biases, and ensure robust, trustworthy DHTs, particularly in AI-driven applications.
(e) A comprehensive checklist to support developers and HTA agencies in assessing the cybersecurity of digital health devices was developed. Its utility and effectiveness will be evaluated through case studies in the project, with refinements based on user feedback.
(f) The project developed a guideline for external validation of Artificial Intelligence-based Clinical Decision Support Systems (AI-CDSS) using local real-world data (RWD) at the pre-deployment, post-regulatory approval stage (VALID-AI - VAlidation of Local Indicators in the pre-Deployment phase of AI-CDSS).
(g) An interim version of the ASSESS-DHT manual collates work across the Work Packages and provides guidance and tools on DHT challenge areas to support piloting activities. The manual provides guidance on conducting assessments of diverse DHTs at different stages of their life cycle. Areas included are traditionally assessed in HTA such as safety, effectiveness and cost-effectiveness and specialist topic considerations such as cybersecurity, AI, data protection and data quality.
(h) Specifications of the repository were defined that serves as a central hub for HTA methodologies and evidence, supporting DHT developers, policymakers, and HTA bodies in making evidence-based decisions while ensuring compliance with evolving regulatory frameworks.
The project presents key Results and Potential Impacts in the form of Deliverables and Achievements. Eight project objectives have been pursued, In the process more than 60 frameworks analysed, 23 HTAs surveyed, identifying major gaps in consistency and expertise for DHT assessment.
Impact Trajectory: KPIs target reduced citizen access time to certified DHTs, improved cross-border recognition of HTA decisions, increased Return on investment for SMEs, and broad regulatory acceptance.
Ongoing WP5 case study piloting of the framework and tools with diverse DHTs and use cases (including AI-driven and telehealth applications) is needed to validate the methodologies and fine-tune evidence generation across different technology types and clinical settings.
Demonstration and Real-World Validation: attempted via the Usability testing in WP5 T5.5. where real-world testing/implementation, especially of the tools developed will be critical to prove impact and guide iterative framework and toolkit refinement.
Commercialisation and Support: Support mechanisms for developer access—through the evidence repository, compliance tools and the related guidance, a platform for sustained engagement with industry to build predictable market and reimbursement systems can be built.
Sustained Outreach: Ongoing dissemination, outreach, usability testing, and policy briefs targeting all stakeholder groups are required for awareness, trust, and transparency.
Impact Trajectory: KPIs target reduced citizen access time to certified DHTs, improved cross-border recognition of HTA decisions, increased Return on investment for SMEs, and broad regulatory acceptance.
Ongoing WP5 case study piloting of the framework and tools with diverse DHTs and use cases (including AI-driven and telehealth applications) is needed to validate the methodologies and fine-tune evidence generation across different technology types and clinical settings.
Demonstration and Real-World Validation: attempted via the Usability testing in WP5 T5.5. where real-world testing/implementation, especially of the tools developed will be critical to prove impact and guide iterative framework and toolkit refinement.
Commercialisation and Support: Support mechanisms for developer access—through the evidence repository, compliance tools and the related guidance, a platform for sustained engagement with industry to build predictable market and reimbursement systems can be built.
Sustained Outreach: Ongoing dissemination, outreach, usability testing, and policy briefs targeting all stakeholder groups are required for awareness, trust, and transparency.