Periodic Reporting for period 3 - ImpleMentAll (Towards evidence-based tailored implementation strategies for eHealth)
Período documentado: 2020-01-01 hasta 2021-03-31
Effective clinical innovations are developed at unprecedented rates. However, only 14% of new evidence-based interventions enter routine practice and those who make it, only do so 17 years after establishing their clinical efficacy. Implementation, seen as a process in which people intend to bring into operation new or modified practices, takes place in a context and faces barriers that vary considerably between settings.
Mental health disorders account for an alarming proportion of the global burden of disease. Internet- based Cognitive Behavioural Therapy (iCBT) is an effective and efficient treatment for common mental health disorders. ICBT services are implemented across the world with varying success. ImpleMentAll examined the effectiveness of tailored implementation compared to usual implementation of iCBT in routine practice. ImpleMentAll pursued the following objectives:
1. To develop a generic Integrated Theory-based Framework for Intervention Tailoring Strategies (the ItFits-toolkit) for tailored implementation of evidence-based clinical eHealth services.
2. To demonstrate the impact of the ItFits-toolkit on implementing iCBT services for common mental disorders in 9 European countries and Australia.
3. To disseminate the validated ItFits-toolkit for tailored implementation across Europe.
The project had a total budget of € 7.1m with € 6m contributed by the European Commission through the Horizon 2020 program and approx. € 1.1m from the Australian government. The consortium united experts in clinical practice, clinical research, and implementation science from 16 organisations in Denmark, the Netherlands, United Kingdom, Germany, Spain, France, Italy, Kosovo, Albania, Belgium, and Australia
Mental health disorders account for an alarming proportion of the global burden of disease. Internet- based Cognitive Behavioural Therapy (iCBT) is an effective and efficient treatment for common mental health disorders. ICBT services are implemented across the world with varying success. ImpleMentAll examined the effectiveness of tailored implementation compared to usual implementation of iCBT in routine practice. ImpleMentAll pursued the following objectives:
1. To develop a generic Integrated Theory-based Framework for Intervention Tailoring Strategies (the ItFits-toolkit) for tailored implementation of evidence-based clinical eHealth services.
2. To demonstrate the impact of the ItFits-toolkit on implementing iCBT services for common mental disorders in 9 European countries and Australia.
3. To disseminate the validated ItFits-toolkit for tailored implementation across Europe.
The project had a total budget of € 7.1m with € 6m contributed by the European Commission through the Horizon 2020 program and approx. € 1.1m from the Australian government. The consortium united experts in clinical practice, clinical research, and implementation science from 16 organisations in Denmark, the Netherlands, United Kingdom, Germany, Spain, France, Italy, Kosovo, Albania, Belgium, and Australia
The project lasted 51 months divided into three phases. In the 1st phase, a novel concept of tailoring was developed based on existing literature and relevant theoretical underpinning, incl. Normalization Process Theory. The conceptual work formed the online ItFits-toolkit featuring an implementer-centred user-interface, flexible workflows, evidence-informed resources, working principles, methods, and a surveying tool. A study protocol and data collection system were developed, and local implementation teams were prepared to commence the trial in the 2nd project phase.
The main research question was twofold: (a) Does the ItFits-toolkit lead to better implementation outcomes than implementation-as-usual (IAU)? and (b) How do implementation teams engage with the ItFits-toolkit?
The primary outcome used for testing the effectiveness of the ItFits-toolkit was the degree of normalisation of the iCBT services as perceived by service delivery staff (i.e. therapists, referrers, IT developers, administrators). Uptake of iCBT (referred and completed treatments) and implementation efficiency were used in a secondary analysis of the effectiveness of the ItFits-toolkit on organisational level. A detailed process evaluation consisting of remotely conducted qualitative interviews (n = 55) and observations from support calls (n = 19) was conducted to study toolkit use.
Following a stepped-wedge trial design, twelve implementation sites in nine countries received access to the ItFits-toolkit in a randomised order. The trial lasted 30 months and ended in November 2020. A repeated measures design with 10 waves was chosen to assess change over time. The data were analysed in the final phase and the toolkit was improved and delivered to the community.
In total 39 implementers in 12 service delivery organisations used the toolkit to implement iCBT services. A total of 456 service delivery staff were included across the 12 organisations. A total of 7,191 patients were referred to the iCBT services. Of those, 20% received adequate exposure to iCBT and 73% ended their treatment prematurely.
The effectiveness study showed that tailored implementation by means of the ItFits-toolkit has a slight but significant positive effect on normalisation in service delivery staff (MItFits=0.09 SE=0.07 p=0.02 Cohen’s d=0.12). Surprisingly, usual implementation efforts show a slight negative normalisation trend in staff. On the longer term, the ItFits-toolkit has the potential to make up for the negative trend under IAU. On an organisational level, toolkit use is not associated with a statistically significant effect on uptake of iCBT, and the toolkit did not require statistically significant different investments in effort of staff in realising their implementation objectives. Implementers found the toolkit useful, shaping their perceived role, collaboration with others, and knowledge concerning barriers, facilitators, and implementation strategies.
The study design intentionally allowed high levels of variations in contextual factors in the implementation sites and many events might have influenced the implementations, incl. natural disasters, legal changes, organisational changes, and staff turnover.
In view of the mission to speed up implementation, our findings warrant modesty about the effectiveness of the specific conceptualisation and operationalisation of tailored implementation in the ItFits-toolkit. Normalisation processes can be enhanced by tailored implementation, but they remain slow with high levels of complexity in which stakeholders and contextual factors play a central role. Although we took a unique approach by integrating a process and methods for tailoring with tailored implementation strategies, these findings are in line with other studies on tailored implementation.
Scientifically, the consortium is expecting to consolidate on the lessons learned and data collected in the coming period. Both the quantitative and the qualitative data are of high quality and of higher density than anticipated. Next to the two main outcome papers (ItFits-toolkit effectiveness and process evaluation), we expect to use the rich data set, as well as the toolkit log-files, for e.g. national or institutional explorations.
The main research question was twofold: (a) Does the ItFits-toolkit lead to better implementation outcomes than implementation-as-usual (IAU)? and (b) How do implementation teams engage with the ItFits-toolkit?
The primary outcome used for testing the effectiveness of the ItFits-toolkit was the degree of normalisation of the iCBT services as perceived by service delivery staff (i.e. therapists, referrers, IT developers, administrators). Uptake of iCBT (referred and completed treatments) and implementation efficiency were used in a secondary analysis of the effectiveness of the ItFits-toolkit on organisational level. A detailed process evaluation consisting of remotely conducted qualitative interviews (n = 55) and observations from support calls (n = 19) was conducted to study toolkit use.
Following a stepped-wedge trial design, twelve implementation sites in nine countries received access to the ItFits-toolkit in a randomised order. The trial lasted 30 months and ended in November 2020. A repeated measures design with 10 waves was chosen to assess change over time. The data were analysed in the final phase and the toolkit was improved and delivered to the community.
In total 39 implementers in 12 service delivery organisations used the toolkit to implement iCBT services. A total of 456 service delivery staff were included across the 12 organisations. A total of 7,191 patients were referred to the iCBT services. Of those, 20% received adequate exposure to iCBT and 73% ended their treatment prematurely.
The effectiveness study showed that tailored implementation by means of the ItFits-toolkit has a slight but significant positive effect on normalisation in service delivery staff (MItFits=0.09 SE=0.07 p=0.02 Cohen’s d=0.12). Surprisingly, usual implementation efforts show a slight negative normalisation trend in staff. On the longer term, the ItFits-toolkit has the potential to make up for the negative trend under IAU. On an organisational level, toolkit use is not associated with a statistically significant effect on uptake of iCBT, and the toolkit did not require statistically significant different investments in effort of staff in realising their implementation objectives. Implementers found the toolkit useful, shaping their perceived role, collaboration with others, and knowledge concerning barriers, facilitators, and implementation strategies.
The study design intentionally allowed high levels of variations in contextual factors in the implementation sites and many events might have influenced the implementations, incl. natural disasters, legal changes, organisational changes, and staff turnover.
In view of the mission to speed up implementation, our findings warrant modesty about the effectiveness of the specific conceptualisation and operationalisation of tailored implementation in the ItFits-toolkit. Normalisation processes can be enhanced by tailored implementation, but they remain slow with high levels of complexity in which stakeholders and contextual factors play a central role. Although we took a unique approach by integrating a process and methods for tailoring with tailored implementation strategies, these findings are in line with other studies on tailored implementation.
Scientifically, the consortium is expecting to consolidate on the lessons learned and data collected in the coming period. Both the quantitative and the qualitative data are of high quality and of higher density than anticipated. Next to the two main outcome papers (ItFits-toolkit effectiveness and process evaluation), we expect to use the rich data set, as well as the toolkit log-files, for e.g. national or institutional explorations.
The main project result is a conceptually and technically innovative ItFits-toolkit facilitating tailored implementation. This is a significant contribution to improving the current state of affairs in implementing evidence-based eHealth interventions in routine practice and its scientific study. The ItFits-toolkit includes evidence-informed materials, e.g. iCBT-relevant determinants of practice and implementation strategies, as well as methods for stakeholder engagement.
During the project, iCBT became accepted and integrated in regular care, and a significant proportion of the persons treated would not have received adequate treatment if not for ImpleMentAll. Without having a comparator (doing nothing), the execution of the project and the emphasis on implementation processes is expected to have had an influence. The ItFits-toolkit and the project as a whole has facilitated that process to the extent that even during the project, other organisations showed interest in using the ItFits-toolkit for implementation purposes and further improving it. Interested organisations received training and signed memoranda of understanding expressing their intentions to use the toolkit for future research and implementation projects.
The toolkit is available at www.itfits-toolkit.com.
Find more information at www.implementall.eu
During the project, iCBT became accepted and integrated in regular care, and a significant proportion of the persons treated would not have received adequate treatment if not for ImpleMentAll. Without having a comparator (doing nothing), the execution of the project and the emphasis on implementation processes is expected to have had an influence. The ItFits-toolkit and the project as a whole has facilitated that process to the extent that even during the project, other organisations showed interest in using the ItFits-toolkit for implementation purposes and further improving it. Interested organisations received training and signed memoranda of understanding expressing their intentions to use the toolkit for future research and implementation projects.
The toolkit is available at www.itfits-toolkit.com.
Find more information at www.implementall.eu