The project lasted 51 months divided into three phases. In the 1st phase, a novel concept of tailoring was developed based on existing literature and relevant theoretical underpinning, incl. Normalization Process Theory. The conceptual work formed the online ItFits-toolkit featuring an implementer-centred user-interface, flexible workflows, evidence-informed resources, working principles, methods, and a surveying tool. A study protocol and data collection system were developed, and local implementation teams were prepared to commence the trial in the 2nd project phase.
The main research question was twofold: (a) Does the ItFits-toolkit lead to better implementation outcomes than implementation-as-usual (IAU)? and (b) How do implementation teams engage with the ItFits-toolkit?
The primary outcome used for testing the effectiveness of the ItFits-toolkit was the degree of normalisation of the iCBT services as perceived by service delivery staff (i.e. therapists, referrers, IT developers, administrators). Uptake of iCBT (referred and completed treatments) and implementation efficiency were used in a secondary analysis of the effectiveness of the ItFits-toolkit on organisational level. A detailed process evaluation consisting of remotely conducted qualitative interviews (n = 55) and observations from support calls (n = 19) was conducted to study toolkit use.
Following a stepped-wedge trial design, twelve implementation sites in nine countries received access to the ItFits-toolkit in a randomised order. The trial lasted 30 months and ended in November 2020. A repeated measures design with 10 waves was chosen to assess change over time. The data were analysed in the final phase and the toolkit was improved and delivered to the community.
In total 39 implementers in 12 service delivery organisations used the toolkit to implement iCBT services. A total of 456 service delivery staff were included across the 12 organisations. A total of 7,191 patients were referred to the iCBT services. Of those, 20% received adequate exposure to iCBT and 73% ended their treatment prematurely.
The effectiveness study showed that tailored implementation by means of the ItFits-toolkit has a slight but significant positive effect on normalisation in service delivery staff (MItFits=0.09 SE=0.07 p=0.02 Cohen’s d=0.12). Surprisingly, usual implementation efforts show a slight negative normalisation trend in staff. On the longer term, the ItFits-toolkit has the potential to make up for the negative trend under IAU. On an organisational level, toolkit use is not associated with a statistically significant effect on uptake of iCBT, and the toolkit did not require statistically significant different investments in effort of staff in realising their implementation objectives. Implementers found the toolkit useful, shaping their perceived role, collaboration with others, and knowledge concerning barriers, facilitators, and implementation strategies.
The study design intentionally allowed high levels of variations in contextual factors in the implementation sites and many events might have influenced the implementations, incl. natural disasters, legal changes, organisational changes, and staff turnover.
In view of the mission to speed up implementation, our findings warrant modesty about the effectiveness of the specific conceptualisation and operationalisation of tailored implementation in the ItFits-toolkit. Normalisation processes can be enhanced by tailored implementation, but they remain slow with high levels of complexity in which stakeholders and contextual factors play a central role. Although we took a unique approach by integrating a process and methods for tailoring with tailored implementation strategies, these findings are in line with other studies on tailored implementation.
Scientifically, the consortium is expecting to consolidate on the lessons learned and data collected in the coming period. Both the quantitative and the qualitative data are of high quality and of higher density than anticipated. Next to the two main outcome papers (ItFits-toolkit effectiveness and process evaluation), we expect to use the rich data set, as well as the toolkit log-files, for e.g. national or institutional explorations.
