Final Report Summary - EIS (Development of a European Implementation Score for measuring implementation of research into healthcare practice using vascular disease as an example)
Executive Summary:
*Key messages:
The problem: Assessing the implementation of research evidence into practice.
This collaborative project set out to develop a European methodology to assess the implementation of research evidence into practice (the European Implementation Score (EIS)), in primary, secondary and specialist care from the perspectives of different target groups (users and carers, voluntary organisations, range of health and social care professionals and health policy makers).
Policy options: General recommendations for measuring effective implementation of research results were developed from the perspectives of health care professionals and health policy, hence widening the theoretical framework for effective implementation from a focus on patients and professionals alone.
* The problem: assessing the implementation of research evidence into practice
The European Implementation Score (EIS) aimed to measure how well new knowledge is implemented into clinical practice in Europe. The EIS addressed implementation of research knowledge at different levels of the health care system (micro-, meso- and macro-level) and in different health care settings (e.g. primary care, hospital, specialised care).
- Focus on stroke
The focus of the project was stroke because of the emerging new evidence of effective new treatments available and because of the national initiatives and governmental policies in this area. We tested the transferability of the developed methods using coronary heart disease as another vascular disease example
- Methods
To develop the project objectives, 8 partners and data from 14 stroke registers and 2 coronary heart disease registers were utilised. A range of methods were used including: literature reviews on implementation, evidence based stroke care and user involvement; multinational surveys; consensus methods for deriving performance indicators; ethnographic methods, statistical modeling of data, including health economic analysis.
- Findings
There are variations in aspects of stroke policy through to patient care across the study countries with half the countries having regulation of care and incentives to practice evidence-based stroke care. All countries have guidelines and national audits in six of the countries. Interviews with 125 informants identified that implementing evidence-based care is easier in the acute period after stroke and highlighted the need for strategic level leadership and collaboration. The EIS score was developed, with weightings to assess policy level and patient level factors that might influence performance. The performance measures represented all aspects of care with representation from 16 countries in a consensus meeting, the results being adopted by the European Stroke Association.
400,000 patients from the national stroke audits of EIS were used to model the relationship between components of the score and performance and there was no homogenous association between the total score or components on performance.
Modeling in 6 European populations datasets demonstrated that the introduction of ‘best practice’ would lead to reductions in mortality and was cost effective.
There is evidence of an EU drive to improve patient participation but little evidence in the area of research, but some good examples were identified. A survey of stroke survivors indicated patients are not well informed about research.
- Transferability to coronary heart disease
When applying the score to coronary heart disease in three countries, the higher the score the greater the improvement in care over time. A comparative effectiveness study of acute myocardial infarction in Sweden and UK showed clinically important differences in uptake of effective treatments and outcomes not explained by known confounders.
* Policy options
General recommendations for measuring effective implementation of research results were developed from the perspectives of health care professionals and health policy, hence widening the theoretical framework for effective implementation from a focus on patients and professionals alone. The EIS benchmarked the current status of implementation of research results in different health care settings at different levels of the health care system using population registers and national audit data.
- Policy option 1: for Health Policy Makers
For health policy makers the concept is complex and the notion of a simple numeric score is not realistic. However the EIS project has identified specific evidence-based factors which should be considered by policy makers when addressing the uptake of new interventions and these do vary widely between EU states.
At the macro-level, we have identified the following key components of the EIS:
1. Input: National Policy (guidelines/strategy),
2. Context: Economic structure and financial incentives,
3. Output: National audit/registry data.
- Policy option 2: for Health Care Professionals
For the Health Care Professionals, a set of criteria has been developed that will facilitate implementation locally and should be considered as part of quality assurance.
At the micro-/meso-level, we have identified the following key components of the EIS:
1. Input: Educational strategies and individual training,
2. Leadership, staff relationships, ‘ownership’, user involvement,
3. Output: Institutional and individual level feedback on performance.
* Implementation considerations
The EIS project has demonstrated that an EU approach to implementing best evidence for stroke is supported by the Stroke Alliance of user groups in different countries and should be considered a facet to implementation plans.
Conclusion
The project has developed methods to identify components of an Implementation Score and generated results that indicate variable quality of stroke care in European populations and a complex relationship between the EIS and stroke performance indicators overall and at the level of policy and individual patient care.
Project Context and Objectives:
Overall aims and objectives of the project
Using a novel theoretical framework for the implementation of research into practice, we aimed to develop an implementation score for stroke and to identify its predictive components for successful diffusion in order to identify areas for improving quality of care in different European countries.
This European Implementation Score (EIS) is intended measure how well new knowledge is implemented into clinical practice in Europe. The EIS will address implementation of research knowledge at different levels of the health care system (micro-, meso- and macro-level) and in different health care settings (e.g. primary care, hospital, specialised care).
General recommendations for measuring effective implementation of research results have been developed from the perspectives of users, health care professionals and health policy, hence widening the theoretical framework for effective implementation from a focus on patients and professionals alone. The different components of the EIS have been refined according to their ability to predict successful implementation of evidence based stroke care in different settings in Europe.
The identification of factors determining effective implementation was data driven, analysing available data from national audits and population-based registers. The EIS benchmarked the current status of implementation of research results in different health care settings at different levels of the health care system and this will inform health policy of a possible set of processes required for closing the research/practice gap. The focus of the project was in stroke because of the emerging new evidence of effective new treatments available and because of the national initiatives and governmental policies in this area. We tested the transferability of the develop methods using coronary heart disease as another vascular disease example.
Settings and multidisciplinary expertise
This project combines expertise from the fields of epidemiology, health services research including implementation research, health policy, clinical stroke care, health economics as well as user groups. Fourteen stroke registers and/or national audits in 9 European countries are participating.
Objectives
1. To coordinate and manage relevant European research expertise and data to address the project objectives (Work Package (WP) 1)
2. To determine currently used methods of implementation of research evidence into practice in different European settings (WP 2, Milestone 1)
3. To develop the European Implementation Score (EIS) to estimate the degree of implementation of research evidence into practice (WP 3, Milestone 3 and Milestone 5)
4. To define performance measures for successful implementation of research evidence into practice (WP 4, Milestone 2)
5. To identify implementation methods determining the successful implementation of research evidence into practice (WP 5, Milestone 4)
6. To model cost effectiveness of different implementation recommendations (WP 6, Milestone 6)
7. To validate the transferability of developed methods to coronary heart disease (WP 7, Milestone 7)
8. To understand the role of users in implementing evidence into practice (WP 8, Milestone 8)
9. To develop recommendations for closing the research-practice gap (WP 9)
Overall strategy of the work plan
Over a four-year period this collaborative project developed a European methodology to assess implementation of research evidence into practice in primary, secondary and specialist care from the perspectives of different target groups (users and carers, voluntary organisations, range of health and social care professionals and health policy makers) using stroke care as an example. For this purpose we developed the European Implementation Score (EIS). The success and timelines of the different components of the EIS were assessed in a range of European settings. We defined European performance measures to monitor the effectiveness of stroke care. The transferability of the developed methodologies to other vascular disease conditions was assessed in coronary heart disease.
The strategy for this ambitious, yet achievable, project is predicated on a truly multidisciplinary research programme involving users of care and international experts executing Work Packages with specific milestones across Europe and with coordination support from the centre in London. All collaborating centres, methodology experts and data collection sub-contractors are involved in Work Packages and will be managed by the EIS Management Board with representation from the Work Packages.
The strategy is to harness the unique opportunity afforded in Europe at this point in time. We plan to utilize the momentum of: emerging research evidence, guideline production across Europe, national audits of stroke care, governmental policy development for stroke, the experience gained in the field of coronary heart disease to develop tools for implementation of evidence into European practice.
The following national audits, population based registers and methodological experts are involved in this project as participants or subcontractors:
National audits or quality of care organisations at national/ regional level
• Belgium: Quality Register of the Flemish Hospital Network of the KU Leuven (Vincent Thijs)
• Germany: German Stroke Register Study Group (Peter Hermanek)
• Poland: POLKARD (Danuta Ryglewicz)
• Spain: Catalan Stroke Audit (Sonia Abilleira)
• Scotland: Scottish Stroke Care Audit (Martin Dennis)
• Sweden: RIKS-Stroke (Bo Norrving)
• UK: National Sentinel Audit of Stroke (Anthony Rudd)
Population-based stroke registers
• France: Dijon Stroke Register (Maurice Giroud)
• Germany: Erlangen Stroke Register (Peter Kolominsky-Rabas)
• Italy: Sesto Fiorentino Study (Domenico Inzitari, Antonio Di Carlo)
• Lithuania: Kaunas Register of Stroke (Daiva Rastenyte)
• Poland: Warsaw Stroke Register (Danuta Ryglewicz)
• Scotland: Glasgow Stroke Register (Peter Langhorne)
• UK: South London Stroke Register (Charles Wolfe, Chris McKevitt)
Additional methodological expertise to underpin all Work Packages
• WHO Steps Stroke Project (Thomas Truelsen)
• Cochrane Stroke Group (Peter Langhorne)
• Stroke Association UK (Jon Barrick)
• Stroke Association Germany (Markus Wagner)
• UK National Institute of Health Research Patient Safety and Service Quality Centre (Naomi Fulop, Annette Boaz)
• Nucleus Epidemiology and Public Health, European Association of Cardiovascular Prevention and Rehabilitation (Harry Hemingway)
Overall, 7 national audits and 7 population-based stroke registers from 9 European countries agreed to participate and to share data within the project. Many of scientists and clinicians participating in the project have worked together on three previous EU Stroke Projects, European Stroke Initiatives (e.g. Helsingborg consensus ), Cochrane collaborations or initiatives from the European Association of Cardiovascular Prevention and Rehabilitation.
Within the consortium experts from governmental and guideline organisations, health policy, social sciences, patient safety, quality management, epidemiology, public health, health services research, clinical medicine, WHO groups, Cochrane collaboration and patient and user groups are represented. The Work Packages are jointly delivered from different members of the consortium, selected for contribution to individual Work Packages according to their expertise in the respective field.
Work Packages (WP)
For fulfilling the overall aims and objectives of the project, the Work Plan uses a stepwise approach:
The successful implementation and completion of the project has been achieved through our management and coordination plans outline in WP 1. All WPs have been led by one of the participants together with a nominated WP Board.
Within WP 2, we aimed to determine currently used methods for implementing research into practice within the new theoretical framework we are proposing. This included reviewing the currently used methods at a national level in the participating European countries and how they cascade evidence to improve implementation of effective stroke care and how this is monitored at a national level.
Based on the current state of implementation practice in Europe, we developed in WP 3 the European Implementation Score (EIS) for assessing to what degree new knowledge is implemented in different European countries. The EIS was developed and considered implementation at different levels of the health care system (micro- [individual], meso- [social] and macro- [policy] level) and in different health care settings (primary care, hospital, and specialized stroke care). We also incorporated the role of users and patient organizations in the process of implementing evidence into practice in the different European countries (WP 8). The development required expertise from health policy, social sciences, as well as clinical and epidemiological input.
Within WP 4 we identified performance measures to monitor the process of implementation of established effective care in stroke based on best current guidance. Performance measures were defined in key areas of stroke care (e.g. acute stroke care, early rehabilitation, secondary prevention), based on standardized methods developed within a previous project in the Fifth Framework Programme (EROS), existing guideline initiatives in different European countries, and the Cochrane Stroke Group.
In WP 5 we identified the association between the different components of the EIS developed in WP 3 and the successful translation of evidence into practice. The success of implementation of research evidence into practice was defined by the performance measures developed in WP 4 and assessed using stroke datasets in Europe from national/regional audits and population based registers. For example, we investigated using appropriate statistical methods, the impact of the different implementation strategies on observed variations in processes (such as percentage of patients receiving thrombolytic therapy, percentage of patients with appropriate secondary prevention), structures (such as stroke unit implementation, multidisciplinary team approach) and outcome (such as death, impairment, activity, participation) in the available datasets. Based on the results from WP 5, the EIS will be refined and according to the abilities of its component in terms of predicting a successful implementation of research into practice.
The transferability of the methodology developed in the area of stroke on other vascular disease conditions will be investigated in WP 7. We used coronary heart disease as an example and replicated the feasibility of collecting information for the EIS in another disease condition. In addition, the predictive abilities of the EIS in coronary heart diseases was assessed. For these purposes, we established collaborations within existing structures of the European Association of Cardiovascular Prevention and Rehabilitation (EACPR) as a research platform. As one of the major Associations within the European Society of Cardiology, current membership 50.000 the EACPR is the only European forum tasked with advancing evidence generation, and implementation, in the field of secondary prevention. We exploited the active 11 member of the Nucleus of Epidemiology and Public Health in the EACPR, with one participant being the current chair (UCL) and one participant being a member (KCL), to further develop existing collaborations and to identify new ones.
In WP 9 recommendations for closing the research-practice gap have been developed based on results from all the Work Packages. The cost effectiveness of different methods of implementing evidence into practice were investigated in WP 6 and contributed to these recommendations. We also identified the costs of introducing optimal implementation strategies at different levels of the health care system. Recommendations for improving the implementation of evidence from research into clinical practice were formulated regarding which implementation strategies are effective at different levels of the health care system. In addition, successful methods for countries at different stages of the implementation process were identified.
The importance of involving health care users in the development of health services is widely recognised, although there is also a need for evidence of the effects of such involvement, as well as guidance about how best this might be achieved. Our project is underpinned by specific strategies to involve user and national stroke patient organisations. This is a novel element to the research paradigm that builds on user involvement in research and service delivery which has been developed over the last three years in the London centre. The user perspective was involved at all stages of the research study, co-ordinated in Work Packages. We maintained close cooperation with the European network of stroke patient groups (Stroke Alliance for Europe (SAFE)), facilitated by the German Stroke Foundation and The Stroke Association UK, an anticipated and a current board member of SAFE.
Project Results:
* Work Package 2: Determine currently used methods of implementation of research evidence into practice in Europe
The design of a new questionnaire evaluating methods used to translate research findings into practice in Europe was considered essential to gain a better understanding of implementation strategies in the field of stroke care. After an extensive review of scientific literature, different methods of implementation were evaluated and selected. The final questionnaire consisted of 12 sections, and included national and regional policies, financial incentives, educational strategies, audits and feedback, reminders, computerised decision support, opinion leaders, multiprofessional collaboration, multifaceted interventions. The contribution of stroke patient associations was also evaluated, together with performance indicators and barriers to evidence and guideline implementation. Implementation strategies were thereafter applied to the complex field of stroke prevention and care, aiming at different levels (national and regional) and settings (primary, hospital and specialist) of care. A methodology was also developed for the administration of the questionnaire. Face-to-face interviews were performed in the 10 European countries, and involved researchers, health professionals, members of governmental organisations and regulatory bodies, members of scientific societies and stroke patients associations.
Regulations having the force of law were present in just half of countries. The main aims were thrombolysis at a national level and networks of care, stroke units, prevention and care, quality certification, and evaluation of performances at regional level. Non-mandatory policies were present in almost all countries, and aimed mainly at stroke prevention and care, stroke guidelines and thrombolysis.
Public financial incentives for stroke care were reported in 7 countries at national level, while private financial incentives were reported just in 2 countries.
Among educational strategies, continuing professional education was mandatory for specialists in 8 countries, and for primary care and hospital physicians in 7 countries. At national level, clinical practice guidelines were distributed in all countries, free internet access to medical websites was provided in 8 countries, articles, reviews, and trials results were distributed in 7 countries, as well as journals, newsletters, bulletins. Electronic publications and stroke scales were distributed in 6 countries, and educational material on counselling in 5 countries. Conferences and courses were organised in all countries and in all settings of primary, hospital and specialist care, followed by large group meetings, small group meetings, lectures, rounds, case discussion, seminars and tutorials, organised in all countries and in almost all clinical settings. Educational outreach visits are face-to-face visits by a trained person to health professionals in their own setting. This implementation modality was reported in England and France both national and regional levels.
Educational campaigns to increase knowledge of stroke among the general population were present at national level in all countries. Radio, newspapers, leaflets and booklets were the media used in all countries; television, Internet and posters were used in 9; billboards in 7; Internet social networks in 5 and social networks in 4. Educational campaigns were aimed at knowledge and behaviour changes toward stroke risk factors, stroke as a medical emergency, requiring urgent help, and need to call an ambulance if a stroke is suspected. In almost all countries, campaigns promoted acknowledgment of early stroke symptoms, thrombolysis, and medical screenings.
Guidelines based on best current scientific evidence were available at national level in all countries, and at regional level in Belgium, France, Italy, Spain, and Sweden. In all countries, guidelines were developed by scientific committees. A guideline version for patients was available in England, France Germany, Scotland, Spain, and Sweden. The most commonly used strategies to implement guidelines were educational meetings, distribution of educational material, opinion leaders’ actions and interactive educational strategies. Internet was used in 60% of countries at national level, and in 40% at regional level, audit and feedback and local consensus processes were used in 40% of countries at national level, and in 60% at regional level, mass media in 40% of countries at both national and regional level.
Audits and feedbacks were present at national level, in England, France, Poland, Scotland, Spain and Sweden, whereas at regional level this implementation strategy was more diffuse (all countries, except Lithuania and Scotland). In 60% of countries, national audits and feedbacks were used to gain information on referrals to stroke unit, acute care, compliance with guidelines and stroke outcomes; in 50% of countries to gain information on specialist care and appropriate drug indications; in 40% of countries for primary prevention, innovations in practice and costs of stroke care. Nurse-physician collaboration was considered by 30% of countries.
Electronic reminders for stroke care were used by healthcare professionals at national level in Scotland and England, and written reminders only in England. At regional level, both electronic and written reminders were used only in Scotland. At national level, both countries used reminders for primary prevention and vascular risk assessment, prescribing drugs, disease management and secondary prevention.
Computerised decision support systems were used at national and regional levels only in England and Scotland. On a regional basis, they were also reported in Germany and Spain. At national level, computerised decision support systems were used for primary prevention, prescribing drugs and ordering tests and exams, administrative and clinical records, follow-up and secondary prevention, and guideline implementation.
The activity of opinion leaders to help implementation of research evidence in stroke care was reported in all countries at both national and regional levels. In all countries, opinion leaders were likely to be academics and healthcare professionals, followed by celebrity stroke survivors (60% of countries), patient associations representatives (50%), policymakers (40%), and healthcare administrators (20%). In most countries, opinion leaders’ activities included educational campaigns, formal and informal education, distribution of educational material.
Multiprofessional collaboration is meant to improve work interactions and processes between two or more types of healthcare professionals, including medical, nursing, therapy (physiotherapy, speech therapy, occupational therapy) staff, and social workers.
At national level, multiprofessional collaboration was present in 70% of EIS countries, and at regional level in 80% of EIS countries.
Multifaceted interventions are actions that combine different strategies of implementation:
At national level they were used in 70% of countries and at regional level in 80%. Strategies more frequently used included financial incentives and audits and feedbacks (80% of countries), distribution of printed and electronic educational material, educational meetings and workshops, guidelines production and dissemination (70% of countries), continuing medical education, national non-mandatory policies, educational campaigns, opinion leaders’ actions, and multiprofessional collaboration (60% of countries).
Stroke Patients’ Associations (SPA) were present at national level in all countries except Lithuania, and at regional level in 70% of countries. SPA promoted campaigns for stroke awareness, healthy lifestyles, psychological support/support groups for stroke survivors and carers (in 90% of countries), implementation of secondary prevention strategies (in 80% of countries), stroke prevention awareness strategies and establishment of stroke units (in 70% of countries), provision of rehabilitation to stroke patient (in 60% of countries). SPA actively offered services, such as information, advice and support services, communication support, reablement of social inclusion and carer support (in 70% of countries), and stroke prevention services (in 50% of countries). SPA contributed to stroke research, mainly in identifying priorities for research, dissemination and implementation of research findings (in 50% of countries). They resulted less involved in designing of research and interpretation of research findings (in 30% of countries), and in preparation of material used to ask for consent and inform stroke research subjects (in 20% of countries). SPA were also involved in planning strategies concerning implementation of stroke services, management of acute stroke, prevention, rehabilitation, evaluation of outcome and quality assessment (in 70% of countries).
Six countries (England, France, Germany, Poland, Scotland and Sweden) showed quite a similar pattern of use of performance measures at both national and regional levels. In Belgium, performance indicators were not used at regional level, and only one indicator was used at national level. In Italy, a systematic use of performance indicators was reported only at regional level. In Lithuania, only 3 quality indicators were reported at both national and regional levels.
Most frequently used indicators included death during hospital period, used in 90% of EIS countries at both national and regional level, and brain imaging, used in 80% of EIS countries at national and in 90% at regional level. Stroke unit care, carotid vessels imaging, thrombolytic therapy, and anticoagulants in patients with atrial fibrillation were reported at national level in 70% of countries.
Among quality indicators of outcome, death or disability at 1-3 months, or long-term death or disability, were both reported at national level only in France, Spain and Sweden. England reported only the use of death or disability at 1-3 months, and Scotland only the use of long-term death or disability. Results of cost-benefit analyses were reported only at national level, and only by England and France. Information to patients and relatives was a quality indicator at national level in England, France, Germany and Sweden. Patient satisfaction with services and quality of life measures were reported at national level only in France and Sweden.
At national level, among possible organisational/contextual barriers, organisational constraints were reported in 90% of countries, followed by lack of resources (in 80% of countries), unclear division of tasks, lack of time/time pressure and bureaucratic inertia (in 70% of countries), lack of collaboration among specialists, communication problems and lack of reimbursement (in 50% of countries). Among possible knowledge and attitude barriers, lack of knowledge among general physicians, lack of awareness and resistance to changes were reported in 80% of countries, followed by lack of motivation (in 60% of countries), lack of knowledge among hospital physicians and inefficient dissemination of guidelines (in 50% of countries). Lack of knowledge among specialists and lack of agreement with guideline interpretation were reported in 10% of countries.
* Work Package 3: Develop the EIS to estimate the degree of implementation of research evidence into practice
In the first twelve months of the project two separate literature reviews were undertaken. The aim was to assess the current state of knowledge around implementation of evidence based practice in healthcare in both the medical literature and the social science literature. The medical literature review was an update of a classic overview of systematic reviews and has subsequently been published. The review identified very few systematic reviews looking exclusively and explicitly at implementing research findings into practice; conversely 43 reviews either focused on the implementation of non-evidenced based findings or were not explicit about the nature of the findings and were thus excluded. The 13 studies included in this overview of systematic reviews highlights the small effects of single interventions such as audit and feedback, computerised decision support and opinion leaders. Multifaceted interventions are frequently used to promote the use of research in practice. Systematic reviews of multifaceted interventions claim an improvement in effectiveness over single interventions, with effect sizes ranging from small to moderate9.
For the social science literature review we drew up our own search strategy and searched for studies which included empirical analysis across social science databases from 2000-2010 to reflect the dates of the search of medical literature. We selected 41 papers overall. The majority of which were Anglophone; 13 came from the UK, 6 from the USA, 6 from Australia, 5 from Canada. There were 2 from the Netherlands, 1 from France, Spain, Italy, Norway, Sweden, Japan and Belgium respectively and 2 more which cut across different countries. Most of the studies used predominantly qualitative methodologies (25) others were quantitative (8) and the remainder used mixed methods of data analysis. We used the Dixon-Woods et al (2004) system to appraise the quality of these studies. In stark contrast to the medical literature which prizes high external validity across multiple settings to test methods of implementation delivery, the social science papers emphasise high internal validity relating to their findings.
We spent the following 12 months designing and performing a series of comparative international case studies across five separate sites in three different countries. Building on the insights gained from the medical literature review (recognised modalities of implementation) and the social science review (local contextual factors linked to epistemic, discursive and economic elements) which shed light on why and how informants construct stories about the ways implementation modalities ‘work’ or not, we designed a semi-structured interview schedule that we used to interview 125 informants representing the main clinical and managerial groups involved in stroke care policy in modern stroke care. Two of the key themes which emerged from the case study work are firstly; the importance of ‘normative integration’ and how implementation of EBHC is more successful at the acute phase of the stroke patients’ journey than following Stroke Unit (SU) discharge and secondly; the importance of strategic level leadership and collaboration between clinical and managerial communities.
We spent the next six months analysing the case study data alongside the incoming results from the WP 2 survey data, all the time interpreting both alongside the findings from the medical and social science literature reviews in an attempt to combine our understandings around the key implementation typologies and how they function at the macro-, meso- and micro-levels and which implementation typologies function as inputs (e.g. research dissemination), and which as outputs (e.g. performance data) whilst emphasising the active role played by context factors which may (or may not) aid this flow of knowledge in either direction. We wanted to show how knowledge may be conceived of flowing in a multi-directional, sometimes messy non-linear way: hence the variety of multi-directional arrows and the importance of different ‘forms’ of knowledge (e.g. research knowledge; institutional knowledge). This led to the initial version of the EIS (Fig. 1).
The early EIS (Fig 1) was based on the following implementation methods based primarily on the findings of the medical literature review combined with a number of contextual factors that came from our qualitative case study work: national and regional policy, financial incentives, educational strategies, audit and feedback, reminders, computerised decision support, opinion leaders, multiprofessional collaboration, multifaceted interventions, stroke patient associations, performance indicators, contextual factors. We initially struggled to account for macro-, meso- and micro-levels of analysis within one EIS, and at this point the macro-level elements predominated. This led to further work around ‘weighting the EIS’ in recognition that our early approach was insufficiently sophisticated to attain the data we were interested in and a particular interest in issues around quantification and relationships of and between the elements of the EIS. This work led to the final version of the EIS (Fig. 2).
The green ‘knowledge flow’ arrows represent the various levels and directions of travel of information which connect the multiple aspects of the model thus highlighting the relationships between the constituent elements of the model. The model suggests that knowledge flows in implementation are not linear – rather they are complex, iterative and dynamic. The key relationships here are shaped by human interaction (context) with information flows in two distinct settings; firstly the dissemination of research findings, and secondly the interpretation of performance data.
Macro-level
Our data suggests there are three key elements at the macro-level which may drive implementation;
Input: National Policy (guidelines/strategy)
Context: Economic structure & financial incentives
Output: National audit/registry data
Work Package 3 case study work highlighted the crucial macro-level role of a well publicised national, coordinated policy in relation to stroke care. National policies provide the guidelines to drive national educational goals and local policy decision-making. National policies reflect wider policy goals. These reflect funding priorities and institutional remuneration strategies, highlighting governmental commitments. Alongside this we see statements of intent regarding staffing levels and investment in research and organisational developments such as hyper-acute stroke unit (HASU), stroke unit (SU) care and the institution of specialist networks. At the macro-level, the key contextual factor relates to economic commitment. Requisite financing needs to be in place and furthermore this needs to be directed via organisational level incentives in order to encourage hospital management to prioritise stroke care along evidence based practice lines. The final key factor at the macro-level is successful measurement regimes (i.e. audit/registry data) which make performance visible and are comprehensive enough to paint a national/regional picture, yet simultaneously sensitive enough to highlight local areas of excellence and failure. The knowledge derived from this audit/registry data then flows back at the macro-level to policy makers who can reinterpret national policy goals in light of these findings and commissioners who can reward or penalise organisations in financial terms with the aim of improving organisational factors that impact on patient care. With reference to Fig 1; Knowledge flow 1 shows the relationship between the information contained in national policy/strategy and audit data. Knowledge flow 2 shows the flow of knowledge from audit data back towards national policy/strategy level. These insights shaped our hypothesis construction work with Work Package 5 around the predictive qualities of the EIS related to the effective implementation of new knowledge into practice at the macro-level.
Micro-/meso-levels
At these levels, our data suggest the following elements are key:
Input: Educational strategies & individual training
Context: Leadership, staff relationships, ‘ownership’ user involvement
Output: Institutional and individual level feedback on performance
At the level of micro - and meso-level delivery multi-faceted interventions involving educational strategies and robust formal and informal feedback on individual and organisational performance are the key ‘input’ and ‘output’ factors. In stroke care this needs to include a commitment to multi-professional collaboration. The key contextual factors are committed leadership coordinating and evaluating these processes, good staff relations and a culture of trust between clinical teams and management, and a sense of ‘ownership’ of these processes for staff. Also, a context in which patient and carer views are respected is one which is likely to reflect best care goals. Again, knowledge flows actively (knowledge flow 1) from research (input) to feedback (output) and back again reflectively (knowledge flow 2) so that feedback informs future education strategies. Significantly too, we suggest that (knowledge flow 3) research findings, disseminated (either formally via guidelines/strategies or informally via academic research publication) from the macro level are interpreted at the meso-level via the contextual factors listed above and that evaluative strategies reflect the efficacy of these findings and should flow back to the centre. Likewise (knowledge flow 4) suggests that individual feedback on performance should be linked in to national and institutional feedback in well led, high functioning teams. These insights shaped our hypothesis construction work around the predictive qualities of the EIS related to the effective implementation of new knowledge into practice at the micro-/meso-level with relation to our self-completion tool pilot work in Scotland and Germany.
* Work Package 4: Define performance measures for successful implementation of research evidence into practice
After a literature review Work Package 4 concluded that there are no clear distinctions between the terms performance measure and quality indicator (or simply indicator), and either term can be used.
Methodological requirements for quality indicators include the following issues:
• A quality indicator must be meaningful to patients and society, or be closely linked to such an outcome
• the measure must be valid and reliable
• the measure can be adjusted for patient variability
• the measure may be modified by improvements in the process of care
• it is feasible to measure the performance of healthcare providers. There should be a balance on practicality and costs/efforts
Work Package 4 have reviewed the literature of framework of indicators, and conclude that the framework of indicators should ideally represent all different aspects of care, and ideally also include patient-oriented outcome measures. Whereas different frameworks are complementary, the one developed by the American Heart Association and the American College of Cardiology has some advantages.
A preferred methodology for identification of quality indicators is to start with a systematic literature review of potential indicators, independent external evaluation, and a pilot study on the use of an indicator in clinical practice have yielded good transparency, acceptance and sustainability of indicators. Multiple stakeholders should be involved in the process. In registers starting directly with selection of a set of indicators, a structured approach of identifying potential indicators from guidelines has usually been the starting point, followed by a review and rating of different indicators from different aspects (cf section above on methodological requirements). The final selection has thereafter been made by a multi-stakeholder/multi-professional panel or through a Delphi process. However, even a standardized methodology to identify quality indicators may come up with different results. Differences in items may reflect that a subjective element of judgement cannot be excluded. Differences may also arise from differences in judgement on the extent of the registry that is feasible from practical/cost aspects. There are substantial differences between the domains of care currently monitored in the different registers and domains that ideally should be covered by a registry. Current joint data elements largely focus only on the early (hospital) phase of stroke care. There is a need to include also other domains of care that may be more challenging with respect to data collection.
European set of performance measures for effective implementation of research into practice in stroke care and prevention: The inaugural version of a European set of performance measures for effective implementation of research into practice in stroke care and prevention was prepared by the European consensus group with representatives from 16 countries at the meeting on April 19, 2011. The starting point was agreed to be the survey that had been conducted by Work Package 4 and Work Package 5, with a main focus on indicators that were present in the majority of the 7 registers studied. To this list were added items that the participants felt were missing from the list, based on therapies and actions with a strong evidence-based support.
Items were grouped according to the domains recommended by WP4: (1) coordination of care, (2) diagnosis, (3) preservation of neural tissue, (4) prevention of complications, (5) initiation of secondary prevention, (6) restoration of function, and (7) survival and functional outcome.
For different items a hierarchical scale was added when appropriate, with steps ranging from the easiest and basic items (1st tier) to more complex or demanding data collections (2nd tier(s)).
The meeting agreed on the following proposal of performance measures.
1 Coordination of care
Stroke unit care
1st tier Proportion treated at stroke unit
2nd tier Time (days) in stroke unit
2 Diagnosis
Brain imaging
1st tier Proportion of patients with stroke examined with CT scan or MRI
2nd tier Time to imaging
2nd tier*: Proportion of patients with TIA examined with CT scan or MRI
Carotid/vessel imaging
1st tier Proportion of patients with ischemic stroke examined with carotid/vessel imaging
2nd tier*: Proportion of patients with TIA examined with carotid/vessel imaging
Assessment by PT/OT
Proportion of patients with stroke seen or treated by appropriate members of the multidisciplinary team (response alternatives: yes, no, not indicated)
Cardiac arrhythmia detection
Proportion of patients with ischemic stroke investigated for paroxysmal AF*
3 Preservation of neural tissue
Thrombolytic therapy
1st tier Proportion of patients with ischemic stroke treated with intravenous thrombolysis
2nd tier Proportion of patients with ischemic stroke aged 18 to 80 years treated with intravenous thrombolysis
Time to thrombolytic therapy (door-to-needle time)
Mean value (minutes)
4 Prevention of complications
Proportion of patients with stroke screened with swallowing test
5 Initiation of secondary prevention
Antiplatelet therapy
1st tier Proportion of patients with ischemic stroke treated with antiplatelets at discharge
2nd tier Proportion of patients with ischemic stroke treated with antiplatelets at 48 hours after admission
Anticoagulants in patients in atrial fibrillation
1st tier Proportion of patients with ischemic stroke and atrial fibrillation treated with anticoagulants at discharge
2nd tier Proportion of patients with ischemic stroke and atrial fibrillation in whom treatment with anticoagulants is planned after discharge
2nd tier Proportion of patients with ischemic stroke and atrial fibrillation and no contraindication treated with anticoagulants at discharge
Statins
Proportions of patients with ischemic stroke discharged on statins
Blood pressure lowering
Proportions of patients with stroke discharged on blood pressure lowering
Carotid surgery
1st tier: proportion of patients with ischemic stroke subject to carotid surgery
2nd tier: proportion of patients with TIA subject to carotid surgery
Time to carotid surgery from vessel imaging
2nd tier: time from admission to carotid imaging
Anti smoking advice
Proportion of patients who were smokers on admission that was given anti-smoking advice as documented in medical records
6 Restoration of function
No item in the majority of the registers and no new proposals of items
7 Survival and functional outcome
Death at 30 days
Death or disability at 3 months
Proportion of patients stroke dead or dependent (mRS 3-5) at 3 months (break-down by stroke type ischemic/hemorrhagic, death, and disability)
Patient safety
Symptomatic intracerebral hemorrhage after intravenous thrombolytic therapy*
Back ground (base line) data
age
sex
stroke type
discharge destination
base line severity
1st tier LOC
2nd tier NIHSS, Scotish model, Catalonia model
* Work Package 5: Identify implementation methods determining successful implementation of research into practice
The main objective of the Work Package was to investigate the association between implementation methods and a successful transfer of research evidence into practice within different European regions in three phases with the main results of the three phases described below:
Phase 1: Combining information from existing national and regional stroke audits in Europe and outputs from Work Package 2 and Work Package 3 on currently used implementation methods
For assessing a potential association between different methods of implementing research evidence into practice on a macro-level and quality of acute hospital care provided, information from existing regional or national stroke audits was utilized. National or regional audits were identified by review of the literature and personal communication that fulfilled the following criteria: running in 2006; focus on monitoring and improving the quality of acute stroke care; and data collection not based on hospital administrative data only. Five national (German Stroke Registers Study Group [ADSR], Germany; the Scottish Stroke Care Audit [SSCA], Scotland; the National Stroke Register in Sweden [Riks-Stroke], Sweden; the National Sentinel Audit of Stroke, England/Wales/Northern-Ireland; and the Polish National Stroke Prevention and Treatment Registry [POLKARD], Poland) and two regional (the Quality Register of Flemish Hospital Network of the K.U.Leuven Flanders, Belgium; and the Catalan Stroke Audit, Catalonia, Spain) audits were identified that fulfilled the inclusion criteria.
For preparing the combination of information from participating audits across Europe, a comprehensive review of the characteristics of the participating audits was performed including standardized written surveys on: details on history, funding and development, patient characteristics; specification of data collection, data management and data quality; and development and selection process of quality indicators and quality standards. Information provided was validated by: interviews with audit representatives; official websites, public reports and scientific publications; and review by representatives of the respective audit. The outputs of this review identified substantial variations in measuring performance of acute stroke care in Europe. For example, overall 123 different indicators were identified that were used in the participating audits to measure performance of hospital in acute stroke care. Of those, only two indicators (brain imaging; anticoagulation in patients with atrial fibrillation) were used in all audits and 13 indicators in at least 2 of the audits. The results of this review also built the ground for informing a European consensus meeting on the development of common quality indicators across Europe organized by Work Package 4. In addition, the results were used for developing the methodology of pooling the datasets from the seven national or regional audits agreed to participate within the EIS project with details given below. Based on this experience, quality standards for developing and documenting quality indicators were proposed that shall be included in future initiatives for a standardized data collection across stroke registers.
Phase 2: Defining appropriate statistical methods for measuring the association between different implementation methods and a successful implementation of research into practice
As next step a theoretical framework for investigating the association between implementation methods and a successful transfer of research evidence into practice within different European regions was developed. General implementation methods were defined by the macro-level components of EIS derived by Work Package 3 and a successful implementation of research evidence into practice by indicators for measuring performance of acute care hospitals available in the participating national or regional audits with definitions delivered by Work Package 4. For example a potential association of the scoring of the countries within the EIS score component “policy” and the percentage of “ischemic stroke patients receiving tPA” was assessed. The scoring of regions or countries in the EIS macro-level components was assigned in cooperation with Work Package 2 and Work Package 3 representatives based on outputs of Work Package 2 survey and Work Package 3 case studies and was classified into three categories of implementation (low – medium – high). The performance of regions or countries in acute stroke care was determined by quality indicators based on Work Package 4 outputs. Data for calculating defined quality indicators were available in participating audits for the years 2007/ 2008, including e.g.: thrombolytic therapy; dysphagia screening; anticoagulants in patients with atrial fibrillation; and death at day 7.
A detailed statistical analysis plan was developed for assessing the predictive properties of the EIS components for quality of care documented in the participating audits by comparing the variance explained by the EIS components with the estimated variance by the country or region effect alone using hierarchical logistic regression models. The EIS components were included as distinct variables in the model. In addition, different ways of aggregating the EIS score components were defined a priori for informing the further development of the EIS Score by Work Package 3 including: unweighted aggregated score of each region/ country; weighted aggregation of each region/ country with weighting of importance of single components of EIS score by weights derived from Work Package 3 based on literature review, case study data, and expert opinion; and weighted aggregation of each region/ country with theoretical hypothesis derived from Work Package 3. The following steps for investigating predictive properties of the macro-level EIS Score were undertaken: exploring the kind of association between a quality indicator and the EIS score components; comparing the explained variance by the EIS components/ aggregated scores and country level; and defining a EIS component to be relevant if a higher variance was explained by the EIS compared to country level and clear type of association was observed.
Phase 3: Determining a potential association between macro-level implementation methods and a successful implementation of research into practice.
Within the first analytic step, data from seven national or regional audits comprising information from more than 400,000 patients in more than 1,000 hospitals in the years 2007 and/or 2008 were included for assessing a potential association between the EIS and quality of care provided. The main results of these analyses were summarized as follows: complex patterns were found in the available datasets on predictive properties of components of macro-level EIS and quality of acute care provided on a national or regional level; for a number of EIS macro-level score components variance explained was found to be greater compared to variance explained by country/ region alone; no clear trend between distinct components or between different ways of aggregating the EIS components (e.g. unweighted, weighted, theory driven) and indicators of quality of care could be revealed; for some EIS components (e.g. “reminders”) there was not enough variation across countries to produce reliable estimates; and no component of the macro-level EIS or different ways of aggregating the EIS were associated with defined indicators in all domains. In conclusion, no homogenous association between components of the macro-level EIS as well as between different ways of aggregating the EIS with quality of care provided was identified. These findings were extensively discussed with EIS consortium members and a number of potential explanations for these findings were identified, e.g. including: insufficient level of detail of information in datasets; potential selection of participating centres and heterogeneous ways of data collection; variations in level of implementation in regions across participating countries; limited information on micro- and meso-level data available (e.g. on hospital characteristics); contribution of reverse causality as information on quality of care and level of implementation were documented at a similar time period.
Based on these considerations, subsequent analyses of the dataset were planned. In addition to assess the impact of components of general implementation methods on quality of care provided also a potential association of key macro-level implementation events in a specific country or region (e.g. introduction of a new policy) with potential changes in stroke care over time was investigated. For the time trend analysis data from 5 national audits participating in the EIS project over an up to 6-year period between 2004 and 2009 were included comprising more than 900,000 patients from more than 1,400 hospitals. Based on variables documented in a comparable way in at least four of the five audits, information on changes over time for five different measures of quality of acute stroke care were provided. The year of specific key macro-level implementation events were identified by Work Package 3 for the countries of the participating audits. Most of the defined key events were around the components “policy” (e.g. implementation of a national stroke strategy or guideline development/ update) as well as “audit and feedback” (e.g. implementation of evidence based quality indicators or improvement of stroke registers). Within most of the audits a general trend towards a better quality of care over the study period was observed for specific indicators, e.g. for dysphagia screening, thrombolytic therapy and treatment on a stroke unit. However, for some of the investigated quality indicators (e.g. antithrombotic therapy) no clear patterns over time were documented. Similar to previous analysis on the predictive abilities of general macro-level implementation methods on quality of care, complex patterns were found regarding a potential link between key macro-level implementation events in a specific country or region and potential changes of quality of care provided over time. There was evidence that the definition of a specific performance measure within an audit activity might increase adherence to this measure over time. In general, it was not possible with the available information to link key events of macro-level implementation with changes in quality of care provided in all audits included in this analysis.
* Work Package 6: Model cost effectiveness of different implementation recommendations
Aim: To model cost effectiveness of different stroke treatment implementation recommendations
We examined the costs and effects of the current standards of stroke care across Europe and contrasted these costs and effects with those that would be incurred if best-practice care in the area of stroke was undertaken. An incremental cost-effectiveness model comparing best-practice with standard care was developed and modelled across a number of European countries. The optimal implementation strategies were partly identified through earlier deliverables, reported in WP5 and the cost-effectiveness of these strategies identified. The incremental costs are estimated from secondary data published literature. The incremental effects of implementation are estimated using data from a previous set of EC stroke registry data collected through the BIOMED EROS project, which were fully compatible with the EIS score developed in this EIS project.
Methodology
The cost-effectiveness of the stroke treatment implementation recommendations is assessed through an incremental cost-effectiveness model which compares best-practice stroke treatment, as identified by earlier Work Packages in this project, with the current standard of stroke care in a number of European countries. The result is expressed as the incremental cost per unit of effect attributable to the recommendations strategy (see Equation 1).
The fundamental method then is to attempt to measure the change in stroke outcomes through a predictive model. This allows identification of the change in treatment outcomes associated with the implementation of best-practice versus the outcomes associated with current treatment levels. This is done through changing the treatment input levels, as specified by and consistent with our implementation scores defined in WP5, at the individual level through the risk adjustment equation to quantify the effect on stroke outcomes. The methodological approach is suggested in Figure 3.
The first stage was to map implementation recommendations onto a set of services to be provided to a stroke patient. This gave an estimate of the resource use requirements if the strategy under investigation was to be implemented. A schematic example of the approach is given in Figure 3. This best-practice care was then defined in terms of treatment service provision. These treatment service provisions could then be identified within the registry databases, and the impact of their treatment effect on stroke outcomes (here defined solely in terms of mortality) was assessed through a predictive model.
Statistical Estimation
Our statistical plan was to model the impact of best-practice stoke treatments on outcomes was predicted from a risk-adjustment model. This was based on logit estimation with Equation 2.
Where different mortality indicators (M) are expressed as a function of patient characteristics (only age and sex shown in the equation here, but co-morbidities were also included in some of the empirical specifications), treatments and hospital characteristics (such as whether or not the hospital was a teaching hospital) Stroke type, baseline severity and discharge destination were also included in some regression runs. In some cases indicator variables relating to hospital characteristics were excluded and treatment centre fixed effects were entered instead.
The effectiveness of the implementation recommendations were then estimated from a predictive equation using the coefficients from the risk adjustment equation above.
This method allowed us to simulate the mortality outcomes associated with current treatment levels from the observed risk equations which can then be compared with the mortality outcomes associated with idealised treatment levels as gained from our predictions/simulations. By attaching treatment costs to best-practice care and standard treatment, this then allowed the estimation of an incremental cost-effectiveness of idealised treatment compared to observed treatment to be calculated.
The incremental cost-effectiveness of best-practice stroke treatment was run across our full European population and individually for 6 European countries. In all cases the introduction of best-practice in the treatment of stroke, as defined by our project, led to reductions in mortality and was shown to be highly cost-effective.
* Work Package 7: Validate the transferability of developed methods to coronary heart disease
The main results of the Work Package can be summarized as the following:
1. Transferability of EIS methodology developed in stroke is applicable to coronary heart disease
The conceptual framework of EIS methodology for successful implementation involves the coordination of three levels of health care (macro, meso, and micro) and three settings of health care (primary, hospital, and specialized care). Collaborating with other Work Packages (WP2, WP4), we identified the macro level in EIS framework as applicable to coronary disease, with ten key aspects in measuring evidence-based care implementation among European countries: national policy, financial incentive, education strategies, audit and feedback, reminders, computerised decision support, opinion leaders, multiprofessional collaboration, multifaceted interventions, and patient organization. These ten aspects can be assessed by the following means: (1) literature review, (2) country narrative, and (3) a structured questionnaire survey on key informants in coronary care. Cross-validation of information gathered by these three methods can ensure the accuracy of the data. The crude implementation score is calculated by verified results from the questionnaire, and further calibration of crude score is done by weighting, where informants are invited to assign importance (weight) to the ten aspects of the questionnaire, and a weighted score is calculated by taking into account the different importance judged by the experts.
The external validity of coronary EIS could be evaluated by analysing the association between score and quality indicators in patients with coronary heart disease in European countries. Useful quality indicators defined by the Work Package are five indicators for treatment (primary percutaneous coronary intervention (PCI) for myocardial infarction (MI) patients with ST-segment elevation on electrocardiography (STEMI), discharge medication used among patients who survived beyond hospital discharge, variables included the use of at least antiplatelet therapy (aspirin or thienopyridines), beta-blocker, angiotensin-converting enzyme inhibitor or angiotensin receptor blocker, and statin) The clinical endpoint suggested by the Work Package is 30-day all-cause mortality. The association is best assessed in countries where nationwide data for coronary disease are available. However, since very few countries acquire such data - in Europe, only in Sweden (Register of Information and Knowledge about Swedish Heart Intensive care Admissions (SWEDEHEART/RIKS-HIA)), Poland (PL-ACS) and the UK (Myocardial Ischaemia National Audit Project (MINAP)). The main challenge in external validation of the EIS methodology for coronary heart disease is the balance between quality of data and sufficient between-country variation to draw robust inference. Given such limitation, results from the Work Package showed that country with higher implementation score for acute coronary syndrome had a greater improvement in the use of evidence-based medicine (primary PCI among STEMI patients) over time, but not significantly associated with change in mortality outcome.
2. Further application of EIS methodology: comparative effectiveness study
The EIS methodology also provided important context for the growing field in international comparative effectiveness for evidence-based care, showed in the international comparative effectiveness study for the care of acute myocardial infarction (AMI) in Sweden and the UK conducted by the Work Package.
We used nationwide registry data from all hospitals providing care for acute coronary syndrome in Sweden (SWEDEHEART/RIKS-HIA) and the UK (MINAP), and included AMI patients (n=119,786 in Sweden and n=391,077 in the UK) registered between 2004 and 2010. The primary outcome was death from any cause at 30-days after admission. We assessed comparative effectiveness by indirect case-mix standardisation. Results of the study showed that the 30-day mortality was 7.6% in Sweden and 10.5% in the UK. The higher mortality in the UK was consistent across strata defined by troponin values, ST elevation and non-ST elevation MI, age, sex, heart rate, systolic blood pressure, diabetes and smoking status. Sweden compared to the UK had earlier and more extensive uptake of primary percutaneous coronary intervention for ST elevation MI patients (59% versus 22%), and beta-blockers at hospital discharge (89% versus 78%). After standardisation to the Swedish case-mix (17 variables), 30-day mortality among UK patients was higher than that in Sweden (standardised mortality ratio, UK versus Sweden: 1.37 95% CI: 1.30-1.45) corresponding to 11,263 (95% CI: 9,620, 12,827) excess deaths. The discrepancy in standardised mortality was only partly explained by differences in hospital treatments between the two countries. However this standardised mortality ratio declined with time, from 1.47 (95% CI: 1.38 1.58) in 2004 to 1.20 (95% CI: 1.12 1.29) in 2010 (P=0.01).
Clinically important differences in the uptake of effective treatments for, and outcomes from, AMI were observed between Sweden and the UK, and cannot by fully explained by difference in casemix and treatment. Aspects of healthcare, not measured in clinical registries but addressed by the EIS methodology, may contribute to the mortality difference.
* Work Package 8: Understand the role of users in implementing evidence into practice
Objective 1: To identify current national policies and strategies to promote participation of users in health research across project centres
To investigate the current status of citizen participation in health research across Europe we undertook a systematic search and synthesis of peer reviewed and grey literature, using Critical Interpretive Synthesis (CIS) method.
The systematic search of peer reviewed and grey literature identified over 5000 papers, of which 73 were included in the review. Documents were included if they discussed the participation of citizens in relation to national or supranational polices concerning health research practices; reviewed the status of citizen participation activities at the macro, meso or micro level; provided examples of citizen involvement in shaping health research-related policy or in national research-related organisations; provided examples of research studies which incorporated citizen participation.
Included papers were from the UK (n= 31), Germany (n=12), France (n=8), Italy (n=7), Spain (n=6), Lithuania (n=4), Sweden (n=4) and Belgium (n=1). No papers were found from Poland.
The following theoretical categories were derived, to explain how citizen participation in health research has been conceived and implemented in various countries across Europe
Policy and legislation Policies related to patient participation are a relatively recent development, dating to the 1990s onwards. With the exception of Poland, there is universal recognition of the importance of equity and patients’ rights in healthcare decision making. Participation is assumed to be beneficial and to lead to improved quality of health services. National policies do not prescribe methods of participation. There is limited focus on participation in research processes with the exception of the UK, the only country included in the review to have specific policy relating to this.
Civil society Participation in healthcare is often seen as an aspect of civil society; community development processes are hindered where there is a limited tradition or culture of participation.
Significant events In three EIS countries (UK, France, Italy) the occurrence of ‘health scandals’ was thought to precipitate questions concerning the role of ‘experts’, the relationship between doctors and patients and the ‘gap between scientific and patient expectations. This may have facilitated the emergence of practices and policies adopting an active participation of citizens in health research and care in these countries.
Practices of citizen participation Numerous activities at individual and group levels are regarded as participation practices, including shared decision-making; participation in clinical guideline development; practices to shape citizens into participatory citizens; state organised or driven participation; participation through patient associations; participation in research processes.
At the supra-national and national levels there is a strong drive to increase participation, education and individual responsibility in healthcare across Europe. However, in practice participation is largely limited to individual patient shared decision-making. With the exception of the UK, there is little evidence of practices enabling participation in health research processes including implementation of research evidence.
Objective 2: To determine current activities where users participate in stroke research across project centres and assess professionals’ perceptions of the role of users in research implementation.
In early meetings of EIS project collaborators, it became clear that engagement of stroke service users (intended here to include stroke patients/stroke survivors/family care-givers) in research processes and implementation of evidence into practice was a novel concept to most partners.
Work Package 8 researchers collaborated with researchers from Work Package 2 and Work Package 3 to design questions for use in the Work Package 2 survey and Work Package 3 case studies to investigate professionals’ perceptions of the role of users in research implementation.
In Work Package 2 structured interviews were conducted with researchers, health professionals, members of governmental organisations and regulatory bodies, members of scientific societies and Stroke Patients Associations in partner centres with the aim of identifying methods of implementing research results. This study reported that Stroke Patient Associations were present at national level in all EIS countries except Lithuania, and at regional level in 70% of countries. All Stroke Patient Associations promoted specific initiatives at national and regional level, including campaigns to promote stroke awareness and risk factor monitoring, to promote secondary prevention strategies (in 80% of countries), and campaigns to promote the establishment of stroke units (in 70% of countries). It was also reported that Associations contributed to stroke research, mainly in identifying priorities for research, dissemination and implementation of research findings (in 50% of countries). Associations were also involved in planning strategies concerning implementation of stroke services, management of acute stroke, prevention, rehabilitation, evaluation of outcome and quality assessment (in 70% of countries) and in information campaigns and the development of clinical guidelines (70% of countries), stroke services development, quality assessment, audits, educational programmes and guidelines for patients (50% of countries) They were less involved in designing of research and interpretation of research findings (in 30% of countries), and in preparation of material used to ask for consent and inform stroke research subjects (in 20% of countries).
However, the case study work conducted in Work Package 3 suggests a more nuanced picture in relation to the perceived importance of this sector in implementation processes. Here it was reported that while Stroke Patient Groups and Associations were identified by case study participants as one of a number of methods thought to be used in implementation processes, these were considered less useful policy tools for implementation.
At the outset of the EIS project across all participating centres, there was only one example of an initiative specifically designed to engage stroke survivors/care-givers in in research and implementation processes. This is the King’s College London Stroke Research Patients and Family group, established in 2005.
During the course of the study, this group served as a model for the development of another patient research group in Berlin. With no existing stroke users’ representative group in Germany the necessary framework for the active involvement of users in the planning, implementing and monitoring of stroke research at the Centre for Stroke Research(CSB), Charité Berlin was established in 2010. A study providing a description of the events and experiences of establishing the group ran from June 2011 to August 2013 during which time a group including approximately 20 patients and carers was built up and met regularly (approximately every 2 months) with individual researchers from the CSB.
The study showed an increase in the level of consumer involvement over the study period and demonstrated active involvement of users with the research process at the CSB. Quantitative and qualitative indicators of patient participation were identified from the literature and showed modest improvement over the period of the study. Experiences from the group and input from stroke service users could inform good practice guidelines to be used by research groups and others wishing to implement evidence based practice and seeking to establish a productive dialogue with patients. Practical recommendations based on our experience of establishing the group may serve as a model for similar groups elsewhere in Germany and Europe.
Further evaluation of user involvement in research is warranted and important aspects of user involvement had to remain unexplored during the project phase, including the impact on research management, outputs and implementation, the economic analysis and changes in attitudes in participants and researchers. Ongoing funding of the user group is being sought from the German Federal Ministry of Education and Research.
Objective 3: To assess stroke service user readiness to participate in quality assurance and development of strategies to implement research evidence and priorities for implementation.
To meet this objective we designed and conducted a cross-sectional questionnaire survey of stroke survivors participating in on-going stroke research EIS project centres. These were four European population stroke registers: the Dijon Stroke Registry, the Erlangen Stroke Project, the Sesto Fiorentino Study, and the South London Stroke Register; a hospital based register - the Glasgow Stroke Register; and RIKS Stroke, a register of stroke patients in Sweden primarily used for audit purposes.
We designed this new questionnaire using a combination of questions from existing questionnaires and newly devised questions developed in conjunction with stroke survivors. Over three sessions, researchers and four stroke survivors, members of an existing stroke patient research group, discussed the concept of research implementation to identify domains to be included in a questionnaire and then developed new questions for each domain or adapted existing questions for inclusion in the appropriate domain. The questionnaire included 14 closed questions covering the following domains: attitudes towards taking part in research; knowledge and use of research evidence and clinical guidelines; willingness to be involved in research processes. Participants’ demographic and outcome data were available from the register in which they were taking part. The questionnaire was developed in English, was translated (and back translated) into French, German Italian and Swedish for use with the Dijon, Erlangen, Sesto Fiorentino and Riks-Stroke registries respectively.
The questionnaire was sent to 647 stroke survivors, and a response rate of 74% was achieved (481/647). Reasons for register participation included responding to clinician request (56%) and to ‘give something back’ (19 %). However, 20% were unaware that they were participating in a register. Research awareness was generally low: 57% did not know the purpose of the register they had been recruited to; 73% reported not having received results from the register they took part in; 60% did not know about any research on stroke care. Few participants (7.6%) used research evidence in their consultations with a doctor. Thirty-four percent of responders were interested in being involved in research processes while the remainder were not interested or unsure. Those who were interested were younger, more highly educated and already research aware. Unexplained differences across countries were observed.
The survey results suggest that across these European centres, stroke survivors already participating in research are not well informed about stroke research. This has implications for their engagement in their own care, but also that increased efforts will be required if researchers and policy makers are to engage patients in research and implementation.
Objective 4: To develop guidance to involving patients/care-givers and the public in implementing research evidence into practice
The following recommendations for development or practice related to engaging citizens in implementation practices were proposed:
1. Further research is needed to improve implementation of national and international policies related to patient engagement in research, implementation and quality improvement.
2. There is a need to develop and evaluate a methodology and standards for involving users and patient advocacy groups into research framework programmes of the EU-Commission.
3. There is a need to identify and evaluate methods of education and empowerment of patients and their organisations to participate into research processes.
4. There is a need to develop and test methods of increasing opportunities to draw on evidence based practice in individual patient care.
5. There is a need to develop and test methods for increasing research literacy among stroke patients.
6. Greater efforts on the part of researchers and research organisations are required to disseminate research findings to stroke survivors and carers.
7. Health services and systems should promote increased client awareness of the need for patient and carer engagement in research and implementation processes.
8. Stroke patient organisations should promote increased client awareness of the need for patient and carer engagement in research and implementation processes.
* Work Package 9: Develop recommendations for closing the research-practice gap
Description of work
The EIS consortium represents an exceptional multidisciplinary mix of experts from the fields of health services research, epidemiology, health policy, clinical stroke care, health economics and quality of care research in Europe. In addition, representatives from patient and user organisations participated. A novel integrated translational research approach was utilised combining qualitative and quantitative methods for identifying what methods are successful in translating new knowledge into practice.
A Dissemination Committee (DC) was formed comprising the Project Director, the Project Manager, and representatives of the beneficiaries with the aim of carrying out peer review of the projects results as they occurred during the lifetime of the project, with a view to determining what advances are capable of scientific dissemination or other exploitation and agreed arrangements for intellectual property arising from the project.
A draft dissemination plan was developed by the DC and discussed at the Project’s inaugural meeting. The Dissemination Plan was refined and expanded after that meeting and ratified by the EIS Management Board (EMB).
The general aim was to disseminate the results of the EIS project through the networks already established by each beneficiary at national and international level and to seek out new ones. Dissemination was be achieved through a mix of internet, website (public and private spaces), presentations at national and European conferences, and publication of study results in scientific journals and through direct presentations to user groups and other interested parties.
Presentations at meetings
Conference-based dissemination began in year 2 of the project. It was envisaged that presentations would be made at the European Stroke Conference, the Meeting of the European Society of Cardiology, and health economic and health policy meetings starting from 2010, which was the case.
Websites
A dedicated project website was created and provides detailed information about current health services research on stroke together with planned events and activities for the scientific community and general public. The dissemination and transfer of results have been communicated through the website and conferences by members of the EMB. This has been particularly important in the final stages of the Project in order to exploit the scientific results for further translational gain. The website is be maintained by administrative staff of the Division of Health and Social Care Research (based at King’s College London). The website address is http://www.eisproject.com
Network communication
The EIS network formed the core activity of horizontal dissemination with new dissemination activities being reviewed regularly by the EMB. The various connections and networks of the partners were explored at the Project’s inaugural Meeting and each Partner assumed the responsibility for their individual expertise and allocated particular dissemination activities.
External communication
The DC submitted press releases, within the guidelines provided in the Grant Agreement and Consortium Agreement, as and when appropriate on new results. Common material with logos for oral and poster presentations were developed to support formal and informal presentations.
Publications
The results of the different Work Packages were published in international peer-review scientific journals. The lead applicants of each Work Package were responsible for drafting the publications. The rules for drafting the publication and authorships of the publications followed the ‘Uniform Requirements for Manuscripts Submitted to Biomedical Journals’ of the ‘International Committee of Medical Journal Editors’ (http://www.icmje.org).
Potential Impact:
Expected impacts listed in the work programme
Current State of Implementation of research evidence into practice
The implementation of research evidence into practice is sub optimal for many disease conditions and systematic reviews of the literature have highlighted the need for new theoretical frameworks to address how to better to implement research into practice. There is a paucity of routine data on the effective diffusion of evidence into practice for vascular disease which we believe to be a key area of public health and clinical care. Cardiovascular Disease, including stroke and coronary heart disease, represent not only the predominant cause of death in the developed and developing world but also contribute significantly to levels of impairment, loss of activity and reduced quality of life. The long-term consequences of cardiovascular diseases for the individual and the society are also substantial.
This collaborative project aimed to improve the health of European Union citizens, particularly the elderly, for a global health issue of public health and clinical importance, namely vascular disease. The incidence of stroke will rise dramatically in the next 20 years with an ageing population and this will affect all countries, particularly those in Eastern Europe where the incidence rates are increasing dramatically. There are effective interventions such as stroke unit care and thrombolysis that can reduce the burden of the disease but these require equitable diffusion into healthcare systems in Europe. Stroke patients who survive their stroke have high levels of impairment, reduced activity and participation and poorer quality of life. These adverse outcomes can be reduced for example, by effective secondary prevention and rehabilitation. So in terms of the clinical and Public Health impact of stroke, by developing new methods and generating the necessary scientific basis to underpin informed policy decisions on health systems for measuring the effective implementation of evidence into practice, this project will have significant benefits to patients, their families and society.
The implementation of research into healthcare practice is complex, current methodologies are insufficient to prevent the adverse outcomes (poor quality care, poor outcomes, side effects of medications). Developing a new conceptual framework for implementation in the field of a chronic vascular disease was challenging and but we believe this has been addressed by our international multidisciplinary team which enabled us to take an innovative approach to tackle this issue. We believed we were much more likely to succeed by addressing this issue at the macro-, meso- and micro-level in different health care settings and brought the different scientific perspectives to bear when designing and executing the project. Key to its likely impact is the involvement of a European network of user organisations in stroke, thereby addressing issues of patient empowerment.
The project aimed to implement policy-driven research at a European level and in particular through benchmarking, comparisons and analysis of models, systems and data of national databases, and generated new knowledge to underpin informed policy decisions on efficiency and responsiveness of health care systems in Europe, and more effective strategies in treatment for vascular disease. We developed a tool that identified how effectively implementation is in different countries. The components of the EIS tool identified what aspects of implementation need to be addressed to improve the likelihood of improved implementation. We demonstrated which factors regarding implementation lead to better outcomes.
In summary the project addressed many components of the call in Area 3 and specifically 3.1-1:
• Cooperation between researchers and users across Europe to promote integration and excellence of European research.
• Findings scientifically validated in different settings and applicable beyond the national level.
• Development and validation a tool for measuring how new knowledge is implemented in different healthcare settings (Work Package 3)
• The study focussed on stroke but examined coronary heart disease as well (Work Package 7)
• Scientific methodologies allowed tools for benchmarking and comparative analysis at the European level to be developed (Work Package 4, Work Package 5)
• The project identified and validated factors determining successful and timely implementation (Work Package 4, Work Package 5)
• Recommendations for closing the research-practice gap in the implementation of clinical guidelines in Europe will influence policy in Europe.
The project plan detailed how these impacts will be realised. There was a unique opportunity both in time and in regard to the number of countries that wished to collaborate in this project. The emerging strong evidence base in stroke and coronary heart disease means that this was the opportune time to identify what factors influence implementation of evidence into practice.
We have gathered together those countries that we collaborated with for nearly 20 years in the field of health services/public health research in the area of stroke. In addition, we also involved countries with national audits of stroke and those that have ongoing population based registers. We involved key figures in the development of the evidence base in stroke and coronary heart disease who have been central to developing European guidelines, Cochrane reviews and national guidelines. We involve those countries that have national audits of these guidelines and in addition populations that have detailed unbiased data which we can investigate. Therefore, the countries and datasets are ready to be involved in the Work Packages and this project seeks funding to bring the multidisciplinary group together (Work Package 1) and to develop and validate the tool for implementation (EIS), develop performance indicators, model cost effective implementation options and make recommendations. The project requires comparison of different implementation strategies in different settings in order to identify which factors are effective in the implementation process. This would not be possible within countries where policies and processes are often similar in many areas. Additionally, stroke is a global issue and needs solutions at a European level.
The research team includes experts in the breadth of applied science, stroke medicine and cardiology, along with user groups. We are unaware of other groups that have published or are involved in developing such detailed tools, using as many large data sets, as we are planning. The literature does not identify research on this level addressing the issues we plan to investigate in stroke and coronary heart disease.
As stated, this project is timely with many governments attempting to address the issues of improving the quality of care, particularly for conditions of significant public health importance. The results of the study will identify how well implementation is being achieved in this area in Europe, and will provide a tool to assess implementation in other European settings. We do not envisage any barriers to this impact being achieved. Naturally having identified a tool and current gaps in implementation, it will be necessary for individual countries to use the data constructively to develop systems to improve implementation. We believe the robustness of this study and its timeliness will mean the project is highly likely to have an influence on policy at a European level.
Dissemination and/or exploitation of project results, and management of intellectual property
Management of knowledge and of intellectual property
The Consortium Agreement will provide that the locus for the management of knowledge is the
Dissemination Committee which has, as one of its permanent members, a representative of the Technology Transfer Office of the coordinating institution, King’s College, London. The members on the Committee are particularly qualified to assess the on-going results of the project because of their previous experience of intellectual property protection. The Committee is keen to be able to identify when a process or system is developed which is capable of further development or manufacture, probably by one of the consortium, at the earliest opportunity rather than at the end of the project and, consequently, periodic technology audits of the project’s progress will be carried out.
Involvement on a continuing basis of a technology transfer expert will enable the necessary licensing arrangements to be put in place quickly allowing that participant, or an outside organisation, to proceed with such development or manufacture. Similarly, where a new technique or process is developed, arrangements for training of individuals inside, and outside, the project in those techniques or processes can be put into effect. It is likely that, as the need arises, an ad hoc sub-group comprising technology transfer personnel from some of the other participants, particularly the industrial partners, will be convened where there are pan-project developments worthy of consideration.
In addition to the pro-active, monitoring role of the Dissemination Committee participants will ensure that in their periodic progress reports mention is made of any advances in their areas that could lead to exploitation or commercialisation, even where this is beyond the current scope of the project.
Members of the consortium recognise the desirability of producing publications for the purpose of promoting scientific and technical progress but are also aware of the requirement to protect project technology. Prior to submission, project-related publications will be provided to members of the consortium and any objection to publication from any partner will be made within a period to be determined in the consortium agreement.
The grounds for such an objection will be that disclosure of the information would adversely affect protection of knowledge and publication will only be delayed for a reasonable period to allow filing of patent applications. This period of time and a maximum time of postponement of publication will also be decided (in the light of Commission rules) and stated in the consortium agreement. These considerations will also apply in the case of submission of degree theses where the person submitting has needed to include preexisting know-how or knowledge of another party.
Major deliverables expected from the project that have potential for exploitation are:
• The final versions of the EIS Score (D3.2)
• Stroke datasets across Europe, subject to individual agreement with the subcontractors
It is estimated that there is a market for these project deliverables. The outputs will be of particular interest to policy makers involved in developing health care services for stroke and other vascular diseases. The generated data and results are also relevant for pharmaceutical companies regarding post-marketing surveillance.
Dissemination of knowledge beyond the consortium
Subject to the requirements, indicated above, for protection of project technology, data and findings from the project will be submitted for publication in peer reviewed journals in the normal way and members of the consortium will present work at scientific meetings. Any such communication shall indicate the relevant specific programme and the support provided by the European Union but shall also state that the author(s) is (are) solely responsible for it. Major findings may also be publicised by means of press releases which may be discussed in advance with the Commission.
The consortium will also seek opportunities to hold workshops or satellite symposia in association with scientific meetings. It will be particularly appropriate for these to be held in conjunction with the international conferences (e.g. European Stroke Conference, Annual Meeting of the European Association of Cardiovascular Prevention and Rehabilitation) that will be held annually during the project’s lifetime. The involvement of representatives and advocates for potential users of the EIS Score will provide additional opportunities for wider dissemination of project findings.
We will also disseminate the results to a wider audience of the general public and to users of stroke and vascular services to promote greater public engagement and dialogue between the scientists and society and to increase awareness about involvement of users in implementing research into practice, e.g. by using existing collaborations with user groups across Europe.
A project website has been established following normal practice. The website will be hosted at the coordinator’s institution and it will be necessary to hire a programmer on a fixed term contract to establish and maintain the site. There will be a restricted area in which members of the consortium can contribute records, experimental data and reports as well as holding discussions. There will also be a freely accessible section that will be used to publicise the project and disseminate non-confidential information and results. Members of the consortium are also aware of the rights of the Commission to disseminate general information regarding the project and have contributed to such dissemination in earlier frameworks.
Work Packages
Work Package 2
Stroke is a costly disease, from patient, family and societal perspectives. Total number of stroke deaths in 48 European countries is currently estimated at 1,239,000 per year (508,000 per year in the 27 EU members). In the 27 EU countries, total annual cost of stroke is estimated at €27 billion: €18.5 billion (68.5%) for direct and €8.5 billion (31.5%) for indirect costs. A further sum of €11.1 billion is calculated for the value of informal care.
Significant variations in incidence, mortality disability rates and stroke care organization have been found among European countries, as well as inequalities in priorities given to stroke management, and disparities in stroke care and stroke prevention, even at local level. These variations may depend in part on different levels of implementation of stroke research evidence into practice.
Work Package 2 of the EIS Project described methods used in 10 European countries (Belgium, England, France, Germany, Italy, Lithuania, Poland, Scotland, Spain and Sweden) to implement research results in the fields of stroke care, considering different levels of the health care systems and different clinical settings.
We found that differences exist among European countries, but also different levels of implementation can be found at regional level in single countries. While some implementation methods were used in the majority of countries participating to the Project, there are areas of stroke care that still are neglected, even by those methods. For instance, rehabilitation, counselling, cooperation with general physicians, and continuity of care still require further development. National or regional regulations having the force of law are not frequent, but their efficacy in comparison with non-mandatory policies may be debated. The role of private financial incentives is negligible, and mostly limited to medical education or guideline implementation. Educational meetings and workshops are widely used, and cover most areas of stroke care, but counselling is less frequently part of educational strategies. National audits are present in 60% of countries, and educational outreach visits are performed in just 2 countries.
The increasing involvement of Stroke Patients’ Associations not only in traditional activities of the voluntary sector, such as information campaigns, but also in service development and stroke research, indicates a growing awareness among European citizens.
Organisational constraints, lack of resources, bureaucratic inertia, and resistance to change were considered among the highest barriers for implementation of evidence on stroke care, and should also be considered among the main targets for health professionals, administrators, policy makers, opinion leaders and Stroke Patients’ Associations, if the final aim is to offer a better level of care to stroke patients in Europe.
In conclusion, Work Package2 of the EIS Project evaluated, for the first time in Europe, with a new dedicated questionnaire, methods of implementation of research results in stroke care in 10 countries. Main objectives were linked to the perceived necessity to narrow the gap between research and everyday practice, and to improve quality of stroke care in Europe, reducing human and societal costs, since studies show that about 30-40% of patients do not receive appropriate care according to present scientific evidence, with underuse of effective interventions.
Our work should be considered as a first step for improvement of knowledge in this field, requiring to be integrated and updated with future contributions, as the relevance of implementation science in the field of stroke care become more apparent.
Work Package 3
Publications, conference and poster presentations
Work Package 3 have published three articles aimed at various academic (medical, managerial, social science and policy) audiences so far from this work (Boaz et al, 2011; Baeza et al, 2012a; Baeza et al, 2012b). We also have three further papers which are shortly to be submitted to Social Science journals focused on inter-professional aspects of EBHC implementation, leadership and EBHC implementation, and ‘bad behaviour’ impacting upon EBHC implementation. We also published an article aimed specifically at the healthcare management community (Fraser et al, 2012). In addition to these papers we also have a further two papers to be shortly submitted in collaboration with Work Package 5 and Work Package 1 team members – the first of these is a paper which documents the process of producing the final version of the EIS (D3.2) and the second which demonstrates the predictive ability of the final version EIS by drawing on the quantitative data analysis of the Work Package 5 team in relation to the effectiveness of different implementation methods to augment selected performance indicators across five selected national audits/registries. In addition to this, we also presented findings from this project at the XXI European Stroke Conference, Lisbon (2012) and the 16th Annual Conference of the International Research Society for Public Management, Rome (2012) and the at European Health Management Association Annual Conference, Porto (2011). Finally, we gave poster presentations at the UK Stroke Forum Conference, Harrogate (2012), the XXI European Stroke Conference, Lisbon (2012), the XX European Stroke Conference, Hamburg (2011) and the European Health Services Research Conference, The Hague (2010).
Measurement tools
In addition to the traditional academic dissemination methods listed above, we also produced two self-completion ‘implementation measurement tools’. The first of these was in collaboration with colleagues at the Royal College of Physicians (RCP) in London. We produced 12 questions designed to measure facets of leadership which were included as part of the RCP National Sentinel Organisational Audit 2012 and completed by all 190 NHS organisations which treat acute stroke patients in England, Wales and Northern Ireland. The results of these questions were then fed back to the organisations. Building on this work we also designed a self-completion meso-/micro-level implementation measurement tool with 38 questions which was piloted at six Scottish hospitals and eight German hospitals through 2012-13 and produced promising results related to the success or otherwise of implementation methods employed at the micro-/meso-level. We plan to build on this pilot work and roll out the application of this tool to further sites in more countries so that stroke professionals and managers may learn from our findings when applied to their own work practices.
Collaborations & further work
We have held a number of meetings with representatives from other EU FP7 projects focused on implementation – specifically with FIRE Project and QUASER Project team members and produced a paper for publication focused on common challenges, strengths and pitfalls of international knowledge mobilisation work, as well as discussing the possibility of future collaborations drawing on our different, but connected experiences in this field. Finally, using the insights developed as part of the EIS Project, key Work Package3 team members have submitted a research proposal to the National Institute of Health Research in England for funding for a further two year project focused on the effectiveness and variability of Early Supported Discharge for stroke patients in the NHS.
Work Package 4
The Work Package 4 has identified standards for requirements of quality indicators for stroke, and has described a suitable process to identify suitable indicators. Together with Work Package 5 we have shown that the variables in different European quality registers on stroke vary substantially. Work Package 4 has for the 1st time identified a set of European quality indicators developed by consensus. This set has been presented at scientific meetings, and has been adopted by the European Stroke Organization. The set of indicators is also considered within the World Stroke Organization subcommittee for Guidelines and Quality of Care.
The recommendations for data collection for the European set of performance measures, as well as the consensus set of indicators, have been published in the webpage of the project and will be disseminated free to the public. A publication of the consensus variables is in preparation.
The work of Work Package 4 has highlighted the differences between different European Quality Registers for stroke, and has led to increased recognition of issues surrounding quality variables. Cooperation between the European Stroke Registers have increased through Work Package 4.
Work Package 5
Potential impact of results
Within Work Package 5 quality standards for developing and documenting quality indicators to measure and to compare quality of acute stroke were defined. These recommendations might lead to reduce the huge variation in measuring performance of acute stroke care in Europe currently hampering the comparison of acute stroke care delivery across Europe in many domains. The experiences of the project and the initiation of standards for data collection methods across European audits might increase comparability of audit datasets in future. A better comparability of stroke audit data in Europe will improve our understanding on patterns driving delivery of appropriate acute stroke care on the population level. In addition, the project provided evidence of a potential link between the definition of a specific quality target and an increase in quality of acute hospital care on a national/ regional level, yielding towards new insights for driving an improvement of delivery of acute stroke care in Europe. Finally, a standardized tool for measuring degree of implementation also on a micro-/ meso-level was developed by Work Package 3 based on the research platforms established in Work Package 5 that can be used in future studies.
Main dissemination activities
The results of this Work Package were mainly disseminated using traditional dissemination techniques including: publication in scientific journals; presentation at national and international conferences; and workshops. In addition, the experiences of the results were disseminated to representatives of participating regional and national European audits during EIS consortium meetings as well as in direct communications for further distributing methodology to define data collection standards. Representatives of Work Package 5 also participated within workshops and meetings of ongoing initiatives from professional stroke organisation (e.g. ESO, WSO) on standardisation of register methodology and quality standard development to further disseminate outputs of the EIS project. Finally, the main results of this Work Package were also presented to a national user group by a Work Package 5 representative.
Work Package 6
This project, and specifically this Work Package, will support the argument for improving stroke care across Europe. As such it has potentially a high impact factor, as it can help shift stroke care to best-practice care across a number of countries.
The results will be disseminated as a range of scientific papers in leading clinical and health economic journals. To pursue fast delivery an LSE Health Working Paper will be delivered by the end of the calendar year 2013.
Work Package 7
Findings of the work group suggest that differences in the implementation of evidence into clinical practise in coronary care can be measured by EIS methodology. The difference in implementation may contribute to international differences in patient mortality.
EIS methodology could be a helpful tool in measuring these aspects of healthcare, which are currently unmeasured in these registries, and which are likely to contribute to explaining the mortality gap, for instance national policies, financial incentives, organization and leadership in delivering care.
By further applying EIS methodology, the international outcome comparisons presented in our work suggest a novel research agenda. First, what additional patient level health care factors, currently unmeasured, might explain these differences? There is a need to harmonise internationally more detailed measures of quality of care in clinical registries, for example pathways of care. Second, how does outcome compare in other countries with national, albeit voluntary, registries including the US, Poland, Hungary, and France? Third, with linkage to national electronic health records, there are opportunities to extend comparisons to include ambulatory care before and after the admission for heart attack, non-fatal events, and longer-term outcomes. The lasting impact of the international comparisons of care and outcome registries may inform new research and policy initiatives to improve the quality of health systems.
Work Package 8
We have identified low levels of stroke patient engagement in research processes generally, in the use of research evidence in on-going health management at individual level and low levels of patient/citizen engagement in implementation of evidence into practices. There was mixed evidence of the role of stroke patient associations in research and implementation processes. This is despite some recognition at the level of European and national policy and professional discourse of the benefits and ethical imperative of such engagement.
This suggests that there is great potential to develop and evaluate strategies to improve the implementation of national and international policies related to patient engagement in research, implementation and quality improvement. EIS investigators are developing plans to take his work forward.
The work undertaken for Work Package 8 is novel. In particular, leading payers in the implementation of stroke research evidence have never before been asked to comment on the role of patients and patient organisations in research implementation. The survey of stroke survivors in EIS participating centres represents the first attempt to quantify levels of research awareness and readiness to be involved in research processes.
Findings from Work Package 8 have been presented at local meetings to professionals and patients in the UK and in Germany and at international scientific and patient association meetings. Two papers have been submitted for publication and two others are in preparation.
The Work Package also led to the development of the first ever stroke patient research group in Germany, thanks to the collaboration between the two Work Package 8 leads (MW and CM). In 2010 a stroke patient group to engage in the planning, implementing and monitoring of stroke research at the Centre for Stroke Research (CSB), Charité Berlin was established. This group took as its model the existing Stroke Research patients & Family Group established at King’s College London in 2005. A study documenting the processes and experiences of establishing the group was carried out from June 2011 to August 2013 during which time a group including approximately 20 patients and carers was built up and met regularly (approximately every 2 months) with individual researchers from the CSB.
List of Websites:
http://www.eisproject.com
