Supporting the Use of Research Evidence (SURE) for Policy in African Health Systems

Informacje na temat projektu

SURE

Identyfikator umowy o grant: 222881

Projekt został zamknięty

Data rozpoczęcia 1 Czerwca 2009

Data zakończenia 31 Maja 2014

Finansowanie w ramach

• Specific Programme "Cooperation": Health

Koszt całkowity € 3 757 838,94

Wkład UE € 2 988 536,00

Koordynowany przez

NASJONALT KUNNSKAPSSENTER FOR HELSETJENESTEN
Norway

Final Report Summary - SURE (Supporting the Use of Research Evidence (SURE) for Policy in African Health Systems)

Executive Summary:
The goal of the Supporting the Use of Research Evidence in African Health Systems (SURE) project was to improve decisions about health systems in Africa by improving policymakers’ access to and use of research evidence that is relevant, reliable, accessible and timely. The project built upon national and international initiatives that aim to improve the use of research evidence in policy decisions about health systems. These partnerships between policymakers, researchers and civil society included the Evidence-Informed Health Policy Network (EVIPNet), the Regional East African Community Health (REACH) Policy Initiative, and the Zambia Forum for Health Research (ZAMFOHR). The project developed and evaluated the following:
• Thirty-five evidence briefs for policy - relevant, reliable, accessible and timely research syntheses that address priority problems from seven African partner countries (Burkina Faso, Cameroon, CAR, Ethiopia, Mozambique, Uganda and Zambia). Each brief includes a description of the problem, options for addressing the problem and implementation considerations.
• User-friendly formats for evidence briefs for policy; deliberative forums involving policymakers, researchers and civil society; support for public access to evidence that informs health system decisions; and capacity building for evidence-informed health policymaking in Africa – We have compiled these resources in the SURE Guides for Preparing and Using Evidence Briefs for Policy and an accompanying series of videos.
• Two clearinghouses for research syntheses and policy relevant research – Pretty Darn Quick Evidence (PDQ-Evidence) is a database of health system evidence that is updated regularly by searching multiple sources for relevant systematic reviews. It includes over 3200 systematic reviews and all of the studies included in those systematic reviews (over 28,000). The Evidence Informed Policy Network Virtual Health Library (EVIPNet VHL) is a meta-search engine that simultaneously searches multiple databases, including Health Systems Evidence, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE) and LILACS.
• Rapid response services to meet policymakers’ needs for research evidence within short time frames (hours or days)
• Evaluation of collaborative initiatives between policymakers and researchers when using the above and other strategies to support evidence-informed health policy – including evaluations of two primary activities (evidence briefs and policy dialogues), annual profiles of the seven African initiatives, structured reflection and an evaluation of an international forum on evidence-informed health policymaking organized by the SURE project.The key findings from the evaluation of evidence briefs and policy dialogues, were that as well as each of their features, they were very highly regarded and that policy dialogues (informed by evidence briefs) led to an intention to act on what participants learned.

Project Context and Objectives:
The goal of the Supporting the Use of Research Evidence in African Health Systems (SURE) project was to improve decisions about health systems in Africa by improving policymakers’ access to and use of research evidence that is relevant, reliable, accessible and timely. The SURE project was a collaborative project that linked to and built upon the Evidence-Informed Health Policy Network (EVIPNet) and the Regional East African Community Health (REACH) Policy Initiative – two international initiatives that aim to improve the use of research evidence in policy decisions about health systems. Country level initiatives in seven African countries were partners in the SURE project: Burkina Faso, Cameroon, Central Afrique Republique, Ethiopia and Mozambique, Uganda and Zambia. Each initiative is a partnership between policymakers, researchers and civil society. Other partners in the SURE project included the World Health Organization, the McMaster Health Forum, the Health Systems and Policy research Group at Karolinska Institutet, and the Global Health Unit at the Norwegian Knowledge Centre for the Health Services.

The specific scientific/technological objectives of SURE were to:
1) Produce relevant, reliable, accessible and timely research syntheses (evidence briefs for policy)
2) Develop and evaluate the effectiveness of five strategies for improving access to and use of research evidence in policy development: user-friendly formats for evidence briefs for policy, clearing houses for research syntheses and policy relevant research, rapid response mechanisms to meet policymakers’ needs for research evidence within short time frames (hours or days), deliberative forums (policy dialogues) involving policymakers and researchers with the involvement of civil society and the general public, and supporting civil society’s and the general public’s access to and use of research evidence
3) Develop capacity for evidence-informed health policy development in Africa
4) Evaluate collaborative initiatives between policymakers and researchers when using the above and other strategies to support evidence-informed health policy

Evidence briefs for policy
Since the start of the project in 2009, the African partners of the SURE project have developed research syntheses on topics identified as priorities within their setting. We refer to these reports as “evidence-based policy briefs” or “evidence briefs for policy”. Methods for setting priorities for evidence briefs and examples of how the SURE partners have done this are described in the SURE Guides for Preparing and Using Evidence Briefs for Policy and the SURE Guide Videos. Each evidence brief includes a description of the problem, options for addressing the problem and implementation considerations. The seven African partner countries have produced 35 evidence briefs. Examples include: task shifting to improve the delivery of maternal and child healthcare, improving access to skilled attendance at delivery, improving patient safety, improving governance of district health services, increasing health insurance coverage
Retaining human resources for health in rural areas, increasing antenatal care coverage, quality of care in emergency rooms, reducing maternal mortality, reducing neonatal mortality, improving allocation of the budget for health, reducing mortality and morbidity due to cervical cancer, managing public health emergencies, improving the effectiveness of community health worker programmes, strengthening the health system for mental health, and access to adolescent health services.

Policymakers, researchers and other stakeholders discussed each completed evidence brief in a policy dialogue. We surveyed participants who read the policy briefs and attended policy dialogues before the start of the dialogues – to collect their views on the evidence briefs – and at the end of the dialogues – to collect their views on the dialogues. Respondents viewed the evidence briefs and deliberative dialogues – as well as each of their key features – very favourably, regardless of the country, issue or group involved. Respondents generally reported strong intentions to act on what they had learnt.

Strategies for improving access to evidence briefs and policy-relevant research evidence

User friendly formats for evidence briefs
We carried out user tests in four African countries to explore users’ experiences with user-friendly formats for evidence briefs. We modified the format iteratively based on feedback from policymakers and other stakeholders using the evidence briefs in policy dialogues and the results of the user tests. We also obtained feedback from an advisory group consisting of members of the SURE Collaboration.
Policymakers and other stakeholders found the overall format to be attractive. They found the simple language, Institution logos, key messages and information box (explaining what the evidence brief was) to be very useful. We identified a need to distinguish very clearly between the short and full reports on the cover pages, as the difference was not instantly recognisable. Views regarding whether there should be separate documents for short and full reports or a single document varied across countries.
Users found the format of the evidence briefs useful and usable. The revised format features distinctive title pages for three versions of the evidence briefs - Key Messages, Executive Summary and Full Report. Templates for alternative formats with two separate documents or a single document allow countries to tailor the presentation to accommodate the preferences of users of the evidence briefs.

Clearinghouses
We used multiple methods to develop and evaluate the clearinghouses. We reviewed existing sources of research evidence to inform the design of the clearinghouses. We used brainstorming workshops to generate ideas and solutions to problems uncovered through feedback and testing. We used advisory group feedback to inform development of the clearinghouses from a stakeholder perspective. We pilot tested the clearinghouses to identify problems and test solutions. We conducted user testing to inform development of the clearinghouses from a user perspective. We conducted a comparative evaluation to compare one of the clearinghouses to six other sources of research evidence, and we conducted a content analysis of that database compared to six other database.
We developed two clearinghouses using different approaches to providing rapid access to research evidence about health systems. Pretty Darn Quick Evidence (PDQ-Evidence) is a database of health system evidence that is updated regularly by searching multiple sources for relevant systematic reviews. It includes over 3200 systematic reviews of health system arrangements, implementation strategies and public (population) health policies, all of the studies included in those systematic reviews (over 28,000), over 240 overviews of systematic reviews (including policy briefs or evidence briefs for policy) and over 1000 structured summaries. Users found it easy to use and quick. The Evidence Informed Policy Network Virtual Health Library (EVIPNet VHL) is a meta-search engine that simultaneously searches multiple databases, including Health Systems Evidence, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE) and LILACS. It provides access to both health system and other systematic reviews and other types of research evidence. It is possible to search PDQ-Evidence and the EVIPNet VHL in English, French, Portuguese and Spanish.
The two clearinghouses developed by the SURE project, PDQ-Evidence and the EVIPNet VHL, provide ‘one stop shopping’ for health systems research. In addition, three partner countries (Uganda, Zambia and Cameroon) had developed country-specific clearinghouses with funding obtained from alternative sources. Those clearinghouses provide access to local (national) evidence and links to PDQ Evidence and the EVIPNet VHL. PDQ-Evidence is quick and easy to search and provides rapid access to systematic reviews, overviews of reviews, structured summaries and primary studies. It enables users filter search results by the type of publication, year of publication and location where studies were done, and to generate a matrix that enables users to compare the studies that are included in related systematic reviews. The EVIPNet VHL provides a useful online tool for policymakers and stakeholders who need timely access to evidence about health system arrangements. Providing a comprehensive, single-entry point to this evidence supports efforts to support the use of research evidence in policymaking.

Rapid response services
Findings from a literature review of services with similar objectives informed the development of the rapid response services. We pilot tested the services to evaluate their feasibility, identify problems and test solutions. Subsequently, we expanded the services and assessed the viability of the service at a wider scale. Throughout this time, we user tested the rapid responses and obtained feedback on the experience of users of the service.
The service responded to over 70 questions. About 30 policymakers in Uganda regularly used the service. Several other partner countries piloted similar services, based on the experience in Uganda and Burkina Faso, and received technical assistance from Uganda. User testing results informed improvements to the template for rapid response summaries. The evaluation of the rapid response services identified crucial components of such a service and informed improvements to the rapid response service. We found that rapid response services designed to meet policymakers’ urgent needs for research evidence about health systems are feasible and desirable to policymakers in low-income countries.

Stakeholder involvement
We developed guidelines for deliberative forums (policy dialogues), which were organised to discuss each of the 35 evidence briefs and for informing and engaging stakeholders in preparing and using evidence briefs. The seven African partners used these guides for each of the evidence briefs they prepared. We revised the guides based on this experience, evaluation of the policy dialogues and feedback on the guides. We have also used examples from this experience to illustrate the guides, including videos that capture reflections of participants.

Resources for capacity building
We conducted annual workshops, additional national and international workshops, traineeships, exchanges and an international conference. The workshops and the conference were evaluated using a structured evaluation form and verbal (group) feedback. We used and evaluated resources to support capacity development at the workshops and by other users and revised the resources based on feedback from users. Structured assessments of the needs of each of seven African partner countries, the evaluations and feedback from participants and other users guided the content of the workshops and resources. Based on this experience and feedback we developed and revised a package of capacity-building resources that use examples from Africa as illustrations and build on the experience of the seven African SURE partners, but are universally relevant. The resources are primarily targeted at researchers and technical support staff, but some are also suitable for building capacity amongst policymakers and other stakeholders. An effective strategy for building capacity amongst policymakers and other stakeholders is through their involvement in developing and using evidence briefs and, particularly, participation in policy dialogues, as well as through their interaction with rapid responses services. African SURE partners increasingly taking lead roles in organising and facilitating both national and international training illustrates the impact of these activities. The positive responses of policymakers and other stakeholders to rapid responses, evidence briefs and policy dialogues, also illustrate this.

Evaluation
The evaluation had three objectives: 1) to evaluate evidence briefs and policy dialogues, 2) to assess the advantages and disadvantages of the different approaches taken by the seven African country initiatives, and 3) to assess the impact of the initiatives. To address the first objective, we completed evaluations of evidence briefs and policy dialogues by surveying over 300 policymakers, stakeholders and researchers before and after policy dialogues. Key findings included that evidence briefs, policy dialogues, and each of their features, were very highly regarded, and that policy dialogues (informed by evidence briefs) led to an intention to act on what participants learned.
To address the second objective, we conducted annual evaluations of each of the country initiatives and undertook a structured-reflection exercise. We found that the governing bodies of those initiatives that had one were composed of a good balance of policymakers, stakeholders and researchers. More than half of the initiatives were housed within a research institution. Most initiatives focused on the production of evidence briefs and organising policy dialogues. Priority-setting events were the most reported activity to identify specific policy issues that could be informed by research evidence. Clearinghouses and websites were the most common activity undertaken to facilitate policymakers' ability to acquire, assess, adapt, and apply research evidence.
In the structured-reflection exercise, participants identified policy dialogues informed by evidence briefs as the most commendable tools for enhancing evidence-informed health policymaking (EIHP) and perceived these as the most impactful of the activities and outputs in all countries. The country initiatives contributed to increased awareness of the importance of EIHP and strengthened relationships among policymakers, stakeholders, and researchers. Support from policymakers and international funders facilitated activities, while the lack of skilled human resources to conduct EIHP activities impeded them. Ensuring the sustainability of EIHP initiatives after the end of funding was a major concern. The ‘institutionalization’ of initiatives within the government helped to retain human resources and secure funding, whereas initiatives hosted by universities highlighted the advantage of autonomy from political interests.
To address the third objective, we conducted seven case studies involving three initiatives in two separate studies using multiple case study designs. We sampled four cases from Uganda and Zambia (two from each country) and three cases, with a particular focus on social networks, from Burkina Faso. In three of the four cases in Uganda and Zambia, the initiatives’ evidence briefs influenced ideas through a longitudinal pathway by initiating potential shifts in the way actors perceived various aspects of the policy issue. Issue characteristics, and in particular whether an issue was familiar or not, emerged as an important determinant in explaining the nature of the influences of evidence briefs. Factors in the political context were associated with evidence briefs’ influence through the longitudinal pathway.

Final package of resources
We developed and evaluated a set of resources (the SURE Guides) for preparing and using evidence briefs for policy and rapid responses. These resources are available through the project’s website. The resources are universally relevant, use examples from Africa as illustrations, and build on the experience of the seven African SURE partners.

Project Results:
The SURE project included eight work packages:
1) Relevant research syntheses to address priority policy questions
2) Strategies for improving access to research syntheses and policy-relevant research evidence
3) Mechanisms to respond rapidly to policymaker needs for research evidence
4) Strategies for facilitating the use of research evidence to inform policy decisions
5) Building capacity for evidence-informed health policy
6) Evaluation of African initiatives to improve the use of research to inform health policy decisions
7) Dissemination of project results
8) Project management

We present a description of the main results from each of the work packages here.

Work package 1: Relevant, reliable, accessible and timely research syntheses

Objectives: To produce research syntheses for policymakers that are relevant, reliable, accessible and timely

Since the start of the project, the African partners of the SURE project have developed research syntheses on topics identified as priorities within their setting. We refer to these reports as “evidence-based policy briefs” or “evidence briefs for policy” (1). The seven African partner countries each identified priority policy questions on which to focus the evidence briefs, using different approaches to engage policymakers and stakeholders in this process. They prepared evidence briefs to address priority topics using the methods described in the SURE Guides. This included systematically searching for and summarising the best available evidence to:
• Describe the size of the problem and causes of the problem
• Identify options to address the problem and summarise the effects of those options
• Identify facilitators of and barriers to implementing those options and implementation strategies that take account of those

As part of the development process for the evidence briefs, country teams consulted various stakeholders including policymakers, non-government organizations, and representatives of civil society, to engage them in development of the evidence briefs and ensure the relevance of the evidence briefs. The country teams used a user-friendly template developed and tested by the SURE project for the evidence briefs. Each evidence brief includes a description of the problem, policy options and implementation strategies based on the best available global and local evidence. The evidence briefs use a graded entry format that accommodates the needs of different users, including a 1-page summary of the key message, an executive summary and the full report. This format was user tested and evaluated following each policy dialogue. External referees review each evidence brief in each country and at least two international experts review each evidence brief. All of the evidence briefs are published on the SURE website after they have been peer reviewed. In addition, the evidence briefs are accessible through the EVIPNet Virtual Health Library and PDQ-Evidence clearinghouses, developed by the SURE project. Versions of some of the evidence briefs have been published in the International Journal of Technology Assessment in Health Care (2 - 11).

The following evidence-based evidence briefs have been prepared:
Uganda (Partner 2, REACH)
• Task shifting to optimize the roles of health workers to improve the delivery of maternal and child healthcare
• Improving access to skilled attendance at delivery
• Advancing the integration of palliative care in the national health system
• Improving patient safety in Uganda
• Sustaining knowledge translation for health policymaking in Uganda
Cameroon (Partner 3, MSP-Cam)
• Improving governance of district health services
• Increasing health insurance coverage
• Retaining Human Resources for Health in Remote Rural Areas
• Increasing antenatal care coverage
• Financial accessibility of emergency rooms
• Quality of care in emergency rooms
• Community health workers programme
Burkina Faso (Partner 4, MS-BF)
• Implementation of promising strategies for reducing maternal mortality in Burkina Faso
• Strategies for improving allocation of the budget for health
• Strategies for achieving universal health insurance
• Strategies for the reduction of smoking in Burkina Faso
• Sustaining a rapid response service in Burkina Faso
Centrafique (Partner 5, FACSS-RCA)
• Strategies for reducing maternal mortality in Centrafique
• Reduction of infant and child mortality through better management of malnutrition and HIV/AIDS in Centrafique
• Strategies to reduce mortality and morbidity due to cervical cancer
• Strategies for managing public health emergencies
Ethiopia (Partner 6, EHNRI)
• Human Resource Capacity to Effectively Implement Malaria Elimination in Ethiopia
• Prevention of Postpartum Haemorrhage in Rural Ethiopia
• Improving skilled birth attendance in Ethiopia
• Improving health care financing in Ethiopia
• Improving the effectiveness of the health extension workers programme in Ethiopia
Zambia (Partner 8, ZAMFOHR)
• Integrating mental health in primary care in Zambia
• Reducing deaths due to postpartum haemorrhage in homebirths
• Implementation of health worker retention strategies
• Strengthening human resource for Mental health – A response to challenging trends
• Sustaining evidence-informed health policymaking in Zambia
Mozambique (Partner 13, UEM)
• The retention of health workers in rural and remote areas
• Pay for performance
• Reducing neonatal mortality in Mozambique
• Access to adolescent health services

Policymakers, researchers and other stakeholders discussed each completed evidence brief in a policy dialogue. We surveyed participants who read the evidence briefs and attended policy dialogues before the start of the dialogues – to collect their views on the evidence briefs – and at the end of the dialogues – to collect their views on the dialogues. We investigated the respondents’ assessments of the evidence briefs and dialogues and the respondents’ intentions to act on what they had learned in descriptive statistical analyses and regression models (12). Of 530 individuals who read the evidence briefs and attended dialogues, 304 (57%) and 303 (57%) completed questionnaires about the policy briefs and dialogues, respectively. Respondents viewed the policy briefs and deliberative dialogues – as well as each of their key features – very favourably, regardless of the country, issue or group involved.

Deliverables for WP1
1. Research syntheses

Work package 2: Strategies for improving access to research syntheses and policy-relevant research evidence

Objectives: To develop and evaluate two strategies, user-friendly formats and clearinghouses, for improving policy-makers’ access to research syntheses and policy-relevant research evidence.

User-friendly formats
We iteratively designed a user-friendly format for evidence briefs based on input from researchers, policymakers from the African partners and a graphic designer. Our objective was to develop a format that includes key information that is needed to facilitate the use of research evidence in policy decisions, and is easily accessible to policymakers.

We prepared examples of formatted evidence briefs using evidence briefs prepared by the seven African SURE partners. An advisory group, made up of policymakers and researchers from the SURE collaboration, reviewed these and provided feedback and suggestions. Participants in policy dialogues, where the evidence briefs were discussed, also provided feedback and suggestions.

We performed usability tests of the initial design and of a subsequently revised design with policymakers in the partner African countries. Examples of evidence briefs prepared by the seven African SURE partners were formatted using the user-friendly format. We included a purposive sample of policymakers and other stakeholders from the African partner countries. We chose the sample purposively to ensure a breadth of perspectives from relevant target audiences. User testing of the user-friendly evidence briefs consisted of providing the policymakers and stakeholders with a relevant scenario and the evidence brief followed by a series of tasks and questions using a semi-structured approach and a structured format for recording responses and observations (13-17). Observers recorded how participants used the evidence briefs any comments they made. Audio and video recordings were transcribed and reviewed by members of the research team and compiled. We made subsequent adjustments to the format based on the results of the user testing and the advisory group was consulted again.

At least two members of the research team reviewed all of the transcripts. To counteract possible biases that our involvement in the development of the user-friendly formats might represent we also included a researcher with no connection to the project to analyse the data. We used the framework analysis approach (18), involving five key stages of analysis: familiarization with the data; identification of a thematic framework; indexing; charting; and finally mapping and interpretation, was used to analyse the data from the focus groups. Each researcher independently carried out the first three stages, where themes were identified and data coded. The researchers then met to discuss and agree upon a common thematic framework, and re-coding was carried out where necessary. All of the researchers collaborated in the final two stages, where themes were charted, mapped and interpreted. Quotes were chosen that express common experiences, attitudes or topics or because they show the breadth of observed experiences. The informants were given fictitious names in order to protect their identity.

We undertook quantitative and qualitative evaluations of the evidence briefs by participants in policy dialogues (12), which are reported as part of Deliverable 6 (Evaluation of African initiatives to improve the use of research to inform health policy decisions).

Subsequent adjustments were made based on the results of the evaluation and the advisory group was consulted again before finalising the changes and producing the final version of this deliverable. Policymakers and other stakeholders found the overall format to be attractive. They found the language simple and the format of the evidence briefs useful and usable. Institution logos, key messages and information box (explaining what the evidence brief was) to be very useful. We identified a need to distinguish very clearly between the short and full reports on the cover pages, as the difference was not instantly recognisable. Views regarding whether there should be separate documents for short and full reports or a single document varied across countries.

The revised format features distinctive title pages for three versions of the evidence briefs - Key Messages, Executive Summary and Full Report - with unique colour palettes. Templates for alternative formats with two separate documents or a single document allow countries to tailor the presentation to accommodate the preferences of users of the evidence briefs.

We revised the formats to reflect the changes from the user testing in Uganda. Each of the African partners then used these formats for developing their evidence briefs. We conducted additional user testing of the revised draft formats of evidence briefs with national stakeholders and policymakers from three additional African countries: three additional tests from Uganda (Partner 2), three tests from Zambia (Partner 8), three tests from Mozambique (Partner 7) and one test from Cameroon (Partner 3). The results of these tests are summarised here.

The feedback from the users was largely positive. The overall format is attractive for users and the language is simple for the full report. Institution logos, key messages and information box were found to be very useful by six participants across countries.

There was a need to distinguish very clearly cover pages for both short and full reports, as the difference was not instantly recognisable. Differences in size of reports was a more helpful indicator than information on the cover page. The new format will feature distinctive title pages for the three versions of the report, Key Messages, Executive Summary and Full Report with unique colour palettes.

Four users thought short report should have references, two users thought otherwise and suggested that it should not, to maintain overall simplicity and brevity. However, some users suggested that both the short and full reports are rather long. It may not be helpful to lengthen this further with references. A workable compromise here would be to have the reference numbers within the text of the executive summary match with those in the main report, which contains the full bibliography. 8 Two Uganda and two Mozambican users prefer to maintain separate documents for short and full reports so as not to put off busy readers. All the Zambian readers preferred to have one report. The revised format was amended to follow Partner Country preferences.

As part of the evaluation of the evidence briefs (work package 6), some questions reflect on the user experience of the evidence brief. An analysis of the comments received from various stakeholders that participated in several policy dialogues was conducted and suggestions made on how to improve the content and layout of the evidence brief. Below is a summary that highlights some of the feedback obtained and suggestions that are under consideration for the improvement of the evidence brief layout:

Add more local information in the problem description section
• Case-boxes with text scenarios, testimonials illustrations/photos when describing the problem
• More local information/facts about the problem, more “detailed situations” report
• Reframing the desire for recommendations

Be clearer about providing process guidance rather than recommendations
Could include a process illustration at the start showing where one is in a larger process: (Problem definition, Options, Barriers/Facilitators, Dialog, and Decisions – and where different types of feedback/information come into this process). This could help clarify where the decision-making happens and could make the methodology of the brief more clear, for those who felt that was unclear.

Suggestions for specific types of content included the following
Changes to graphics or evidence tables, definitions. Suggest including bibliographic references in order of importance, with short summaries of content (annotated bibliographic reference list).

The need for involving more stakeholders
Consider the need for decision-aid type of material – like decision cards, or presenting each option on one sheet of paper or power point presentation for local settings, in order to involve/gather feedback from more people. This could build on the same content in the brief, but just be organized differently so that it was more like a poster (or a set of posters) than a report.

Revisions of the user-friendly formats
The evidence brief templates were revised following feedback from the African partners and the results of the evaluation of the evidence briefs undertaken as part of Work package 6. These new templates were tested with the same participants from the previous testing before being finalised.

Clearing Houses
We developed two clearinghouses using different approaches to providing rapid access to research evidence about health systems. Pretty Darn Quick Evidence (PDQ-Evidence) is a database of health system evidence that is updated regularly by searching multiple sources for relevant systematic reviews. It includes over 3200 systematic reviews of health system arrangements, implementation strategies and public (population) health policies, all of the studies included in those systematic reviews (over 28,000), over 240 overviews of systematic reviews (including policy briefs or evidence briefs for policy) and over 1000 structured summaries. Users found it easy to use and quick (Box 1).

The Evidence Informed Policy Network Virtual Health Library (EVIPNet VHL) is a meta-search engine that simultaneously searches multiple databases, including Health Systems Evidence, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE) and LILACS. It provides access to both health system and other systematic reviews and other types of research evidence. It is possible to search PDQ-Evidence and the EVIPNet VHL in English, French, Portuguese and Spanish. The EVIPNet VHL is an online repository, and was launched at the 66th World Health Assembly to support country teams in undertaking evidence-informed health policymaking efforts (e.g. preparing evidence briefs for policy). It aims to ensure that global policymakers and stakeholders have timely access to the full range of policy-relevant research evidence through the establishment of a comprehensive online “one-stop shop.” It is the result of a partnership between the Pan American Health Organization (PAHO) and the McMaster Health Forum and involves the integration of the Health Systems Evidence database with PAHO’s Virtual Health Library, which is maintained by the Latin American and Caribbean Centre on Health Sciences Information (known by the acronym BIREME). The integration of these two sources allows for both the timely identification of research on the programs, services and drugs that can be adopted to improve health, and the health systems arrangements and implementation strategies that can support getting cost-effective programs, services and drugs to those who need them. Furthermore, the EVIPNet VHL is available in English, French, Portuguese and Spanish, facilitating access to its resources in a number of languages that are used in African and Latin American countries. Box 2 provides a snapshot of the EVIPNet VHL.

In addition, three partner countries (Uganda, Zambia and Cameroon) have developed country-specific clearinghouses with funding obtained from alternative sources. Those clearinghouses provide access to local (national) evidence and links to PDQ Evidence and the EVIPNet VHL.

Deliverables for WP2
2. User-friendly formats for research syntheses
3. Clearing houses for research syntheses and policy-relevant research evidence

Workpackage 3: Mechanisms to respond rapidly to policymaker needs for research evidence

Objectives: To develop and evaluate mechanisms for responding rapidly when policymakers need research evidence to inform the responses within hours or days.

Policymakers occasionally require quick access to research evidence in order to make decisions. Rapid response services, as part of the SURE Project were established in order to support policymakers in the health sector to access and use research evidence when it is needed in hours or days. These services provide an opportunity to develop a better understanding of policymakers needs for research evidence in their daily work, potential impacts on decisions that are made urgently when there is a real or perceived need to respond, and potential impacts on attitudes and the general climate in which policy is developed.

Development of the rapid response services was guided by findings from a literature review of
services with similar objectives. This was followed by pilot testing of the service structure to
evaluate its feasibility, identify problems and test solutions for these. Phase II sought to expand the service to include all policymakers and managers in the country level health system and assess the viability of the service on a wider scale. Throughout this process, user testing was done to receive feedback on the experience of the users of the service with its products. All complete reports from this service are now accessible through the Uganda Clearing House for Health Systems and Policy Research.

A set of tools has been developed for the rapid response service. Box 3 summarises the tools that have been utilised during the process of developing the response services.

A process evaluation of the Uganda rapid response service was conducted during the third period (see Appendix 1). At the time of evaluation, about 30 active users had utilized the service. The majority of the users are from the Ministry of Health, the others include bi-/multi-lateral organizations, civil society, private sector. There is variation in qualifications of users including policymakers with a medical background, those with research experience at a bachelor’s, master’s and doctorate degree level.
None of those approached and offered the service out rightly declined the service. However it is difficult to establish whether those that knew of the service and did not use it declined quietly or just genuinely did not get a chance to use it.

Annually, on average the service has delivered/disseminated 19 briefs. Specific outputs for each year are as follows: 2010 – 21 briefs, 2011 – 36 briefs, 2012 – 12 briefs, 2013 (until August) – 10 briefs. The majority (26.1%) of the questions were about governance issues in the health system, followed by those about organizational arrangements (21.5%). There were 18.5% health technology questions, 16.9% about implementation strategies, 13.8% on financial arrangements and 6.2% on other topics like public health.
The average duration in which answers were needed was 13 (12.8) days. While the modal time for response was 21 days, two questions required responses in 24 and 48 hours each and another six questions required response in 5 days. The maximum time that was given for a request was 28 days. Most (81%) of responses were returned on time, 11% were returned later than the allotted time and 8% were not followed through to the end. Some of the reasons given for responses returned late were delayed review process and delivery of reports through a third party, for example a secretary. For those not followed through to the end, one of the reasons noted was that the policymakers were not available to clarify the question.

Other rapid response services
Burkina Faso has continued to develop its rapid response service, modelled on the Ugandan service. The staff person leading the service in Burkina Faso spent several months at Makerere University during the initiation phase. Fourteen rapid responses have been provided by the time of developing this repor. An important difference between the two services is that the one in Burkina Faso is based in the Ministry of Health, whereas the one in Uganda is based at a university. The Cameroon rapid response service has been operational from May 2012 and has completed six rapid responses so far. These cover the following topics: Increasing PMTCT uptake; increasing coverage of skilled attended deliveries; the effectiveness of antimalarial vaccine candidates; increasing immunization coverage rate; financial support and protection against HIV infection, improving access to care of underprivileged citizens. The Zambian rapid response service piloted two rapid responses for the Ministry of Health.

Deliverables for WP3
4. Mechanisms for responding rapidly to policymakers’ needs for evidence

Work package 4: Strategies for facilitating the use of research evidence to inform policy decisions

Objectives: To develop and evaluate methods for conducting deliberative processes that are informed by research syntheses and for involving civil society and the general public in policy development and supporting their access to and use of research evidence.

As part of the SURE Project, guidelines for the deliberative forums (policy dialogues) were developed. One guide is entitled: Informing and engaging stakeholders; and has been developed to guide decisions about how to inform and engage stakeholders in preparing and using a research synthesis (evidence briefs). Four main questions are used to guide this process:
1. Which stakeholder groups should be informed and engaged in preparing and using the policy brief?
2. What contextual factors might affect efforts to engage stakeholders?
3. How will different stakeholder groups be engaged in preparing and using the evidence brief?
4. What difference will informing and engaging stakeholders make and how will this be evaluated?

Another SURE guide entitled Organizing and running policy dialogues addresses the following questions:
1. What are the objectives of the policy dialogue?
2. Who will participate in the dialogue?
3. How will the dialogue be organised?
4. What needs to be done following the policy dialogue?

Policy dialogues have been held in the seven African partner countries to discuss the evidence-based evidence briefs. Policy dialogues are interactive, knowledge sharing mechanisms that facilitate interaction between researchers and policy makers. They provide an opportunity for stakeholders to review the research evidence and consider the policy options and implementation strategies suggested in the evidence briefs. The policy dialogues have brought together key stakeholders in the country. This includes:
• Policymakers – including elected officials, political staff and civil servants – at national and sub-national levels of government
• Managers in districts or regions, healthcare institutions (e.g. hospitals), non-governmental organisations, and other relevant types of organisations
• Civil society groups – including consumer groups, health professional associations and other relevant groups
• Researchers in national research institutions, universities, and from other countries

All of the African partners have engaged stakeholders in setting priorities for evidence briefs, using different approaches to do this. For example, Zamfohr in Zambia (Partner 8) has networks of stakeholders in six key areas and these networks have set priorities for evidence briefs in their respective areas. One advantage of this approach has been to create opportunities to put problems on the policy agenda, such as mental health, which might otherwise not have received attention. Other partners have consulted and met with policymakers and other stakeholders to elicit and prioritise topics.

Partners have also engaged stakeholders in developing the evidence briefs. REACH Uganda (Partner 2), for example, has included relevant mid-level policymakers from the Ministry of Health and researchers with relevant expertise in the teams that have developed evidence briefs. They have also consulted key stakeholders while developing the evidence briefs. They have also engaged stakeholders by sending each section of an evidence brief for review or meeting with stakeholders to discuss each section.

The public has been informed about evidence briefs and policy dialogues by including journalists as participants in policy dialogues, discussing evidence briefs with journalists and making the evidence briefs and reports of policy dialogues open to the public. Partner 3 (Cameroon) conducted a survey of 40 media and civil society representatives to assess their needs for improving the use of research evidence in decision-making. Guided by this needs assessment, two workshops were held in Yaoundé (25 – 26 November 2011 for 18 participants) and Douala (2-3 December 2011 for 22 participants) to introduce concepts and principles of evidence-informed heath policymaking to media and civil society organisations.

A report of each policy dialogue has been prepared, summarising the deliberations. The template used to structure these reports includes the background, the problem, the policy options, implementation considerations and next steps. Examples of outcomes from policy dialogues are presented in Box 4.

Deliverables for WP4
5. Methods for stakeholder involvement

Work package 5: Building capacity for evidence-informed health policy

Objectives: To build capacity among researchers, policy makers and civil society for developing and implementing evidence-informed health policies.

Annual workshops
We organised a series of workshops to build capacity of the SURE project partners and others to prepare and support the use of evidence-based evidence briefs within the SURE project partners (Table 1).

Participants evaluated the content and delivery of each workshop using a structured form. They have consistently assessed the workshops and workshop materials positively, with a median overall assessment of 6 (on a scale from 1 to 7). They have made numerous suggestions for ways to improve the workshops and the workshop materials. We have used this information to guide the organisation of subsequent workshops and to make improvements in our resources for capacity building.

International Forum
In year four, we organized 35 workshops as part of the International Forum on Evidence Informed Health Policy in Low- and Middle-Income Countries 27 – 31 August 2012. The International Forum on Evidence-Informed Health Policy (EIHP) in Low- and Middle-Income Countries (LMICs) brought together 119 policymakers, stakeholders and researchers from 27 countries to:
1. Share experiences related to EIHP in LMICs
2. Identify opportunities for improvement in EIHP initiatives
3. To provide an opportunity for networking among initiatives that support EIHP

The event consisted of plenaries, and small group sessions that were organized around five key themes: 1) Evidence informed health policy in action; 2) Skills; 3) Tools; 4) People; and 5) Collaboration and innovation. The Forum as a whole - and some of the collaboration and innovation sessions in particular - helped to further establish and promote a climate that supports the use of research in LMICs by bringing together a range of policymakers, stakeholders and researchers from several countries.

Additional workshops, traineeships and exchanges
Additional workshops, traineeships and exchanges have been organised each year. These have included in country workshops, in the seven African partner countries, and international workshops. For example, Partners 1 (the Norwegian Knowledge Centre for the Health Services) and 2 (Makerere University) organised a workshop on systematic reviews in Kampala, Uganda and the principal investigator (Partner 1) was a guest researcher with the Ugandan partner (Partner 2 - REACH) during the first year of the project.

Examples of traineeships and exchanges include someone from Burkina Faso spending several months in Uganda receiving training and practical experience with rapid responses, training of others in Cameroon and Zambia to prepare rapid responses by the Ugandan and Burkina Faso teams, visits by the Ethiopian team to Oslo for training and support preparing evidence briefs, visits to the African partners by researchers from the European and Canadian partners, and course work by people from Cameroon and Uganda at McMaster University and the Karolinska Institute.

Five people have or are completing PhD studies based on SURE Project activities (Table 2). They have obtained technical support from universities that are partners in this project (Makerere University (Partner 2), McMaster University (Partner 12), and Karolinska University (Partner 11).

SURE partners have hosted several other workshops, including ones that have targeted participants beyond the SURE project group . These have largely focused on building the capacity of researchers to develop and interpret findings from systematic reviews and knowledge translation. Several African partners have highlighted this need, which we therefore had as a priority. Policymakers have attended some of the workshops that we have offered together with researchers.

Capacity building resources
We have developed capacity-building resources and workshop materials. These are resources that are intended primarily for those people responsible for preparing and supporting the use of evidence briefs and ensuring that decisions about health systems are well-informed by research evidence. The guides focus specifically on these issues in the context of African health systems and the examples used are taken from evidence briefs that address important problems in African countries. These include 16 packages of material for workshops. The packages include objectives, reading material, examples, worksheets, notes for facilitators and PowerPoint presentations. These resources are incorporated in the SURE Guides (Figure 1).

The 16 workshops cover:
1. What is evidence-informed policymaking?
2. Improving how your organisation supports the use of research evidence to inform policymaking
3. Introduction to evidence briefs
4. Setting priorities for evidence briefs
5. Clarifying the problem
6. Finding and using evidence about local conditions
7. Deciding on and describing options to address the problem
8. Finding systematic reviews
9. Deciding how much confidence to place in a systematic review
10. Judgements about the quality of evidence and summarising findings
11. Finding and using research evidence about resource use, costs and cost-effectiveness
12. Identifying and addressing barriers to implementing the options
13. Incorporating considerations about equity in evidence briefs
14. Clarifying uncertainties and needs for monitoring and evaluation
15. Organising and running policy dialogues
16. Informing and engaging stakeholders

Led by WHO (Partner 10), the SURE Guide and the SUPPORT Tools (developed as part of FP6 project) were adopted to an e-learning platform. This online tool helps users to develop an evidence brief for policy of their chosen topic and teaches users how to organize a policy dialogue. In addition, the course provides a learning opportunity about evidence informed policy-making and knowledge translation. The e-learning platform is adapted and transformed from Moodle.

The course includes the five modules, with the SURE Guide content:
• Module 1 – What you need to know before starting?
• Module 2 – How to prepare an Evidence Brief for Policy?
• Module 3 – How to use an Evidence Brief for Policy and organize a stakeholders dialogue?
• Module 4 – How to become facilitator in Evidence Informed Policy-Making (EIPM)
• Module 5 – Know more about EIPM in WHO and EVIPNet programme

Deliverables for WP5
6. Resources for capacity building

Workpackage 6: Evaluation of African initiatives to improve the use of research to inform health policy decisions
Objectives: The objectives of the evaluation were to:
1) Evaluate each of the African partners’ initiatives annually and provide them with feedback from their own evaluation and in comparison with the other initiatives
2) Assess the advantages and disadvantages of the different approaches that are taken and examine contextual factors that may influence how well different approaches work in different settings
3) Assess the impact of two initiatives: EVIPNet in Cameroon and REACH in Uganda

A number of units have begun to experiment with efforts to address the challenges of linking research evidence to policy about health systems. These units, or ‘knowledge-translation platforms’ (KTPs), seek to convert the knowledge arising from research into the types of action on the ground that can mean the difference between life and death or the difference between good health and bad health.

A comparative evaluation of the KTPs in seven African countries – Burkina Faso, Cameroon, Centrafrique, Ethiopia, Mozambique, Uganda and Zambia was conducted as part of the SURE Project.

To address objective 1, we completed evaluations of the two primary outputs/activities of KTPs in Africa – evidence briefs and policy dialogues – and specifically we surveyed 304 policymakers, stakeholders and researchers about briefs (n=18) and 303 about dialogues (n=17). The results from this component of the evaluation were recently published in the Bulletin of the World Health Organization (12). Key findings included that evidence briefs and policy dialogues, as well as each of their features, were very highly regarded and that policy dialogues (informed by evidence briefs) led to an intention to act on what was learned. We supplied each of our SURE partner countries with an updated country evidence brief and dialogue evaluation report, whenever a new evidence brief was prepared or dialogue convened (19-24).

To address objective 2, we conducted annual KTP profiles and activities and outputs inventories, undertook a critical interpretive synthesis, empirically assessed how context- and issue-related factors influenced views about evidence briefs, and undertook a structured-reflection exercise.

The profiles and inventories identified that: 1) the governing bodies of those KTPs that had them (i.e. all but Cameroon, Centrafique and Mozambique) were composed of a good balance of policymakers, stakeholders and researchers, and more than half of the KTPs were housed within a research institution; 2) presentations were the most often reported activity to support the creation of a climate conducive to the use of research 3 )evidence in policymaking; 3) priority-setting events were the most reported activity to identify specific policy challenges/issues that could be informed by research evidence; 4) most KTPs were focused on the production of evidence briefs (rather than systematic reviews, new primary research, or the training of researchers); 5) giving presentations at conferences were the commonly most reported activity to disseminate research evidence; and 6) research resource listings, clearinghouses, databases and websites were the most common activity undertaken to facilitate policymakers' ability to acquire, assess, adapt, and apply research evidence to policy priorities.

The key findings from the critical interpretive synthesis were as follows: 1) contextual factors, particularly the institutions, interests, and values within a given context, can influence views of evidence briefs; 2) whether an issue is polarizing and whether it is salient (or not) and familiar (or not) to actors in the policy arena can influence views of evidence briefs prepared for that issue; 3) influential factors can emerge in several ways (as context driven, issue driven, or a result of issue-context resonance); 4) these factors work through two primary pathways, affecting either the users or the producers of briefs; and 5) these factors influence views of evidence briefs through a variety of mechanisms. The full critical interpretive synthesis, including the resulting model, was published in the Milbank Quarterly (25).
The empirical assessment found that country (a proxy for many unmeasured contextual factors) and the number of evidence briefs prepared in the country were each significant predictors of views about briefs in six of nine regression models. The findings are currently available as a chapter in a doctoral thesis and will soon be submitted as a manuscript to a journal (26).

The key findings from the structured-reflection exercise were as follows: 1) policy dialogues informed by evidence briefs were identified as the most commendable tools for enhancing evidence-informed health policymaking (EIHP) and were perceived as the most impactful of the activities and outputs in all countries; 2) KTPs contributed to increased awareness of the importance of EIHP and strengthened relationships among policymakers, stakeholders, and researchers; 3) support from policymakers and international funders facilitated KTP activities, while the lack of skilled human resources to conduct EIHP activities impeded KTPs; 4) ensuring the sustainability of EIHP initiatives after the end of funding was a major concern for KTPs; and 5) the ‘institutionalization’ of KTPs within the government helped to retain human resources and secure funding, whereas KTPs hosted by universities highlighted the advantage of autonomy from political interests (27). These findings have already been published in an article in Health Research Policy and Systems (27).

To address objective 3, we conducted seven case studies involving three KTPs in two separate studies using multiple case study designs. Four cases were sampled from Uganda and Zambia (two from each country) (28) and three cases, with a particular focus on social networks, were sampled from Burkina Faso (29). In three of the four cases in Uganda and Zambia, the KTPs’ evidence briefs influenced ideas through a longitudinal pathway by initiating potential shifts in the way actors perceived various aspects of the policy issue. In one case (health human resources in Zambia), the KTP’s evidence brief influenced the policy process through a cross-sectional interaction with existing ideas and institutions, and resulted in an incremental policy change. Issue characteristics, and in particular whether an issue is familiar or not, emerged as an important determinant in explaining the nature of the influences of evidence briefs. Factors in the political context were associated with evidence briefs’ influence through the longitudinal pathway. Of the cases studied in Burkina Faso, the one in which the KTP was most active– home management of malaria – was the one where research evidence played the least role (although select interviewees suggested that the KTP’s evidence brief did play a key role in the framing of the country’s application to the Global Fund to Fight AIDS, Tuberculosis and Malaria). These results have been published as doctoral theses and will soon be submitted to peer-reviewed journals.

We also undertook and published three complementary studies, which were not formal SURE deliverables but which complemented these deliverables and in most cases explicitly cited the SURE project as a source of funding for the data and/or analysis being presented: 1) a print media analysis about whether and how policymakers, stakeholders, and researchers talk in the media about three topics: policy priorities in the health sector, health research evidence, and policy dialogues regarding health issues (published in Health Research Policy and Systems in 2011) (30); 2) a description of systematic-review production (published in the Journal of Health Services Research and Policy in 2012) (31); and 3) an evaluation of the SURE-supported international conference (published in Health Research Policy and Systems in 2014) (32).

While also not formally funded by SURE, a number of doctoral students have been able to capitalize on the SURE project and will in the coming year or two defend their doctoral theses on the following aspects of the SURE project: 1) rapid-response services; 2) one-stop shops / clearinghouses (33); 3) national ‘workbooks’ that allow for the local contextualization of global guidance through the use of evidence briefs and policy dialogues; and 4) research conducted by Ugandan masters and doctoral students.

In order to conduct the baseline assessments and subsequent evaluations mentioned above, ethical approval was sought and obtained from the McMaster Research Ethics Board (REB), country REBs (specifically Burkina Faso, Ethiopia, Uganda, and Zambia), as well as the WHO REB.

Deliverables for WP7
7. Evaluation

Workpackage 8: Dissemination of project results

Objectives:To ensure wide dissemination and use of the results of this collaborative project.
During the first year of the project, the SURE Website was built as an ‘add on’ to the global
WHO EVIPNet website based both at the WHO website (http://www.who.int/rpc/evipnet/en/) and in the EVIPNet Portal (http://www.evipnet.org/sure). This was done not only to take advantage of the existing infrastructure and resources, but also to target the audience already accessing the EVIPnet website. These would be the same audience we hope to reach with the information generated by the SURE project. Subsequently, the SURE website was migrated to a new site: http://www.who.int/evidence/sure/en The World Health Organisation (partner 10) continues to work with the coordinator (Partner 1) to update the SURE website. Updates to the website include newly completed evidence briefs.

Highlights and news from the SURE project have been disseminated in several publications of the EVIPNet Newsletter. At the time of developing this report, the EVIPNet Newsletter had 960 members that are registered and receive email alerts with every publication of the Newsletter. In addition, EVIPnet news is circulated to 488 members of the HIFA-EVIPNet francophone group (Global discussion forum for francophone countries).

The SURE videos that were produced during the first period provided an overview of planned activities of the SURE project in relation to the on the ground realities at the start of the project. We developed an additional set of videos that can be used alongside the SURE guides as training materials but also for dissemination of SURE project activities. These videos share examples of products from the project for example how to conduct a policy dialogue and from the process of developing evidence briefs among others. These videos are integrated with the SURE guides and the 28 videos, all of which
are available in both English and French, have been posted on YouTube (https://www.youtube.com/user/TheSUREGuides).

Dissemination activities have also been conducted in languages other than English. In addition to the videos that were translated to French and Portuguese during the first period of this project, some of the evidence briefs (by Cameroon, the Central African Republic and Burkina Faso) have been produced in French, and the SURE Guides and SURE Guide videos were translated to French. In addition, SURE products have been disseminated through HIFA-EVIPNet-Fr, a discussion list in French that promotes the dissemination of relevant scientific literature for evidence-informed policymaking and SURE activities.

Led by Partner 1, we successfully organised a session at the Health Systems Symposium in Beijing (31 Oct – 3 Nov 2012). Approximately 30 participants attended this session and participated in discussions with the SURE partners. The main objectives of this session were to:
• Share experiences from supporting the use of research evidence in healthcare policymaking in Africa,
• Discuss the lessons learned while developing and evaluating three key strategies – evidence briefs, policy dialogues and rapid response services, and
• Discuss the possible way forward for further development and evaluation of these strategies.

A working group led by Partner 10 (WHO) successfully organized an International conference with the title: ‘Evidence-Informed Health Policymaking in Low- and Middle-Income Countries: An International Forum’ that took place from 27 to 31 August 2011, in Addis Ababa, Ethiopia. The main aims were:
• To share and further develop resources to support evidence-informed health policymaking (EIHP) in low and middle-income countries
• To build capacity for EIHP in low and middle-income countries
• To provide an opportunity for networking among initiatives to support EIHP
• To promote EIHP in low and middle-income countries

The forum built on lessons learned from previous international conferences in the past five years, including the Ministerial Summit on Health Research, IDEAHealth (International Dialogue on Evidence- Informed Action in Health in Developing Countries), the Global Ministerial Forum on Research for Research for Health planned for 2008, and experiences with workshops and deliberative forums from SURE, REACH, and EVIPNet in Africa, Asia, Latin America, and the Eastern Mediterranean regions.

The conference-targeted policymakers, researchers and civil society representatives linked to initiatives to support EIHP in low- and middle-income countries. One hundred and nineteen participants from 27 countries took part in the forum that predominantly used small group, interactive approaches rather than large plenary sessions and didactic presentations. It was organised around five key themes: evidence informed health policy in action, skills, tools, people, collaboration and innovation. The summary of the activities implemented at the forum and main outcomes of this conference can be found in a report that has been published. (http://www.who.int/evidence/Addisreport2012.pdf).

Each of the SURE partners also utilised opportunities to disseminate our activities at local and international fora. Several presentations and workshops have been held, and some of our work published in peer-reviewed journals (the lists of workshops, presentations and publications have been uploaded onto the online reporting tool).

Deliverables for WP8
8. SURE website




Workpackage 8: Project coordination and management

Objectives: To ensure that the project runs smoothly and achieves its aims. It will also ensure that the project’s tools and resources are packaged into an easily accessible form and produce the final report.

A project management group (Partner Leads of 1, 2, 3 and 10) maintained overall coordination and decision making for the SURE Project. Email communication enabled regular communication and discussion of issues as they arose. The Partner Leads of each country team were responsible for managing the day-to-day project activities within partner countries. The principal investigator (Dr. Andy Oxman) maintained overall leadership of all work packages. Additional support was provided to the Management group by Kite Innovation (Europe) Ltd (www.kiteinnovation.com). This support facilitated the financial management of the project for example by maintaining an overview of the consortium’s budget, and supporting partners with the financial reporting process.

Several telephone meetings were held between partners during preparations for the international forum and on other instances when the coordinator needed to discuss issues with partners. Face-to-face meetings were held during the annual meetings over the course of the project. At each of these meetings, we discussed progress towards project objectives, strategies to improve delivery of our tasks and other project management issues. In addition, meetings were held between the coordinator and each partner lead to discuss use of funds and reporting, and any issues specific to the partner country.

As part of Work package 8, the resources which we developed in other work packages, were brought together in a way that ensures that the results of the SURE project reach researchers and policymakers who are the end user. The resources have been collated into an online package to support the development of evidence briefs, use of formats for research syntheses; guidance for responding rapidly to policymakers’ needs. These resources are available through the project’s website.

We continue to work closely with the EVIPNet and REACH groups as well as two other FP7 projects - African Regional Capacity Development for Health Services and Systems Research (ARCADE-HSSR); Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE) - and a DFID project (HRPC09 Evidence Building and Synthesis Research), as well as the other partners in each of these projects. Other groups with which we cooperate include WHO and the Cochrane Collaboration, among others. The SURE partner countries have also worked closely with several stakeholders, such as Ministries of Health and national programmes, some of which are funded by international organizations and non-governmental organizations.

Deliverables for WP8
9. Final package of resources







Potential Impact:
Supporting strong scientific partnerships with LMIC
The World Health Report 2003 stated that there will be only limited progress towards the United Nations Millennium Development Goals and other national health priorities without the development of health care systems that respond to the complexity of current health challenges (34). The main barrier to improving health in LMIC may be poverty but there are also grave problems with healthcare systems, in selecting and implementing effective health policies and in the delivery of care. Simply providing more aid is not sufficient - healthcare systems in some of the poorest countries are unable to absorb more than a few extra Euros per person (35). Healthcare systems in LMICs need support to ensure that their existing healthcare budgets are spent on designing and implementing effective healthcare policies that help the greatest number of people.

SURE involved an excellent partnership between LMIC, European and Canadian scientists, WHO and two LMIC networks (EVIPNet and REACH) that promote evidence-based policymaking. Its scientists are at the cutting edge of international research into how best to transfer research into policy and its LMIC policymakers are in a position to both influence and adopt SURE’s methods, tools and best practice. For example, three of SURE’s partners are African health ministries: these could make significant changes to how health policy is developed as a result of SURE. Our WHO link provides a solid body of LMIC experience and unrivalled global dissemination channels; links to EVIPNet and REACH provide policymaking contacts and experience, together with regional dissemination channels. This provides proven dissemination channels through which to spread SURE resources and experience to policymakers and researchers in partner countries and beyond. Finally, our partnership was strongly anchored in LMIC, which ensures that these countries have principal ownership of the outputs of the project.

Putting the evidence into evidence-based policymaking
The process of making health policy is highly complex. Policymaking is influenced by a wide range of interests at local, national and international levels; by institutions such as ministries of health and service delivery agencies; ideas, including beliefs about the effects of different policy choices and how policies have been received elsewhere; and events, such as changes in government and in access to resources. Many of these issues are specific to a particular context and can only be understood (and addressed) through careful discussion and deliberation by stakeholders familiar with that context. However, the contribution of reliable research evidence to this discussion in LMICs is smaller than in rich countries, and most important, smaller than it could be. LMIC health budgets of less than €80 per person (36) make it essential that every health euro is spent wisely. The WHO Commission on Macroeconomics and Health had as the first of its four main recommendations for choosing healthcare interventions that interventions should be technically efficacious and able to be delivered effectively (37). This requires research evidence to make a greater contribution to the selection and implementation of healthcare interventions that flow from policy decisions. The often very limited influence on policy making in LMICs of reliable evidence undermines the potential for economically rational, criterion based and transparent decision-making, thus jeopardising sustainable development in the health sector and preventing good governance.

There is research evidence on health issues of importance to LMIC, though more is needed. What has been lacking, however, are methods of communicating this evidence in ways that effectively inform policy, public health and healthcare decision making, and on timescales that are relevant to policymakers. Factors that influence the use of research have been studied (38, 39), but strategies that attempt to make use of these factors have not been evaluated. SURE has evaluated six strategies in seven African countries over a period that is long enough (five years) to provide knowledge of the effectiveness of these strategies. This knowledge did not previously exist, which seriously hindered efforts to transfer research evidence into frontline policies that improve the lives of LMIC citizens. Having this knowledge can change the ability of LMICs to meet, and the European Community to support, the attainment of international initiatives such as the MDGs and the Mexico Statement on Health Research (40). Moreover, by involving active regional networks like EVIPNet and REACH, and global players such as WHO, SURE was extremely well-placed to disseminate its knowledge and tools within Africa and beyond.

Building evidence-based policymaking capacity in LMIC
The quality of health policy and health service delivery in LMICs is unlikely to improve, or will improve only slowly, without a significant increase in the capacity of LMIC researchers to generate policy-relevant research and of LMIC policymakers able to access, appraise and apply this evidence to their decision-making. The importance of capacity has been emphasised by others, not least by the ministers and other participants from the 52 countries that produced the Mexico Statement of Health Research in 2004 (40). The impact of research initiatives such as SURE would be disappointing without parallel capacity-building initiatives.

SURE delivered a series of workshops and other capacity building activities (including traineeships, exchanges and an international conference) to promote the use of research evidence to inform health policy decisions and the production of relevant, reliable, accessible and timely research syntheses. Such workshops helped policymakers to disseminate their policy needs to the research community and guide researchers towards selecting research questions that are of national and regional importance. These workshops also informed the development of SURE’s tools. Perhaps more importantly, the day-to-day interactions between policymakers and researchers in the seven African partner countries, in connection with priority setting processes, evidence briefs, policy dialogues and rapid responses, helped to develop a climate that is supportive of evidence-informed health policymaking in those countries. Our evaluation of those country initiatives and linkages with similar initiatives in other countries has provided opportunities for other countries to learn from those experiences and to utilise SURE resources to develop capacity in other countries.

Supporting development policy
The central aim of the European Union’s development aid policy is the reduction of poverty. That poor people have worse health is well-described (41). Around 70% of deaths in LMIC occur below the age of 70, compared to less than 40% in rich countries (42). For children below the age of five, 98% of deaths in 2002 were in LMIC (42). Reducing these gaps between rich and poor, both within and between countries, is not only a question of poverty reduction but of social justice and equity (43). Ill health is not only the result of poverty but also generates it, with ill health often having a substantial impact on household incomes (44-46). Improving health, through the development of health systems and the implementation of effective health policy, therefore has the potential to break the cycle of poverty and disease that contributes to underdevelopment in LMICs (37). LMICs must be able to use research to inform policy decisions on the selection and delivery of effective interventions if they are to improve the accessibility, effectiveness and efficiency of their health systems, and the quality of care that these systems provide to citizens.

The SURE project’s goal was to improve healthcare delivery and health systems by increasing the proportion of care that can be regarded as best practice and the extent to which health care policies are based on rigorous evidence of intervention effectiveness. We did this by providing relevant, reliable, accessible and timely research syntheses of existing best evidence on the effectiveness of health systems interventions. We evaluated strategies for making this information available in a form (and in language) that is accessible and useful to decision makers. Up to now, use of strategies to support evidence-informed health policymaking has been ad hoc and not informed by rigorous evaluations. In particular, we aimed to develop and evaluate methods that make it easier for researchers, civil society and policymakers to work together to improve the quality of health policies. By strengthening LMIC health systems at the policymaking level, SURE supported a more stable and rational environment in which to implement European development policy. It also strengthened the resources available to policymakers in LMICs for debating policy choices and for engaging with stakeholders at regional and international levels regarding policy development and implementation.

In this way, SURE contributed to improving health status, reducing poverty and improving quality of life and the ability of individuals to generate incomes for their household. SURE also helped to prevent the wastage of valuable financial and human resources and to reduce health and social inequalities by sharing knowledge and experience.

European added value
Development is crucial. Avoidable mortality accounts for about 87% of the total chance of death among children up to the age of five in LMIC; for males aged between five and 29 years this figure is 60%, and for females it is 82% (37). Some organisations in LMIC, including those within the SURE partnership, are making progress towards improving the healthcare systems in their countries. However, improvements of the kind envisioned by the authors of the MDGs, the Mexico Statement on Health Research and the EC’s 2002 Communication on Health and Poverty Reduction in Developing Countries (47) require greater and sustained support from rich regions. LMICs cannot do this on their own: they need to work in partnership with rich countries (37, 48). Initiatives of the kind described in this report will only succeed if combined with the weight and support of a major player in the development arena such as the European Union. This partnership will promote European values such as solidarity while also directly supporting European Union development policies such as sustainability, good governance and long-term economic growth.

The transfer of knowledge on how to improve healthcare in LMICs can be considered a global public good since the benefits extend beyond a country’s borders. Investments in this sort of global public good are beyond the means of a single government and beyond the sum of national research programs. Such work requires coordinated, international research, the results of which will be of benefit to all. Moreover, the European Commission has said that the European Research Area should explicitly aim to develop scientific and technical activities useful to the implementation of European Union foreign policy and development aid (49). The SURE project, combined with the EVIPnet and REACH networks and the experience and dissemination potential of WHO, created a critical mass of expertise on how to improve access to and use of research evidence in policy development in African health systems, which has the potential to lead to greater policy coherence. The global reach of WHO also means that developments in African health systems can be disseminated to and implemented in other countries in Africa and other regions.

Helping to solve societal problems
The quality of life of every citizen in both rich countries and LMIC is directly affected by his or her health status. Citizens in poor health may not be able to participate fully in their local community and may be excluded from all or some areas of the labour market. As health improves parents invest more in education. Improved intellectual development and physical wellbeing leads to higher labour productivity increased per capita income and extends the economically productive life of individuals (47). SURE has helped to improve the quality of care by helping to increasing the extent to which policy decisions are informed by research evidence. Moreover, rather than ad hoc use of one or another strategy for improving access to and use of research evidence in policy, SURE evaluated strategies in seven African health systems over a period of five years. An explicit aim of this work was to consider issues of applicability, scaling-up and equity; these are the real-life factors that policymakers must consider. Effective healthcare interventions delivered appropriately and built into policy decisions will improve the quality of life of many citizens together with that of their dependants.


List of Websites:
http://www.who.int/evidence/sure/en/