Final Report Summary - PERFORM (Supporting decentralised management to improve health workforce performance in Ghana, Uganda and Tanzania)
Executive Summary:
The biggest barrier to achieving the health-related Sustainable Development Goal (number 3) in many countries is the lack of an adequate and well-performing health workforce. There are numerous initiatives in Sub-Saharan Africa to scale up the workforce, but fewer effective initiatives to improve the performance of the existing workforce. Integrated human resource management strategies, complemented by strategies related to other health system areas are needed to improve workforce performance. These strategies need to be based on a thorough analysis of the problems that analyse root causes.
District-level health managers in decentralised contexts are well-positioned to take a system-wide approach to improving the performance of their frontline health workers. They are close to the problems and know what they can achieve with their available resources. The PERFORM project used a participatory action research process as management strengthening intervention to facilitate the development and implementation of work plans to improve workforce performance in Ghana, Tanzania and Uganda in three districts in each country.
Researchers in each country carried out a situation analysis in each study district jointly with district health management teams (DHMT) to identify and prioritise workforce performance and service delivery problems. The research teams facilitated a process of detailed problem analysis and developed appropriate work plans through using consultative meetings and two short workshops. The teams then followed up the implementation of the work plans with the DHMTs, continuing to facilitate the action research initiative and collecting process evaluation data. An endline evaluation was carried out after an 18-month period, using interviews, focus group discussions and routine data and document analysis to identify the effects of the intervention on management strengthening and the impact of the work plans on health workforce performance and service delivery.
The action research approach to management development appeared to be an appropriate intervention and DHMTs demonstrated the ability to carry out a thorough root cause analysis of problems identified and to develop appropriate integrated work plans.
In spite of limited official authority for managing human resources, most DHMTs were able to implement their work plans using available resources, though some were entrepreneurial about obtaining financial support from development partners in their district. Some of the strategies for improving workforce performance were beginning to show positive results at the time of the endline evaluation. DHMTs were realising the importance of monitoring the effects of the work plans and were making their own decisions about modifying them as necessary and learning which strategies worked best for improving workforce performance and service delivery.
Through the development of work plans for which they felt ownership, the DHMTs appeared to make better use of what they had perceived to be limited decision-space in the management of human resources – and other areas of management. This increased their confidence in improving not only health workforce problems, but also wider service delivery challenges.
Many DHMT members wanted the action research management strengthening approach to be extended or expanded. Plans for this have been discussed with relevant stakeholders including the Ministry of Health in each country to explore ways of building on the PERFORM project. Engagement with international stakeholders will continue exploring the use of the approach in other countries and facilitation materials are being made available through the project website.
Project Context and Objectives:
1.Project context
'Supporting decentralised management to improve health workforce performance in Ghana, Uganda and Tanzania’ (PERFORM) was a multidisciplinary and multi-partner project implemented over four years (September 2011 – August 2015) with the financial support from the European Commission FP7.
The PERFORM project research framework outline and action plan were defined in a Grant Agreement with the EC (Ref: 266334) as part of the work programme HEALTH. 2010.3.4-1 (Develop and assess key interventions and policies to address the human resource crisis in the health sector).
2. Background to thinking about workforce performance and systems thinking
In sub-Saharan Africa (SSA) the single biggest barrier to scaling up the necessary health services for addressing Millennium Development Goals 4, 5 and 6, Universal Health Coverage, and for the achievement of the health related aspects of the 2030 Sustainable Development Agenda (goal number 3) is the lack of an adequate and well-performing health workforce. Many donors and international initiatives are focusing on increasing the numbers of available health professionals through advocacy (Scaling Up, Saving Lives) and country-specific initiatives such as the Emergency Human Resources Programme in Malawi, the health extension workers programme in Ethiopia, the Emergency Hiring Plan in Kenya. However, the deficit in health professionals needs to be addressed both by scale-up through funding and training more new health personnel and also by improving the performance of the existing and future workforce.
Most development and research emphasis has been on the first of these – increasing numbers (such as the PEPFAR programme to train 140,000 new health workers in Africa and the expansion of the use of community health workers). There has been a serious neglect of initiatives to address the complex area of existing workforce performance. Examples of poor workforce performance include high vacancy and turnover rates at the management level and poor clinical behaviour, frequent absenteeism at the individual level and low health worker productivity. The PERFORM project focused on this aspect by providing district managers with new knowledge and skills for effectively intervening within their current contexts of decentralisation to improve the performance of their staff as a response to the workforce crisis.
A number of complex factors affect workforce performance. Of particular importance are the maldistribution of staff, inappropriate task allocations and poor working conditions (including training, management and support). These factors may also lead to poor staff retention. Understanding the nature of these factors and developing appropriate responses will both improve the performance of the existing workforce and may reduce staff losses. It will also increase the effectiveness of future new health personnel trained and deployed as part of workforce scale-up to meet the health–related SDGs and the drive towards Universal Health Coverage.
There are a wide range of measures which managers can use to address human resource (HR) performance issues, such as supervision, appraisal and improved work organisation. Research has shown however, that effectiveness of these HR strategies relates to the nature of the factors that influence employee behaviour e.g. gender, career stage, level of responsibility. It is therefore necessary to select strategies that respond to the factors that influence the behaviour of particular health personnel.
In addition to choosing the right strategies, there is also a need for their integration across human resource management (HRM) practices. For example, the effectiveness of the recruitment system may benefit from changes in the remuneration system. Current thinking on health systems strengthening also suggests the importance of integration of health workforce strategies with the five other health system building blocks (i.e. service delivery, information, financing, leadership & governance, medical products, vaccines and technologies). It is imperative to consider potential unintended as well as intended consequences of a strategy within the wider health system. For example, while the use of incentives for the provision of HIV services is expected to result in better access for the user, it can also lead to a reduction in access in other areas of service provision.
The decentralisation of the planning and management of health systems continues in SSA (e.g. in Kenya under the national devolution policy). Decentralisation has frequently been to the level of district. Research has been conducted to understand how to allocate resources more efficiently at this level. However, there has been no equivalent research to understand how such decentralised authority can be effectively used, within the constraints of available resources, to improve health workforce performance at the district level.
This research project was designed to enhance our understanding of how, and under what conditions, a management strengthening intervention can improve workforce performance within districts in decentralised settings. A guiding principle of the intervention was that most effective management strengthening approaches address real problems that managers face and use planning and management tools that managers are familiar with and for which they are likely to get support in future.
Partners with health systems research expertise were selected from countries with decentralised health systems: Ghana (School of Public Health, University of Ghana); Tanzania (Institute of Development Studies, University of Dar-es-salaam); and Uganda (Makerere School of Public Health). Each of these countries face major problems of inadequate health workforce distribution and performance. They also have decentralised management structures that offer management teams greater and more relevant decision-making opportunities including in the area of human resources. The research project was led by Liverpool School of Tropical Medicine (UK) with support from the Swiss Tropical and Public Health Institute and the Nuffield Centre for International Health and Development, University of Leeds (UK). All three European institutions had substantial health systems expertise and experience of working in decentralised contexts in SSA.
3. Description of the conceptual framework
The research has been conducted in Ghana, Tanzania and Uganda. The research was designed to study how management strengthening interventions can be used, and under what conditions, to enhance workforce performance. A comparative analysis of the findings from three study districts in each country added new knowledge as to the effect of different country contexts on these interventions. This provided insights into the application of the new approaches in different district contexts across three African country settings.
The theory of change for PERFORM project was: action research in decentralised decision making environments provides district management teams with opportunities to focus on human resource and health systems management by using a systematic process of identifying relevant problems and developing customised strategies to improve workforce performance; action research also provides opportunities for reflecting on successes and failures in planning and implementation, and if sustained and coupled with adequate “decision space” ultimately strengthens the managerial competences of the DHMT for improving workforce performance and other areas of health management.
The project uses a systems approach to address the workforce problems by ensuring that HR strategies to address the problem are integrated; that complementary health systems strategies are used, and unintended effects are identified. The systems approach is operationalised through the management strengthening intervention – based on the principles and four stages of action research – which included a problem analysis leading to appropriate strategy design (1. Plan: the application of the strategy (2. Act); and the monitoring and evaluation (3. Observe; and 4. Reflect) of the strategy leading to modification or re-design (1. Plan) – and so on. The new knowledge provided by the project relates to both the use of the workforce strategies and the management strengthening intervention (see Figure 1 in attachment).
4. Main objectives
The overall aim of the project was to identify ways of strengthening decentralised management to address health workforce inadequacies by improving health workforce performance in sub-Saharan Africa.
The specific objectives were:
1. To support health managers to carry out a situation analysis on the health workforce, with a particular focus on performance, in the study districts.
2. To develop and test context-specific management strengthening processes, focused on improving workforce performance, which will:
a. identify areas of health workforce performance to be improved
b. implement integrated HR and health systems strategies feasible within the existing context, to improve health workforce performance;
c. monitor the implementation of the strategies and evaluate the intermediate processes and impact on health workforce performance, and the wider health system.
3. To conduct comparative analyses across districts and countries of:
a. the management strengthening intervention to support improved workforce performance, and
b. processes of implementing the integrated HR and health systems strategies and intended and unintended effects on health workforce performance and the wider health system.
4. To provide ongoing communication of the research process, findings and conclusions, in order to raise awareness and change attitudes of sub-national, national and international stakeholders.
5. To consolidate research capacity of partners on integrated approaches to workforce performance improvement and contribute to strengthening capacities of decentralised management of district health systems.
6. To establish and maintain effective partnerships amongst academia, civil society, policy-makers, and health managers in study countries and amongst partners.
5. Work packages
Eight Work Packages were used to achieve the PERFORM objectives. These included Study design and methodology development (package 1) which preceded the three country case studies (packages 2-4) to deliver the intervention which was followed by the comparative analysis (package 5). Research communication (package 6) and capacity building (package 7) ran throughout the project and the project management package (8) supported the other 7 work packages (see Figure 2 in attachment).
Project Results:
1. Introduction
The research was led in each country by the respective Country Research Team (CRT) with support from a paired EU partner (EP). The CRTs worked with selected district health management teams (collectively referred to in the project as DHMT and specifically in Ghana as Budget Management Centre (BMC) and in Tanzania as Council Health Management Team (CHMT)). This part of the report describes the PERFORM methodology and then presents the findings with some details at country level, but the main findings are from the comparative analysis.
2. PERFORM research methodology
This section introduces the concept of action research. It then describes the three projects phases and the related research methods.
2.1 Action research
The PERFORM project used an action research (AR) approach. The definition of AR that we applied during PERFORM was as follows:
“Action Research (AR) is an enquiry which is conducted by a group on a problem which is of importance to them. Its aim is two-fold; to improve practice and to generate knowledge about the processes and strategies that work best to create that improvement.”(Reason and Bradbury 2001)
The DHMTs, facilitated by the CRTs, worked through systematic cycles of planning, acting, observing and reflecting to:
1. Describe and analyse the problem they face
2. Identify and plan strategies to improve situation or solve problem
3. Implement the changes needed
4. Observe, explain and reflect on the process and the effects of changes made.
The DHMTs then continued with subsequent cycles (see Figure 3 in attachment) to continue to create improvements. The CRTs worked with the group to build participation, provide research methods support during observation phases of the cycles, and to record and analyse the process and strategies of change. Successive cycles are beneficial as they can deepen the learning about a problem and its solutions. In practice these may be more like a spiral, or a cycle with smaller cycles spinning off and feeding back into the main study.
The following aspects of AR in PERFORM were used to build research evidence:
• An initial situation analysis to collect research data on the problem, this feeds into the first cycle of AR.
• Robust use of research and analysis methods during the observation phases of the AR cycle(s) guided by the CRTs.
• Record and reflect on the change process throughout AR cycles using diaries.
• Recollect core data for evaluation including context, to identify within-district changes.
To support the process of implementing the AR cycle, two guidance documents were developed in the first part of the project. The first was the Research Methods Manual (D1.1) which provided tools for conducting the initial situation analysis and then tools were developed and added during the life of the project. The first edition of the Research Methods Manual was produced in August 2012. The second guidance document produced was the DHMT methods manual to explain the research process. It also included a set of guidelines on selecting HR/HS bundles to improve workforce performance (D7.2) to help DHMTs to make their work plans to address workforce problems they had analysed – mainly in the second of the DHMT workshops. This was later incorporated into the Research Methods Manual, which is being made available as part of a package of capacity building materials developed by the project.
2.2 Project phases
The project had had 3 phases spread out over the 48 months, starting in September 2001, and a timeline of key activities for three phases are given in Figure 4 in attachment:
2.2.1 Phase 1: preparation (September 2011-May 2012)
Phase 1 covered the project start-up and selection of the three districts in country, the establishment of Country Research Advisory Groups, made up of key stakeholders at national level, to support the CRTs and assist with research uptake. Project management systems were established. The methodology was further developed with input from all partners resulting in the production of a methodology manual (Deliverable D1.1). The overall management strengthening intervention using concepts from health systems thinking and action research applied within the district context was agreed in this phase. The methodology and agreement is based on a common understanding amongst all members of the research teams of the proposed research approach.
Ethics approval in each of the three study countries and for the European partners was sought and obtained during this phase of the project. Capacity of the consortium partners was strengthened through a series of activities including a workshop in Tanzania for carrying out the research and for facilitating the action research process with the DHMTs.
Three districts were selected in each country Ghana, Tanzania and Uganda using pre-defined criteria. Because of the collaborative nature of the management strengthening intervention, one important criterion for the selection of study sites was a motivated and reasonably staffed district management team with which to work. A second criterion was the inclusion of a mix of types of districts reflecting different contexts including a mix of rural and urban. In practice the selection was negotiated with the health service administration at national level and in Ghana and Tanzania this led to selection of districts in the same regions. In Uganda, district selection also took into consideration a range of levels of performance of the districts over the past three years as indicated in by the performance rank in the Ministry of Health league table. The list of study districts by country is given in the attachment.
No ‘control’ districts were selected in any of the countries, because the project did not seek to attribute potential changes in management processes exclusively to the AR approach. However, a point was made of ensuring the project methods and tools captured information about wider contextual changes in each setting so that these could be taken into consideration when assessing the impact of the interventions.
2.2.2 Phase 2: Intervention (June 2012-August 2014)
Phase 2 started in 2012 with an initial situation analysis carried out together by the CRTs and the DHMTs. This was the beginning of the problem identification and analysis phase which was carried through to a series of short national workshops that brought the three DHMTs together. In the second National Workshop the DHMTs developed an action plan to address the problems that identified. These plans were later refined, resourced and implemented. The CRTs provide facilitation support during this period through visits, joints meetings for the DHMTs and communication by phone and e-mail.
a. Situation analysis
The purpose of the initial situation analysis was to serve as the baseline for the project as well as
Inform the subsequent action research cycles in each district through identification of priority challenges to be addressed. It focused on some common core HR and health systems indicators across districts and countries to allow for the comparative analysis. As much as possible, these indicators used routinely collected data and performance indicators (gender disaggregated where possible) to simplify the collection process, minimize disruption and to increase the chance of expanding and sustaining such an approach beyond the project period. The objectives of the (initial) situation analysis were to identify areas of exceptional service delivery performance, areas of staffing shortage, problems of health workforce performance, health systems and contextual factors affecting performance, and current management and communication process used by the DHMTs.
b. Methods
The initial situation analysis was a two-step process in each district. The purpose of Step One was to collect information on the structure, staffing and health service activities of the district, while Step Two focused on identifying human resource problems and their causes. The method used in Step Two was group discussion involving DMHT members facilitated by the CRT. The output was an initial list of prioritised problems that the DHMT considered they had the capacity to influence or change.
c. Data collection
Data was collected in the three countries during several visits by the CRT between August and September 2012. As part of the action research approach, the DHMT members worked with the CRT to collect the data for the situation analysis. The CRT made several visits to the districts between July and August 2012. A generic questionnaire was used to collect data on the health workforce, DHMT, health system, local and national contexts. The CRT in consultation with the European partners concurred that some additional customized data collection tool was necessary to collect data missed by using the generic data collection tool. A data collection tool focusing on the specific district contexts was used to collect the missing data. Data was also extracted from reports and databases held at the district.
d. Data Analysis
Data analysis was done by the DHMTs with the support of the CRTs. This offered an opportunity for the DHMTs to learn or improve their skills in analysing data. Both the DHMTs and CRTs reflected on the data and emerging issues such as power dynamics in the DHMT, district problems, decision-making, and verification of data. These were valuable health workforce situation indicators to focus on in Step Two of the situation analysis. Triangulation of secondary data, and the data generated through discussions was intended to strengthen understanding of the district situation and provide a firm basis for the next step of the initial situation analysis.
2.2.2 Problem identification and prioritisation
At National Workshop 1 in October 2012 each DHMT presented the findings of the situation analysis and their initial problem analyses to the other DHMTs as well as other resource persons (e.g. civil society representatives) and further root cause analysis was carried out. On Day 2 some guidance was given from CRTs and peers on refining the problem trees and prioritising the problems to be addressed. Further discussions were held at district level and the prioritisation and root cause analysis was finalised on the first day of National Workshop 2 that was held in February 2013.
2.2.3 The process of selecting integrated bundles of human resource and health systems (HR/HS) strategies
The concept of the action research process had been explained to the DHMTs from the first exploratory visits in early 2012 and during later visits the DHMT manual which explained the action research processes was shared. An additional section (based on Deliverable 7.2) was later added to provide guidance during National Workshop 2 on selecting appropriate human resource and health systems strategies to address problems identified. This covered the concepts of developing coherent HR strategies and integration with complementary HS strategies –referred to as bundles of HR/HS strategies. The manual also contained a reference table with about 40 strategies, each with sample activities, possible indicators for monitoring and evaluation and suggestions of how each strategy might be linked to other HR or HS strategies to improve integration both within the bundles, but also with wider plans. The work plans containing the HR/HS bundles were developed using the following criteria:
• Focused on improving health workforce performance in the district
• Measurable and observable effect on workforce performance within 12 – 18 months
• Implemented within available resources in the district
• Linked to district plan
• Linked to existing policies / strategies
• Based on the evidence accumulated in the situation analysis
• Bundles are logical.
Each district plan was subjected to peer review by the other DHMTs and then whenever possible incorporated into district planning frameworks that the DHMTs had brought with them – mostly on their laptop computers.
2.2.4 Implementation of HR/HS bundles
The implementation of the HR/HS bundles phase was from March 2013 (after National Workshop 2) until September 2014 (when the evaluation took place). During this period the DHMTs first refined their plans and in some cases sought extra resources from development partners for implementing the plans. The DHMTs in Tanzania had to wait some time until the budget allocation for the next financial year arrived before implementing some of their plans.
2.2.5 Action Research process recording
Diaries, and in Ghana a documentation template, were used to record and reflect on the change process throughout the AR cycles. The importance of the learning developed in particular from the observation and reflection parts of the action research cycle was explained on the final day of National Workshop 2 and the DHMTs were all given diaries to keep a record of the implementation of the plans they had developed and in particular their reflection on the implementation. CRTs made regular visits to the districts to discuss implementation of the bundles of strategies including their reflections on implementation and effects.
2.3 Phase 3: evaluation (August 2014 to August 2015)
2.3.1 Evaluation of management strengthening and HR/HS bundles at country level
In order to build the knowledge base of the overall effectiveness of the approach used in PERFORM, ongoing data collection and analysis is needed. Two stages of analysis were undertaken:
1. Analysis of data collected by the CRTs in collaboration with the DHMTs throughout the project as part of the action research cycle.
2. Overall evaluation which will include comparative analysis following the period of implementation of the HR/HS bundles.
The evaluation took place in September 2014 after the HR/HS bundles had been implemented for about 18 months. This situation analysis was a one stage process, with data on the health workforce, DHMT, health systems, local and national context being collected.
a. Methods
The evaluation sought to document processes of both the management strengthening intervention (the action research cycle) and the development and implementation of the bundles of HR/HS strategies. It also aimed to identify the effects (both intended and unintended) of the management strengthening and bundles of HR/HS strategies. The study, using qualitative methods, gathered the perceptions of a mixed group of stakeholders (the DHMT, sub-district managers, health staff, relevant stakeholders and the researchers themselves) to obtain a retrospective account of processes and change. In Uganda, obtaining the perceptions of community members was originally considered, but rejected after piloting because all HR/HS bundles were addressing health workforce performance problems for which the impact at the level of service delivery after a period of 18 months would not have been easily detectable. However, in Ghana and Tanzania, service users’ views were sought: client exit interviews were used in Ghana, and focus group discussions were used in Tanzania. A range of documents produced during the project (such as CRT visit reports, DHMT diaries, workshop reports) were analysed for information on processes and effect. Selected health systems and health services indicators were used to see whether any before and after effects could be identified or associations of impact made through linkages to the qualitative data.
b. Sampling
Uganda: In depth interviews with DHMT members: 4-5 members were purposively selected ensuring a range of participants with regard to gender, length of service in DHMT and position in DHMT.
Focus group discussion with DHMT members: 6-8 members were purposively selected ensuring a range of participants with regard to gender, length of service in DHMT and position in DHMT.
In depth interviews with sub district managers: two sub districts were selected in each district, and then within each sub district one health centre IV, one health centre III, and one HC II were selected. The manager from each of these facilities was selected for interview.
In depth interviews with sub district staff: using the same facilities as outlined in the sampling for the in depth interviews with the sub district managers, one health worker from each facility was selected, ensuring a range of participants across the sample for the district with regard to cadre, gender and length of service.
In depth interviews with stakeholders: the Local Council human resources officer, a Health Unit Management Committee member, a development partner and the council secretary for health were selected in each district.
Ghana: All respondents of the in-depth interviews (IDIs) were purposively selected. They were made up of the District Directors of Health Services (DDHSs), District Public Health Nurses (DPHNs), District Health Information Officers (DHISs) and District Disease Control Officers (DDCOs) at the DHMTs. At the district hospital, Matrons and heads of Reproductive and Child Health (RCH) and Antenatal Care (ANC) units were interviewed. At the sub-district level, facility heads and in-charges of RCH and ANC units were interviewed, whilst Community Health Officers (CHOs) were interviewed at the CHPS Zone level. The IDIs were conducted by CRT at the respective facilities of respondents. Clients were randomly selected with the assistance of the in-charges of the ANC and Child Welfare Clinics (CWCs) for the exit interviews. Client exit interviews were conducted by the research team at CWC sites.
Tanzania: 5 health facilities involved in the provision of CTC services were sampled in Iringa Urban; 10 facilities involved in male circumcision, PMTCT and CTC services were sampled in Mufundi; and 10 facilities that provided RCH services were sampled in Kilolo. All DHMT members were purposively selected for interviews and FGD, and health facility managers of the sampled facilities were purposively selected for in-depth interviews. One health facility staff was randomly selected for in-depth interview at each of the sampled facility in each district. Users of the services were randomly selected for FGDs as they sought services at the facilities.
All countries: In-depth interviews were carried out with the CRT and the European partners to explore in particular the support provided between the paired partners and to the DHMTs.
All relevant project documents were reviewed. These included the DHMT and DHT management meetings, work plans, CRT visit reports, diaries, inter-district meetings, national workshops and partner visit reports
c. Data collection tools
The data collection tools were developed based on an analysis of key questions about the process and effects of the management strengthening and workforce improvement components of the intervention. These included: topic guides for the FGD with DHMT, the interview with DHMT, the interview with manager, the interview with staff, interview with stakeholder; and data collection tool for health systems and health services indicators. With the exception of the DHMT topic guide, all tools used at district level were piloted in May 2014 in Luwero district and modified in consultation with all PERFORM consortium members.
d. Data collection
The CRTs in each country visited each study district during September and October 2014 and carried out the data collection using the tools. Interviews with DHMT members, health facility managers and staff, and stakeholders were conducted in quiet rooms at the district health offices, health care facilities or offices and lasted between 30 and 60 minutes. FGDs with DHMTs were conducted in the District offices and lasted 60 to 90 minutes. Focus group discussions and client exit interviews with service users were conducted in quite areas of the health facilities. Before each interview and FGD, the CRT went through an informed consent process and obtained written consent from each participant, which included recording of the interview or discussion.
Selected health systems and health services indicators: data on the district was collated at the DHO office from the HMIS and in discussion relevant staff; data on the health facilities was also collected from the HMIS, and where gaps were identified, the CRT and DHO office staff telephoned the health facility managers.
Document review: documents were collated from the district health office which included: the DHMT diary; and the District annual health work plans and reports, DHMT minutes and DHT minutes for preceding years (back as far as 2011 in Uganda). In addition to these district level documents the following national and international level documents were collated: National Workshop Reports, Inter District Meeting Reports, CRT visit reports and Consortium Workshop Reports.
e. Data Analysis
Interviews, FGDs and documents: the recordings were transcribed verbatim and checked for accuracy by the CRT. All documents were then uploaded to NVIVO qualitative analysis software (Version 10). A coding framework was agreed by the consortium, based on the questions relating to the management strengthening and workforce performance improvement interventions. All documents were coded using this framework. Next, queries were made to extract the data by theme for each district and then summarised with key supporting quotes. These were then combined to compile the reports for each of the districts. Finally these were combined to allow for inter district analysis and to produce an overall set of findings and recommendations.
Client exit interviews: simple descriptive statistics were used to analyse this data.
Data on health facilities activities (e.g. consultations, vaccinations) and workforce (i.e. number of staff by categories) were obtained from health facility reports for the previous calendar year or were provided by staff. Minimum, maximum, means, percentiles 25 and 75 and medians across health facilities were calculated for each district and for each country. Medians and Inter-Quartile Ranges (IQR = p25 to p75) are reported in order to exclude data errors produced by missing values or outliers. As the health facilities provided a different mix of activities (e.g. some may have had maternities to carry out deliveries, while others did not), in order to facilitate comparability across districts and countries a Units of Activity (UA) approach was used as a weighed estimate of the amount of activities being carried out across the health facilities. Note: the results of the analysis of health facility activities are reported in the Comparative Analysis report (D5.1) but not in this report.
f. Output from the evaluation
An evaluation report was produced in February 2015 for each country as project deliverable (D2.1 D3.1 and D4.1). The findings were presented to the DHMTs at National Workshop 3 both for the purposes of validation and as part of the research communication process, and reports were revised based on this feedback.
2.3.2 Comparative analysis
The comparative analysis built upon the three country reports which presented the project’s results from the application of the overall PERFORM methodology in the three country contexts.
As a first step a table was elaborated according to the project’s thematic and analytical framework, which traces for each country in a single table the various project processes (initial situation analysis, problem identification etc.), the effects of the AR approach on DHMT management practices, and the effects of the HR/HS strategies on workforce performance and health systems. For each of these rubrics this analytical framework contains main themes and sub-themes, questions to be answered, indicators and suggested methods to answer the questions, and the sources where the information about each can be found. The sheet was filled out for each country in turn, and a brief description of the differences and similarities between them was provided in the fourth column. The comparative analysis essentially relies on secondary data from the country reports. Taking these as the basis, in an iterative process that started at the comparative analysis workshop held in Basel in April 2015 the key areas of interest for comparison were agreed.
The comparative analysis report was produced as a publicly available project deliverable (D5.1) and is being used as a key source document for comparative journal publications and presentations.
3. Main S&T results
The programme was implemented as planned. All deliverables were submitted on time although extensions were requested and granted for the country reports and comparative analysis report.
The remainder of this section, based on the country reports and the comparative analysis report provides key results on the problem analysis and strategies developed by country and then the results from the comparative analysis.
3.1 S&T results at country level
This section presents both the problem analysis and work plans developed for each country in tabular form. Full details of the findings in each country are available in the country reports (D2.1 D3.1 and D4.1).
3.1.1 Ghana
The key problems identified by district, with their underlying causes, are presented in Table 2 in attachment. The strategies developed in National Workshop 2 to address problems identified are in Table 3 in attachment.
3.1.2 Tanzania
The key problems identified by district, with their underlying causes, are presented in Table 4 in attachment. The strategies developed in National Workshop 2 to address problems identified are in Table 5 in attachment.
3.1.3 Tanzania
The key problems identified by district, with their underlying causes, are presented in Table 6 in attachment. The strategies developed in National Workshop 2 to address problems identified are in Table 7 in attachment.
3.2 Main S&T results from the comparative analysis
This section draws on findings across the three countries from the comparative analysis report produced under work package 5. It starts with a brief review of the context including the forms of decentralisation, human resource management at district level and the structure of the DHMT. The findings related to the intervention are presented in two parts. The first covers the process, which describes the stages of the problem analysis, the design of the strategies to address priority problems and the experiences implementing the strategies. The second part covers the effects of the intervention on management strengthening, health workforce performance and service delivery, and finally the effects on other areas. The section is completed by a set of conclusions from the study.
3.2.1 Context
In Uganda and Tanzania the responsibility for the management of services at district level – including health services – has been devolved to local government and so the District Health Office is part of the local council structure, which develops plans and manages resources. In Ghana decentralisation takes the form of ‘deconcentration’ - the autonomous Ghana Health Service through the levels of the regional and district health offices with no direct linkage with local government.
In spite of the decentralised structures in each of the countries, the DHMTs officially had limited powers for managing the health workforce they were responsible for. In Tanzania and Uganda the local council had a human resource management function which dealt with the major human resource management functions including recruitment. The control on staff establishment (the number of permanent posts) is controlled centrally in both countries. In Ghana, the national level that has the core management authority related to pay levels, recruitment, regional postings, performance assessment, training schedules, promotion, disciplinary action, and termination for permanent senior levels of staff. The district has control over recruiting contract staff and intra-district postings, performance assessments, in-service training, and leave and sick-days.
The DMHTs across the three countries ranged from 8-12 members each – most core managers, supervisors and some administrative staff at the district level and in the sub-district level in Uganda. The composition of male and female DHMT members was balanced in Ghana and Uganda; there was a lower proportion of females in Tanzania. The concept of the DHMT seemed to be a little fluid and in the case of Uganda is built around a smaller District Health Team based at the district office which meets on a more frequent basis. DHMTs in Uganda and Tanzania could co-opt members as necessary and so at times could be quite big.
3.1.2 The Process of PERFORM
a. Problem analysis
Although the concept of analysing problems as part of the process of planning was not new to most participants, all DHMTs were actively involved in the process starting around September 2012 through to the first part of National Workshop 2 in February 2013. Some participants said that the process helped them become more questioning, see things in a different way and recognising that in the past they did not always prioritise the problems to be addressed. The problem trees (showing links between cause and effect) produced and subsequently revised demonstrated the DHMTs’ engagement with the process of root cause analysis which for many participants was a new part of the problem analysis process. The process of refining the problem trees was time-consuming, but proved a good investment for developing appropriate strategies.
The aim of PERFORM was to address, amongst other problems, those related to health workforce performance. This was approached differently with the DHMTs in Tanzania and Ghana identifying problems with service delivery e.g. poor service coverage and then identifying the causes in the system including those related to health workforce performance. The Uganda DHMTs, on the other hand, started directly with workforce performance problems. The two approaches yielded some common causes of health workforce performance problems across the three countries, for example supervision support, staff capacity and staff attitude. However, what seemed important to the DHMTs was that, unlike in some similar programme-specific interventions, they had ownership of the process of identifying, prioritisation and analysis of the problems. This provided the motivation for developing effective strategies that would address these problems in an appropriate way.
b. Strategy development
A frequent failing of the development of management strategies is that they are not clearly based on a clear understanding of the problems. An analysis of the strategies developed by the DHMTs shows a clear correlation to their problem analyses. For example, to address the problem of high dropout rate in Akuapim North district in Ghana, their work plan included strategies to: intensify monitoring and supervision; intensify EPI defaulter tracing; community mobilisation and use of work plans. See other examples in Tables 3, 5 and 7 in attachment. This clear correlation was achieved through the use of tools and processes that ensured this clear linkage was made – in some cases physically developing an objectives tree (work plan in diagram form) alongside the problem tree to make the linkages visible.
There was some evidence form observation during National Workshop 2 that the district teams consulted the reference table provided in the DHMT methods manual with sample HR strategies, supporting activities and possible indicators, though this was not well documented.
A number of participants reported in the feedback on National Workshop 2 that the concept of the integration of human resource related strategies and wider health system related strategies into an integrated bundle was difficult to comprehend. Nevertheless they were able to do this in practice, integrating both complementary HR strategies (e.g. attendance monitoring and appraisal) with strategies addressing wider health systems problems such as ensuring medical supplies are made available alongside competence building. In addition, they followed the criteria for selecting strategies that would be both feasible and effective ensuring their affordability within available resources and synergies with interventions of other programmes/projects in the district.
The template for developing the workplan included a column for activities to support the strategies and indicators for monitoring progress. All strategies were supported in the plan by multiple activities. Indicators were developed for all plans, but only DHMTs in Ghana specified targets. One DHMT in Tanzania reported on the difficulty of developing their own indicators – this is normally something done at higher levels of management or by external agencies.
Most DHMTs were able to integrate work plans developed in National Workshop 2 in their wider funded district-level annual plans or to schedule some activities for the following financial year.
c. Implementation monitoring and modification of the interventions
The experience of implementation of the work plans developed by the DHMTs was only monitored from the end of National Workshop 1 to September 2014 (18 months), but DHMTs continued implementation after that period.
Many of the DHMTs used diaries or other monitoring tools provided by the project for reviewing the implementation of their plans. It took the districts some time to get used to the diaries and the value of their use in tracking progress. Several difficulties were mentioned including lack of clarity about which information to record in them, and how to ensure that all DHMT members including those based at the sub-district level gained the opportunity to write in them. To overcome these challenges, some DHMTs adapted the format of the diary so it could be more easily taken with them when they went to the districts. The content of the diaries tended to be mainly factual rather than reflective, as intended. However, the continued support through visits and e-mail/phone contacts by the CRTs, inter-district meetings and the DHMTs’ own meetings all provided the opportunity the teams to reflect and consequently for the work plans to evolve according to need.
In the design of the work plans teams kept “affordability” in mind when developing the strategies assuming that activities could be funded from the existing district budget because the budget headings were sufficiently broad. In other cases, where this was not possible or funds from the regular budget were delayed, they either reprogrammed activities for the next budget cycle or approached stakeholders to identify alternative sources of funding. For example, in Tanzania one DHMT approached a non-government organisation to fund training for community health workers and renovation of service delivery facilities. However, DHMTs in Tanzania, were regularly hampered by the late disbursement of the regular government budget and this was particularly late in 2014, delaying the implementation of the work plans.
Some activities were modified during the implementation period and some abandoned. For example, one DHMT in Uganda decided to allocate mentors to the new supervisors in order to improve supervision. One district in Ghana abandoned the task of “develop and implement ppm [Planned Preventive Maintenance] for solar panels and fridges” as the DHMT felt it was beyond their capacity to implement.
The DHMTs in Ghana who had set indicators with clear targets were able to monitor the impact of their work plans. Others had given more priority to developing the strategies and related activities at the design stage, rather than the indicators. However, because of the strong ownership of the plans DHMTs began to be curious about how well they were working. For example, a DHMT in Uganda started to monitor staff absence using an attendance register.
3.1.2 The Effects of PERFORM
a. Management strengthening:
From an analysis of their actions and from testimonies during the final situation analysis, the DHMTs appear to have developed or improved their management competencies related to problem analysis, strategy development and implementation with only attending about 4 days of workshop time and up to 3 days of inter-district meetings. Regarding the action research cycle, evidence of the first two steps – ‘plan’ and ‘act’ – was clear. There is less evidence of the ‘observe’ stage, which relates to both monitoring the process of implementation and in particular the effects of the plan. Nevertheless, there were indications that the DHMTs were beginning to appreciate the importance, especially as they were implementing plans that they were developing themselves. There was even less clear evidence of ‘reflection’ in the action research cycle. This is a crucial step as if this does not take place there is no learning and the DHMTs will continue to make the same mistakes. The diaries were designed to help with reflection, but DHMTs had difficulty with this function and it was challenging for facilitators to help them. However, the fact that plans were changed based on observations and reflections implies that some reflection did take place. For instance after their experience implementing the work plan one DHMT in Uganda introduced spot check supervision visits but also to put more emphasis on rewarding the best performing staff rather than using sanctions against bad performers. Respondents implied that the reflection part of the action research cycle is more challenging, and identified the need for more support in recording reflection. Nevertheless both some of the changes made to plans, as already mentioned, and some of the discourse in the regular meetings attended by CRTs suggested that reflection was taking place.
In addition to the development of DHMTs’ management, there was evidence of changes in the way in which the DHMTs operated. This included empowerment, increased initiative taking, increased team working and collaboration and application of lessons learnt to other areas.
The process of selecting priority problems, analysing them and developing strategies provided a sense of empowerment because they were doing it all themselves. One manager in Tanzania felt his team was now empowered to bring about change. The process also encouraged some DHMTs to develop and implement strategies that they might not have previously considered due to their perceived limitations of their roles and level of decision-making autonomy. This contributed to their self-confidence.
Health service managers were largely dependent on funding from government or external development partners and PERFORM provided no additional funds for implementing the work plan. In spite of initial complaints, some DHMTs saw this as an opportunity to make better use of existing resources and others showed initiative and creativity in searching for funding. For example, one DHMT in Uganda approached a bank to fund orientation of newly recruited staff in return for the opportunity to advertise their banking services. A number of other examples of co-funding were reported and some DHMT members said they developed skills in advocacy and proposal writing.
Perhaps the most striking impact the process had on the DHMTs was the stimulation of greater collaboration and teamwork within DHMTs. This was made possible through the workshops – both being together and working on a common task – and because the problem analysis showed the interdependence of many members of the team. The workshops and inter-district meetings also stimulated collaboration between DHMTs which do not otherwise have the opportunity to meet.
Finally, the DHMTs saw the possibility of applying the lessons learnt in developing strategies to improve health workforce performance to other areas of their work. In Tanzania some CHMTs used the diaries to record implementation of other areas of work.
b. Health workforce performance and service delivery
There is mixed evidence of the impact of the strategies for improving health workforce performance. There were several major challenges in measuring the impact of the health workforce improvement strategies. First, in most cases the lack of baseline data and in some cases the reliance on qualitative data at the endline. Second, the problem of attribution, given the multiple ongoing workforce and workforce related initiatives in the study districts. Having said that the study has provided examples of strategies that are likely to have had a positive impact on health workforce performance and to have contributed to improved service delivery.
The findings from both Akwapim North and Kwahu West districts in Ghana show improvements in vaccination and drop-out rates during the intervention period. There was a national scale-up of one of the vaccines, Rotarix, going from 2012 onwards which could have contributed to this increase. However, DHMTs claimed convincingly that improved supervision contributed to making the Community Health Officers more effective through better record-keeping and defaulter tracing.
Iringa Urban DHMT in Tanzania used the introduction of more supportive rather than fault finding supervision as one strategy for addressing low coverage and quality of Care and Treatment Clinic (CTC) services. The frequency and scope of the visits (previously focussed on the clinic in-charge, but now included all facility staff) had increased. Staff and CTC users said that quality of services had improved. This was partially attributed to improved supportive supervision, but also to strong efforts on the part of the CHMT to avoid stock-outs of drugs and supplies.
In Jinja district in Uganda the original strategy of supervision was focussed on managing absenteeism as the DHMT gained a deeper understanding of the problem. The team needed to develop a tool for tracking absenteeism – a simple attendance register - and to form a disciplinary committee to deal with offenders. Although no quantitative data on the reduction absence was available, the change was clearly noticed and one respondent claimed that having a fuller complement of staff at the beginning of the day enabled clinics to finish their work earlier in the afternoon.
Even just the fact of focussing on strategies for improving health workforce performance seems to have helped the DHMTs. However, the strategies may not always have been effective. For example, the introduction of the incident book in Upper Manya Krobo in Ghana seems to have got a negative reaction from one of the health workers interviewed; in Kabarole district in Uganda the selection of some junior staff as supervisors caused some friction amongst senior staff. There may have been other complaints from staff about strategies introduced by the DHMTs that were not picked up by the study. Getting feedback is a good way for the DHMT members to learn and future initiatives could consider the management of change to support implementation of work plans.
c. Other effects
There were also other wider health systems effects from PERFORM. For example, in Ghana the work plans focussed on encouraging and appreciating stronger interaction with the communities played an important role. In Tanzania the CHMTs considered that the general availability of drugs, medical equipment and supplies had improved during the projects timeframe. Several references were made that the CHMT members’ efforts had certainly led to greater availability of condoms in the districts. HMIS data confirms that the availability of ARVs at the CTCs was improved. In Uganda, supervisions schedules had been drawn up and shared with partners to avoid duplication of visits and to share resources generating greater efficiency across systems.
Finally, with respect to health system governance, PERFORM seems to have helped catalyse a more participatory approach to the planning/management of the decentralized health system, and the implementation of interventions in all the settings. By encouraging the engagement of other stakeholders and stronger community involvement it can be argued that PERFORM may have enhanced transparency, and ultimately contributed to greater accountability of the DHMTs.
The only perceived negative effect, voiced by a number of respondents though not forcefully, was the time needed for implementing their work plans – and particularly for monitoring and reflection. This is understandable, given how busy the DHMT members are, but on the other hand their role is, by definition, to manage which takes time.
4. Conclusions
In spite of the decentralised context in which the DHMTs are operating, the DHMTs have limited official authority for managing their human resources, though in Tanzania and Uganda the district level councils have more autonomy. Even so, establishment control and levels of pay remains a national level responsibility.
The action research approach to management development appeared to be an appropriate intervention and DHMTs demonstrated the ability to carry out a thorough root cause analysis of problems identified and to develop appropriate integrated work plans to address the problems.
Most DHMTs were able to implement their work plans using resources from the existing district funding allocation, though some were entrepreneurial about obtaining financial support from development partners in the district. The lack of implementation funds from the project was actually seen by some participants as beneficial for management development.
Some of the strategies for improving workforce performance were beginning to show positive results at the time of the final evaluation. More important from a management strengthening perspective is that the DHMTs were realising the importance of monitoring the effects of the work plans and were making their own decisions about modifying them as necessary. Through this process they were beginning to find out what strategies worked best for improving workforce performance and service delivery.
Through the development of work plans for which they had ownership, the DHMTs appeared to make better use of what they had perceived to be limited decision-space in the management of human resources – and other areas of management. This increased their confidence in improving not only health workforce problems, but also wider service delivery challenges.
In general, DHMTs wanted the programme to be extended or expanded. It will be important to build on the critical mass and support for the approach that has been developed at the district level, and the knowledge developed at the regional level in Ghana and Tanzania, to continue the management development process. Plans for this have been discussed with relevant stakeholders including the Ministry of Health in each country.
Potential Impact:
1. Impact
1.1 The findings from PERFORM enhances the state of the art in this under-researched area in the following ways:
• The study has confirmed the appropriateness, acceptability and effectiveness of the action research approach for management strengthening at district level to improve health workforce and potentially other health systems areas. The most challenging aspect of the action research approach appeared to be the ‘reflection stage’, though this might be because of the difficulty of monitoring it.
• The study provides new knowledge about the context – enablers and barriers – to the development and implementation of the bundles of human resource and health systems strategies for improving the performance of the health workforces and, in some cases, health services.
• The study provides new knowledge about the actual and perceived ‘decision space’, or managerial room-for-manoeuvre, for improving workforce performance at district level in decentralised contexts.
• The study has not been able to prove that work plans of integrated bundles of human resource (HR) and other health systems strategies work better than single HR interventions. However, it has shown that with minimal input it is possible for DHMTs to develop these work plans based on a thorough problem analysis, some of which seemed to lead, or could feasibly lead to, improved workforce performance.
• The study provided useful insights into the range of strategies that DHMTs can develop to improve health workforce performance and provides examples of the kind of steps that DHMTs can go through to incrementally address problems and monitor progress.
1.2 Further research questions that would contribute to the state of the art arise from the study, including:
• How to scale up the programme to other districts while maintaining fidelity to the action research approach? What role could DHMTs from the study districts play?
• How long does the management strengthening intervention need to be to achieve sustainable improvements?
• What degree of decentralisation is needed for improving health workforce performance and other service delivery improvements using an action research approach?
1.3 The capacity of consortium partners was developed to support research using action research for management strengthening in the areas of study design, process evaluation, facilitation as well as improving our understanding of decentralisation and management, and methods for improving health workforce performance. These research teams are now well-placed to undertake similar or more advanced studies. A paper on building capacity to facilitate action research in health systems is currently being developed.
1.4 Partnerships have been developed in each country amongst academia, civil society, policy-makers during the life-time of the project. At the international level we have collaborated with other research teams using similar interventions on platforms at conferences and through social media engagements. We have collaborated with WHO AFRO’s Director of Health Systems and Services Cluster and are building relationships with the Global Health Workforce Alliance, a number of bilateral donor and implementing NGOs.
2. Dissemination and exploitation of results
Dissemination has taken place at district and national level in the study countries and at international level.
2.1 Study district level
• The DHMT manual containing a description of the action research process and guidance and resource materials for developing bundles of human resource/health systems strategies was distributed to all DHMT members attending National Workshop 2 in all three countries.
• 2-3 inter-district meetings were organised for study districts in each country to share experiences of the developing and implementing their work plans.
• In addition, in Uganda four editions of a district newsletter called ‘The Performer’ were produced by the CRT in collaboration with the DHMTs and a closed Facebook page was established for DHMTs to share their experiences.
• All countries had dissemination meetings to share the results of the Country report in early 2015. Tanzania had a final dissemination meeting with the districts in August.
2.2 Neighbouring districts and regional level
• Kabarole district in Uganda was visited by a DHMT not participating in PERFORM to learn more about the programme and the Kabarole DHMT asked to go to other districts to talk about the PERFORM approach
• The research officer from Eastern Region in Ghana collaborated with the CRT throughout the programme ensuring the Regional Health Office was constantly appraised of PERFORM’s progress and impact. The Regional Health Administration is carrying the programme’s approach forward by ensuring experiences shared by study districts during performance review served as a reference point for other districts.
• Representatives of the Regional Office in Tanzania frequently attended workshops and meetings of the PERFORM districts
2.3 National level:
• All three countries established Country Research Advisory Groups who have been kept informed of the progress of the project. The Uganda team has also had engagement with the Senior Management Committee at the Ministry of Health.
• The Uganda team reported on the National Workshops in “Public Health Matters”, the newsletter of the Makerere University School of Public Health. The team has also produced a number of blogs about the project.
• The Uganda team made a presentation on PERFORM at the Joint Annual Scientific Health Conference 2014.
• A national level dissemination workshop took place at the end of the project in Kampala, Uganda. This was attended by the Ministry of Health, NGOs, development partners and the press. Flyers were distributed and an 11-minute video on the PERFORM project was shown as well as a presentation on the overall findings from the PERFORM consortium and a presentation on the specific findings from Uganda. The event was reported through two news broadcasts and one newspaper article. The presentations were well-received and the approach used by PERFORM was judged to be appropriate, cost-effective and the view was that it should be extended to other districts. Recommendations included: 1) refine the action research facilitation tools and make them available for those who may want to replicate PERFORM in other districts; 2) provide documentation of the whole process - success stories, good practices, recommendations for policy; and 3) incorporate this approach into district staff orientation and induction of new staff for creation of a common vision. 4) share the country report with the Director General of Health Services and arrange a meeting to discuss how to take PERFORM forward (she was scheduled to attend the dissemination meeting, but at the last minute was unable to come).
• A national level dissemination workshop was held in the Iringa Region in Tanzania in August 2015. The workshop took place with 57 participants. These included the Regional Health Management Team (RHMT) for Iringa Region, Council Health Management Teams (CHMTs) for Kilolo, Mufindi and Iringa Urban districts, District Medical Officers (DMOs) from the neighbouring Iringa Rural and Wanging’ombe districts, members of the Country Research Team (CRT), European Partners from Leeds (UK) and representatives of the Ministry of Health (MoH), Tanzania Commission for AIDS (TACAIDS) and Tanzania Network for Women Living with HIV/AIDS (NETWO). There were presentations, including the video about the implementation of the PERFORM project in Ghana, powerpoint presentations on overview, design and methods of PERFORM and the PERFORM study and findings in Tanzania. After the dissemination meeting participants agreed to make available to the Ministry of Health the executive summary of PERFORM’s work on how to strengthen district performance and a concept paper on how to train trainers for scaling up the PERFORM approach to other districts in Tanzania.
2.4 International level
• We have established an internal working group to steer research uptake at the global level that will continue to operate for some time after the end of the project
• The website (www.performconsortium.com) has been updated and the structure simplified for on-going posting of materials.
• A Twitter account (@PERFORMtug) has been established for promoting events and products and has been used in a multi-project twitter chats engaging with academics and policy makers. The first twitter chat, held on 20th August 2015, was on the topic of People Centred Health Systems Research Methods and was hosted by Health System Global. #HSR2015 - Twitter Search. PERFORM made a significant contribution to this event as a panellist and a Storify was produced. On the 29th October PERFORM took part in a second twitter chat organised by the Global Health Workforce Alliance about their new human resources for health strategy and this was an opportunity to raise some of the findings from our work.
• Early on in the project a case study was produced for the practical guide on implementation research produced by the Alliance for Health Policy and Systems Research: http://www.who.int/alliance-hpsr/resources/implementationresearchguide/en/
• Information materials (leaflets, briefing notes and case studies) have been produced in hard copy and electronic form (the latter posted on the web).
• Four short videos have been produced and are available on the website.
• A set of facilitation materials (research manual, DHMT manual, workshop materials, follow up checklists, etc.) developed during the project will be made available on the website and promoted through relevant websites and networks for use by people working on similar projects. We are planning a webinar with the support of Health Systems Global and relevant Thematic Working Groups to present the facilitation materials.
• We are engaging with stakeholders to share the findings of the project and to discuss how they may be taken up in other initiatives. We have engaged in dialogue with the Global Health Workforce Alliance and the human resources for health department of WHO HQ; the directorate of Health Services and Systems of the regional WHO office in Brazzaville; NGOs including Equinet and Medicus Mundi; funding agencies including USAID, DFID and the Swiss Development Co-operation; Health Systems Global and the Thematic Working Group on teaching and learning; and the EC-funded CHEPSAA project for depositing resources.
• Our policy engagement has included formally and informally inputting into the Global Health Workforce Alliance consultation on the new WHO human resources strategy.
2.5 Presentations
• We have engaged with other projects using action research including sharing a platform with the District Innovation and Action Learning for Health Systems project (DIALHS) project from South Africa and the MANIFEST project in Uganda at the second Global Symposium for Health Systems Research in Beijing in 2012. We got funding from the conference organisers to bring a member of one of our participating DHMTs in Ghana.
• We have been successful in presenting our work recently at a number of important conferences. Of particular note was the panel at the Third Global Symposium for Health Systems Research in Cape Town in 2014 where we were able to bring a member of the DHMT from Uganda, funded by the conference organisers, to speak about his experience as part of the overall presentations and we included as a discussant the Director of Health Services and Systems, WHO, Brazzaville. A blog on the lessons from this panel was posted on the website of Health Systems Global (link on PERFORM consortium website).
• Tim Martineau (LSTM) and Stephen Maluka (IDST) presented at the Swiss TPH Spring Symposium, 23rd April 2015: “ ‘We choose the problems - that's the difference!’ Strengthening district health management in Sub-Saharan Africa.”
• Comfort Mshelia (UNIVLEEDS) presented “Practising Action Research” conference at, Action Research Group Ireland, 5th Action Research Colloquium University College, Dublin, 18-19 June, 2015: “What can we learn from using action research to strengthen district health management in African countries? Experiences from PERFORM”.
• Joanna Raven presented at the 2015 International Health Conference, St Hugh's College, Oxford 25th -27th June: “Improving health workforce performance in Uganda: linking research and practice through action research”.
• Following the end of PERFORM, Tim Martineau also presented on the PERFORM project at the 9th European Congress of Tropical Medicine and International Health in Basel, September 2015: “Management strengthening using health workforce performance problems in decentralised contexts: lessons from Ghana, Tanzania and Uganda”.
2.6 Publications
2.6.1 Published articles:
• Mshelia C, Huss R, Mirzoev T, Elsey H, Baine SO, Aikins M, et al. Can action research strengthen district health management and improve health workforce performance? A research protocol. BMJ Open. 2013;3(8):e003625.
• Bonenberger M, Aikins M, Akweongo P, Wyss K. The effects of health worker motivation and job satisfaction on turnover intention in Ghana: a cross-sectional study. Hum Resour Health. 2014;12(1):43.
• Bonenberger M, Aikins M, Akweongo P, Bosch-Capblanch X, Wyss K. 2015. What do district health managers in Ghana use their working time for? A case study of three districts. Plos One. DOI: 10.1371/journal.pone.0130633. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130633
2.6.2 Articles currently under development include the following topics:
• Decision space in decentralized district health workforce management
• Using Diaries for Action Research
• Action research as a means of developing management competencies
• How to improve efficiency and performance of district health managers in Ghana
• The role of context in management strengthening
• Building capacity to facilitate action research in health systems.
List of Websites:
http://performconsortium.com/
The biggest barrier to achieving the health-related Sustainable Development Goal (number 3) in many countries is the lack of an adequate and well-performing health workforce. There are numerous initiatives in Sub-Saharan Africa to scale up the workforce, but fewer effective initiatives to improve the performance of the existing workforce. Integrated human resource management strategies, complemented by strategies related to other health system areas are needed to improve workforce performance. These strategies need to be based on a thorough analysis of the problems that analyse root causes.
District-level health managers in decentralised contexts are well-positioned to take a system-wide approach to improving the performance of their frontline health workers. They are close to the problems and know what they can achieve with their available resources. The PERFORM project used a participatory action research process as management strengthening intervention to facilitate the development and implementation of work plans to improve workforce performance in Ghana, Tanzania and Uganda in three districts in each country.
Researchers in each country carried out a situation analysis in each study district jointly with district health management teams (DHMT) to identify and prioritise workforce performance and service delivery problems. The research teams facilitated a process of detailed problem analysis and developed appropriate work plans through using consultative meetings and two short workshops. The teams then followed up the implementation of the work plans with the DHMTs, continuing to facilitate the action research initiative and collecting process evaluation data. An endline evaluation was carried out after an 18-month period, using interviews, focus group discussions and routine data and document analysis to identify the effects of the intervention on management strengthening and the impact of the work plans on health workforce performance and service delivery.
The action research approach to management development appeared to be an appropriate intervention and DHMTs demonstrated the ability to carry out a thorough root cause analysis of problems identified and to develop appropriate integrated work plans.
In spite of limited official authority for managing human resources, most DHMTs were able to implement their work plans using available resources, though some were entrepreneurial about obtaining financial support from development partners in their district. Some of the strategies for improving workforce performance were beginning to show positive results at the time of the endline evaluation. DHMTs were realising the importance of monitoring the effects of the work plans and were making their own decisions about modifying them as necessary and learning which strategies worked best for improving workforce performance and service delivery.
Through the development of work plans for which they felt ownership, the DHMTs appeared to make better use of what they had perceived to be limited decision-space in the management of human resources – and other areas of management. This increased their confidence in improving not only health workforce problems, but also wider service delivery challenges.
Many DHMT members wanted the action research management strengthening approach to be extended or expanded. Plans for this have been discussed with relevant stakeholders including the Ministry of Health in each country to explore ways of building on the PERFORM project. Engagement with international stakeholders will continue exploring the use of the approach in other countries and facilitation materials are being made available through the project website.
Project Context and Objectives:
1.Project context
'Supporting decentralised management to improve health workforce performance in Ghana, Uganda and Tanzania’ (PERFORM) was a multidisciplinary and multi-partner project implemented over four years (September 2011 – August 2015) with the financial support from the European Commission FP7.
The PERFORM project research framework outline and action plan were defined in a Grant Agreement with the EC (Ref: 266334) as part of the work programme HEALTH. 2010.3.4-1 (Develop and assess key interventions and policies to address the human resource crisis in the health sector).
2. Background to thinking about workforce performance and systems thinking
In sub-Saharan Africa (SSA) the single biggest barrier to scaling up the necessary health services for addressing Millennium Development Goals 4, 5 and 6, Universal Health Coverage, and for the achievement of the health related aspects of the 2030 Sustainable Development Agenda (goal number 3) is the lack of an adequate and well-performing health workforce. Many donors and international initiatives are focusing on increasing the numbers of available health professionals through advocacy (Scaling Up, Saving Lives) and country-specific initiatives such as the Emergency Human Resources Programme in Malawi, the health extension workers programme in Ethiopia, the Emergency Hiring Plan in Kenya. However, the deficit in health professionals needs to be addressed both by scale-up through funding and training more new health personnel and also by improving the performance of the existing and future workforce.
Most development and research emphasis has been on the first of these – increasing numbers (such as the PEPFAR programme to train 140,000 new health workers in Africa and the expansion of the use of community health workers). There has been a serious neglect of initiatives to address the complex area of existing workforce performance. Examples of poor workforce performance include high vacancy and turnover rates at the management level and poor clinical behaviour, frequent absenteeism at the individual level and low health worker productivity. The PERFORM project focused on this aspect by providing district managers with new knowledge and skills for effectively intervening within their current contexts of decentralisation to improve the performance of their staff as a response to the workforce crisis.
A number of complex factors affect workforce performance. Of particular importance are the maldistribution of staff, inappropriate task allocations and poor working conditions (including training, management and support). These factors may also lead to poor staff retention. Understanding the nature of these factors and developing appropriate responses will both improve the performance of the existing workforce and may reduce staff losses. It will also increase the effectiveness of future new health personnel trained and deployed as part of workforce scale-up to meet the health–related SDGs and the drive towards Universal Health Coverage.
There are a wide range of measures which managers can use to address human resource (HR) performance issues, such as supervision, appraisal and improved work organisation. Research has shown however, that effectiveness of these HR strategies relates to the nature of the factors that influence employee behaviour e.g. gender, career stage, level of responsibility. It is therefore necessary to select strategies that respond to the factors that influence the behaviour of particular health personnel.
In addition to choosing the right strategies, there is also a need for their integration across human resource management (HRM) practices. For example, the effectiveness of the recruitment system may benefit from changes in the remuneration system. Current thinking on health systems strengthening also suggests the importance of integration of health workforce strategies with the five other health system building blocks (i.e. service delivery, information, financing, leadership & governance, medical products, vaccines and technologies). It is imperative to consider potential unintended as well as intended consequences of a strategy within the wider health system. For example, while the use of incentives for the provision of HIV services is expected to result in better access for the user, it can also lead to a reduction in access in other areas of service provision.
The decentralisation of the planning and management of health systems continues in SSA (e.g. in Kenya under the national devolution policy). Decentralisation has frequently been to the level of district. Research has been conducted to understand how to allocate resources more efficiently at this level. However, there has been no equivalent research to understand how such decentralised authority can be effectively used, within the constraints of available resources, to improve health workforce performance at the district level.
This research project was designed to enhance our understanding of how, and under what conditions, a management strengthening intervention can improve workforce performance within districts in decentralised settings. A guiding principle of the intervention was that most effective management strengthening approaches address real problems that managers face and use planning and management tools that managers are familiar with and for which they are likely to get support in future.
Partners with health systems research expertise were selected from countries with decentralised health systems: Ghana (School of Public Health, University of Ghana); Tanzania (Institute of Development Studies, University of Dar-es-salaam); and Uganda (Makerere School of Public Health). Each of these countries face major problems of inadequate health workforce distribution and performance. They also have decentralised management structures that offer management teams greater and more relevant decision-making opportunities including in the area of human resources. The research project was led by Liverpool School of Tropical Medicine (UK) with support from the Swiss Tropical and Public Health Institute and the Nuffield Centre for International Health and Development, University of Leeds (UK). All three European institutions had substantial health systems expertise and experience of working in decentralised contexts in SSA.
3. Description of the conceptual framework
The research has been conducted in Ghana, Tanzania and Uganda. The research was designed to study how management strengthening interventions can be used, and under what conditions, to enhance workforce performance. A comparative analysis of the findings from three study districts in each country added new knowledge as to the effect of different country contexts on these interventions. This provided insights into the application of the new approaches in different district contexts across three African country settings.
The theory of change for PERFORM project was: action research in decentralised decision making environments provides district management teams with opportunities to focus on human resource and health systems management by using a systematic process of identifying relevant problems and developing customised strategies to improve workforce performance; action research also provides opportunities for reflecting on successes and failures in planning and implementation, and if sustained and coupled with adequate “decision space” ultimately strengthens the managerial competences of the DHMT for improving workforce performance and other areas of health management.
The project uses a systems approach to address the workforce problems by ensuring that HR strategies to address the problem are integrated; that complementary health systems strategies are used, and unintended effects are identified. The systems approach is operationalised through the management strengthening intervention – based on the principles and four stages of action research – which included a problem analysis leading to appropriate strategy design (1. Plan: the application of the strategy (2. Act); and the monitoring and evaluation (3. Observe; and 4. Reflect) of the strategy leading to modification or re-design (1. Plan) – and so on. The new knowledge provided by the project relates to both the use of the workforce strategies and the management strengthening intervention (see Figure 1 in attachment).
4. Main objectives
The overall aim of the project was to identify ways of strengthening decentralised management to address health workforce inadequacies by improving health workforce performance in sub-Saharan Africa.
The specific objectives were:
1. To support health managers to carry out a situation analysis on the health workforce, with a particular focus on performance, in the study districts.
2. To develop and test context-specific management strengthening processes, focused on improving workforce performance, which will:
a. identify areas of health workforce performance to be improved
b. implement integrated HR and health systems strategies feasible within the existing context, to improve health workforce performance;
c. monitor the implementation of the strategies and evaluate the intermediate processes and impact on health workforce performance, and the wider health system.
3. To conduct comparative analyses across districts and countries of:
a. the management strengthening intervention to support improved workforce performance, and
b. processes of implementing the integrated HR and health systems strategies and intended and unintended effects on health workforce performance and the wider health system.
4. To provide ongoing communication of the research process, findings and conclusions, in order to raise awareness and change attitudes of sub-national, national and international stakeholders.
5. To consolidate research capacity of partners on integrated approaches to workforce performance improvement and contribute to strengthening capacities of decentralised management of district health systems.
6. To establish and maintain effective partnerships amongst academia, civil society, policy-makers, and health managers in study countries and amongst partners.
5. Work packages
Eight Work Packages were used to achieve the PERFORM objectives. These included Study design and methodology development (package 1) which preceded the three country case studies (packages 2-4) to deliver the intervention which was followed by the comparative analysis (package 5). Research communication (package 6) and capacity building (package 7) ran throughout the project and the project management package (8) supported the other 7 work packages (see Figure 2 in attachment).
Project Results:
1. Introduction
The research was led in each country by the respective Country Research Team (CRT) with support from a paired EU partner (EP). The CRTs worked with selected district health management teams (collectively referred to in the project as DHMT and specifically in Ghana as Budget Management Centre (BMC) and in Tanzania as Council Health Management Team (CHMT)). This part of the report describes the PERFORM methodology and then presents the findings with some details at country level, but the main findings are from the comparative analysis.
2. PERFORM research methodology
This section introduces the concept of action research. It then describes the three projects phases and the related research methods.
2.1 Action research
The PERFORM project used an action research (AR) approach. The definition of AR that we applied during PERFORM was as follows:
“Action Research (AR) is an enquiry which is conducted by a group on a problem which is of importance to them. Its aim is two-fold; to improve practice and to generate knowledge about the processes and strategies that work best to create that improvement.”(Reason and Bradbury 2001)
The DHMTs, facilitated by the CRTs, worked through systematic cycles of planning, acting, observing and reflecting to:
1. Describe and analyse the problem they face
2. Identify and plan strategies to improve situation or solve problem
3. Implement the changes needed
4. Observe, explain and reflect on the process and the effects of changes made.
The DHMTs then continued with subsequent cycles (see Figure 3 in attachment) to continue to create improvements. The CRTs worked with the group to build participation, provide research methods support during observation phases of the cycles, and to record and analyse the process and strategies of change. Successive cycles are beneficial as they can deepen the learning about a problem and its solutions. In practice these may be more like a spiral, or a cycle with smaller cycles spinning off and feeding back into the main study.
The following aspects of AR in PERFORM were used to build research evidence:
• An initial situation analysis to collect research data on the problem, this feeds into the first cycle of AR.
• Robust use of research and analysis methods during the observation phases of the AR cycle(s) guided by the CRTs.
• Record and reflect on the change process throughout AR cycles using diaries.
• Recollect core data for evaluation including context, to identify within-district changes.
To support the process of implementing the AR cycle, two guidance documents were developed in the first part of the project. The first was the Research Methods Manual (D1.1) which provided tools for conducting the initial situation analysis and then tools were developed and added during the life of the project. The first edition of the Research Methods Manual was produced in August 2012. The second guidance document produced was the DHMT methods manual to explain the research process. It also included a set of guidelines on selecting HR/HS bundles to improve workforce performance (D7.2) to help DHMTs to make their work plans to address workforce problems they had analysed – mainly in the second of the DHMT workshops. This was later incorporated into the Research Methods Manual, which is being made available as part of a package of capacity building materials developed by the project.
2.2 Project phases
The project had had 3 phases spread out over the 48 months, starting in September 2001, and a timeline of key activities for three phases are given in Figure 4 in attachment:
2.2.1 Phase 1: preparation (September 2011-May 2012)
Phase 1 covered the project start-up and selection of the three districts in country, the establishment of Country Research Advisory Groups, made up of key stakeholders at national level, to support the CRTs and assist with research uptake. Project management systems were established. The methodology was further developed with input from all partners resulting in the production of a methodology manual (Deliverable D1.1). The overall management strengthening intervention using concepts from health systems thinking and action research applied within the district context was agreed in this phase. The methodology and agreement is based on a common understanding amongst all members of the research teams of the proposed research approach.
Ethics approval in each of the three study countries and for the European partners was sought and obtained during this phase of the project. Capacity of the consortium partners was strengthened through a series of activities including a workshop in Tanzania for carrying out the research and for facilitating the action research process with the DHMTs.
Three districts were selected in each country Ghana, Tanzania and Uganda using pre-defined criteria. Because of the collaborative nature of the management strengthening intervention, one important criterion for the selection of study sites was a motivated and reasonably staffed district management team with which to work. A second criterion was the inclusion of a mix of types of districts reflecting different contexts including a mix of rural and urban. In practice the selection was negotiated with the health service administration at national level and in Ghana and Tanzania this led to selection of districts in the same regions. In Uganda, district selection also took into consideration a range of levels of performance of the districts over the past three years as indicated in by the performance rank in the Ministry of Health league table. The list of study districts by country is given in the attachment.
No ‘control’ districts were selected in any of the countries, because the project did not seek to attribute potential changes in management processes exclusively to the AR approach. However, a point was made of ensuring the project methods and tools captured information about wider contextual changes in each setting so that these could be taken into consideration when assessing the impact of the interventions.
2.2.2 Phase 2: Intervention (June 2012-August 2014)
Phase 2 started in 2012 with an initial situation analysis carried out together by the CRTs and the DHMTs. This was the beginning of the problem identification and analysis phase which was carried through to a series of short national workshops that brought the three DHMTs together. In the second National Workshop the DHMTs developed an action plan to address the problems that identified. These plans were later refined, resourced and implemented. The CRTs provide facilitation support during this period through visits, joints meetings for the DHMTs and communication by phone and e-mail.
a. Situation analysis
The purpose of the initial situation analysis was to serve as the baseline for the project as well as
Inform the subsequent action research cycles in each district through identification of priority challenges to be addressed. It focused on some common core HR and health systems indicators across districts and countries to allow for the comparative analysis. As much as possible, these indicators used routinely collected data and performance indicators (gender disaggregated where possible) to simplify the collection process, minimize disruption and to increase the chance of expanding and sustaining such an approach beyond the project period. The objectives of the (initial) situation analysis were to identify areas of exceptional service delivery performance, areas of staffing shortage, problems of health workforce performance, health systems and contextual factors affecting performance, and current management and communication process used by the DHMTs.
b. Methods
The initial situation analysis was a two-step process in each district. The purpose of Step One was to collect information on the structure, staffing and health service activities of the district, while Step Two focused on identifying human resource problems and their causes. The method used in Step Two was group discussion involving DMHT members facilitated by the CRT. The output was an initial list of prioritised problems that the DHMT considered they had the capacity to influence or change.
c. Data collection
Data was collected in the three countries during several visits by the CRT between August and September 2012. As part of the action research approach, the DHMT members worked with the CRT to collect the data for the situation analysis. The CRT made several visits to the districts between July and August 2012. A generic questionnaire was used to collect data on the health workforce, DHMT, health system, local and national contexts. The CRT in consultation with the European partners concurred that some additional customized data collection tool was necessary to collect data missed by using the generic data collection tool. A data collection tool focusing on the specific district contexts was used to collect the missing data. Data was also extracted from reports and databases held at the district.
d. Data Analysis
Data analysis was done by the DHMTs with the support of the CRTs. This offered an opportunity for the DHMTs to learn or improve their skills in analysing data. Both the DHMTs and CRTs reflected on the data and emerging issues such as power dynamics in the DHMT, district problems, decision-making, and verification of data. These were valuable health workforce situation indicators to focus on in Step Two of the situation analysis. Triangulation of secondary data, and the data generated through discussions was intended to strengthen understanding of the district situation and provide a firm basis for the next step of the initial situation analysis.
2.2.2 Problem identification and prioritisation
At National Workshop 1 in October 2012 each DHMT presented the findings of the situation analysis and their initial problem analyses to the other DHMTs as well as other resource persons (e.g. civil society representatives) and further root cause analysis was carried out. On Day 2 some guidance was given from CRTs and peers on refining the problem trees and prioritising the problems to be addressed. Further discussions were held at district level and the prioritisation and root cause analysis was finalised on the first day of National Workshop 2 that was held in February 2013.
2.2.3 The process of selecting integrated bundles of human resource and health systems (HR/HS) strategies
The concept of the action research process had been explained to the DHMTs from the first exploratory visits in early 2012 and during later visits the DHMT manual which explained the action research processes was shared. An additional section (based on Deliverable 7.2) was later added to provide guidance during National Workshop 2 on selecting appropriate human resource and health systems strategies to address problems identified. This covered the concepts of developing coherent HR strategies and integration with complementary HS strategies –referred to as bundles of HR/HS strategies. The manual also contained a reference table with about 40 strategies, each with sample activities, possible indicators for monitoring and evaluation and suggestions of how each strategy might be linked to other HR or HS strategies to improve integration both within the bundles, but also with wider plans. The work plans containing the HR/HS bundles were developed using the following criteria:
• Focused on improving health workforce performance in the district
• Measurable and observable effect on workforce performance within 12 – 18 months
• Implemented within available resources in the district
• Linked to district plan
• Linked to existing policies / strategies
• Based on the evidence accumulated in the situation analysis
• Bundles are logical.
Each district plan was subjected to peer review by the other DHMTs and then whenever possible incorporated into district planning frameworks that the DHMTs had brought with them – mostly on their laptop computers.
2.2.4 Implementation of HR/HS bundles
The implementation of the HR/HS bundles phase was from March 2013 (after National Workshop 2) until September 2014 (when the evaluation took place). During this period the DHMTs first refined their plans and in some cases sought extra resources from development partners for implementing the plans. The DHMTs in Tanzania had to wait some time until the budget allocation for the next financial year arrived before implementing some of their plans.
2.2.5 Action Research process recording
Diaries, and in Ghana a documentation template, were used to record and reflect on the change process throughout the AR cycles. The importance of the learning developed in particular from the observation and reflection parts of the action research cycle was explained on the final day of National Workshop 2 and the DHMTs were all given diaries to keep a record of the implementation of the plans they had developed and in particular their reflection on the implementation. CRTs made regular visits to the districts to discuss implementation of the bundles of strategies including their reflections on implementation and effects.
2.3 Phase 3: evaluation (August 2014 to August 2015)
2.3.1 Evaluation of management strengthening and HR/HS bundles at country level
In order to build the knowledge base of the overall effectiveness of the approach used in PERFORM, ongoing data collection and analysis is needed. Two stages of analysis were undertaken:
1. Analysis of data collected by the CRTs in collaboration with the DHMTs throughout the project as part of the action research cycle.
2. Overall evaluation which will include comparative analysis following the period of implementation of the HR/HS bundles.
The evaluation took place in September 2014 after the HR/HS bundles had been implemented for about 18 months. This situation analysis was a one stage process, with data on the health workforce, DHMT, health systems, local and national context being collected.
a. Methods
The evaluation sought to document processes of both the management strengthening intervention (the action research cycle) and the development and implementation of the bundles of HR/HS strategies. It also aimed to identify the effects (both intended and unintended) of the management strengthening and bundles of HR/HS strategies. The study, using qualitative methods, gathered the perceptions of a mixed group of stakeholders (the DHMT, sub-district managers, health staff, relevant stakeholders and the researchers themselves) to obtain a retrospective account of processes and change. In Uganda, obtaining the perceptions of community members was originally considered, but rejected after piloting because all HR/HS bundles were addressing health workforce performance problems for which the impact at the level of service delivery after a period of 18 months would not have been easily detectable. However, in Ghana and Tanzania, service users’ views were sought: client exit interviews were used in Ghana, and focus group discussions were used in Tanzania. A range of documents produced during the project (such as CRT visit reports, DHMT diaries, workshop reports) were analysed for information on processes and effect. Selected health systems and health services indicators were used to see whether any before and after effects could be identified or associations of impact made through linkages to the qualitative data.
b. Sampling
Uganda: In depth interviews with DHMT members: 4-5 members were purposively selected ensuring a range of participants with regard to gender, length of service in DHMT and position in DHMT.
Focus group discussion with DHMT members: 6-8 members were purposively selected ensuring a range of participants with regard to gender, length of service in DHMT and position in DHMT.
In depth interviews with sub district managers: two sub districts were selected in each district, and then within each sub district one health centre IV, one health centre III, and one HC II were selected. The manager from each of these facilities was selected for interview.
In depth interviews with sub district staff: using the same facilities as outlined in the sampling for the in depth interviews with the sub district managers, one health worker from each facility was selected, ensuring a range of participants across the sample for the district with regard to cadre, gender and length of service.
In depth interviews with stakeholders: the Local Council human resources officer, a Health Unit Management Committee member, a development partner and the council secretary for health were selected in each district.
Ghana: All respondents of the in-depth interviews (IDIs) were purposively selected. They were made up of the District Directors of Health Services (DDHSs), District Public Health Nurses (DPHNs), District Health Information Officers (DHISs) and District Disease Control Officers (DDCOs) at the DHMTs. At the district hospital, Matrons and heads of Reproductive and Child Health (RCH) and Antenatal Care (ANC) units were interviewed. At the sub-district level, facility heads and in-charges of RCH and ANC units were interviewed, whilst Community Health Officers (CHOs) were interviewed at the CHPS Zone level. The IDIs were conducted by CRT at the respective facilities of respondents. Clients were randomly selected with the assistance of the in-charges of the ANC and Child Welfare Clinics (CWCs) for the exit interviews. Client exit interviews were conducted by the research team at CWC sites.
Tanzania: 5 health facilities involved in the provision of CTC services were sampled in Iringa Urban; 10 facilities involved in male circumcision, PMTCT and CTC services were sampled in Mufundi; and 10 facilities that provided RCH services were sampled in Kilolo. All DHMT members were purposively selected for interviews and FGD, and health facility managers of the sampled facilities were purposively selected for in-depth interviews. One health facility staff was randomly selected for in-depth interview at each of the sampled facility in each district. Users of the services were randomly selected for FGDs as they sought services at the facilities.
All countries: In-depth interviews were carried out with the CRT and the European partners to explore in particular the support provided between the paired partners and to the DHMTs.
All relevant project documents were reviewed. These included the DHMT and DHT management meetings, work plans, CRT visit reports, diaries, inter-district meetings, national workshops and partner visit reports
c. Data collection tools
The data collection tools were developed based on an analysis of key questions about the process and effects of the management strengthening and workforce improvement components of the intervention. These included: topic guides for the FGD with DHMT, the interview with DHMT, the interview with manager, the interview with staff, interview with stakeholder; and data collection tool for health systems and health services indicators. With the exception of the DHMT topic guide, all tools used at district level were piloted in May 2014 in Luwero district and modified in consultation with all PERFORM consortium members.
d. Data collection
The CRTs in each country visited each study district during September and October 2014 and carried out the data collection using the tools. Interviews with DHMT members, health facility managers and staff, and stakeholders were conducted in quiet rooms at the district health offices, health care facilities or offices and lasted between 30 and 60 minutes. FGDs with DHMTs were conducted in the District offices and lasted 60 to 90 minutes. Focus group discussions and client exit interviews with service users were conducted in quite areas of the health facilities. Before each interview and FGD, the CRT went through an informed consent process and obtained written consent from each participant, which included recording of the interview or discussion.
Selected health systems and health services indicators: data on the district was collated at the DHO office from the HMIS and in discussion relevant staff; data on the health facilities was also collected from the HMIS, and where gaps were identified, the CRT and DHO office staff telephoned the health facility managers.
Document review: documents were collated from the district health office which included: the DHMT diary; and the District annual health work plans and reports, DHMT minutes and DHT minutes for preceding years (back as far as 2011 in Uganda). In addition to these district level documents the following national and international level documents were collated: National Workshop Reports, Inter District Meeting Reports, CRT visit reports and Consortium Workshop Reports.
e. Data Analysis
Interviews, FGDs and documents: the recordings were transcribed verbatim and checked for accuracy by the CRT. All documents were then uploaded to NVIVO qualitative analysis software (Version 10). A coding framework was agreed by the consortium, based on the questions relating to the management strengthening and workforce performance improvement interventions. All documents were coded using this framework. Next, queries were made to extract the data by theme for each district and then summarised with key supporting quotes. These were then combined to compile the reports for each of the districts. Finally these were combined to allow for inter district analysis and to produce an overall set of findings and recommendations.
Client exit interviews: simple descriptive statistics were used to analyse this data.
Data on health facilities activities (e.g. consultations, vaccinations) and workforce (i.e. number of staff by categories) were obtained from health facility reports for the previous calendar year or were provided by staff. Minimum, maximum, means, percentiles 25 and 75 and medians across health facilities were calculated for each district and for each country. Medians and Inter-Quartile Ranges (IQR = p25 to p75) are reported in order to exclude data errors produced by missing values or outliers. As the health facilities provided a different mix of activities (e.g. some may have had maternities to carry out deliveries, while others did not), in order to facilitate comparability across districts and countries a Units of Activity (UA) approach was used as a weighed estimate of the amount of activities being carried out across the health facilities. Note: the results of the analysis of health facility activities are reported in the Comparative Analysis report (D5.1) but not in this report.
f. Output from the evaluation
An evaluation report was produced in February 2015 for each country as project deliverable (D2.1 D3.1 and D4.1). The findings were presented to the DHMTs at National Workshop 3 both for the purposes of validation and as part of the research communication process, and reports were revised based on this feedback.
2.3.2 Comparative analysis
The comparative analysis built upon the three country reports which presented the project’s results from the application of the overall PERFORM methodology in the three country contexts.
As a first step a table was elaborated according to the project’s thematic and analytical framework, which traces for each country in a single table the various project processes (initial situation analysis, problem identification etc.), the effects of the AR approach on DHMT management practices, and the effects of the HR/HS strategies on workforce performance and health systems. For each of these rubrics this analytical framework contains main themes and sub-themes, questions to be answered, indicators and suggested methods to answer the questions, and the sources where the information about each can be found. The sheet was filled out for each country in turn, and a brief description of the differences and similarities between them was provided in the fourth column. The comparative analysis essentially relies on secondary data from the country reports. Taking these as the basis, in an iterative process that started at the comparative analysis workshop held in Basel in April 2015 the key areas of interest for comparison were agreed.
The comparative analysis report was produced as a publicly available project deliverable (D5.1) and is being used as a key source document for comparative journal publications and presentations.
3. Main S&T results
The programme was implemented as planned. All deliverables were submitted on time although extensions were requested and granted for the country reports and comparative analysis report.
The remainder of this section, based on the country reports and the comparative analysis report provides key results on the problem analysis and strategies developed by country and then the results from the comparative analysis.
3.1 S&T results at country level
This section presents both the problem analysis and work plans developed for each country in tabular form. Full details of the findings in each country are available in the country reports (D2.1 D3.1 and D4.1).
3.1.1 Ghana
The key problems identified by district, with their underlying causes, are presented in Table 2 in attachment. The strategies developed in National Workshop 2 to address problems identified are in Table 3 in attachment.
3.1.2 Tanzania
The key problems identified by district, with their underlying causes, are presented in Table 4 in attachment. The strategies developed in National Workshop 2 to address problems identified are in Table 5 in attachment.
3.1.3 Tanzania
The key problems identified by district, with their underlying causes, are presented in Table 6 in attachment. The strategies developed in National Workshop 2 to address problems identified are in Table 7 in attachment.
3.2 Main S&T results from the comparative analysis
This section draws on findings across the three countries from the comparative analysis report produced under work package 5. It starts with a brief review of the context including the forms of decentralisation, human resource management at district level and the structure of the DHMT. The findings related to the intervention are presented in two parts. The first covers the process, which describes the stages of the problem analysis, the design of the strategies to address priority problems and the experiences implementing the strategies. The second part covers the effects of the intervention on management strengthening, health workforce performance and service delivery, and finally the effects on other areas. The section is completed by a set of conclusions from the study.
3.2.1 Context
In Uganda and Tanzania the responsibility for the management of services at district level – including health services – has been devolved to local government and so the District Health Office is part of the local council structure, which develops plans and manages resources. In Ghana decentralisation takes the form of ‘deconcentration’ - the autonomous Ghana Health Service through the levels of the regional and district health offices with no direct linkage with local government.
In spite of the decentralised structures in each of the countries, the DHMTs officially had limited powers for managing the health workforce they were responsible for. In Tanzania and Uganda the local council had a human resource management function which dealt with the major human resource management functions including recruitment. The control on staff establishment (the number of permanent posts) is controlled centrally in both countries. In Ghana, the national level that has the core management authority related to pay levels, recruitment, regional postings, performance assessment, training schedules, promotion, disciplinary action, and termination for permanent senior levels of staff. The district has control over recruiting contract staff and intra-district postings, performance assessments, in-service training, and leave and sick-days.
The DMHTs across the three countries ranged from 8-12 members each – most core managers, supervisors and some administrative staff at the district level and in the sub-district level in Uganda. The composition of male and female DHMT members was balanced in Ghana and Uganda; there was a lower proportion of females in Tanzania. The concept of the DHMT seemed to be a little fluid and in the case of Uganda is built around a smaller District Health Team based at the district office which meets on a more frequent basis. DHMTs in Uganda and Tanzania could co-opt members as necessary and so at times could be quite big.
3.1.2 The Process of PERFORM
a. Problem analysis
Although the concept of analysing problems as part of the process of planning was not new to most participants, all DHMTs were actively involved in the process starting around September 2012 through to the first part of National Workshop 2 in February 2013. Some participants said that the process helped them become more questioning, see things in a different way and recognising that in the past they did not always prioritise the problems to be addressed. The problem trees (showing links between cause and effect) produced and subsequently revised demonstrated the DHMTs’ engagement with the process of root cause analysis which for many participants was a new part of the problem analysis process. The process of refining the problem trees was time-consuming, but proved a good investment for developing appropriate strategies.
The aim of PERFORM was to address, amongst other problems, those related to health workforce performance. This was approached differently with the DHMTs in Tanzania and Ghana identifying problems with service delivery e.g. poor service coverage and then identifying the causes in the system including those related to health workforce performance. The Uganda DHMTs, on the other hand, started directly with workforce performance problems. The two approaches yielded some common causes of health workforce performance problems across the three countries, for example supervision support, staff capacity and staff attitude. However, what seemed important to the DHMTs was that, unlike in some similar programme-specific interventions, they had ownership of the process of identifying, prioritisation and analysis of the problems. This provided the motivation for developing effective strategies that would address these problems in an appropriate way.
b. Strategy development
A frequent failing of the development of management strategies is that they are not clearly based on a clear understanding of the problems. An analysis of the strategies developed by the DHMTs shows a clear correlation to their problem analyses. For example, to address the problem of high dropout rate in Akuapim North district in Ghana, their work plan included strategies to: intensify monitoring and supervision; intensify EPI defaulter tracing; community mobilisation and use of work plans. See other examples in Tables 3, 5 and 7 in attachment. This clear correlation was achieved through the use of tools and processes that ensured this clear linkage was made – in some cases physically developing an objectives tree (work plan in diagram form) alongside the problem tree to make the linkages visible.
There was some evidence form observation during National Workshop 2 that the district teams consulted the reference table provided in the DHMT methods manual with sample HR strategies, supporting activities and possible indicators, though this was not well documented.
A number of participants reported in the feedback on National Workshop 2 that the concept of the integration of human resource related strategies and wider health system related strategies into an integrated bundle was difficult to comprehend. Nevertheless they were able to do this in practice, integrating both complementary HR strategies (e.g. attendance monitoring and appraisal) with strategies addressing wider health systems problems such as ensuring medical supplies are made available alongside competence building. In addition, they followed the criteria for selecting strategies that would be both feasible and effective ensuring their affordability within available resources and synergies with interventions of other programmes/projects in the district.
The template for developing the workplan included a column for activities to support the strategies and indicators for monitoring progress. All strategies were supported in the plan by multiple activities. Indicators were developed for all plans, but only DHMTs in Ghana specified targets. One DHMT in Tanzania reported on the difficulty of developing their own indicators – this is normally something done at higher levels of management or by external agencies.
Most DHMTs were able to integrate work plans developed in National Workshop 2 in their wider funded district-level annual plans or to schedule some activities for the following financial year.
c. Implementation monitoring and modification of the interventions
The experience of implementation of the work plans developed by the DHMTs was only monitored from the end of National Workshop 1 to September 2014 (18 months), but DHMTs continued implementation after that period.
Many of the DHMTs used diaries or other monitoring tools provided by the project for reviewing the implementation of their plans. It took the districts some time to get used to the diaries and the value of their use in tracking progress. Several difficulties were mentioned including lack of clarity about which information to record in them, and how to ensure that all DHMT members including those based at the sub-district level gained the opportunity to write in them. To overcome these challenges, some DHMTs adapted the format of the diary so it could be more easily taken with them when they went to the districts. The content of the diaries tended to be mainly factual rather than reflective, as intended. However, the continued support through visits and e-mail/phone contacts by the CRTs, inter-district meetings and the DHMTs’ own meetings all provided the opportunity the teams to reflect and consequently for the work plans to evolve according to need.
In the design of the work plans teams kept “affordability” in mind when developing the strategies assuming that activities could be funded from the existing district budget because the budget headings were sufficiently broad. In other cases, where this was not possible or funds from the regular budget were delayed, they either reprogrammed activities for the next budget cycle or approached stakeholders to identify alternative sources of funding. For example, in Tanzania one DHMT approached a non-government organisation to fund training for community health workers and renovation of service delivery facilities. However, DHMTs in Tanzania, were regularly hampered by the late disbursement of the regular government budget and this was particularly late in 2014, delaying the implementation of the work plans.
Some activities were modified during the implementation period and some abandoned. For example, one DHMT in Uganda decided to allocate mentors to the new supervisors in order to improve supervision. One district in Ghana abandoned the task of “develop and implement ppm [Planned Preventive Maintenance] for solar panels and fridges” as the DHMT felt it was beyond their capacity to implement.
The DHMTs in Ghana who had set indicators with clear targets were able to monitor the impact of their work plans. Others had given more priority to developing the strategies and related activities at the design stage, rather than the indicators. However, because of the strong ownership of the plans DHMTs began to be curious about how well they were working. For example, a DHMT in Uganda started to monitor staff absence using an attendance register.
3.1.2 The Effects of PERFORM
a. Management strengthening:
From an analysis of their actions and from testimonies during the final situation analysis, the DHMTs appear to have developed or improved their management competencies related to problem analysis, strategy development and implementation with only attending about 4 days of workshop time and up to 3 days of inter-district meetings. Regarding the action research cycle, evidence of the first two steps – ‘plan’ and ‘act’ – was clear. There is less evidence of the ‘observe’ stage, which relates to both monitoring the process of implementation and in particular the effects of the plan. Nevertheless, there were indications that the DHMTs were beginning to appreciate the importance, especially as they were implementing plans that they were developing themselves. There was even less clear evidence of ‘reflection’ in the action research cycle. This is a crucial step as if this does not take place there is no learning and the DHMTs will continue to make the same mistakes. The diaries were designed to help with reflection, but DHMTs had difficulty with this function and it was challenging for facilitators to help them. However, the fact that plans were changed based on observations and reflections implies that some reflection did take place. For instance after their experience implementing the work plan one DHMT in Uganda introduced spot check supervision visits but also to put more emphasis on rewarding the best performing staff rather than using sanctions against bad performers. Respondents implied that the reflection part of the action research cycle is more challenging, and identified the need for more support in recording reflection. Nevertheless both some of the changes made to plans, as already mentioned, and some of the discourse in the regular meetings attended by CRTs suggested that reflection was taking place.
In addition to the development of DHMTs’ management, there was evidence of changes in the way in which the DHMTs operated. This included empowerment, increased initiative taking, increased team working and collaboration and application of lessons learnt to other areas.
The process of selecting priority problems, analysing them and developing strategies provided a sense of empowerment because they were doing it all themselves. One manager in Tanzania felt his team was now empowered to bring about change. The process also encouraged some DHMTs to develop and implement strategies that they might not have previously considered due to their perceived limitations of their roles and level of decision-making autonomy. This contributed to their self-confidence.
Health service managers were largely dependent on funding from government or external development partners and PERFORM provided no additional funds for implementing the work plan. In spite of initial complaints, some DHMTs saw this as an opportunity to make better use of existing resources and others showed initiative and creativity in searching for funding. For example, one DHMT in Uganda approached a bank to fund orientation of newly recruited staff in return for the opportunity to advertise their banking services. A number of other examples of co-funding were reported and some DHMT members said they developed skills in advocacy and proposal writing.
Perhaps the most striking impact the process had on the DHMTs was the stimulation of greater collaboration and teamwork within DHMTs. This was made possible through the workshops – both being together and working on a common task – and because the problem analysis showed the interdependence of many members of the team. The workshops and inter-district meetings also stimulated collaboration between DHMTs which do not otherwise have the opportunity to meet.
Finally, the DHMTs saw the possibility of applying the lessons learnt in developing strategies to improve health workforce performance to other areas of their work. In Tanzania some CHMTs used the diaries to record implementation of other areas of work.
b. Health workforce performance and service delivery
There is mixed evidence of the impact of the strategies for improving health workforce performance. There were several major challenges in measuring the impact of the health workforce improvement strategies. First, in most cases the lack of baseline data and in some cases the reliance on qualitative data at the endline. Second, the problem of attribution, given the multiple ongoing workforce and workforce related initiatives in the study districts. Having said that the study has provided examples of strategies that are likely to have had a positive impact on health workforce performance and to have contributed to improved service delivery.
The findings from both Akwapim North and Kwahu West districts in Ghana show improvements in vaccination and drop-out rates during the intervention period. There was a national scale-up of one of the vaccines, Rotarix, going from 2012 onwards which could have contributed to this increase. However, DHMTs claimed convincingly that improved supervision contributed to making the Community Health Officers more effective through better record-keeping and defaulter tracing.
Iringa Urban DHMT in Tanzania used the introduction of more supportive rather than fault finding supervision as one strategy for addressing low coverage and quality of Care and Treatment Clinic (CTC) services. The frequency and scope of the visits (previously focussed on the clinic in-charge, but now included all facility staff) had increased. Staff and CTC users said that quality of services had improved. This was partially attributed to improved supportive supervision, but also to strong efforts on the part of the CHMT to avoid stock-outs of drugs and supplies.
In Jinja district in Uganda the original strategy of supervision was focussed on managing absenteeism as the DHMT gained a deeper understanding of the problem. The team needed to develop a tool for tracking absenteeism – a simple attendance register - and to form a disciplinary committee to deal with offenders. Although no quantitative data on the reduction absence was available, the change was clearly noticed and one respondent claimed that having a fuller complement of staff at the beginning of the day enabled clinics to finish their work earlier in the afternoon.
Even just the fact of focussing on strategies for improving health workforce performance seems to have helped the DHMTs. However, the strategies may not always have been effective. For example, the introduction of the incident book in Upper Manya Krobo in Ghana seems to have got a negative reaction from one of the health workers interviewed; in Kabarole district in Uganda the selection of some junior staff as supervisors caused some friction amongst senior staff. There may have been other complaints from staff about strategies introduced by the DHMTs that were not picked up by the study. Getting feedback is a good way for the DHMT members to learn and future initiatives could consider the management of change to support implementation of work plans.
c. Other effects
There were also other wider health systems effects from PERFORM. For example, in Ghana the work plans focussed on encouraging and appreciating stronger interaction with the communities played an important role. In Tanzania the CHMTs considered that the general availability of drugs, medical equipment and supplies had improved during the projects timeframe. Several references were made that the CHMT members’ efforts had certainly led to greater availability of condoms in the districts. HMIS data confirms that the availability of ARVs at the CTCs was improved. In Uganda, supervisions schedules had been drawn up and shared with partners to avoid duplication of visits and to share resources generating greater efficiency across systems.
Finally, with respect to health system governance, PERFORM seems to have helped catalyse a more participatory approach to the planning/management of the decentralized health system, and the implementation of interventions in all the settings. By encouraging the engagement of other stakeholders and stronger community involvement it can be argued that PERFORM may have enhanced transparency, and ultimately contributed to greater accountability of the DHMTs.
The only perceived negative effect, voiced by a number of respondents though not forcefully, was the time needed for implementing their work plans – and particularly for monitoring and reflection. This is understandable, given how busy the DHMT members are, but on the other hand their role is, by definition, to manage which takes time.
4. Conclusions
In spite of the decentralised context in which the DHMTs are operating, the DHMTs have limited official authority for managing their human resources, though in Tanzania and Uganda the district level councils have more autonomy. Even so, establishment control and levels of pay remains a national level responsibility.
The action research approach to management development appeared to be an appropriate intervention and DHMTs demonstrated the ability to carry out a thorough root cause analysis of problems identified and to develop appropriate integrated work plans to address the problems.
Most DHMTs were able to implement their work plans using resources from the existing district funding allocation, though some were entrepreneurial about obtaining financial support from development partners in the district. The lack of implementation funds from the project was actually seen by some participants as beneficial for management development.
Some of the strategies for improving workforce performance were beginning to show positive results at the time of the final evaluation. More important from a management strengthening perspective is that the DHMTs were realising the importance of monitoring the effects of the work plans and were making their own decisions about modifying them as necessary. Through this process they were beginning to find out what strategies worked best for improving workforce performance and service delivery.
Through the development of work plans for which they had ownership, the DHMTs appeared to make better use of what they had perceived to be limited decision-space in the management of human resources – and other areas of management. This increased their confidence in improving not only health workforce problems, but also wider service delivery challenges.
In general, DHMTs wanted the programme to be extended or expanded. It will be important to build on the critical mass and support for the approach that has been developed at the district level, and the knowledge developed at the regional level in Ghana and Tanzania, to continue the management development process. Plans for this have been discussed with relevant stakeholders including the Ministry of Health in each country.
Potential Impact:
1. Impact
1.1 The findings from PERFORM enhances the state of the art in this under-researched area in the following ways:
• The study has confirmed the appropriateness, acceptability and effectiveness of the action research approach for management strengthening at district level to improve health workforce and potentially other health systems areas. The most challenging aspect of the action research approach appeared to be the ‘reflection stage’, though this might be because of the difficulty of monitoring it.
• The study provides new knowledge about the context – enablers and barriers – to the development and implementation of the bundles of human resource and health systems strategies for improving the performance of the health workforces and, in some cases, health services.
• The study provides new knowledge about the actual and perceived ‘decision space’, or managerial room-for-manoeuvre, for improving workforce performance at district level in decentralised contexts.
• The study has not been able to prove that work plans of integrated bundles of human resource (HR) and other health systems strategies work better than single HR interventions. However, it has shown that with minimal input it is possible for DHMTs to develop these work plans based on a thorough problem analysis, some of which seemed to lead, or could feasibly lead to, improved workforce performance.
• The study provided useful insights into the range of strategies that DHMTs can develop to improve health workforce performance and provides examples of the kind of steps that DHMTs can go through to incrementally address problems and monitor progress.
1.2 Further research questions that would contribute to the state of the art arise from the study, including:
• How to scale up the programme to other districts while maintaining fidelity to the action research approach? What role could DHMTs from the study districts play?
• How long does the management strengthening intervention need to be to achieve sustainable improvements?
• What degree of decentralisation is needed for improving health workforce performance and other service delivery improvements using an action research approach?
1.3 The capacity of consortium partners was developed to support research using action research for management strengthening in the areas of study design, process evaluation, facilitation as well as improving our understanding of decentralisation and management, and methods for improving health workforce performance. These research teams are now well-placed to undertake similar or more advanced studies. A paper on building capacity to facilitate action research in health systems is currently being developed.
1.4 Partnerships have been developed in each country amongst academia, civil society, policy-makers during the life-time of the project. At the international level we have collaborated with other research teams using similar interventions on platforms at conferences and through social media engagements. We have collaborated with WHO AFRO’s Director of Health Systems and Services Cluster and are building relationships with the Global Health Workforce Alliance, a number of bilateral donor and implementing NGOs.
2. Dissemination and exploitation of results
Dissemination has taken place at district and national level in the study countries and at international level.
2.1 Study district level
• The DHMT manual containing a description of the action research process and guidance and resource materials for developing bundles of human resource/health systems strategies was distributed to all DHMT members attending National Workshop 2 in all three countries.
• 2-3 inter-district meetings were organised for study districts in each country to share experiences of the developing and implementing their work plans.
• In addition, in Uganda four editions of a district newsletter called ‘The Performer’ were produced by the CRT in collaboration with the DHMTs and a closed Facebook page was established for DHMTs to share their experiences.
• All countries had dissemination meetings to share the results of the Country report in early 2015. Tanzania had a final dissemination meeting with the districts in August.
2.2 Neighbouring districts and regional level
• Kabarole district in Uganda was visited by a DHMT not participating in PERFORM to learn more about the programme and the Kabarole DHMT asked to go to other districts to talk about the PERFORM approach
• The research officer from Eastern Region in Ghana collaborated with the CRT throughout the programme ensuring the Regional Health Office was constantly appraised of PERFORM’s progress and impact. The Regional Health Administration is carrying the programme’s approach forward by ensuring experiences shared by study districts during performance review served as a reference point for other districts.
• Representatives of the Regional Office in Tanzania frequently attended workshops and meetings of the PERFORM districts
2.3 National level:
• All three countries established Country Research Advisory Groups who have been kept informed of the progress of the project. The Uganda team has also had engagement with the Senior Management Committee at the Ministry of Health.
• The Uganda team reported on the National Workshops in “Public Health Matters”, the newsletter of the Makerere University School of Public Health. The team has also produced a number of blogs about the project.
• The Uganda team made a presentation on PERFORM at the Joint Annual Scientific Health Conference 2014.
• A national level dissemination workshop took place at the end of the project in Kampala, Uganda. This was attended by the Ministry of Health, NGOs, development partners and the press. Flyers were distributed and an 11-minute video on the PERFORM project was shown as well as a presentation on the overall findings from the PERFORM consortium and a presentation on the specific findings from Uganda. The event was reported through two news broadcasts and one newspaper article. The presentations were well-received and the approach used by PERFORM was judged to be appropriate, cost-effective and the view was that it should be extended to other districts. Recommendations included: 1) refine the action research facilitation tools and make them available for those who may want to replicate PERFORM in other districts; 2) provide documentation of the whole process - success stories, good practices, recommendations for policy; and 3) incorporate this approach into district staff orientation and induction of new staff for creation of a common vision. 4) share the country report with the Director General of Health Services and arrange a meeting to discuss how to take PERFORM forward (she was scheduled to attend the dissemination meeting, but at the last minute was unable to come).
• A national level dissemination workshop was held in the Iringa Region in Tanzania in August 2015. The workshop took place with 57 participants. These included the Regional Health Management Team (RHMT) for Iringa Region, Council Health Management Teams (CHMTs) for Kilolo, Mufindi and Iringa Urban districts, District Medical Officers (DMOs) from the neighbouring Iringa Rural and Wanging’ombe districts, members of the Country Research Team (CRT), European Partners from Leeds (UK) and representatives of the Ministry of Health (MoH), Tanzania Commission for AIDS (TACAIDS) and Tanzania Network for Women Living with HIV/AIDS (NETWO). There were presentations, including the video about the implementation of the PERFORM project in Ghana, powerpoint presentations on overview, design and methods of PERFORM and the PERFORM study and findings in Tanzania. After the dissemination meeting participants agreed to make available to the Ministry of Health the executive summary of PERFORM’s work on how to strengthen district performance and a concept paper on how to train trainers for scaling up the PERFORM approach to other districts in Tanzania.
2.4 International level
• We have established an internal working group to steer research uptake at the global level that will continue to operate for some time after the end of the project
• The website (www.performconsortium.com) has been updated and the structure simplified for on-going posting of materials.
• A Twitter account (@PERFORMtug) has been established for promoting events and products and has been used in a multi-project twitter chats engaging with academics and policy makers. The first twitter chat, held on 20th August 2015, was on the topic of People Centred Health Systems Research Methods and was hosted by Health System Global. #HSR2015 - Twitter Search. PERFORM made a significant contribution to this event as a panellist and a Storify was produced. On the 29th October PERFORM took part in a second twitter chat organised by the Global Health Workforce Alliance about their new human resources for health strategy and this was an opportunity to raise some of the findings from our work.
• Early on in the project a case study was produced for the practical guide on implementation research produced by the Alliance for Health Policy and Systems Research: http://www.who.int/alliance-hpsr/resources/implementationresearchguide/en/
• Information materials (leaflets, briefing notes and case studies) have been produced in hard copy and electronic form (the latter posted on the web).
• Four short videos have been produced and are available on the website.
• A set of facilitation materials (research manual, DHMT manual, workshop materials, follow up checklists, etc.) developed during the project will be made available on the website and promoted through relevant websites and networks for use by people working on similar projects. We are planning a webinar with the support of Health Systems Global and relevant Thematic Working Groups to present the facilitation materials.
• We are engaging with stakeholders to share the findings of the project and to discuss how they may be taken up in other initiatives. We have engaged in dialogue with the Global Health Workforce Alliance and the human resources for health department of WHO HQ; the directorate of Health Services and Systems of the regional WHO office in Brazzaville; NGOs including Equinet and Medicus Mundi; funding agencies including USAID, DFID and the Swiss Development Co-operation; Health Systems Global and the Thematic Working Group on teaching and learning; and the EC-funded CHEPSAA project for depositing resources.
• Our policy engagement has included formally and informally inputting into the Global Health Workforce Alliance consultation on the new WHO human resources strategy.
2.5 Presentations
• We have engaged with other projects using action research including sharing a platform with the District Innovation and Action Learning for Health Systems project (DIALHS) project from South Africa and the MANIFEST project in Uganda at the second Global Symposium for Health Systems Research in Beijing in 2012. We got funding from the conference organisers to bring a member of one of our participating DHMTs in Ghana.
• We have been successful in presenting our work recently at a number of important conferences. Of particular note was the panel at the Third Global Symposium for Health Systems Research in Cape Town in 2014 where we were able to bring a member of the DHMT from Uganda, funded by the conference organisers, to speak about his experience as part of the overall presentations and we included as a discussant the Director of Health Services and Systems, WHO, Brazzaville. A blog on the lessons from this panel was posted on the website of Health Systems Global (link on PERFORM consortium website).
• Tim Martineau (LSTM) and Stephen Maluka (IDST) presented at the Swiss TPH Spring Symposium, 23rd April 2015: “ ‘We choose the problems - that's the difference!’ Strengthening district health management in Sub-Saharan Africa.”
• Comfort Mshelia (UNIVLEEDS) presented “Practising Action Research” conference at, Action Research Group Ireland, 5th Action Research Colloquium University College, Dublin, 18-19 June, 2015: “What can we learn from using action research to strengthen district health management in African countries? Experiences from PERFORM”.
• Joanna Raven presented at the 2015 International Health Conference, St Hugh's College, Oxford 25th -27th June: “Improving health workforce performance in Uganda: linking research and practice through action research”.
• Following the end of PERFORM, Tim Martineau also presented on the PERFORM project at the 9th European Congress of Tropical Medicine and International Health in Basel, September 2015: “Management strengthening using health workforce performance problems in decentralised contexts: lessons from Ghana, Tanzania and Uganda”.
2.6 Publications
2.6.1 Published articles:
• Mshelia C, Huss R, Mirzoev T, Elsey H, Baine SO, Aikins M, et al. Can action research strengthen district health management and improve health workforce performance? A research protocol. BMJ Open. 2013;3(8):e003625.
• Bonenberger M, Aikins M, Akweongo P, Wyss K. The effects of health worker motivation and job satisfaction on turnover intention in Ghana: a cross-sectional study. Hum Resour Health. 2014;12(1):43.
• Bonenberger M, Aikins M, Akweongo P, Bosch-Capblanch X, Wyss K. 2015. What do district health managers in Ghana use their working time for? A case study of three districts. Plos One. DOI: 10.1371/journal.pone.0130633. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130633
2.6.2 Articles currently under development include the following topics:
• Decision space in decentralized district health workforce management
• Using Diaries for Action Research
• Action research as a means of developing management competencies
• How to improve efficiency and performance of district health managers in Ghana
• The role of context in management strengthening
• Building capacity to facilitate action research in health systems.
List of Websites:
http://performconsortium.com/