Universal coverage in Tanzania and South Africa: Monitoring and evaluating progress

Final Report Summary - UNITAS (Universal coverage in Tanzania and South Africa: Monitoring and evaluating progress)

Executive Summary:
There has been growing attention in low- and middle-income countries to the goal of universal health coverage (UHC), further bolstered by the inclusion of UHC in the recently adopted Sustainable Development Goals. Tanzania and South Africa, a low-income country and middle-income country respectively, are reforming their health systems to promote universal coverage. This provided a valuable opportunity for the UNITAS project to track policy development and monitor from an early stage the implementation of UHC policies.

A range of qualitative and quantitative research was undertaken to meet the project objectives, some at the national level and some in selected districts, some focusing on overall efforts to move towards UHC and some assessing in detail specific reforms or innovations.

While the two countries are pursuing very different approaches to financing universal health coverage, they have adopted a similar initial reform emphasis, namely wide-ranging efforts to improve service delivery and management. Attention has been focused on improving the resourcing of public sector services, in terms of physical infrastructure, equipment, human resources, drugs and other medical supplies, and on improving the quality of health services. There has been a particular focus on the primary health care (PHC) level, including expanding community level services. Although both countries intend introducing fundamental health financing reforms, these have proved more difficult to progress, not least of all because they have clear redistributive implications.

UNITAS has been an unusual, cross-country and multi-level evaluation of UHC reform implementation, which has consistently fed back findings to policymakers and managers at national and district levels. We have found that, in both countries, there has been some progress towards UHC through the implementation of specific reforms intended to strengthen the public sector, as the first step on the path towards UHC. However, progress is uneven and has been challenging in both countries. Our findings demonstrate both the hard political work of UHC reform implementation - requiring not only continuous high level political leadership, including public presentations of ideas and justifications for reform, but also strong frontline management and action to bring these ideas alive in daily service improvements. The findings show that UHC reform requires attention not only to reform design but also to implementation processes, including reform sequencing, awareness of reform interactions, managerial engagement and strengthening, support for frontline manager flexibility and innovation in addressing context-specific challenges, and real time monitoring and evaluation processes that inform implementation. Further analysis of findings will draw out more specific lessons from comparisons across the two countries and identify their implications for other settings. Synthesis of findings is ongoing, with a number of papers in preparation.

While UNITAS contributes to international discussions about how best to promote movement to universal health coverage and to understanding universal coverage policy implementation bottlenecks, critically examining the implementation process of health system reforms was explicitly envisaged as supporting policy implementation at country level. Conveying to senior managers and policy makers the experiences, views and concerns of those working at the frontline in districts and health facilities is seen as a valuable contribution of the project. The project has also provided an opportunity to promote learning around implementation across districts, an important mechanism for promoting effective UHC policy implementation.

Project Context and Objectives:
In recent years, there has been growing attention in low- and middle-income countries to the goal of universal health coverage (UHC), defined as universal access to needed health services without the risk of financial catastrophe or impoverishment associated with obtaining care. The 2010 World Health Report highlights that universal coverage can only be achieved through pre-payment financing mechanisms, particularly tax funding and/or social health insurance. The combination of financing sources and provision arrangements within a universal health system, and the degree of equity sought and achieved, vary widely amongst countries that have achieved universal coverage. Given that universal coverage requires cross-subsidies of various types, especially from richer to poorer groups, political dynamics and broader social influences are very important in affecting both the choice of strategies and the speed of progress.

Very few African countries have achieved what can be termed universal health systems. Most African countries face challenges in raising sufficient domestic revenues, with large subsistence agriculture and informal sectors that constrain the tax base and make promoting universal coverage particularly difficult in these countries. There are particularly severe constraints in many African countries in terms of access to quality health services. So, at the time this research was initiated, there was very limited experience of how to move towards universal health systems within the African context, and inadequate documentation of the considerable challenges facing African countries trying to pursue this goal. Nevertheless, there has been greater interest and commitment since the 2010 World Health Report to pursuing universal systems in African countries, and UHC is now included in the newly agreed Sustainable Development Goals. As Tanzania and South Africa are both currently reforming their health systems in order to promote universal coverage, these country experiences provide an important opportunity for globally-relevant learning.

South Africa has a very inequitable and divided health system, with limited cross subsidies across population groups. Less than 16% of the population is covered by private health insurance schemes and yet about 44% of total health care expenditure is attributable to these schemes. The remainder of the population (84%) is heavily dependent on tax funded health services (which also account for about 44% of total expenditure). The other 12% of expenditure takes the form of out-of-pocket payments to public and private providers. Previous research found that there are inequities in both health care financing (particularly in terms of regressive out-of-pocket payments and lack of cross-subsidies from the rich to the poor in the overall health system) and the distribution of benefits from using health care (with both public and private sector services being strongly pro-rich).

The 2007 policy conference of the ruling African National Congress (ANC) committed the South African government to introducing what the ANC termed a national health insurance (NHI) system. The Green Paper (i.e. the preliminary policy proposal document) for the NHI was only published some four years later, in the second half of 2011. It clearly indicated that the goal of the NHI reforms was to move towards universal coverage. While many had anticipated that there would be substantive reforms in health care financing, i.e. revenue collection, pooling and purchasing, at an early stage the Green Paper indicated that this would not be the focus of NHI reforms for many years. Instead, the current priority is to improve management and the availability and quality of services within the public health sector, particularly at the PHC level. This emphasis recognises the serious challenges currently facing the public health system, with widespread concerns about quality of care; it seeks to create the conditions for the equitable and efficient provision of quality services within the public health system. At least one district in each of the nine provinces in South Africa was selected as “NHI pilot districts”, within which priority service delivery and management reforms linked to UHC have been introduced. Many of these reforms are being introduced in other districts as well.

In December 2015, the NHI White Paper (generally the final version of a policy) was released. This again highlights the importance of service delivery and management improvements, but provides clearer guidance on health care financing reforms. A series of work streams have been established to undertake detailed planning for the implementation of a NHI Fund (NHIF).

In Tanzania, inequities in health have been on the rise since the end of the 1990s, with one of the major causes being out-of-pocket payments for health care increasing from 26% of total health expenditure in 2005/06 to 32% in 2009/10. Tanzania also remains highly dependent on external funding from donors (40% of total health expenditure in 2009). The government is committed to universal health coverage and expanding financial protection through the deployment of several health insurance schemes targeting different population segments, and improving service coverage through expanding primary health care services.

From a financial protection perspective, the organisation of health insurance schemes remains fragmented, with a national health insurance fund (NHIF) covering civil servants, and community health fund (CHF) and TIKA covering those outside the formal sector in rural and urban areas respectively. Overall, insurance scheme coverage is low, reaching less than 10% of the population nationally. More effective risk management, better coordination and enhanced harmonisation between the different schemes have been sought over the past 5 years. As a step in that direction, from 2009 the monitoring and oversight of CHF activities that had been done by the Ministry of Health and Social Welfare (MOHSW) was taken over by the NHIF. The MOHSW aims to submit a proposal for a Single National Health Insurance Scheme to Cabinet by June 2016.

In terms of service coverage, the government is keen to extend the provision of primary healthcare facilities to every village in the country, and is in the process of constructing and rehabilitating facilities to meet this target, together with funding and training Community Health Workers (CHWs). The MOHSW is also attempting to expand service coverage by contracting out the delivery of priority interventions to faith-based providers where public facilities are lacking; and by contracting referral facilities to deliver services to CHF members. The government is also rolling out a results-based financing scheme in order to improve the quality and efficiency of healthcare delivery.

These two countries, one a middle-income country (South Africa) and the other a low-income country (Tanzania), are adopting very different approaches to financing for universal coverage. South Africa is planning to introduce a system of universal health care entitlements to be funded from general tax and additional dedicated tax revenue. Tanzania is integrating existing health insurance schemes for formal and informal sector workers under the management of a single insurer and is seeking to expand insurance coverage of informal sector workers. In both countries, these reforms are being phased in, with an initial emphasis on wide-ranging efforts to improve the delivery and management of health services rather than on substantive reforms in the health financing system. Combined, these reforms intend to reduce existing health system inequalities and improve population, health service and health care cost coverage, i.e. to make progress towards universal coverage.

When conceptualising this project, it was recognised that the policy formulation phase would be challenging in both countries. It is well established from previous research that the design of large-scale health care reforms is influenced by powerful actors. Whilst it may be necessary to make design compromises to ensure that the policy is acceptable to critical policy actors, care must be taken to avoid such compromises undermining the achievement of overall objectives. Tracking the policy formulation process from an early stage can support policy makers in managing these processes.

Policy implementation is even more challenging; translating health policies into the intended changes ‘on the ground’ can be extremely difficult. Factors that usually contribute to poor implementation include: changes in the policy design from what was originally agreed, as it is implemented; resistance to the policy changes, either from within or outside the health service; a lack of resources (financial, human, etc.) required to implement the policy; and a lack of appropriate organisational structure or management skills necessary for implementation. National systems for monitoring the implementation of health systems reform are often weak, due to poor data capture and/or management, limiting the capability of national governments to ascertain the impact of such reforms, or be alerted to implementation bottle-necks or constraints during the roll-out process. Effective policy intervention requires a well functioning monitoring and evaluation system which provides data that allow policies to be improved over time, and consequently strengthen their potential to achieve universal health coverage.

Given the importance placed internationally on promoting universal health systems, and the need to identify ways of successfully pursuing this goal in African countries, this project intended to support monitoring and evaluation of policy implementation, and identifying implementation difficulties at an early stage. The team recognised that there was an element of risk involved in initiating this project when policy formulation was still underway. However, by structuring the project in a flexible way to focus on any initiatives that will contribute to the ultimate goal of universal coverage, initiating research at an early stage has the benefit of capturing vital information on policy formulation and implementation issues that will contribute not only to strengthening the policy process in South Africa and Tanzania but also to supporting similar policies in other low- and middle-income countries.

The overall aim of the research was to support the formulation and implementation of policy for universal coverage in South Africa and Tanzania through monitoring and evaluating these policy processes and regularly feeding findings back to policy makers and implementers.

The specific objectives for the research were to:
1. track the formulation of policy for the proposed health care reforms, particularly considering the processes of policy development, design and planning for policy implementation and the role and influence of key actors, in order to support policy makers in managing these processes;
2. monitor the progress of policy implementation at both the national and district levels, with particular emphasis on identifying implementation problems and, hence, serving as an ‘early warning system’ for policy-makers and implementers;
3. evaluate interventions aimed at progressing towards the goal of universal coverage;
4. engage with policy makers and implementers at all levels about the research findings throughout the study period; and
5. synthesise the results from the studies in South Africa and Tanzania, in combination with information drawn from monitoring developments in other countries pursuing universal coverage, to draw out policy implications about health financing mechanisms and implementation strategies supporting the achievement of universal coverage and sustainable quality health care in low- and middle-income countries.

A range of qualitative and quantitative research was undertaken to meet these objectives, some at the national level and some in selected districts, some focusing on overall efforts to move towards UHC and some assessing in detail specific reforms or innovations. Brief descriptions of some of the research methods are provided where relevant in the next section outlining the main results. It is important to note that extensive ethics review processes were applied to this research, with ethical review of research protocols and methodological tools being undertaken within each of the partner institutions as well as approval obtained from the relevant national and provincial health department or ministry research review panels.

Project Results:
UNITAS has been an unusual, cross-country and multi-level evaluation of UHC reform implementation, which has consistently fed back findings to policymakers and managers at national and district levels. We have found that, in both countries, there has been some progress towards UHC through the implementation of specific reforms intended to strengthen the public sector, as the first step on the path towards UHC. However, progress is uneven and has been challenging in both countries. Our findings demonstrate both the hard political work of UHC reform implementation - requiring not only continuous high level political leadership, including public presentations of ideas and justifications for reform, but also strong frontline management and action to bring these ideas alive in daily service improvements. The findings show that UHC reform requires attention not only to reform design but also to implementation processes, including reform sequencing, awareness of reform interactions, managerial engagement and strengthening, support for frontline manager flexibility and innovation in addressing context-specific challenges, and real time monitoring and evaluation processes that inform implementation. Synthesis of findings is ongoing, with a number of papers in preparation.

Key findings in relation to policy formulation (objective 1)
External influences on the universal coverage reforms being taken forward in South Africa and Tanzania
One of the UNITAS work packages focused on tracking the evolution of policy design and implementation preparation through a review of documentary sources and key-informant interviews. The range and nature of the reforms being implemented in South Africa and Tanzania have been influenced by various factors, not least the global health policy context and international actors. For this reason, a key focus of our analysis was on:
• The role of external influences in national policy formulation and implementation preparation processes; and
• The process of policy transfer (i.e. drawing ideas and experiences from other countries and the international context) in universal coverage ideas between international and national jurisdictions.

There has been active participation by a variety of international actors in processes of policy development and implementation preparation in both countries, albeit with greater involvement of international actors in Tanzania. International actors have provided funding and given operational support and technical support, particularly in Tanzania. However, in addition, there have been important elements of policy transfer in both countries. International ideas and experiences have been drawn on by domestic actors when framing the policy choices, justifying specific reforms, and highlighting both good practice and what not to do. In particular, international debates have been used to bolster justification for introducing UHC reforms and attempting to leverage health system change locally. For example, extensive reference has been made to the prioritisation of UHC reforms in global health policy discussions, such as through World Health Assembly and UN General Assembly resolutions, as a means of arguing that the reforms being introduced in these countries are in line with global trends. Specific elements of these global discussions, such as encouraging movement away from out-of-pocket payments to mandatory pre-payment funding mechanisms, are also frequently referred to in order to justify country-specific reforms.

A key finding is, thus, that such international ideas and experiences have been useful to national-level policy makers within often contested and conflictual health reform processes linked to the goal of UHC. Not only has there been an openness among national policy makers to international information, but they have also actively sought out such information.

Overview of universal coverage reforms being implemented
Countries wishing to make progress towards universal health coverage usually need to implement a wide array of health system reforms. South Africa and Tanzania are no exception, with a large number of reform elements across the WHO health system building block spectrum being introduced (see Table 1).

Interestingly, the initial emphasis in the path towards UHC in both countries has been on improving the resourcing of public sector services, in terms of physical infrastructure, equipment, human resources, drugs and other medical supplies, and on improving the quality of health services. There has been a particular focus on the PHC level, including expanding community level services. There have also been efforts in both countries to draw on private health care providers to improve access to health services. While both countries intend introducing fundamental health financing reforms, these have proved more difficult to progress, not least of all because they have clear redistributive implications.

Table 1: Overview of the key universal coverage reforms in South Africa and Tanzania, by WHO health system building blocks
South Africa Tanzania
Service delivery • Audit of public health facilities (assessment of physical infrastructure, equipment and staffing)
• Health facility improvement (infrastructure and service quality)
• Ward-based PHC outreach teams (community health workers)
• School-based PHC services
• Drawing doctors into public facilities (initially involving private GPs in HIV & TB services; then contracting with GPs to provide range of PHC services in clinics; then private company recruiting full-time doctors to work in public clinics)
• Various initiatives to improve maternal and child health
• Various initiatives to improve chronic disease management, including antiretroviral treatment (e.g. ICDM / Integrated Chronic Disease Management; and alternative access to chronic medication) • Improve primary health care infrastructure through construction of new facilities & upgrading existing facilities (MMAM)
• Rolling out CHWs and linking formally to the health system
• Public-private partnerships in the health sector, through service level agreements; also used to extend the benefit package available to CHF members
• Hospital reform programme
• Quality Improvement framework implemented for facility accreditation
Health workforce • Increasing full-time public sector staff (employment)
• Expanding training capacity and increasing training outputs
• Strengthening capacity of DHMT and hospital managers • Initiatives to increase availability of human resources (MMAM)
• Improved training capacity for training institutions (MMAM)
• Payment for Performance / Results Based Financing - individual rewards to providers based on achievement of performance targets
• Capacity building activities to support local government authorities in relation to health
• Shorten training period for nurses
Information • Establishment of the National Health Information Repository and Data Warehouse (NHIRD) and roll-out in provinces & districts • Computerisation of health information systems (introduction and roll out of the DHIS)
Medical products, vaccines, technologies • Various initiatives to reduce stock-outs in facilities (ordering & distribution systems)
• Establishment of central drug procurement system • Initiatives to improve drug supply in facilities, including improved flow of funds for drugs and other supplies
• Additional funds from WB for improved drug supply (MMAM)
Financing • NHI pilot district conditional grant
• NHI pilot hospital conditional grant
• Design of NHIF (including benefit entitlement specification, accreditation of providers, purchasing from private providers, provider payment mechanisms, aligning with RAF, COIDA) (still to occur)
• Funding sources for NHI (still to occur)
• Change in tax subsidy for medical scheme contributions (tax credits initially, removal later) (still to occur) • Increased funding of LGA ($0.30/person/year) to support service delivery
• P4P - Health facility bonus
• Introduce performance and equity indicators into the allocation formula for the basket fund
• Revision of cost sharing guidelines, based on new national costing study: user fees may increase further, MOHSW/NHIF reimbursements to facilities are likely to increase
Leadership and governance • District Clinical Specialist Teams (DCSTs)
• Office of Health Standards Compliance (OHSC)
• Hospital management
• Ideal clinic initiative
• District management (Establishment of District Health Authorities)
• Defining role of medical schemes under NHI and during transition
• Creation of the South African Health Products Regulatory Authority
• Establishment of NHIF (still to occur) • Introduction of lower level facility accounts for more efficient use of complementary financing
• NHIF management of CHF - pro-poor /vulnerable strategies

Selection of districts and innovations of focus for detailed monitoring and evaluation and brief overview of methods (objectives 2 and 3)
The above findings derive from research undertaken at a national level and considering the overall range of UHC reforms. The majority of UNITAS research was undertaken within selected districts, three in each country, and on a limited number of reforms or innovations, to allow detailed monitoring and evaluation of the implementation of these innovations. The innovations selected for detailed research are summarised below.

Innovations of focus: South Africa
Within South Africa, the reforms (innovations) researched are:
• The introduction of District Clinical Specialist Teams (DCSTs), which aim to reduce child and maternal mortality and (ideally) comprise 3 sets of doctor-nurse specialist dyads in obstetrics and gynaecology, paediatrics, and family medicine, plus an anaesthetist located at the district level;
• Drawing doctors into public sector primary care facilities, initially through the contracting in of private general practitioners (GPs); and
• Strengthening district management.

In addition to these specific reforms, we also looked at the strengthening of referral pathways within health districts and sought to understand if and how the various reforms work together to strengthen primary health care as a fundamental strategy for moving towards achieving universal coverage.

Innovations of focus: Tanzania
Within Tanzania, the reforms researched are:
• Extending insurance coverage through the Community Health Fund (CHF) and TIKA (voluntary insurance for the rural and urban informal sectors respectively). In addition to direct efforts to increase membership, other interventions to make membership more attractive and improve member benefits include:
o The Service Agreements, a mechanism through which district councils enter into a contract with providers to extend the benefits available to CHF/TIKA members and expand enrolment, for example referral services, including specialised outpatient care and inpatient care at hospital level (faith-based or regional government hospitals), or extending the range of registration points by contracting with faith-based primary care providers where government providers are not available.
o Facility bank accounts which are being encouraged to support the use of cost sharing funds for service delivery improvements at the facility as well as to supplement staff income and pay staff allowances, thus granting additional autonomy to primary level facilities in the management of CHF resources.
• The Primary Health Services Development Programme, locally known as the MMAM, which aims to improve the availability and quality of public primary health care through a range of initiatives including facility construction and renovation.

Brief overview of qualitative research methods
The broad methodological approach that we adopted for our qualitative research is that of a ‘theory of change’ (TOC) approach, which allows us to track the unfolding experience of policy reform through engaging in collaborative and reflective inquiry with those who are managing and implementing these reforms in each selected district. This approach enables us to tell the story of a reform as it is implemented, reflect on what it achieves and how that relates to the assumptions and rationale guiding its design. It also allows us to identify challenges arising during implementation, and through feeding this information back to managers and implementers, enables strategic thinking about how to address these challenges and promote more effective implementation. Finally, a TOC approach provides a methodological framework to document the thinking and experience of implementation, and to feed into comparisons across programmes, sites and countries, which will generate a wider understanding of policy implementation and its management.

A first step was to undertake a context mapping in each of the selected districts, which provided a detailed understanding of the district setting, and facilitated the purposive selection of health facilities, in which all or some of the reforms of focus were actively being implemented, to include in the detailed research. In South Africa, we carried out detailed interviews with those ‘doing’ the implementation of reforms: district and sub-district managers, local area and clinical managers, operational/facility managers, as well as frontline staff delivering services such as nurses, doctors and data capturers. In Tanzania, a combination of focus group discussions and in-depth interviews were undertaken with a similar range of implementers. These interviews and discussions have allowed us to document and analyse the practice of managing and implementing reforms at sub-district/district- and facility-levels. In addition, in South Africa, we undertook detailed qualitative interviews with about 15 women per sub-district who had experienced the maternal referral system, to supplement our analysis of the innovations to improve referral systems.

Brief overview of quantitative research methods
Data were collected at all of the primary health care (PHC) facilities in which in-depth qualitative work was undertaken as well as at other PHC facilities in the sub-district/district. As the primary goal of many of the district level foundational reforms for universal coverage in both countries relate to improving the quality of care in public sector facilities and access to care (see Table 1), the quantitative evaluation research focused on:
• Health worker motivation (as health workers are the front-line implementers of many of the reforms and hence are affected by and influence implementation processes, and quality of care is strongly affected by health worker motivation);
• Quality of care provided in facilities (assessed using a range of tools such as facility checklists around availability of equipment and drugs, gathering information on staffing and staff skills, interviews with staff around use of standard treatment guidelines, and review of records on a set of tracer services); and
• Patient experience of care (through exit interviews, again focusing on patients using specific tracer services).

Two rounds of qualitative and quantitative data collection were undertaken at least a year apart.

Key findings in relation to monitoring policy implementation: South Africa (Objective 2)
The key policy implementation findings are presented for each of the innovations of focus.

District-based Clinical Specialist Teams (DCSTs)
In 2011, District-based Clinical Specialist Teams (DCSTs) were introduced into the South African health system to strengthen clinical governance at district level. Teams are ideally comprised of seven members: an anaesthetist, plus doctor-nurse dyads in three disciplines (Obstetrics and Gynaecology, Family Medicine and Paediatrics). However, over the course of the UNITAS project, none of the study site DCSTs was fully constituted, ranging instead from 3-6 members. As a new reform, the role of the DCST is yet to be fully understood and defined in practice. We monitored how this role has been defined, communicated and integrated at different levels. We also identified some of the barriers and enablers to the successful implementation of the reform, drawing on perceptions and experiences of a range of actors, including district managers, DCST team members and intended beneficiaries (facility managers and staff).

Activities and scope of work
Despite differences in (i) district contexts (demographic, geographic and institutional) and (ii) team composition (affecting their reach and scope), DCSTs across the three study sites generally engaged in similar activities, including annual situational analyses, continuous quality improvement programmes, education and training of health workers, engagement with the district management team, clinical outreach visits, and support for other streams of primary health care reengineering. As anticipated by policy, DCSTs in all study sites have predominantly focused on strengthening maternal and child health services, although some stakeholders would like them to adopt a wider ‘primary care’ focus. Many would also like DCSTs to devote more time to clinical practice, alongside clinical governance and supervision. However, DCS team members have noted multiple demands on them, coupled with limited capacity to provide direct support to all facilities within their districts. As a result, they have tried to prioritise and respond to identified district-specific needs.

Role clarification, conflict and boundary disputes
Although tasked with ‘clinical governance’, there is no clarity on the boundaries and relationships between DCSTs and other clinical governance actors, including existing hospital specialists, local area managers, clinic supervisors, general practitioners and doctors in facilities, programme managers (especially maternal child and women health coordinators) and facility managers. There is consequently high potential for duplication of time and resources by different programme coordinators and managers visiting facilities to provide supportive supervision roles. Furthermore, there are no formalised means for DCSTs to engage with existing clinical supervisors, cadres or teams except, less directly, via district managers. In some cases, this had resulted in rivalry and duplication of roles, to the frustration of all involved.

Teamwork and team dynamics
In at least two of the three sites, there are indications of disciplinary fragmentation in the functioning of the DCSTs. Role conflict within and across the dyads has been identified, related to doctor versus nurse complexities, as well as relationships of ‘expertise’ between specialities in the team and at a broader district level, influencing levels of trust and support amongst DCST members, and with other specialists and teams. Conversely, there are also accounts of positive teamwork and relationship-building, including with ward-based outreach teams and data management teams in all of the study districts.

Paradoxical effects: limited improvement in maternal and child health (MCH) indicators
It is difficult to ascribe changes in MCH indicators to the DCSTs, given the multitude of MCH-focused programmes and processes underway in each study site. In one district (and sub-district in particular), despite concerted DCST attention, MCH outcomes have deteriorated over the course of our monitoring, largely due to high volumes of cross-(district)border patients – a challenge for the referral system. The DCST has also raised concerns that the impact of their training and mentorship is not clear and does not always seem to translate into improved clinical practice.

Adding value, authority and ‘getting things done’
Despite expressed challenges in all of the study sites, most managers, implementers and intended beneficiaries have identified value in the DCSTs. Over time, the teams have been building trust and developing relationships through their contribution to mentorship and problem solving. Positioned at the interface of different layers of the district system, they are able to use their authority, as well as clinical expertise, to strengthen collaboration, build teamwork and ‘get things done’.

Increasing staffing by doctors in clinics
This innovation initially centred on the contracting in of private general practitioners (GPs) to support clinic-based public services but has since evolved to include different models of contracting and managing doctors in clinics, particularly the full-time employment of doctors by a private company who then assigns them to work in designated public PHC facilities.

Early implementation lessons
A key challenge to the implementation of contracting with private GPs related to inadequate communication and co-ordination of efforts across levels of government. Advertising for and contracting with GPs was undertaken at the national level, while district managers were responsible for ensuring that public facilities were made ‘GP ready’ by ensuring that clinics had the full range of essential equipment that doctors would require. More importantly, the GPs’ contract is with the national Department of Health posing challenges for monitoring and accountability at facility and district level. Another challenge was the geographic distribution of private GPs; the greatest uptake by GPs was in districts that are largely urban. Low levels of remuneration were reported to be a key factor in influencing uptake rates.

Although there were difficulties in attracting GPs to work in public clinics, where this has been possible, nurses and patients have responded very positively. Nurses value the additional expertise and clinical capacity and the doctors appear to welcome opportunities to share their knowledge and contribute to skills development. Nurses reflected that having doctors in clinics will reduce unnecessary hospital referrals and will reduce the need for chronic patients to go the hospital for prescriptions. This could in turn reduce the costs of transport for patients who need to see a doctor. Although the availability of equipment in public clinics has improved, and in some cases additional consulting rooms have been built as part of the process of getting clinics ‘GP ready’, some contracted GPs have expressed a need for a wider range of drugs and additional equipment.

The evolution of change
The main reason for moving away from GP contracting was the inability to recruit the required number of GPs to meet national targets. Subsequently the innovation changed to a model using a third party private company to recruit and employ doctors on a full-time basis who are then placed in public sector clinics. This model has proved more successful in recruiting doctors nationally, likely to be at least partly due to the relatively high salaries paid to these doctors.

While adapting the innovation to address implementation challenges has improved the ability to achieve the goal of this innovation, namely to increase the number of doctors working in public clinics, there are several concerns around the model that has been adopted. Given the relatively high salary levels and the cost of a third party intermediary, the financial sustainability of the model is one concern. It is also not clear where the budget for salaries will come from or whether doctors will be motivated to stay in the public service after the two-year contract, considering that they may not have the opportunity to utilise their skills fully at the clinic level. Another concern is that doctors being recruited under the new arrangement may in fact be from public sector hospitals rather than from the private sector, which undermines the public-private sector distributional effect hoped for.

Management strengthening
The National Health Insurance White Paper released in December 2015 again highlights the need for strengthening management and governance at the district and primary healthcare level in South Africa. In the medium- to long-term, it is planned that District Health Management Offices (DHMOs) will be developed for the function of providing personal and non-personal health services, with some purchasing functions potentially also being devolved to this level over time. This will require new management competencies as well as skills to implement and lead new governance arrangements.

In our research, we found highly experienced managers who fully understand the problems and opportunities that are present in their districts. Many senior managers have attended or are attending different national and international university accredited training programmes. However, because of the vast array of training opportunities, there seems to be inadequate coordination of training for senior managers to ensure that the needed competencies that should be represented in an effective district management team are present. A challenge consistently identified across all districts was the need for more capacity in financial services and supply chain management.

Accountability and Motivation
Within the context of inadequate management capacity and the complexity of district health systems, it is hard to hold people accountable. Our research identified accountability and staff motivation as key challenges within districts. There appears to be an imbalance in focus, with a strong emphasis on clinical performance and accountability and little attention paid to support broader performance and accountability. Generally in public services, the process of holding serial underperformers to account is experienced as an administratively heavy task, and facility managers indicated that they do not have the skills base to “have difficult conversations with staff”. Staff members are also sceptical of the current general staff performance management system which requires considerable paperwork to justify good performance ratings. In terms of motivation, many social activities are arranged within the district and at facilities; these are welcomed by staff and are seen as contributing to keeping staff motivated. However, these activities are self-funded, either by staff in the facilities or district managers, and are not necessarily adequate to motivate staff to the point of significantly improved performance.

Implementing positive change
Within this context, managers have nevertheless been able to institutionalise some changes to promote stronger management and leadership in their districts. For example, some critical senior management posts have been filled, there are improved working relationships with non- state partners, information is being used more effectively in district management team meetings and support services are being asked to communicate their performance (or under performance) more effectively. In one district, areas of weakness are being identified and sub-district and facility level managers are being targeted for training in these areas by an existing regional training centre. In some cases, changes appear to be small, but they are incremental and critical for achieving the necessary management capacity in districts.

A key implementation success story for the Ministry of Health was requiring all hospital Chief Executive Officers (CEOs) to reapply for their posts in competition with others, leading to the replacement of those without the requisite qualifications and experience. All hospital CEOs in the country then participated in a short and once-off induction training program, which was highly valued by hospital CEOs interviewed in our districts. However, the longer term impact of the hospital CEO programme still needs to be assessed fully. Linking and building strong relationships between the hospital and the PHC service delivery platforms is critical. In this regard, the DCSTs have been acting as useful conduits in bringing the hospital and the PHC platform together through information sharing and capacity building.

Further work is needed
Based on our research, which was undertaken in three relatively poorly performing districts, it is clear that there is a lot of work that still needs to be done to prepare the health system for future changes, particularly in the areas of finance and supply chain management, both in terms of skills of all level of staff and the number of staff available to perform these critical tasks. A continuing focus on strengthening management at the facility level is also needed. It is also clear that districts have quite different histories and challenges and so a flexible approach to implementing change is needed in which district managers can apply their tacit knowledge to resolve challenges and take up opportunities.

District-level referral functioning
A key goal of current South African reforms is to improve ‘rational’ referral systems and their functioning. We have tracked progress in this area by looking at experiences of, and changes in, district-level referral systems from a range of key actors’ perspectives (including patients, frontline staff at referring/receiving facilities and district health managers). Our monitoring activities have focused on referrals between primary health care facilities (e.g. PHC clinics) and those providing the next level of health care (e.g. district hospitals) with a particular emphasis on maternal health.

Patient experience
Across all three sites, maternal patients reported positively on the clinical care that they had received at both referring and referral facilities. However, transport difficulties and a general lack of communication about their health condition/reasons for their referral were identified as barriers to overall patient satisfaction with district-level maternal referral systems.

Geographical access (transportation) to referral facilities
In all three study districts, lack of, or delayed, emergency transportation emerged as a key challenge for referral functionality. Emergency response times are reportedly slow due to vehicle shortages, budget cuts and deficiencies in the management of human resources. One district is ‘coping’ by extending the lifespan of vehicles (which increases maintenance costs and the risk of vehicles breaking down) and by utilising private emergency transport services, often at great cost. However, there are efforts to improve the availability and functionality of district-level emergency transport systems. For example, in another district, the emergency services department recently received resources to improve delivery of care – 95 new vehicles were procured for the province, with 17 allocated to the study site. In another site, the inter-facility transfer process has been revised and streamlined with support from the DCST, in order to improve the efficiency of emergency personnel. Here too, planned patient transport services for non-emergency cases are being piloted in the most rural sub-district and two obstetric ambulances have been introduced as part of the Campaign on Accelerated Maternal, Newborn and Child Mortality in Africa (CARMMA).

Guidance, supervision and monitoring by district health administrators
In the three study sites, district-level referral protocols have been revised and updated, with assistance from the DCSTs. Referral protocols now include the role of community health workers in supporting appropriate and timeous referrals. However, in one site, inadequate communication by district administrators has meant that people at sub-district and facility levels are unaware of the referral policy or have not noticed any differences as a result of the protocol. In another district, district hospitals continue to receive a large number of maternity patients from beyond the district borders and no coordination or communication mechanism exists for referrals of this type. Clear guidance from the district and/or provincial administration is required to address the problem.

Capacity (technical, human resources, infrastructure) in referring and referral facilities
The increase in the number and availability of medical doctors at clinics (referring facilities) has had a positive effect on the district referral systems in all study sites. Previously, patients requiring examination by a doctor were referred by nurses to the nearest referral hospitals or community health centres. Now, given that medical doctors at clinics can treat less severe cases, the number of ‘unnecessary’ referral cases has been reduced, alleviating some pressure on hospitals (referral facilities). This improved functioning of district-level referral systems is closely linked to the concurrent implementation of various other reforms, particularly the placement of doctors in clinics and the community health worker/ward-based outreach team programme, both of which contribute to identifying appropriate cases for referral.

Communication and feedback systems between referring (transferring) and referral (receiving) facilities
In all three sites, reforms to improve inter-facility communication have been developed, including patient booking systems and ‘referral appropriateness checks’ from doctors based in clinics. However, inter-facility communication still remains a challenge, particularly for back referrals: despite revised protocols, staff members at referral facilities (district hospitals) often do not complete back-referral forms and fail to provide feedback to referring facilities. This has consequences for staff at clinics, as well as patients who are sometimes made to travel back and forth to relay information between referring and referral facilities, resulting in additional costs and time wasted.

Key findings in relation to monitoring policy implementation: Tanzania (Objective 2)
As indicated earlier, implementation monitoring at district level focused on two main reforms, the main objectives of which were to promote progress towards UHC by expanding both service coverage through the MMAM and enhancing financial protection through the CHF and TIKA pre-payment schemes, and two reforms supporting the expansion of CHF, service agreements and facility bank accounts.

Community-based health insurance: the Community Health Fund (CHF) and Tiba Kwa Kadi (TIKA)
The CHF and TIKA are voluntary health insurance schemes for rural and urban dwellers respectively. While the CHF has been in existence in rural areas since 2001, a more recent decision was taken to introduce voluntary insurance in urban councils. Since 2009, the CHF and TIKA have been under the management of the National Health Insurance Fund (NHIF). The NHIF mandate was to support districts to increase CHF enrolment in rural areas, and to introduce TIKA in urban councils. As part of this research study, we tracked the implementation of strategies to increase CHF enrolment in two rural districts, and the process of introducing TIKA in one urban council.

The Community Health Fund
The two rural districts reported using a series of strategies to increase CHF enrolment and associated premium revenue at the district level. Strategies to stimulate demand included: increasing CHF awareness through mass sensitisation; increasing user fees; changing the CHF premium; allowing premium payment in instalments, allowing group enrolment at schools; and identifying mechanisms to finance the premiums of the poor. Districts also attempted to introduce strategies to improve the availability of health services covered by CHF. Supply side strategies included: working together with the programme on construction and rehabilitation of primary care facilities (the MMAM); allowing members to obtain care at any public facility within the Council and implementing service agreements between Local Government Authorities (LGAs) and facilities offering referral care to expand the CHF benefit package in terms of range of services; using premium revenue to procure drugs to sustain the district buffer stock; paying provider incentives from CHF premia; requesting CHF matching grants (a subsidy provided by government) in a timely manner; and increasing provider autonomy by opening facility bank accounts at primary care facilities, which can retain 33% of CHF revenue.

Districts had mixed experience in implementing these strategies. The pressure to perform relative to other districts was an important enabler for implementation of the strategies, and made districts more willing to learn from each other. The NHIF acted as an intermediary between districts, facilitating the sharing of information on best practice, and generating ideas on potential strategies to enhance enrolment. The NHIF also provided financial support for sensitisation campaigns and conducted regular facility visits to monitor enrolment levels. Support from top-level district officials also created momentum, together with favourable structures that existed in villages that enabled regular communication about the CHF with communities. Non-governmental organisations, like World Vision, also lent support to strategies by, for example, helping to finance the CHF premiums of the poor.

Facilities also had a degree of autonomy in how they promoted the CHF within their communities, and how they managed the funds collected. This allowed facilities to develop their own initiatives or approaches that were best suited to their local context.

A further factor facilitating the implementation of CHF enrolment strategies was exploiting the potential for synergy with other programmes operating in the district. For example, the programme to construct primary care facilities (the MMAM) was seen as an opportunity to leverage CHF enrolment, by making registration points (facilities) more accessible. The funds provided by MMAM for facility rehabilitation were also seen as an opportunity to improve service quality and again encourage enrolment. The service agreement mechanism, which had been available to districts for some time, was deliberately used to extend the benefit package available to CHF members, in terms of referral care and, in some cases, providing greater choice of primary care providers. However, out of four contracts that were signed only two of the contracted facilities were reported to have treated CHF members, due to the geographical location of the contracted providers and limited understanding of benefits among members and providers. Where CHF members were seeking care at contracted providers, there were concerns from providers about the level of reimbursement of services, and high levels of utilisation in one of the contracted facilities led the LGA to suspend the contract, as the LGA was not able to provide sufficient reimbursement.

Despite these successes, insufficient funding has been a major hindrance in the implementation of a variety of CHF enrolment strategies. For example, in both rural districts, there were plans to conduct training for facility managers on the management of facility bank accounts; however, this did not happen due to a lack of funds to pay for the training. Other challenges affecting the bank account policy included variation in terms of the share of funds that should be deposited into the facility account compared to the district account, and variation in practice in terms of district oversight over facility accounts.

Recent changes in the CHF matching grant procedures was highly time and resource intensive which delayed the districts’ requests and the subsequent receipt of matching funds. Other limitations include the potential burden of user fees on the population, and the sustainability of community based health insurance contributions in a subsistence economy.

Tiba Kwa Kadi (TIKA)
We also monitored the introduction of TIKA in one urban district, specifically the development of a bylaw which defines the premium and benefit package offered, and is the first step required prior to introduction of the scheme.

The NHIF supported the process of bylaw development by providing funds for sensitisation meetings in the Council and undertaking a survey to collect community views. The NHIF also printed leaflets which aimed to educate the community about TIKA. The process of bylaw development for the introduction of TIKA in the Municipal Council has been very slow. The reasons for this include: a lack of clarity on the role of different actors and specifically who was to lead the process (the NHIF or the Municipal Council); the bylaw development process was very time consuming and resource intensive involving consultation with a wide range of stakeholders; once drafted, the bylaw had to go through a process of approval at the President’s Office and this seems not to have been prioritised by central government ahead of the November 2015 general election. There were mixed views about TIKA within the urban council and the scheme was considered to be a threat by the growing number of private health insurance companies.

Mpango wa Maendeleo ya Afya ya Msingi (MMAM)
In 2007, the newly elected president of Tanzania, J.K. Kikwete, requested the Ministry of Health and Social Welfare (MOHSW) to prepare a Primary Health Care (PHC) development plan in order to keep his electoral promise of “one village, one health dispensary; one hamlet, one health centre”, a slogan inspired by the primary education development plan. The MOHSW developed a comprehensive plan including 18 broad objectives. Most of these objectives perfectly matched the strategic lines of the Health Sector Strategic Plan 2, reframed from a PHC perspective. One – the facility construction component of the PHC development plan – was, however, new and more contentious.

In the early 2000s, the donor community established a common funding mechanism – the Joint Rehabilitation Fund (JRF) – in order to support the Government of Tanzania in its endeavour to rehabilitate and extend the network of primary health care facilities. However, concerns around the administration and capacity of the JRF eventually led many donors to rather invest in complementary health inputs – e.g. development and deployment of human resources for health, supply of drugs and other equipment – rather than health infrastructure. Two donors – the Danish Cooperation Agency (DANIDA) and the German Development Bank (KfW) –continued to support facility construction. In 2011, they channelled resources directly to districts via basket funding initially as an earmarked lump sum, and subsequently as un-earmarked capital funds. In 2013, DANIDA and KfW stopped this financial support, following the decision of KfW to pull out of the health sector.

The lack of a consistent flow of funds to support the MMAM is one of the main barriers to effective implementation. In 2015, at the time of the second round of data collection, only one of the 7 facilities that were under construction in 2013 was operational.

There were a variety of other factors affecting implementation. First, the implementation process was unclear and varied from one location to another. For example, in some cases, the district was in charge of identifying the contractors to undertake the construction work. In other places, the village authorities decided who would be in charge of the construction work. There was also a lack of oversight and poor compliance enforcement. Communities are expected to follow-up the construction work under the supervision of the district managers, despite limited enforcement capacity at village level. In many cases, contractors left the construction site without completing the work they had been contracted for.

Issues were also noted in relation to communication and coordination of funding. For example, most of the district officials were unable to indicate whether there had been any allocation for MMAM.

Another issue relates to the community contribution. Each community was requested to start the construction work and to bring the building up to lintel level. Local leaders would sometimes impose specific contributions of time and resources to households within their community. Once this level was reached, the government committed to completing the construction work, and contributing the necessary equipment, supplies and staffing to deliver care. However, in many places communities struggled to complete their contributions, for various reasons:
• The contribution was deemed too high, and many villagers were unable to provide the expected contribution. This was a function of resource capacity as well as the degree of community cohesiveness.
• Communities were subject to numerous calls for contributions to public works and were unable to support all of them. During the course of the research, communities were also asked to contribute towards building school laboratories, which was seen as a greater government priority, and as a result many communities decided to put the MMAM contribution on hold.
• Community distrust and fatigue vis-à-vis initiatives requiring contributions from the community. In many places, people were sceptical about the government’s promise to match their contribution. Many community leaders therefore preferred to adopt a ‘wait-and-see’ stance, and to observe what was happening in the neighbouring communities. In the case of MMAM, since the government was not fulfilling its promises in terms of support to those who completed their contribution (see below), many got discouraged and stopped contributing.

Some communities, however, managed to elicit contributions and meet the required level for government support. Some did so by using innovative strategies to raise contributions. For example, in one case, the community members had agreed to contribute in cash (rather than in kind) in order to get things done more rapidly.

A further constraint was the frequent turnover of district officials. This was identified as one of the main difficulties in building a coherent, long-term infrastructure development strategy. There were also some concerns regarding the fairness in the selection of sites for facility construction/rehabilitation. The selection is left to the discretion of district officials. While all communities could take the initiative to begin the construction process, district officials had oversight of MMAM funds, and could decide where to prioritise the investment, as received funds were insufficient to support all facilities. In one district, most of the resources were allocated to the villages where the council leaders – chairperson, vice-chairperson, head of the finance committee – were from. The research team also noted that in some cases, villages receiving priority in resource allocation were already less than 5 km away from a health facility.

Where facilities were constructed there were delays in equipping facilities to deliver services, with some communities waiting for more than 2 years to be assigned adequate staffing to run their facilities.

Village implementers tried to cope with these challenges. Some expressed resentment towards this policy since they had to bear the consequences of and be held responsible for implementation failures.

Key findings of policy intervention evaluation: health worker motivation and quality of care in South Africa (Objective 3)

Overall, quality of care as judged by a range of process and structural indicators was found to be generally good, and did not change significantly over time. Staff motivation levels were generally high too, but staff absenteeism dropped between survey rounds, albeit with variation between sub-districts, as a possible indicator of increased commitment to work. Staff satisfaction with management and supervision also increased over time. Although there were concerns about financial incentives and issues of fairness in relation to workload, the most important motivation factor was community relationships. These findings highlight staff motivational issues as being particularly important to continue to track over time, as reform implementation proceeds.

Sample sizes and key characteristics
The different data collection tools were administered in 40 health facilities (6 health centres and 34 clinics) across 3 sub-districts. Table 2 shows the sample sizes for each of the rounds and surveys.

Table 2: Sample size for each of the surveys
Type of survey Round 1 Round 2
Health facility survey 40 40
Exit interviews 2,599 3,028
Health worker survey 173 203

Patients
Three types of patients were interviewed: patients receiving antenatal care (ANC), postnatal care (PNC) and those with a non-communicable disease (NCD). ANC and PNC patients were included because of the focus of the PHC reforms on maternal and child services. Non-communicable diseases (diabetes and/or hypertension care) were included in the study as additional tracer conditions because they affect a large proportion of the population (high incidence and prevalence) and contribute significantly to morbidity and mortality. There was also a concern that the heavy focus on maternal and child health services may result in NCD services being neglected. Table 3 shows the sample sizes of patients interviewed for each round and tracer condition. For each tracer condition in each health facility, we interviewed on average 22 patients in round 1 and 25 patients in round 2.

Table 3: Types of patients / services interviewed for each survey round
Characteristics Round 1 (n=2,599) Round 2 (n=3,028)
n % n %
Antenatal care 782 30.1 917 30.3
Postnatal care 827 31.8 990 32.6
Non-communicable diseases care 990 38.1 1,121 37.1
Diabetes care1 103 10.4 123 11.0
Hypertension care1 696 70.3 776 69.2
Diabetes and hypertension care1 191 19.3 222 19.8
Note: 1 As a % of NCD

Health workers
The characteristics of health workers that were interviewed are shown in Table 4. We interviewed nurses only at the health facilities since these are the largest group of health service providers in South Africa. There are 3 categories of nurses in South Africa (from highest to lowest level of qualification): professional (registered) nurses, enrolled nurses and assistant enrolled nurses (or nurse auxiliaries). More than half of the interviewed nurses were professional nurses. The average age was 40 years, 87% were female and had been working at the health facility for on average 4 years.

Table 4: Health worker characteristics of each round
Characteristics Round 1 (n=173) Round 2 (n=203)
n n
Age (mean, std) 173 41.2 [15.3] 203 39.5 [10.3]
Months worked at facility (mean, std) 156 49.6 [47.5] 203 47.2 [51.0]
Professional title
Assistant enrolled nurse (%) 30 17.3 38 18.7
Enrolled nurse (%) 33 19.1 47 23.1
Professional nurse (%) 95 54.9 101 49.8
PHC or other specialized nurse (%) 15 8.7 17 8.4

Access to services and quality of care
Overall, patients reported relatively positive experience with care in both rounds, with some indicators improving over time (e.g. cleanliness of health facilities, staff attitudes towards patients).

Time spent traveling to the health facilities, and the cost associated with transportation, remained the same between the 2 rounds (Table 5). This is not surprising since no new health facilities were built in the study districts. Travel time was greatest in the most rural of the three sub-districts, at around 45 minutes; transportation costs were also higher by 25% in this sub-district compared to the other sites (data not shown). In all sites, the consultation time remained the same between the two rounds. The waiting time however showed a significant increase between round 1 and round 2, from 82 to 140 minutes (i.e. a 1 hour difference). The increase was observed for all tracer conditions, except postnatal care in one sub-district where the average waiting time remained the same at just over 40 minutes. As expected, the patients' attitude towards waiting time was consistently more negative when waiting times were higher.

Table 5: Patient experience with care
Characteristics Round 1 (n=2,599) Round 2 (n=3,028)
Transport cost in Rand (mean, std)* 12.1 [7.7] 11.6 [8.0]
Time to reach health facility (mean, std) 33.3 [24.7] 30.4 [24]
Wait time in minutes (mean, std) 81.8 [77.8] 139.9 [103.1]
Consultation time in minutes (mean, std) 15.8 [11.9] 15.3 [10.7]
Considered waiting time to be acceptable (%) 69.7% 72.0%
Considered health facility to be clean (%) 87.0% 91.3%
Were treated with respect and dignity (%) 96.7% 98.4%
Would return to health facility for care (%) 96.2% 96.9%
* Among those that incurred a transport cost (n=1,281 in round 1 and n=1,071 in round 2)

Overall, the structural quality of care at health facilities was also good. All health facilities were connected to the electricity grid and water supply system; some health facilities also had a generator and water tanks as reserve. Only 6 (15%) health facilities had no means of sterilizing equipment, while 14 (35%) facilities sent their equipment to the nearest community health centre or hospital. In terms of availability of drugs, 29 (72.5%) facilities reported drug stock-outs at some point during the year. On average 36% of facilities reported a problem with the functioning of general equipment in the facility (e.g. stethoscopes, blood pressure apparatus); one sub-district performed considerably better compared to the other sub-districts in this regard. For all tracer conditions, clinical care guidelines for essential services were adhered to most of the time. In the case of diabetes/hypertension, some health facilities could not provide all services with the recommended equipment (e.g. the use of monofilaments to detect neuropathy), though alternatives were used.

Health worker motivation
Table 6 presents information for both rounds on absenteeism of health workers and on performance appraisal, skills training and quality improvement plans at the health facility.

Table 6: Health worker absenteeism and supervision
Characteristics Round 1 (n=173) Round 2 (n=203)
n n %
Absent in past 30 days 82 46.4 90 42.9
Due to illness 33 19.1 38 18.7
Due to vacation 26 15.0 21 10.3
Due to training 20 11.6 21 10.3
Other absence (e.g. funeral) 25 14.5 27 13.3
Arrived late 48 27.7 45 25.4
Left early 15 8.7 19 10.7
Health workers’ performance is appraised 138 79.8 159 78.3
Facility has skills training plan 125 72.2 163 80.3
Facility has quality improvement plan 145 83.8 182 89.7

Absenteeism among nurses that were interviewed decreased between round 1 and round 2. There were however substantial differences between sub-districts. While absenteeism decreased significantly in two sub-districts, it nearly doubled in the third sub-district mainly due to training activities and to a lesser extent, absences for other reasons. Nurses were more aware about the existence of skills training and quality improvement plans in round 2; we again observed substantial differences between sub-districts.

The most important motivational factor for health workers was their relationship with the community (data not shown). They were also satisfied by the teamwork in the health facility and management. In particular for management and supervision, we observed an increase in health workers' satisfaction between rounds. Health workers also demonstrated high levels of conscientiousness and commitment. They reported being least satisfied by financial aspects followed by issues related to fairness and transparency on the job and the work environment (both physical and workload).

Key findings of policy intervention evaluation: health worker motivation and quality of care in Tanzania (Objective 3)

Overall, quality of care as judged by a range of process and structural indicators was variable across districts and in some regards, e.g. structural factors, poor. Waiting times reduced in one district over time, and an indicator of improved satisfaction was a slight increase across districts in patient willingness to recommend the facility to a friend. Staff absenteeism increased over time, though reported supervision frequency also increased. Despite salary increases, staff reported being least satisfied with financial aspects of their job and the work environment and were most satisfied with their relations with the community. Future tracking of reform impacts should continue to assess all these indicators.

Sample sizes and key characteristics
Data were collected at 69 public primary care facilities (including 11 health centres) in round 1 and 58 facilities (8 health centres) in round 2 in the three districts. The sample sizes for each of the rounds and surveys are shown in Table 7.

Table 7: Sample size for each of the surveys
Surveys Round 1 Round 2
Facility survey 69 58
Exit interviews 729 922
Health worker survey 137 140

We conducted exit interviews with patients from each of the sampled facilities, comprising antenatal care or child vaccination clients, parents of children presenting with one or more of the following symptoms: fever, cough or diarrhoea, and adults presenting with one or more of the same three symptoms. These symptoms were chosen as they are indicative of malaria, acute respiratory infection and diarrhoea, the top three causes of outpatient visits in Tanzania. Table 8 indicates that in each round about a quarter of patients were antenatal care clients, followed by adult fever cases, childhood immunisation visits, and childhood fever cases. Just over 30% of those interviewed were insured by the CHF (Table 8).

Table 8: Types of Patients / Services Interviewed for Each Survey Round
Service type Round 1 (n=729) Round 2 (n=922)
n % n %
ANC care 179 24.8 216 23.6
Child vaccination 111 15.4 149 16.2
Adult fever 210 29.0 188 20.4
Adult cough 62 8.6 70 7.7
Adult diarrhoea 33 4.6 50 5.5
Child fever 69 9.5 127 13.9
Child cough 38 5.3 65 7.1
Child diarrhoea 16 2.2 51 5.6
Insured by CHF 220 30.6 324 35.3

Of the health workers interviewed, the average age was around 40, they had been working at the facility for over 4 years and the majority were nursing cadres, followed by clinical cadres and then support staff (Table 9).

Table 9: Health Worker Characteristics for Each Survey Round
Characteristics Round 1
(n=137) Round 2
(n=140)
n n
Age (mean, std) 130 42.4 [10.2] 137 39.20 [10.87]
Months worked at facility (mean; std) 137 69.8 [97.9] 128 54.84 [76.25]
Medical Officer/Assistant Medical Officer (%) 3 2.3 12 8.5
Clinical Officer (%) 27 20.6 27 19.3
Nursing staff (%) 77 58.8 84 60.0
Other staff (%) 24 18.3 17 12.1
Facility in charge (%) 19 14.3 30 21.4

Access to services, CHF awareness and quality of care
Patients reported spending less time getting to the facility between the first and second round, but transport costs remained broadly constant between rounds at around 1,800 Tsh (Table 10). Time spent travelling to the facility was greatest in the two rural districts, at just over 50 minutes compared to just under half an hour in the urban district, and transport costs were also higher (just over 3,000 Tsh) (data not shown). Waiting time was broadly similar in the three districts (Table 10) with facilities in one district demonstrating a significant reduction between rounds (from 130 minutes to 100 minutes) (data not shown). Consultation time was broadly constant over time and across districts at 10 minutes on average. There was a reduction in the probability of paying for health care across rounds and in each of the districts. Patients were more likely to recommend the facility to a friend in round 2 than round 1. There was an increase in the proportion of patients who had heard of the CHF in the rural districts from 70% to 80%, which is likely indicative of the effect of the sensitisation programme.

Table 10: Patient experience of care
Variables Round 1 (n=729) Round 2 (n=922)
Transport cost-TZS (Mean, std) 1,821.3 [2,656.7] 1,840.4 [2,069.7]
Walking time in minutes (Mean, std) 50.1 [67.1] 41.3 [44.7]
Wait time in minutes (Mean, std) 117.6 [90.4] 104.4 [52.4]
Consultation time (Mean, std) 10.2 [10.1] 10.7 [9.7]
Paid for care (%) 19.8 15.2
Recommend facility to a friend (%) 88.5 90.6
Heard of CHF (in rural districts) 70.0 80.0

Structural quality was generally poor at the surveyed facilities. Less than half of facilities had electricity, piped water, flush toilet, or essential equipment. Antenatal care drugs were more widely available, with vaccines and anti-malarial drugs being the most widely available, and obstetric drugs, and antiretroviral drugs, the least available. In terms of antenatal care patients, clinical care guidelines were adhered to in just over half of cases, with laboratory tests being the least common component of the care guidelines to be carried out, and a physical examination being the most likely component of care to be carried out. Adherence to the guidelines was affected by availability of drugs and supplies, and staffing levels, and health worker characteristics.

Health Worker Motivation
Absenteeism in the sample of health workers was higher in round 2 than round 1, consistent across the three districts, at just over 60% (Table 11). Over three quarters of health workers were supervised as expected within the past 30 days, a slightly higher level in round 2 than round 1. There was a reduction in the proportion of health workers being supervised “more than a month and less than three months prior to the survey” and “never” between rounds. There were substantial variations in average salary levels across the three districts (data not shown). Average salaries were higher in round 2 than round 1 in all districts. There was an increase in the proportion of health workers reporting an increase in salary in the past year by 15 percentage points between rounds. Despite a stated policy of allowing health workers to benefit from CHF funds, very few health workers reported receiving such allowances.

Table 11: Health worker absenteeism and supervision, payment
Variables Round 1
(n=137) Round 2
(n=140)
n % n %
Absent in past 30 days 60 45.5 74 62.2
Has a supervisor 118 86.1 112 80.0
Supervised in past 30 days 92 78.0 91 81.3
Supervised 31-90 days 14 11.9 11 9.8
Supervised more than 90 days ago 3 2.5 3 2.7
Never supervised 7 5.9 5 4.5
Had a salary increase in past 12 months 88 64.2 112 80.0
Received allowance for CHF 3 4.4 3 3.5

In terms of job satisfaction, overall health workers were most satisfied with their relations with the community followed by management and supervision at the facility, followed by team work, and demonstrated high levels of conscientiousness and commitment. Despite reported salary increases, health workers remained least satisfied with financial aspects of the job and the work environment. There was little variation over time in satisfaction scores, with a worsening of satisfaction with financial aspects of the job between rounds 1 and 2.

Engaging with policy makers and implementers and
synthesising the results (Objectives 4 and 5)

As will be apparent from the above, engaging with policy makers and implementers was embedded in the research approach and has been ongoing throughout the project. Similarly, the annual research workshops, and regular skype exchanges, has enabled us to compare and contrast findings and lessons in both countries over time. Thus, for example, we have found that factors that usually contribute to poor implementation include: changes in the policy design from what was originally agreed, as it is implemented; resistance to the policy changes, either from within or outside the health service; a lack of resources (financial, human, etc.) required to implement the policy; and a lack of appropriate organisational structure or management skills necessary for implementation. National systems for monitoring the implementation of health systems reform are often weak, due to poor data capture and/or management, limiting the capability of national governments to ascertain the impact of such reforms, or be alerted to implementation bottle-necks or constraints during the roll-out process. Effective policy intervention requires a well functioning monitoring and evaluation system which provides data that allow policies to be improved over time, and consequently strengthen their potential to achieve universal health coverage.

This is enabling us to develop thinking about how best to phase and sequence reforms, a topic that we have found to be barely addressed in the literature.

In addition, as part of reflexive research practice, we have been able to reflect on the overall UNITAS project, particularly lessons from our ‘on-the-ground’ experience of undertaking district-level implementation research.

Understanding and reflecting on the nature of the research: Research design, purpose, setting and requirements of the health system may enable or impede the research process. In UNITAS, we tried to draw in policy makers and implementers at the planning stage, as well as throughout the research (using a theory of change approach) to ensure relevance and collective ownership. In South Africa, working in three districts that were piloting new reforms and thereby attracting a huge amount of national attention, when we began to discuss the project with district management teams, there were serious concerns about being oversaturated with research demands. Indeed, many proposed research projects were being turned away from these districts. However, because UNITAS was focused on the perspectives and experiences of those managing the change – looking at their theories of change – managers recognised the potential value of our project and agreed that it should go ahead.

Negotiating entry into sites is more than just ‘starting the research’: It is the process that establishes relationships, builds trust and directs researchers to what is happening and where. Implementation research is indeed ‘messy’ and a huge amount of time was spent understanding interventions and their sequencing in individual sites in order to try and do research on actual practice. In UNITAS, we were able to do this through an initial context mapping phase, which served as a means to ‘meet and greet’ the key stakeholders but also to analyse the ‘situation’. However, this process was very time consuming and resource-intensive and even while the context mapping was being undertaken, new initiatives began, which we often only found out about when we started data collection and reflexive engagement.

Anticipating and planning for implementation challenges in the research process itself: While research relevance and implementer ‘buy-in’ are prerequisites to successful implementation research, we found that a shared stakeholder vision (in terms of the broad aims, objectives and vision of the research) did not automatically translate into straightforward or easy implementation of the UNITAS project. Also getting buy-in took numerous visits to districts and repeated engagements. As with policy implementation, system readiness and acceptance of research happens at different levels. On the ground, researchers may meet resistance, especially in stretched and strained systems where the presence of a research team can simply present yet another burden for busy, tired health care practitioners struggling to cope. In one facility included in the UNITAS project, although we had formal permission to do the work, the manager and staff simply refused – through quiet acts of resistance and even open hostility - to work with us (at least initially). Being able to travel regularly to sites, engage in other forums and through email and phone calls ultimately helped to develop and sustain relationships in this facility, as well as across the districts more generally.

It is critical to sustain relationships with district and facility managers over time in projects like this: The research process happens in a context, where history, relationships and networks may assist, or not, research opportunities. So much of the UNITAS team’s work involved establishing and building relationships with people in the system. Embedded in districts over 3-4 years, strong relationships formed the basis for our ongoing work. Furthermore, there is a strong ethical imperative to ensure sound research practice for future work within the system. Implementation research can be enabled or impeded by how it is done and who does it. Researcher reputation and credibility are hugely important for enabling research work in environments where change is rapid but also long-term (with potential for sustained/future work). Including the ‘researched’ through a co-productive/ theory of change approach between researchers and implementers can facilitate people feeling heard and reflected in the process as well as the content of the project.

Site management in complex, large research projects (working across sites and teams) is highly dependent on the pace at which government moves and on inter-governmental relations: In South Africa, at the research permission stage, the process flowed down from national to provincial to district levels. However, once in districts, there were few formal upward flowing mechanisms for feedback and communication on the research. Therefore, the ability of the team to seek out forums for feeding back key findings during a time of rapid change was important for opening spaces of dialogue between different layers of government.

Taking note of the health system itself: There is something specific about working in a clinical environment and the health system as a whole that brings its own language and practices. From a research perspective, working in multidisciplinary teams comprised of public health specialists, health economists, medical doctors, statisticians, health policy and systems analysts, quantitative and qualitative researchers was a benefit for UNITAS because it allowed for smooth translation of professional discourse and fresh insights into the implementation of reforms. However, there are points in time when multidisciplinary teams cannot find agreement as thinking can be quite divergent and this can create some difficulties. Ensuring regular team meetings and engagements across sites was an important contributor to developing a ‘shared’ UNITAS perspective and language.

The need for ongoing reflection on how implementation research is valued: Designed to monitor and evaluate the early implementation of reforms, for UNITAS, the implementation ‘story’ has not really ended although the project is formally over. It is important to recognise that much of the effect of implementation research projects may be seen only after the research grant has come to an end. For example, we have learnt since UNITAS ended in March that following the final meeting with district and provincial managers, where results were disseminated and discussed, managers went back to their districts and began to implement some of the recommendations discussed at the meeting.

Potential Impact:
There is a range of potential impacts of the UNITAS project.

The UNITAS study will make three important contributions to ongoing discussions about how best to promote movement to universal health coverage and to understanding universal coverage policy implementation bottlenecks in low- and middle-income countries. First, insights are being generated into what types of health system reforms best promote progress towards universal coverage in different contexts. Second, there are important insights around universal coverage reform sequencing, arising from the dominant initial focus in both South Africa and Tanzania on improving the physical infrastructure, equipment, human resources, drugs and other medical supplies within public sector services, and on improving the quality of health services. Third, practical issues and processes that need to be considered across settings to strengthen implementation processes to achieve reform goals are being identified, including the value of and approaches to monitoring and evaluating reform implementation.

UNITAS is likely to make a particular impact in relation to international consideration of how to monitor health sector reform implementation. It demonstrates the usefulness of real-time implementation monitoring for tracking health system reforms, and in understanding how and why reform implementation processes unfold and with what consequences. Implementation research remains a relatively small field, and to date very little attention has been paid to explanatory analysis of health system reform implementation experiences. It is anticipated that UNITAS will contribute to building the field of implementation research through such analyses.

Country impact
To date, only a handful of studies have been undertaken with the deliberate goal of supporting policy change. The UNITAS study was explicitly envisaged as critically examining the implementation process of health system reforms in order to support policy implementation. Before initiating the project, there were already strong links between the country research teams and policy makers and managers at the national and district level. This provided a basis for ensuring that the research was policy relevant and focused on priority issues and also increased the likelihood of research being translated into policy and practice.

Particular emphasis was placed on strengthening these links between researchers and policy makers and district managers during the initial phases of the project. Efforts were made to ensure regular engagement with policy makers and district managers around research findings, not only through formal processes, but also through informal engagements. Efforts were also made to interpret research findings and consider potential ways of addressing implementation challenges in discussion with implementers.

A very important element of the research project has been to give voice to those working at the frontline in districts and health facilities. Conveying their experiences, views and concerns to senior managers and policy makers has been seen as a valuable contribution of the project. The project has also provided an opportunity to promote learning around implementation across districts – a need raised by managers in both countries. In March 2016, the South African team facilitated a workshop for senior managers from the three study sites. This was the first time that peers from different NHI pilot districts had reflected together on their experiences as managers of NHI pilot districts. The workshop created an opportunity for cross-site learning, mutual support and the co-production of ideas for future practice, while also enabling joint discussion and collaborative interpretation of key findings from the project overall.

Ultimately, it is hoped that the UNITAS project will contribute to progress towards universal coverage in both South Africa and Tanzania. Reducing inequalities between groups in accessing quality health care and promoting financial protection against the costs of health care in both countries will have major benefits at a societal level.

Dissemination activities
The UNITAS team has engaged, and will continue to engage, in a range of dissemination activities at district, national and global level. One of the first activities undertaken to disseminate information on the project was the development of a project website which provided information on the project, updates on progress with the project and contains relevant project publications (e.g. policy briefs, copies of conference posters and presentations).

District and facility level
As most of the research was undertaken at a district level and within individual facilities, this has been a specific focus for dissemination of findings and further engagement, consistent with a theory of change approach. An important period was that after the preliminary analysis of qualitative and quantitative data from the first round of data collection. The preliminary findings from Round 1 were fed back and discussed through meetings with managers and implementers in each district. In addition, a two-page ‘District Update’ summary of findings was prepared for each study site. It was distributed widely in districts and facilities within the district. This document greatly enhanced the willingness of frontline staff and managers to continue participating in the UNITAS project, and many offered valuable reflections and insights after reading it. Further distribution of this document during the second round of data collection provided an opportunity to feed back to other actors not present at the results’ dissemination meetings.

There was also regular informal contact with key managers in each of the districts to keep them informed about progress with the research and important findings. In addition, the research teams made use of other opportunities presented to engage with district managers, such as at district health system consultative forums.

In South Africa, a workshop was arranged near the end of the project for key managers from each of the study districts. Focused on experiences of managing a NHI pilot site, the workshop was designed in response to an identified need for cross-site exchange and learning. Senior managers were drawn together to share their experiences of managing NHI reforms alongside routine district work. The workshop was also an opportunity for researchers and managers to collectively reflect on, and critically review, project findings. Co-facilitated by an organisational psychologist, participants were introduced to tools for organisational and change management. New ways of engaging people were also demonstrated and, following the workshop, key management resources were provided to deepen knowledge shared. Space was also created for each district to develop ideas for future activities once ‘back home’. Ideas generated included the transfer of ‘best practice’ lessons across sites to enhance strategic planning, strengthen referral and information systems and nurture a pipeline of motivated leaders across all levels of the district hierarchy. There was also a commitment to lobbying the national Department of Health to facilitate similar cross-site learning and sharing platforms between managers from all of the country’s NHI pilot districts.

Senior managers in districts reflected that they found the process of facilitation of the workshop highly encouraging; they appreciated the experiential methods of facilitation that were used such as world cafés, councils and short presentations with insightful questions to promote reflection, thinking and sharing. They noted this was a very different way of sharing research findings with districts and found it much more useful than the long, didactic presentations they typically have to sit through. We are documenting the process followed in order to share different approaches to cross-site learning with other researchers, managers and government officials.

In Tanzania, all the key implementers at the district and facility level participated in a dissemination meeting where key findings were presented and participants had time to discuss these findings and identify areas where the district managers would need to put more efforts to ensure they achieve their goals; for instance limited funding from the central government to LGA was identified as a critical challenge, but the district managers had an opportunity to discuss ways to explore local resources that could be used to implement the incomplete projects. Researchers also used that time to share lessons from other districts.

National level
The research teams in both countries have strong relationships with senior health officials at national level and policy makers. For example, in Tanzania one of the team members regularly attended monthly meetings of the health financing technical working group of the Ministry of Health and Social Welfare. Similarly, members of the South African research team serve on universal health coverage policy committees (such as the National Health Insurance implementation work streams). These mechanisms have allowed UNITAS research findings to be fed more directly into policy processes.

Other opportunities have arisen for making presentations on elements of the UNITAS research findings, either at meetings of the national Department or Ministry of Health, or at national conferences attended by health managers. In addition, a key mechanism adopted by the UNITAS team for disseminating research findings to policy makers, other actors active in developing and implementing universal coverage reforms within each country as well as a broader audience has been that of policy briefs. These are documents of approximately four pages which summarise key research findings and which are written in easily accessible language. A number of policy briefs on key aspects of the UNITAS findings have been developed and disseminated widely; additional policy briefs are planned in the near future.

International level
Research findings have also been disseminated to the international community. This has taken the form of presentations at international conferences, particularly the Global Symposium on Health Systems Research and the International Health Economics Associations Global Congress, and peer-reviewed journal publications. Although a limited number of journal articles have been published to date, a considerable number of articles are currently in preparation and will be submitted for publication review over the next few months.

Members of the research team from all of the partner institutions have also engaged in less ‘formal’ sharing of experience through our engagements in regional and global networks and with key international organisations. For example, some team members have participated in WHO meetings related to the content of and mechanisms for monitoring and evaluating universal health coverage reforms, and in processes around the universal health coverage elements of the Sustainable Development Goals. We have had many opportunities to feed our findings directly into global debates through these and other key ‘entry-points’ to these debates.

List of Websites:
Project public website address:
http://www.unitas-africa.org

Contact details:
Coordinators:
Prof. Anne Mills
London School of Hygiene and Tropical Medicine, United Kingdom
Anne.Mills@lshtm.ac.uk

Prof. Di McIntyre and Prof. Lucy Gilson
Health Economics Unit, University of Cape Town, South Africa
Diane.McIntyre@uct.ac.za and Lucy.Gilson@uct.ac.za

Partners:

Dr Gemini Mtei and Jane Macha
Ifakara Health Institute, Dar es Salaam, Tanzania
gmtei@ihi.or.tz and jmacha@ihi.or.tz

Prof. Bruno Marchal and Fahdi Dkhimi
Institute of Tropical Medicine, Antwerp, Belgium
bmarchal@itg.be and fdkhimi@itg.be

Dr Jane Goudge and Dr Bronwyn Harris
Centre for Health Policy, University of the Witwatersrand, Johannesburg, South Africa
Jane.Goudge@wits.ac.za and Bronwyn.Harris@wits.ac.za

Dr Till Bärnighausen and Dr Natsayi Chimbindi
Africa Centre for Health and Population Studies, University of KwaZulu-Natal, Hlabisa, South Africa
tbarnighausen@africacentre.ac.za and nchimbindi@africacentre.ac.za

Dr Peter Kamuzora and Dr Stephen Maluka
Institute of Development Studies, University of Dar es Salaam, Tanzania
petkamu@udsm.ac.tz and stephenmaluka@yahoo.co.uk

Final Report Summary - UNITAS (Universal coverage in Tanzania and South Africa: Monitoring and evaluating progress)

Related documents

Share this page

Download