Final Report Summary - EQUIP (Expanded Quality management Using Information Power for Maternal and Newborn Health in Africa)
Executive Summary:
EQUIP, an innovative intervention package for maternal and newborn health, was designed in the context of persistently high rates of maternal and newborn mortality, despite the availability of evidence-based, affordable and appropriate technical interventions. Obstacles to improved survival exist on both the demand (e.g. low utilization) and supply side (e.g. low quality and lack of services). Quality management and quality improvement are increasingly promoted to improve the quality of care. We hypothesized that quality management supported by improved data use, reaching out to communities, and including district managers in the process could improve both utilization and quality of care.
Between November 2011 and April 2014, EQUIP worked on improving utilization and quality along the continuum of care in Tandahimba district in southern Tanzania and Mayuge district in eastern Uganda. Using the collaborative approach to quality management, and working with district, health facility, and community quality improvement teams, we initiated the work by highlighting priority areas in ante-, intra-, and postpartum care during “learning session” meetings. Local teams at each level used Plan-Do-Study-Act cycles to identify and test strategies for improvement. Simultaneously, EQUIP implemented a continuous survey in the intervention districts and in one comparison district in each county. Data from these surveys were synthesized on report cards for use by the quality improvement teams, and analyzed for effect evaluation for a set of pre-defined primary indicators as well as for secondary indicators which reflected specific improvement topics.
The results indicated an improved coverage of women receiving uterotonics within one minute after birth in both countries; in Tanzania, the proportion of women who received uterotonics was 26 percentage points higher (95% CI: 25%–28%) at the end of the project period in the intervention compared to comparison district. In Uganda, the difference was smaller but still statistically significant at eight percentage-points (95% CI: 6%–9%). There was some evidence of an increase in preparation of clean birth kits for home deliveries in Tanzania, with the difference adjusted for baseline at 31% (95% CI 2%–60%) and weak evidence of more deliveries in facilities in the intervention than in the comparison district (difference adjusted for baseline 7%, 95% CI: -7% , 21). There was no evidence of an effect of the EQUIP quality improvement intervention on any other primary indicators, immediate breastfeeding, or knowledge of danger signs; however teams did not prioritize work on this.
The learning sessions every four months at the community, health facility and district levels were resource intensive. Although appreciated by the quality improvement teams, the visits were at times difficult to operationalize, as skilled mentors are rare. The teams prioritized improvement topics together with the district and EQUIP mentors, although some areas were pre-determined. We saw improvements where district health managers were strongly involved (supply side improvement), and more mothers coming to deliver in facilities and preparing for birth (demand side improvement). Our work underlines the need for systemic approaches and highlights the need to work on both the demand and supply sides simultaneously.
Our study suggests that teams in communities and health facilities can use both quality improvement techniques and locally generated data to inform decision making. Positive changes were achieved, although not for all the topics that teams worked on. Simultaneous strong involvement of the district managers seems to be the key to having any impact. The difference in response to the intervention between Tanzania and Uganda suggests that the district’s financial resources could be important for quality improvement, particularly having “non-earmarked” funds to overcome local constraints. Quality improvement needs robust support in the form of mentoring and coaching. These systems need to be sustained over time.
Project Context and Objectives:
Introduction
Maternal and newborn mortality remain unacceptably high especially in Sub-Saharan Africa. Over one million newborns and 179,000 mothers died in 2013 in Sub-Saharan Africa alone [1, 2]. This is although many effective and affordable interventions are available [3] but implementation levels vary and is in many countries insufficient [4].
In Tanzania and Uganda, many essential interventions such as active management of third stage of labour or blood pressure taking during antenatal care are not implemented at scale. Reasons are many, among them low availability of essential items in facilities [5, 6] but also substandard practices. Quality management has the potential to improve implementation levels by optimizing the existing health infrastructure rather than on the addition of human and physical resources. It includes the review of processes to improve performance, and monitoring of quality using scientific and statistical thinking with the continuous plan-do-study-act (PDSA) cycles in the centre.
The collaborative approach to quality improvement [7], as supported by the Institute of Healthcare Improvement includes in a package some aspect of training or sensitization towards standards, coaching and mentoring and learning within a team. Quality improvement is not an registered or protected package but rather include adapted support package to thrive towards improvement of quality and sustaining of an improvement change [8].
Quality improvement is increasingly used also in low resource settings to improve the quality of care of selected interventions [9] or the scale-up of new interventions [10, 11]. As such, quality improvement has many links to health system strengthening. In Ethiopia is was employed to strengthen the health extension workers, community health development agents and traditional birth attendants ability to provide maternal and newborn care [12] but with a strong component of district health system improvement [13]. In Malawi, a combined facility quality improvement and community mobilization project reduced perinatal mortality by around 20% [14]. A multicounty trial conducted in Asia, Africa and Latin Americas combining community mobilization and facility staff training but without any explicit component of district strengthening, did not show any effect on perinatal mortality [15].
The Expanded Quality Management Using Information Power (EQUIP) was developed with a view on strengthening the district health system using quality improvement approaches. The development of the quality improvement approach took a similar stand as outlined during the conference on district health systems in Senegal 2013 [16] and focused on all levels of the health care system including the community, health facility and the district level. It respond to the need to build inclusive, patient-centred health systems [17] with a continuum of care approach and spanning from community to primary facility to hospital care.
Objectives
We hypothesize that expanded QM based on high quality information, and implemented in consultation with health planners, health service providers and community will lead to measurable improvements in quality and utilization of maternal and new-born services and practices across the continuum of care, and that this intervention is feasible, affordable and relatively cost-effective for the African health systems.
Our overall goal was to assess the feasibility, cost and community effectiveness of an innovative approach of Expanded QM Using Information Power (EQUIP) that links communities and health facilities to increase the quality and utilization of health care services in order to improve maternal and new-born health in rural Uganda and Tanzania.
Specific objectives
1) To analyse the policy environment of maternal and new-born health and QM approaches as well as bottlenecks to the potential introduction of selected essential evidence-based interventions and innovative technologies
2) To collect data through continuous household and health facility surveys, the health management information system, and qualitative assessments and summarize the information on report cards for use at community, health facility and district level
3) To use this information to drive a QM strategy linking community, health facility and district/regional levels
4) To assess the feasibility, effects and community effectiveness of EQUIP
5) To analyse the cost and cost-effectiveness of EQUIP as well as the implementation cost of selected essential evidence-based strategies and innovative technologies
6) To engage policy makers and key stakeholders throughout the study in a discussion of the intervention for policy relevance and generalization to national scale
Project Results:
Methods
The study design
EQUIP used a quasi-experimental design to compare changes in maternal and newborn health outcomes and output indicators over the implementation period between one intervention district and one comparison district (Figure 1). This study design was chosen because EQUIP targeted the district level, not smaller implementation units such as wards or villages, and therefore randomization was not possible as funds were limited.
The evaluation was based on continuous household and health facility surveys, which provided estimates of changes over time between intervention and comparison districts. To assess whether other changes over time or other factors might have led to changes in indicators or in implementation, we also documented contextual factors in our evaluation process.
Evaluation methodology
Implementation of the EQUIP Continuous Survey
Continuous survey data were used for quality improvement in the intervention districts and for effect evaluation of the intervention. Over the 30 months of intervention (November 2011 to April 2014), EQUIP conducted continuous cross-sectional household and health facility surveys using independent probability samples of household clusters to represent each district each month, and repeat censuses of all government health facilities. All resident women aged 13–49 years (15–49 in Uganda) in selected households were interviewed about recent live births and use of health services (Photo 1). Geographical positioning was collected for all respondents for subsequent spatial analysis. Using repeat samples in this way allowed data to be aggregated at six four-monthly intervals to track progress over time for evaluation, and for continuous feedback to quality improvement teams in intervention districts.
Questionnaires used for the continuous survey were adapted from tools including the Safe Motherhood Needs Assessment, Demographic and health Surveys, Multiple Indicator Cluster Surveys, Service provision Assessments, and others [36-38]. A modular checklist-type questionnaire is used to assess health facilities, including staff employed, drugs, supplies and equipment and implementation of essential interventions for routine childbirth care using a ‘last event’ approach where health workers are asked to report on the care they provided during the last birth they attended. The household questionnaire has a module that includes questions on household assets, housing type, ethnic group and geographical position. Women of reproductive age (13 to 49 years in Tanzania and 15 to 49 in Uganda) are asked about knowledge and use of family planning and a pregnancy history since January 2010. Also, information on perceived quality of care is collected. Women with a live birth in the two years before the survey are asked about care received during the antenatal period, delivery and the post-partum period.
In both countries, one continuous survey team of eight people was employed to interview approximately 300 households, 10 health facilities, and complete 10 health worker interviews per district, per month. Data were collected using personal digital assistants. After every four months of data collection, routine tabulations of all indicators were produced and synthesized into report cards for use to support decision making by the EQUIP quality improvement teams. See Table 1 in the attached report for the EQUIP continuous household survey interviews completed by district.
The continuous surveys were implemented as planned. Completion of household interviews was consistently at 90% in both countries. A total of 1,415 and 2,993 interviews with women who had a live births in the last 12 months in Tanzania and Uganda, respectively, were done throughout the two and a half year of implementation. The higher number of interviews of mothers with a live births in the year prior to the survey in Uganda reflect the larger household size and the higher fertility in Uganda. Indicators across the continuum of care for mothers and newborns were assessed.
Approximately 90% of the planned health facility interviews were done. In most places an interview with a health workers who take care of the last delivery in health facility (last event) was completed to collect information on implementation practices. See Table 2 for the EQUIP health facility census and health care worker interviews completed by district.
The continuous surveys were feasible and provided high quality data throughout the EQUIP implementation period which were use for regular feed-back to the teams and for repord cards.
Contextual Documentation Methodology
To complement the information from the continuous surveys, EQUIP documented changes over time and differences between the implementation and comparison districts through:
• Monthly documentation of important events such as unrest, drought, or policy changes
• Interviews with the district management team in the implementation and comparison districts three times a year to investigate any resource or management changes, such as: breakdown of ambulances; changes in human resource availability and trainings; or existence of other donors
• Interviews with the district management teams once a year about overall health planning and implementation
• Health reports and continuous survey data
Statistical analysis
We defined priory primary indicators to measure the impact of EQUIP. These were the coverage indicators ‘Facility delivery’ and ‘breastfeeding within 1 hour’, the quality indicators ‘active Management of Third Stage of Labor”’ and knowledge indicators of ‘danger signs in pregnancy and for newborns’.
In each country, we estimated coverage estimates for utilization, life-saving interventions, quality of care and knowledge along the continuum of care in the intervention and comparison areas at four-monthly intervals during the period of intervention (November 2011 – April 2014). We use meta-regression analysis to fit a regression line through the six data points in each district, and estimated the change in outcomes over time between baseline (first data collection round) and endline (last data collection round ), comparing this change between intervention and comparison districts using difference-of-differences. We used the delta method to estimate the variance [18].
EQUIP implemented changes in response to a range of improvement topics at district, health facility and community level such as improving facility delivery and Active Management of Third Stage of Labour (AMTSL).
The Study Setting
EQUIP was implemented in two rural areas in Tanzania and Uganda over a period of 30 months (November 2011 to April 2014). The EQUIP intervention was implemented in eastern Uganda (Mayuge District) and southern Tanzania (Tandahimba District); two neighboring districts served as comparison areas (Namayingo District in Uganda and Newala District in Tanzania) (see Figure 2).
All four districts have high maternal and newborn mortality rates, are predominantly rural with small district capitals, and have populations that live on subsistence farming. In Tanzania, less than half of the population live in a house with improved roofing (iron sheeting). In Mayuge district in Uganda, 74% of houses had an iron roof whereas in the comparison district in Namayingo, the level was only 39%. See Table 3 in the attached report for the main health indicators in the intervention and comparison areas.
EQUIP’s Quality Management Approach
EQUIP had a comprehensive quality improvement approach including improvement teams at the district, health facility and community level with a view to strengthen the health system in a comprehensive manner (see Figure 3).
EQUIP’s quality improvement approach was based on the Collaborative Model for Improvement which has been promoted by the Institute for Healthcare Improvement (see Figure 4). The model is a short-term, rapid learning approach to make improvements in specific topics. Teams were guided to work using three questions:
• What are we trying to accomplish?
• What changes can we make that will result in an improvement?
• How will we know that the change is an improvement?
Teams were mentored to use Plan-Do-Study-Act (PDSA) cycles to identify problems, define a strategy that could produce change (“a change idea”), and test the strategy using locally generated data to determine if the change idea resulted in an improvement. Run-charts using the local data were used to monitor progress (Figure 5).
Maternal and newborn health topics for improvement were introduced during “learning sessions” held every three months where teams from multiple sites were encouraged to brainstorm obstacles related to each topic and to develop plausible change ideas to address these problems. The learning sessions also provided an opportunity to refresh knowledge of essential maternal and newborn health interventions, to share experiences, and to learn from each other. The learning sessions were followed by an “action period” where quality improvement teams of health providers or community participants implemented their change ideas and monitored progress. During this action period, quality improvement mentors provided coaching and mentoring to the teams.
The learning sessions and the regular mentoring and coaching visits were highly appreciated. The largest limitation was the volume of resources (human and financial) needed to assist the quality improvement teams. A dedicated and qualified team of mentors and sufficient financial support to facilitate their work were needed.
Coaching and Mentoring Through Existing Government Structures
Teams working on quality improvement need regular support to maintain their efforts and to continuously improve performance. Supervision, mentoring, and coaching is critical but often difficult to operationalize in low resource settings.
The EQUIP project employed two people in both countries to work on designing the overall approach, implementation, and research. Two additional people were involved in conducting the learning sessions and in the coaching and mentoring of the quality improvement teams. EQUIP included several members of the district health teams for the learning sessions and the mentorship of the health facility and community teams (see Figure 6).
In Uganda, two health facility mentors and two district community mentors were trained on quality improvement, and coaching and mentoring. In addition, four health facility sub-district mentors and 30 community mentors were trained to support the health facility and community teams in the sub-district.
In Tanzania, one health facility and one community mentor were trained, together with two more members of the district team engaged in implementation. The community district mentor was part of the community development office and established a structure for community work where extension workers were included in the EQUIP implementation process. Ten community extension workers were carrying out regular visits to community teams.
During the 30-month implementation period, a total of seven learning sessions were held in Uganda and Tanzania for both the health facility and community quality improvement teams. Mentorship took place eight-to-ten times per year, achieving more than 75% of the planned supervision of teams.
It was feasible to align mentoring and coaching to the district’s supervision structures, but the EQUIP project funded transport and communication allowances for the learning sessions and supervision visits to the health facility and community improvement teams. EQUIP was able to build district health team capacity for quality improvement through its own funding, but it additional resources will be needed for sustainability.
Results and experience of the quality improvement intervention
We successfully implemented a comprehensive quality improvement project over 30 months covering two districts in two countries. We included all facilities providing reproductive health services - except six located on Remote Island in Uganda for economic reasons - and reached out to the majority of communities.
In Tanzania and Uganda, mentoring and coaching sessions at health facility level were carried out on average two to three times each quarter. In Tanzania, to supervise the community level so-called cluster meetings are done, generally 2-3 times per quarter. In Uganda similarly, 2-3 coaching and mentoring visits were done per quarter. Over the period of 30 months, 7 and 6 health facility learning session and 6 and 5 community learning session were done in Tanzania and Uganda, respectively.
See Table 4 in the attached report for mentoring and coaching, learning sessions, and District QI team meeting frequencies per quarter.
Experiences on Operationalizing District-Level Quality Improvement Teams for Maternal and Newborn Health
Quality improvement in health facilities needs the support of the district managers because not all barriers for successful implementation can be tackled solely at the facility level. Issues such as stock outs of drugs, lack of human resources, or supervision of the health facilities need to be handled by the district managers. For this reason, district-level quality improvement teams made up of district health managers were formed.
We engaged the district quality improvement teams by:
o Involving them as facilitators and mentors for quality improvement teams in health facilities and at varying degree at community level
o Including them as teachers and facilitators during learning sessions
o Encouraging them to work on district level problems
The district quality improvement teams met every 3–4 months for feedback and updates. During these meetings, each team was informed about the progress of health indicators in the district, using data from the continuous surveys presented as district “report cards”. These report cards contained trends of selected indicators presented in the form of run charts and figures.
Based on these report cards, the district quality improvement team was supported by the EQUIP staff to analyse problems using the fishbone method. Change ideas were generated and tested using plan-do-study-act cycles. Most efforts at the district level were targeted at reducing drug stock-outs, improving supervision of health facilities, improving human resources provision within the facilities, and providing the necessary equipment.
In Tanzania, the improvement topic “regular provision of oxytocin” was successful in reducing stock-outs of the drug (see case study below). However, the improvement topic “supportive supervision” was less successful. Members of the district quality improvement team reported that while they were visiting health facilities together with various collaborating organisations, they had limited time to perform supervision and were unable to supervise more than one health topic at a time. Also, because there was no documentation of the purpose or content of these supervisory visits, no progress could be measured.
In Uganda, the district quality improvement team tackled the lack of human resources by the reallocation of staff within the district as well as recruitment of new staff. The frequency of drug stock-outs was reduced by improved forecasting and the timely requisition of drugs. Convening the district quality improvement teams was a challenge in both countries due to many competing activities of team members. The district QI teams appreciated the reports cards and the intense quality improvement work in the health facilities. Although they committed to performing the continuous oversight of health facility and community quality improvement activities through supportive supervision, a lack of transport, poor documentation, poor coordination, and competing activities within the district hindered their quality improvement activities.
District quality improvement in Tanzania: Supporting the availability of oxytocin
“While working on improving the implementation of active management of the third stage of labour to prevent postpartum bleeding, it became apparent that individual health facility quality improvement teams were not able to solve stock-outs of oxytocin on their own. The immediate problem was that the Medical Stores Department, the official government supplier of drugs in Tanzania, was not able to regularly provide the facilities with the amounts they had requested. Securing the cold chain for proper handling of oxytocin was also identified as a problem.
EQUIP therefore engaged the district quality improvement team to resolve this situation. During a workshop, the district quality improvement team analysed bottlenecks using the fishbone technique. As one immediate solution, the district decided to procure directly from Medical Stores Department using district-owned resources (e.g. the donor supported funding mechanism ‘basket fund’). By November 2012, a total of 100 vials had been purchased, mainly for use in the district hospital, though some were distributed to other health facilities.” (Yovitha Sedekia, EQUIP Tanzania)
District quality improvement in Uganda: Tackling lack of human resources
“EQUIP report cards revealed that one of the key problems faced by Mayuge district in Uganda was insufficient human resources for maternal and newborn health. The district quality improvement team analysed this problem using root-cause analysis and found that some health facilities had a surplus of staff while others did not have sufficiently skilled staff.
To mitigate this problem, the district quality improvement team proposed a change idea of staff reallocations within the district. In March 2012 and February 2013, surplus health workers were transferred to facilities where they were more needed. The district quality improvement team also used the report cards successfully as an advocacy tool for recruitment of more health workers. As a result, five nurses were recruited in July 2012.”(Monica Okuga, EQUIP Uganda)
Health Facility Quality Improvement for Maternal and Newborn Health
Many essential maternal and newborn health interventions, known to be highly cost-effective and recommended by WHO, are not implemented in some countries, including Tanzania and Uganda. Reasons for low implementation include providers not following examination standards or available treatment guidelines, drug stock outs, and a lack of laboratory tests. Using the collaborative approach, EQUIP aimed to assist health workers in overcoming local obstacles so that they could provide maternal and newborn health services according to national standards.
During learning sessions we introduced several improvement topics (see Box 1). Using plan-do-study-act cycles, we encouraged facility quality improvement teams to brainstorm “change ideas” to overcome local obstacles to implementation. We also decided on indicators to best monitor progress. Between learning sessions, during the “action period”, the teams implemented their change ideas and were supported through on-site mentoring and coaching by quality improvement mentors, with a focus on use of the plan-do-study-act cycles to enhance their work.
In Mayuge district in Uganda, a total of 10 trained district mentors, selected from the District Health Team and higher level health facilities, supported 30 facility-level quality improvement teams monthly in problem identification, setting goals, designing and implementing change ideas, and analysing performance using data collected by the team. In Tandahimba district in Tanzania, three trained members of the district health team together with two EQUIP staff provided mentoring and coaching at facilities roughly every six weeks to review progress and encourage further testing of change ideas.
Change ideas were developed for improvement topics (see Box 2). The learning sessions were a platform for quality improvement teams to share experiences and learn from each other. This peer-to-peer learning enhanced rapid spread of solutions and motivated teams to improve their work. The facility quality improvement teams monitored the progress of change ideas over time.
On site mentoring and coaching in combination with learning sessions and peer-to-peer learning was successful in improving the quality of maternal and newborn care at the facility level. This improvement was achieved despite key challenges such as drug stock outs and insufficient human resources for health.
Working on birth preparedness in health facilities in Uganda
“Following the introduction of EQUIP, a change idea used at many health facilities was to employ ‘goal-oriented antenatal care’, whereby what was supposed to happen on individual antenatal care visits was shared with the mothers, and dates for future appointments were fixed. Because of this change idea, an increase in regular antenatal care attendance was observed. There are now special health education sessions during antenatal care, during which the birth preparedness of women is checked. Volunteers also check these preparations in the community. Due to the success of this change idea, women are now more actively preparing for birth.” (Rogers Mandu, EQUIP Uganda)
Working on increasing health facility delivery in Tanzania
“Health facility delivery was an improvement topic implemented by all health facility quality improvement teams. A common change idea was to encourage male involvement, particularly in birth preparedness planning. This was talked about during antenatal care visits and also in the community during volunteers’ visits. In areas where traditional birth attendants were influential, facility quality improvement teams cooperated with local authorities to educate [them] about the importance of delivering in a health facility. The response to this change idea was positive in most of the areas. Most facilities also made use of community volunteers to educate women on the importance of facility delivery at the community level; this was particularly important because the volunteers were trusted by the community.
The synergistic effect of the above change ideas brought tremendous results in the health facility delivery in rural Tanzania.” (Petro Arafumin, EQUIP Tanzania)
Reducing wound infections after caesarean sections in the district hospital
“In Tandahimba district hospital in 2011, wound infections were a big problem, with nearly three quarters of all mothers developing an infection after caesarean sections. Several measures had been taken by the hospital, including relocating the operating theatre to the new maternity wing, replacing worn-out mattresses and disposing of gauze suspected to be contaminated with fungi. But it was not until the implementation of EQUIP, through the systematic testing of change ideas and the measurement of follow up effects that a true impact was made. Prophylactic antibiotics and cleaning of the abdomen and genital area before surgery were introduced to all mothers, reducing the wound infection rate to around 10 percent.
We were conducting operations [caesarean sections] and later on we used to find almost three quarters of them had wound infections and some had septicaemia and others lost life. So after they [EQUIP] came here, we sat and talked to them, they gave us ways to fight against this problem and we have been practicing that and we continue to practice until now.” (Assistant Medical Officer, Tandahimba district hospital)
Experiences of Implementing Community-Level Quality Improvement
We used QI to engage community-level stakeholders in identifying problems and implementing local strategies to address health care seeking and household-level maternal and newborn care practices. Applying the collaborative approach to quality improvement, we worked in all 157 villages in Tandahimba District, Tanzania, and all 72 parishes in Mayuge District, Uganda.
In Tanzania, starting in November 2011, two volunteers from each village in Tandahimba formed a village-level quality improvement team. Ten government-employed extension workers supported volunteers in groups of 15–18 villages. A district mentor supported volunteers and extension workers alongside EQUIP staff.
In Uganda, we conducted two-day trainings of two volunteers from each village to form a parish quality improvement team from 2011 and 2012. Each parish has 6–15 villages. A total of 72 quality improvement teams were formed. The trained volunteers were often also members of pre-existing village health teams. We trained 30 community mentors to support the 72 community quality improvement teams.
During “learning sessions”, the community quality improvement teams learned the basic principles of quality improvement and how to use “plan-do-study-act” cycles. Essential health education in maternal and newborn health topics that could feasibly be addressed at the community level was also given during these sessions (see Box 1). Report cards of indicators linked to these topics, collected from EQUIP continuous survey data, were shared with volunteers at learning sessions to underline the need for action.
Shortly after learning sessions, community quality improvement teams were mentored and coached to support their understanding of improvement topics and to ensure that plausible change ideas related to the topics were created. In Tanzania and Uganda, monthly meetings were held to review learning session content and to allow the community quality improvement teams to share experiences on various issues related to implementation.
Community quality improvement teams were able to learn and apply quality improvement techniques, but ongoing mentoring and coaching was critical to their success. In Tandahimba, throughout the EQUIP intervention, the contribution that community quality improvement teams made to the increases in health facility births and birth preparedness was noted by village leaders and community members. Major challenges included limited literacy of quality improvement team members, dropout, resistance to change ideas among traditional birth attendants or in households, and some health facilities being unable to meet increased demand for services.
It was feasible to conduct quality improvement at the community level. Community quality improvement team members were accepted in their communities and the change ideas generated by them were able to respond to local contexts. Community quality improvement also helped to increase cooperation between communities and health facility staff. However, community-level quality improvement was heavily reliant on extensive mentoring and coaching, which was resource intensive.
Community quality improvement in Uganda: Birth preparedness and health facility delivery
“Initially, when we did orientation and training of the community quality improvement team members, we taught them the steps that we go through: the problem analysis and brainstorming and developing possible change ideas. To increase birth preparedness, some suggested that money should be saved to assist mothers in need to go to the nearest health facility at the time of delivery. This was the start of women’s savings groups initiated by community members. Mothers were informed about these saving groups during home visits and at a special meeting in one village. Now there are over 45 of these groups. They work together to save small amounts throughout each woman’s pregnancy.
Women have expressed their joy and happiness about how they have been assisted. There was one mother who was brought to the health centre to deliver, but the midwife wasn’t there. She didn’t have enough money to get to another health facility, so she used the money from the savings groups to go to a higher-level facility and delivered there safely. In general, many women are able to use the money from the savings groups at the time of birth. Before, if a woman could not prepare for birth, she was afraid to go to the health facility and would instead give birth at home. The women are now comfortably and confidently going to the facility to deliver.” (Rogers Mandu, EQUIP Uganda)
Community quality improvement in Tanzania: Improved collaboration between health facilities and communities
“In one cluster, a staff member from a nearby health facility rushed to be present for a community quality improvement meeting. She wanted to convey that there had been an increase in home births recently, and she wanted to encourage the community quality improvement team to continue with their efforts. Nowadays, even those working in the health facilities see the importance of community involvement and recognise that they can make a difference in maternal and newborn health. People in the community can support each other to make sure that there is no maternal or newborn death. Now health facility staff are cooperating with the community level and seeing it as essential in order to encourage good health-seeking behaviours.” (Albert Majura, EQUIP Tanzania)
Using Report Cards for Feedback and Action
Both quality improvement and health system strengthening programs need health data as the basis for decision-making, planning, and monitoring. High quality population data on health outcomes are rarely available at the local level. The continuous household and health facility surveys implemented by EQUIP produced high quality real-time, district-specific data, which we summarised every three-to-four months for district, health facility, and community quality improvement teams using ”report cards”. We used reports card to introduce new topics for quality improvement during the learning sessions and, at the district level, to discuss progress.
Report cards were specifically adapted for three different levels of care. The community report cards included very few indicators and used a simple graphical display for ease of being understood by community quality improvement teams (see Figure 7). Content of report cards was tailored to the interests of quality improvement teams and designed to trigger action. The health facility and district report cards used bar charts and run-charts, which also allowed the indication of trends over time or levels in different parts of the district (see Figure 8).
The use of the report cards was supported by the EQUIP team. The report cards were introduced during learning sessions and indicators were explained: how they were constructed; which questions were asked of respondents; and which limitations the indicators had.
This explanation led to discussions on the reliability of many common indicators. This process also built capacity to interpret indicators for monitoring of maternal and newborn health at the district level.
The report cards were met with high interest at the community, health facility, and district levels and were able to trigger attention for neglected areas of health care. Help with interpretation of indicators was important so that improvement teams understood what the data could tell them. The capacity to interpret population- and facility-based data improved over time but required considerable facilitation by project staff.
Results / Effect Evaluation
Our analysis included 27,380 households in the two countries and 4,408 interviews with mothers with a live births in the year prior to the survey (see Figure 9).
The results indicate an increase in live births where mothers received uterotonics within one minute after birth in both countries. In Tanzania, the increase over the project period in the proportion of women with a live birth in the year prior to the survey who received AMTSL was 26 percentage points higher (95% CI 25%–28%) in the intervention district compared to the comparison district, adjusted for baseline. In Uganda, the difference was less pronounced at 8% (95% CI 6%–9%). There was some evidence of an increase in preparation of clean birth kits for home deliveries in Tanzania, with the difference adjusted for baseline at 31% (95% CI 2%–60%) (Table 5)
In Tanzania, we saw weak evidence for an increase in the percentage of deliveries (difference-in-difference 7%; 95% CI -7%–21%). We found no evidence of a positive effect for any of our other primary indicators (immediate breastfeeding, or knowledge of danger signs), however the teams did not target these areas specifically.
In Tanzania our analysis also indicated weak evidence of improvement in early postnatal care (17%; 95%CI -8%–17%) and availability of key items for infection prevention (21% difference, 95% CI -4%–46%). In Uganda we found no evidence of a positive change for the improvement topics early postnatal care and early vaccination of newborns. Table 5 in the attached report presents the effects of EQUIP on utilization coverage, quality, and knowledge of danger signs.
The intervention district in Uganda (Mayuge district) has double the population size (400 000 people) than the intervention district in Tanzania (Tandahimba, 220 000 people). The Tanzanian comparison district of Newala reported 12 USD per capita for health expenditure, compared to 7 USD in the intervention district, Tandahimba, in 2013/2014. For Uganda, such data were not available, but availability of funds at the district level is limited. Availability of human resources was similar in both districts in Tanzania, but slightly higher in the intervention district than in the comparison district in Uganda (Table 1). There was little training in the area of maternal and newborn health in both districts in Tanzania, whereas several trainings were provided in the intervention area in Uganda, supported by other partners during the EQUIP implementation period. Availability of drugs and supplies were better overall in Tanzania than in Uganda.
In both countries, health planning was based on information from the health management information system (HMIS), whereas other data from surveys or studies were not used, except in the intervention district in Uganda. The primary health care infrastructure was similar, but the comparison district in Uganda had no district hospital. Referral systems were relatively underdeveloped in Uganda. See Table 6 in the attached report for a context information summary.
Discussion
We successfully implemented a comprehensive quality improvement project over 30 months covering two districts in two countries. We included all facilities providing reproductive health services - except six located on Remote Island in Uganda for economic reasons - and reached out to the majority of communities. The district managers, health providers and communities appreciated the intervention very much [19]. We observed a positive effect of a 26% and 8% increase of the intervention on the quality indicator of uterotonics within one minute of birth in Tanzania and Uganda, respectively. In both countries this was a result of changes within the facilities with strong support from the district health managers (supply side improvement, improved drug management), and more mothers coming to deliver in facilities (demand side improvement). We also observed a positive on facility delivery and availability of infection prevention items in facilities in Tanzania.
The positive effect of the intervention on AMTSL gives some hint that systemic approaches to quality improvement might be better able to yield results. In Malawi, a combined facility quality improvement and community mobilization project also indicated a positive effect [14].
Many quality improvement initiatives are evaluated through data produced by the quality improvement teams themselves, and only in the facilities where such improvement work was done. These often indicate impressive improvement [9, 20]. Maternal audits, a classical quality improvement intervention has often been evaluated using time series and case study approach [21], and few used a trial design [22, 23].
Evaluation without control group or only using data produced by the quality improvement teams, however, cannot provide robust evidence since it cannot account for general improvements in the health care system, or bias. Also, population-level effects (including in those who do not come to health facilities) cannot be estimated. In EQUIP, we aimed to provide high-quality evidence as to whether the intervention was able to produce the changes it aimed to make, by using independent population based household survey data, linked to health facility data, and carefully monitoring the context in both intervention and comparison districts.
Improvements were not observed for all improvement topics. For example, knowledge of danger signs amongst mothers with a recent birth, and immediate breastfeeding. However, these areas were not prioritised by the quality improvement teams over the period of the study. Teams were encouraged to choose their own improvement topics, rather than being bound by the pre-selected primary outcome indicators. One limitation of our approach was an inability to cover many priority areas within a short period of time.
Overall, we observed smaller improvements in Uganda than in Tanzania. Contextual factors are likely to be important for quality improvement. The population was double in size in Uganda compared to Tanzania. Unlike Tanzania, the district could not give any indication of the available budget, suggesting less financial opportunities in Uganda compared to Tanzania. Importantly District Health Teams in Uganda have very limited non-earmarked funds, while in Tanzania roughly one dollar per capita per year is available. Lack of such “fiscal space” may limit the potential of quality improvement. Also, availability of critical drugs and supplies was better in Tanzania than Uganda. As drugs and supplies are crucial not just to provide quality interventions, but also to keep health workers motivated, this could be a factor explaining the differences observed.
Potential Impact:
Impact of district health strengthening and health provider capacity building
We evaluated a comprehensive quality improvement approach and could indicate positive effects on the quality indicator of mothers having received uterotonics within 1 minute after birth. Our results also indicated some evidence of a positive effect on facility delivery and availability of infection prevention items as well as supervision in Tanzania. The positive effects observed provided some evidence that the innovative quality improvement approach applied in EQUIP can improve the quality of care within the existing resources of district health systems in low income countries.
Our approach neither included the introduction of any new technology nor financial support to human resources or drugs and supply. Our approach was based on the hypothesis that gains in implementation of essential interventions can be afforded by supporting the district health system to improve within their capacities. The approach is in line with ‘district health system strengthening’, which recently gained momentum again [16].
Our positive evidence of improvements afforded by quality improvements alone, without new investments, is in contrast to the common thinking that new technologies are needed to save more mothers and their babies. We indicate that important gains can be afforded without new technology but by working with the district managers, health providers, and communities. We were simply trying to support the implementation of interventions known to be effective and widely promoted, and thus to overcome the know-do gap. Our innovative approach to overcome the gap was successful and appreciated by the local teams.
Our results also underscore the importance of implementation research to evaluate whether interventions work or not in respective settings. To note, this goes against the common practice in research funding that overwhelmingly favors development of new technologies over implementation research; for example, the US National Institutes of Health spends 97% of its research funds on the former and 3% on the latter [24]. Yet the priority for reaching MDGs 4 and 5 is implementation research on scale-up issues in complex health systems [25, 26]. However, there are increasing calls for change [27].
Nonetheless, our results also indicate that if local recourses are too thin, and non-earmarked funding to support local quality improvement teams and bottom-up solution are too little, like in Uganda, improvements are more difficult to obtain.
We used the plausibility approach based on independent household and facility surveys to assess an intervention, which included the district as a whole with its management, health provision and consumer functions. As the funds were too small to implement the intervention and the assessment in several districts to enable a randomized design, the chosen ‘plausibility design’ was the most appropriate. Our study also indicates the usefulness using this type of evaluation to rigorously evaluate a health system intervention. The comparison between two different countries added to the strength of our evaluation. The results also underscore the need to invest in a thorough context analysis so that findings can be explained and interpreted within a certain background.
Our evaluation contrasts with the most common quality improvement evaluation approach, that quality improvements are evaluated by data produced by the teams themselves [20, 28] which has to be considered much less robust than our population-based design. However, we faced the limitation that our evaluation period covered only a few months of full implementation, which is why we might have missed changes due to the intervention.
Our evaluation also included an in-depth analysis of the mechanisms and processes in communities and health facilities. Thus our evaluation agrees with the calls to document mechanisms and contexts, while learning from incremental changes in Plan-Do-Study-Act cycles in which researchers and implementers collaborate [29]. Others even suggest integrating participatory action research into controlled trials [30].
Our process evaluation indicates that our intervention had particularly positive effects at the community level. We documented that the community felt empowered through the use of data and their own monitoring. Our intervention initiated a dialogue between the health facility and community, centered on joint efforts and joint evaluation. This could also be seen as a model for mutual accountability. We also documented a positive effect on the way the district managers perceived their task. In Tanzania, we saw improvement in supervision of lower level health facilities, and a change in the way the community was included in the yearly district planning.
The most successful change topic, consistent application of uterotonics, was the result of combined effects: a) more women delivering in facilities, b) health workers were more consistent in adhering to the recommended practice, and c) district managers assisting in the regular procurement of the necessary drug oxytocin, underscores the importance of systemic interventions to improve implementation, such as EQUIP.
Our program was less successful on a few other interventions where a similar effect was thought to be possible, such as for more consistent syphilis testing. Here the country-level supply constraints, such as irregular national purchasing and supply of screening tests, were hindering a more successful implementation. This result points to the important limitation of district quality improvement initiatives: they cannot address the national resource and coordination constraints, nor can they overcome local supply deficits – unless enabled through non-earmarked funding and policies allowing private procurement.
EQUIP also included an innovative capacity building approach by using regular mentoring and coaching rather than conventional in-service training. The World Health Report 2008, titled “Primary health care – Now more than ever”, questioned the traditional form of capacity building through training and planning and management toolkits. EQUIP indicates that a short reorientation of providers, as done during the learning sessions, on essential interventions coupled with joint brainstorming on how implementation can be improved, and mentoring and coaching during the implementation phase, can bring about improvements which are not regularly seen after short term trainings. In addition, our learning sessions were information driven, as we used the data on implementation deficits as provided by our continuous household and health facility surveys. This unique combination initiated successful implementation changes.
Thus our EQUIP approach can be seen as a model of change, where all levels of the health system - community, health provider and district health manager - work together. This work was supported by district-own data generated through the continuous health facility and household surveys and made available regularly in facilitated sessions.
Our QM approach had several positive changes on the district health system, in particular:
• The intervention encouraged local and innovative solution to problems and bottom-up knowledge and experience generation, as promoted in the World Health Report 2008 [17].
• The intervention was largely implemented as a systemic intervention addressing several levels of the health system. The systemic approach was appreciated and our results indicate that the most successful gains can be made when problems at different levels are addressed in a synergistic way.
• Our results indicated that improvements are possible through quality improvement approaches without concurrent additional training or supplies.
• We document the potential to overcome health-system constraints [31] by supporting the operationalization of known essential and effective interventions.
• The intervention improved mutual accountability for health care provision in communities and their health facilities
• Our 1-day learning sessions followed by mentoring and coaching might be an alternative to conventional in-service trainings to improve implementation of known interventions.
The positive effects were communicated through continuous policy exchange channels to policy makers and partners at the national level. Members engaged actively in the discussion of national quality improvement approaches and particularly highlighted the need to engage the community (Fatuma Manzi, keynote speaker, 3rd National Quality Improvement Forum, Dar-es-Salaam).
The film prepared in Uganda on EQUIP received large attention by policy makers in Uganda and NGOS, and the Ministry of Health plans to adopt the approach within another area.
Development to improve measurement of coverage of essential interventions
EQUIP is also contributing to the development for improved measurements of indicators of coverage along the continuum of care for mothers and their newborns. The manuscript by Ulrika Baker, titled “Implementation bottlenecks for effective coverage of maternal and newborn care in rural Tanzania”, which is currently under review by the Bulletin of the World Health Organization, provides an innovative idea of measuring the coverage of essential implementation, particularly in the neglected area of intrapartum care. We combine results from household surveys and health facility assessments from the same area to construct effective coverage as an indicator summarizing utilization of care (population utilizing the respective services) and readiness of the facilities to provide these services (availability of drugs and supplies) and implementation practices. Our analysis has indicated that effective coverage is very low, often below 25% at population level. We hope with this paper to contribute to a re-conceptualization of coverage, which has up to now mainly focused on ‘utilization’ of services (such as four times antenatal care or delivery in a health facility) without taking into consideration the large losses of effective coverage as health facilities are not equipped to provide the service and health workers might be insufficiently trained. Moreover, our approach to estimating effective coverage and identifying implementation bottlenecks provides a generic framework which may help to operationalise measurements and track progress towards universal health coverage, in all areas of health care and in any context.
Individual capacity building impact
EQUIP included the training of four PhD students, out of them one from Uganda and one from Tanzania. The PhD student in Tanzania in particular profited from being part of a larger multi-professional and multi-disciplinary team. She received support from several members of the team for her study design, write-up and presentation of results. Her award as being the best presenter on the 4th National Quality Improvement Forum in Tanzania is giving witness to her strength and also this excellent capacity building environment.
In Uganda, one member of the team finalized his MSc degree and used the EQUIP data for his thesis.
At the district level, several members of the district teams within the health sector and beyond were trained to be mentors and coaches – a rare and much needed skill set in this setting. In addition, health managers, health providers and community members increased their knowledge and skills in several areas: clinical expertise, use of data and quality improvement. More MSc students will use the data compiled by EQUIP for further analysis, and thus more capacity building will be built.
Wider social and economic implication
Besides the direct health impact of the EQUIP intervention (see results), EQUIP with its systemic approach of quality improvement can also be seen as a model for quality improvement of other health priorities. EQUIP can also be seen as a model for district health system strengthening which has lately gained support after being neglected many years [16].
EQUIP might have a wider social implication if it is further piloted or implemented by other partners. We see positive notes that our knowledge generated and the experience gained may have such an impact.
Quality improvement for better implementation of essential interventions is an investment into efficiencies and has a wider economic implication. Systems with insufficient, inefficient, or ineffective implementations are costly as they fail to do what they ought to do.
Effects on EU policy
EQUIP shows the potential and the limitations of quality improvement when applied in health services and communities in low-income settings. This has direct implications for EU policies, as the health sector is supported in low income countries, and also to high income countries in Europe, as quality improvement is increasingly used at different levels of the health system. Methods to involve communities along with health services to address issues of concern to both are much needed, not the least in the field of lifestyle and non-communicable diseases, as well as to improve health system responsiveness and equity. Here the methods developed in EQUIP offer clear, transferable lessons, which include aspects of task shifting to communities and community/patient groups, as well as for governance. There are also applicable lessons on design and methods to evaluate quality improvement interventions.
We would like to propose that a mechanism be found within the EC to facilitate learning from research projects, such as EQUIP, to other relevant parts of the commission, to facilitate cross-learning.
Dissemination and Exploitation of Results
EQUIP did include a model of continuous policy dialogue as an accompanying part of the intervention. This was enabled by regular communication with key stakeholders in the Ministry of Health in Tanzania and Uganda and active participation in steering committees and working groups. Members of the EQUIP team were included or invited to key hearings and discussions on quality improvement and district health strengthening.
We drafted information leaflets at three occasions during the project and had a web-site. We published the EQUIP protocol early on so that our overall approach was more widely known. We also had several occasions were preliminary findings were disseminated, in particular:
• The International forum for quality and safety in Health care (yearly international conference)
• The international conference of Health System Global in Beijing (2012) and Cape town (2014)
• The Global Maternal Health Conference in Arusha
• The Global maternal newborn Conference in Johannesburg
In addition we participated in the conference on “District health in Africa: ,Progress and Prospects 25 years after the Harare Declaration, Dakar, Senegal. This conference was in particular important to disseminate our experience also within a group of people engage into health planning, management and district health strengthening.
We had a close dialogue with our partners at the district level. We used platforms as available in the regions and districts and used every occasion to distribute and discuss results. Our final dissemination briefs facilitated an indepth discussion on explanation of findings at international, national and local (district) level. In addition, the EQUIP dissemination briefs were shared with our technical advisors and raised their interests, so that we believe results will more widely be discussed and might led to new projects.
Link to EQUIP resources: http://ki.se/en/phs/equip-expanded-quality-management-using-information-power-aiming-to-improve-maternal-newborn
EQUIP, an innovative intervention package for maternal and newborn health, was designed in the context of persistently high rates of maternal and newborn mortality, despite the availability of evidence-based, affordable and appropriate technical interventions. Obstacles to improved survival exist on both the demand (e.g. low utilization) and supply side (e.g. low quality and lack of services). Quality management and quality improvement are increasingly promoted to improve the quality of care. We hypothesized that quality management supported by improved data use, reaching out to communities, and including district managers in the process could improve both utilization and quality of care.
Between November 2011 and April 2014, EQUIP worked on improving utilization and quality along the continuum of care in Tandahimba district in southern Tanzania and Mayuge district in eastern Uganda. Using the collaborative approach to quality management, and working with district, health facility, and community quality improvement teams, we initiated the work by highlighting priority areas in ante-, intra-, and postpartum care during “learning session” meetings. Local teams at each level used Plan-Do-Study-Act cycles to identify and test strategies for improvement. Simultaneously, EQUIP implemented a continuous survey in the intervention districts and in one comparison district in each county. Data from these surveys were synthesized on report cards for use by the quality improvement teams, and analyzed for effect evaluation for a set of pre-defined primary indicators as well as for secondary indicators which reflected specific improvement topics.
The results indicated an improved coverage of women receiving uterotonics within one minute after birth in both countries; in Tanzania, the proportion of women who received uterotonics was 26 percentage points higher (95% CI: 25%–28%) at the end of the project period in the intervention compared to comparison district. In Uganda, the difference was smaller but still statistically significant at eight percentage-points (95% CI: 6%–9%). There was some evidence of an increase in preparation of clean birth kits for home deliveries in Tanzania, with the difference adjusted for baseline at 31% (95% CI 2%–60%) and weak evidence of more deliveries in facilities in the intervention than in the comparison district (difference adjusted for baseline 7%, 95% CI: -7% , 21). There was no evidence of an effect of the EQUIP quality improvement intervention on any other primary indicators, immediate breastfeeding, or knowledge of danger signs; however teams did not prioritize work on this.
The learning sessions every four months at the community, health facility and district levels were resource intensive. Although appreciated by the quality improvement teams, the visits were at times difficult to operationalize, as skilled mentors are rare. The teams prioritized improvement topics together with the district and EQUIP mentors, although some areas were pre-determined. We saw improvements where district health managers were strongly involved (supply side improvement), and more mothers coming to deliver in facilities and preparing for birth (demand side improvement). Our work underlines the need for systemic approaches and highlights the need to work on both the demand and supply sides simultaneously.
Our study suggests that teams in communities and health facilities can use both quality improvement techniques and locally generated data to inform decision making. Positive changes were achieved, although not for all the topics that teams worked on. Simultaneous strong involvement of the district managers seems to be the key to having any impact. The difference in response to the intervention between Tanzania and Uganda suggests that the district’s financial resources could be important for quality improvement, particularly having “non-earmarked” funds to overcome local constraints. Quality improvement needs robust support in the form of mentoring and coaching. These systems need to be sustained over time.
Project Context and Objectives:
Introduction
Maternal and newborn mortality remain unacceptably high especially in Sub-Saharan Africa. Over one million newborns and 179,000 mothers died in 2013 in Sub-Saharan Africa alone [1, 2]. This is although many effective and affordable interventions are available [3] but implementation levels vary and is in many countries insufficient [4].
In Tanzania and Uganda, many essential interventions such as active management of third stage of labour or blood pressure taking during antenatal care are not implemented at scale. Reasons are many, among them low availability of essential items in facilities [5, 6] but also substandard practices. Quality management has the potential to improve implementation levels by optimizing the existing health infrastructure rather than on the addition of human and physical resources. It includes the review of processes to improve performance, and monitoring of quality using scientific and statistical thinking with the continuous plan-do-study-act (PDSA) cycles in the centre.
The collaborative approach to quality improvement [7], as supported by the Institute of Healthcare Improvement includes in a package some aspect of training or sensitization towards standards, coaching and mentoring and learning within a team. Quality improvement is not an registered or protected package but rather include adapted support package to thrive towards improvement of quality and sustaining of an improvement change [8].
Quality improvement is increasingly used also in low resource settings to improve the quality of care of selected interventions [9] or the scale-up of new interventions [10, 11]. As such, quality improvement has many links to health system strengthening. In Ethiopia is was employed to strengthen the health extension workers, community health development agents and traditional birth attendants ability to provide maternal and newborn care [12] but with a strong component of district health system improvement [13]. In Malawi, a combined facility quality improvement and community mobilization project reduced perinatal mortality by around 20% [14]. A multicounty trial conducted in Asia, Africa and Latin Americas combining community mobilization and facility staff training but without any explicit component of district strengthening, did not show any effect on perinatal mortality [15].
The Expanded Quality Management Using Information Power (EQUIP) was developed with a view on strengthening the district health system using quality improvement approaches. The development of the quality improvement approach took a similar stand as outlined during the conference on district health systems in Senegal 2013 [16] and focused on all levels of the health care system including the community, health facility and the district level. It respond to the need to build inclusive, patient-centred health systems [17] with a continuum of care approach and spanning from community to primary facility to hospital care.
Objectives
We hypothesize that expanded QM based on high quality information, and implemented in consultation with health planners, health service providers and community will lead to measurable improvements in quality and utilization of maternal and new-born services and practices across the continuum of care, and that this intervention is feasible, affordable and relatively cost-effective for the African health systems.
Our overall goal was to assess the feasibility, cost and community effectiveness of an innovative approach of Expanded QM Using Information Power (EQUIP) that links communities and health facilities to increase the quality and utilization of health care services in order to improve maternal and new-born health in rural Uganda and Tanzania.
Specific objectives
1) To analyse the policy environment of maternal and new-born health and QM approaches as well as bottlenecks to the potential introduction of selected essential evidence-based interventions and innovative technologies
2) To collect data through continuous household and health facility surveys, the health management information system, and qualitative assessments and summarize the information on report cards for use at community, health facility and district level
3) To use this information to drive a QM strategy linking community, health facility and district/regional levels
4) To assess the feasibility, effects and community effectiveness of EQUIP
5) To analyse the cost and cost-effectiveness of EQUIP as well as the implementation cost of selected essential evidence-based strategies and innovative technologies
6) To engage policy makers and key stakeholders throughout the study in a discussion of the intervention for policy relevance and generalization to national scale
Project Results:
Methods
The study design
EQUIP used a quasi-experimental design to compare changes in maternal and newborn health outcomes and output indicators over the implementation period between one intervention district and one comparison district (Figure 1). This study design was chosen because EQUIP targeted the district level, not smaller implementation units such as wards or villages, and therefore randomization was not possible as funds were limited.
The evaluation was based on continuous household and health facility surveys, which provided estimates of changes over time between intervention and comparison districts. To assess whether other changes over time or other factors might have led to changes in indicators or in implementation, we also documented contextual factors in our evaluation process.
Evaluation methodology
Implementation of the EQUIP Continuous Survey
Continuous survey data were used for quality improvement in the intervention districts and for effect evaluation of the intervention. Over the 30 months of intervention (November 2011 to April 2014), EQUIP conducted continuous cross-sectional household and health facility surveys using independent probability samples of household clusters to represent each district each month, and repeat censuses of all government health facilities. All resident women aged 13–49 years (15–49 in Uganda) in selected households were interviewed about recent live births and use of health services (Photo 1). Geographical positioning was collected for all respondents for subsequent spatial analysis. Using repeat samples in this way allowed data to be aggregated at six four-monthly intervals to track progress over time for evaluation, and for continuous feedback to quality improvement teams in intervention districts.
Questionnaires used for the continuous survey were adapted from tools including the Safe Motherhood Needs Assessment, Demographic and health Surveys, Multiple Indicator Cluster Surveys, Service provision Assessments, and others [36-38]. A modular checklist-type questionnaire is used to assess health facilities, including staff employed, drugs, supplies and equipment and implementation of essential interventions for routine childbirth care using a ‘last event’ approach where health workers are asked to report on the care they provided during the last birth they attended. The household questionnaire has a module that includes questions on household assets, housing type, ethnic group and geographical position. Women of reproductive age (13 to 49 years in Tanzania and 15 to 49 in Uganda) are asked about knowledge and use of family planning and a pregnancy history since January 2010. Also, information on perceived quality of care is collected. Women with a live birth in the two years before the survey are asked about care received during the antenatal period, delivery and the post-partum period.
In both countries, one continuous survey team of eight people was employed to interview approximately 300 households, 10 health facilities, and complete 10 health worker interviews per district, per month. Data were collected using personal digital assistants. After every four months of data collection, routine tabulations of all indicators were produced and synthesized into report cards for use to support decision making by the EQUIP quality improvement teams. See Table 1 in the attached report for the EQUIP continuous household survey interviews completed by district.
The continuous surveys were implemented as planned. Completion of household interviews was consistently at 90% in both countries. A total of 1,415 and 2,993 interviews with women who had a live births in the last 12 months in Tanzania and Uganda, respectively, were done throughout the two and a half year of implementation. The higher number of interviews of mothers with a live births in the year prior to the survey in Uganda reflect the larger household size and the higher fertility in Uganda. Indicators across the continuum of care for mothers and newborns were assessed.
Approximately 90% of the planned health facility interviews were done. In most places an interview with a health workers who take care of the last delivery in health facility (last event) was completed to collect information on implementation practices. See Table 2 for the EQUIP health facility census and health care worker interviews completed by district.
The continuous surveys were feasible and provided high quality data throughout the EQUIP implementation period which were use for regular feed-back to the teams and for repord cards.
Contextual Documentation Methodology
To complement the information from the continuous surveys, EQUIP documented changes over time and differences between the implementation and comparison districts through:
• Monthly documentation of important events such as unrest, drought, or policy changes
• Interviews with the district management team in the implementation and comparison districts three times a year to investigate any resource or management changes, such as: breakdown of ambulances; changes in human resource availability and trainings; or existence of other donors
• Interviews with the district management teams once a year about overall health planning and implementation
• Health reports and continuous survey data
Statistical analysis
We defined priory primary indicators to measure the impact of EQUIP. These were the coverage indicators ‘Facility delivery’ and ‘breastfeeding within 1 hour’, the quality indicators ‘active Management of Third Stage of Labor”’ and knowledge indicators of ‘danger signs in pregnancy and for newborns’.
In each country, we estimated coverage estimates for utilization, life-saving interventions, quality of care and knowledge along the continuum of care in the intervention and comparison areas at four-monthly intervals during the period of intervention (November 2011 – April 2014). We use meta-regression analysis to fit a regression line through the six data points in each district, and estimated the change in outcomes over time between baseline (first data collection round) and endline (last data collection round ), comparing this change between intervention and comparison districts using difference-of-differences. We used the delta method to estimate the variance [18].
EQUIP implemented changes in response to a range of improvement topics at district, health facility and community level such as improving facility delivery and Active Management of Third Stage of Labour (AMTSL).
The Study Setting
EQUIP was implemented in two rural areas in Tanzania and Uganda over a period of 30 months (November 2011 to April 2014). The EQUIP intervention was implemented in eastern Uganda (Mayuge District) and southern Tanzania (Tandahimba District); two neighboring districts served as comparison areas (Namayingo District in Uganda and Newala District in Tanzania) (see Figure 2).
All four districts have high maternal and newborn mortality rates, are predominantly rural with small district capitals, and have populations that live on subsistence farming. In Tanzania, less than half of the population live in a house with improved roofing (iron sheeting). In Mayuge district in Uganda, 74% of houses had an iron roof whereas in the comparison district in Namayingo, the level was only 39%. See Table 3 in the attached report for the main health indicators in the intervention and comparison areas.
EQUIP’s Quality Management Approach
EQUIP had a comprehensive quality improvement approach including improvement teams at the district, health facility and community level with a view to strengthen the health system in a comprehensive manner (see Figure 3).
EQUIP’s quality improvement approach was based on the Collaborative Model for Improvement which has been promoted by the Institute for Healthcare Improvement (see Figure 4). The model is a short-term, rapid learning approach to make improvements in specific topics. Teams were guided to work using three questions:
• What are we trying to accomplish?
• What changes can we make that will result in an improvement?
• How will we know that the change is an improvement?
Teams were mentored to use Plan-Do-Study-Act (PDSA) cycles to identify problems, define a strategy that could produce change (“a change idea”), and test the strategy using locally generated data to determine if the change idea resulted in an improvement. Run-charts using the local data were used to monitor progress (Figure 5).
Maternal and newborn health topics for improvement were introduced during “learning sessions” held every three months where teams from multiple sites were encouraged to brainstorm obstacles related to each topic and to develop plausible change ideas to address these problems. The learning sessions also provided an opportunity to refresh knowledge of essential maternal and newborn health interventions, to share experiences, and to learn from each other. The learning sessions were followed by an “action period” where quality improvement teams of health providers or community participants implemented their change ideas and monitored progress. During this action period, quality improvement mentors provided coaching and mentoring to the teams.
The learning sessions and the regular mentoring and coaching visits were highly appreciated. The largest limitation was the volume of resources (human and financial) needed to assist the quality improvement teams. A dedicated and qualified team of mentors and sufficient financial support to facilitate their work were needed.
Coaching and Mentoring Through Existing Government Structures
Teams working on quality improvement need regular support to maintain their efforts and to continuously improve performance. Supervision, mentoring, and coaching is critical but often difficult to operationalize in low resource settings.
The EQUIP project employed two people in both countries to work on designing the overall approach, implementation, and research. Two additional people were involved in conducting the learning sessions and in the coaching and mentoring of the quality improvement teams. EQUIP included several members of the district health teams for the learning sessions and the mentorship of the health facility and community teams (see Figure 6).
In Uganda, two health facility mentors and two district community mentors were trained on quality improvement, and coaching and mentoring. In addition, four health facility sub-district mentors and 30 community mentors were trained to support the health facility and community teams in the sub-district.
In Tanzania, one health facility and one community mentor were trained, together with two more members of the district team engaged in implementation. The community district mentor was part of the community development office and established a structure for community work where extension workers were included in the EQUIP implementation process. Ten community extension workers were carrying out regular visits to community teams.
During the 30-month implementation period, a total of seven learning sessions were held in Uganda and Tanzania for both the health facility and community quality improvement teams. Mentorship took place eight-to-ten times per year, achieving more than 75% of the planned supervision of teams.
It was feasible to align mentoring and coaching to the district’s supervision structures, but the EQUIP project funded transport and communication allowances for the learning sessions and supervision visits to the health facility and community improvement teams. EQUIP was able to build district health team capacity for quality improvement through its own funding, but it additional resources will be needed for sustainability.
Results and experience of the quality improvement intervention
We successfully implemented a comprehensive quality improvement project over 30 months covering two districts in two countries. We included all facilities providing reproductive health services - except six located on Remote Island in Uganda for economic reasons - and reached out to the majority of communities.
In Tanzania and Uganda, mentoring and coaching sessions at health facility level were carried out on average two to three times each quarter. In Tanzania, to supervise the community level so-called cluster meetings are done, generally 2-3 times per quarter. In Uganda similarly, 2-3 coaching and mentoring visits were done per quarter. Over the period of 30 months, 7 and 6 health facility learning session and 6 and 5 community learning session were done in Tanzania and Uganda, respectively.
See Table 4 in the attached report for mentoring and coaching, learning sessions, and District QI team meeting frequencies per quarter.
Experiences on Operationalizing District-Level Quality Improvement Teams for Maternal and Newborn Health
Quality improvement in health facilities needs the support of the district managers because not all barriers for successful implementation can be tackled solely at the facility level. Issues such as stock outs of drugs, lack of human resources, or supervision of the health facilities need to be handled by the district managers. For this reason, district-level quality improvement teams made up of district health managers were formed.
We engaged the district quality improvement teams by:
o Involving them as facilitators and mentors for quality improvement teams in health facilities and at varying degree at community level
o Including them as teachers and facilitators during learning sessions
o Encouraging them to work on district level problems
The district quality improvement teams met every 3–4 months for feedback and updates. During these meetings, each team was informed about the progress of health indicators in the district, using data from the continuous surveys presented as district “report cards”. These report cards contained trends of selected indicators presented in the form of run charts and figures.
Based on these report cards, the district quality improvement team was supported by the EQUIP staff to analyse problems using the fishbone method. Change ideas were generated and tested using plan-do-study-act cycles. Most efforts at the district level were targeted at reducing drug stock-outs, improving supervision of health facilities, improving human resources provision within the facilities, and providing the necessary equipment.
In Tanzania, the improvement topic “regular provision of oxytocin” was successful in reducing stock-outs of the drug (see case study below). However, the improvement topic “supportive supervision” was less successful. Members of the district quality improvement team reported that while they were visiting health facilities together with various collaborating organisations, they had limited time to perform supervision and were unable to supervise more than one health topic at a time. Also, because there was no documentation of the purpose or content of these supervisory visits, no progress could be measured.
In Uganda, the district quality improvement team tackled the lack of human resources by the reallocation of staff within the district as well as recruitment of new staff. The frequency of drug stock-outs was reduced by improved forecasting and the timely requisition of drugs. Convening the district quality improvement teams was a challenge in both countries due to many competing activities of team members. The district QI teams appreciated the reports cards and the intense quality improvement work in the health facilities. Although they committed to performing the continuous oversight of health facility and community quality improvement activities through supportive supervision, a lack of transport, poor documentation, poor coordination, and competing activities within the district hindered their quality improvement activities.
District quality improvement in Tanzania: Supporting the availability of oxytocin
“While working on improving the implementation of active management of the third stage of labour to prevent postpartum bleeding, it became apparent that individual health facility quality improvement teams were not able to solve stock-outs of oxytocin on their own. The immediate problem was that the Medical Stores Department, the official government supplier of drugs in Tanzania, was not able to regularly provide the facilities with the amounts they had requested. Securing the cold chain for proper handling of oxytocin was also identified as a problem.
EQUIP therefore engaged the district quality improvement team to resolve this situation. During a workshop, the district quality improvement team analysed bottlenecks using the fishbone technique. As one immediate solution, the district decided to procure directly from Medical Stores Department using district-owned resources (e.g. the donor supported funding mechanism ‘basket fund’). By November 2012, a total of 100 vials had been purchased, mainly for use in the district hospital, though some were distributed to other health facilities.” (Yovitha Sedekia, EQUIP Tanzania)
District quality improvement in Uganda: Tackling lack of human resources
“EQUIP report cards revealed that one of the key problems faced by Mayuge district in Uganda was insufficient human resources for maternal and newborn health. The district quality improvement team analysed this problem using root-cause analysis and found that some health facilities had a surplus of staff while others did not have sufficiently skilled staff.
To mitigate this problem, the district quality improvement team proposed a change idea of staff reallocations within the district. In March 2012 and February 2013, surplus health workers were transferred to facilities where they were more needed. The district quality improvement team also used the report cards successfully as an advocacy tool for recruitment of more health workers. As a result, five nurses were recruited in July 2012.”(Monica Okuga, EQUIP Uganda)
Health Facility Quality Improvement for Maternal and Newborn Health
Many essential maternal and newborn health interventions, known to be highly cost-effective and recommended by WHO, are not implemented in some countries, including Tanzania and Uganda. Reasons for low implementation include providers not following examination standards or available treatment guidelines, drug stock outs, and a lack of laboratory tests. Using the collaborative approach, EQUIP aimed to assist health workers in overcoming local obstacles so that they could provide maternal and newborn health services according to national standards.
During learning sessions we introduced several improvement topics (see Box 1). Using plan-do-study-act cycles, we encouraged facility quality improvement teams to brainstorm “change ideas” to overcome local obstacles to implementation. We also decided on indicators to best monitor progress. Between learning sessions, during the “action period”, the teams implemented their change ideas and were supported through on-site mentoring and coaching by quality improvement mentors, with a focus on use of the plan-do-study-act cycles to enhance their work.
In Mayuge district in Uganda, a total of 10 trained district mentors, selected from the District Health Team and higher level health facilities, supported 30 facility-level quality improvement teams monthly in problem identification, setting goals, designing and implementing change ideas, and analysing performance using data collected by the team. In Tandahimba district in Tanzania, three trained members of the district health team together with two EQUIP staff provided mentoring and coaching at facilities roughly every six weeks to review progress and encourage further testing of change ideas.
Change ideas were developed for improvement topics (see Box 2). The learning sessions were a platform for quality improvement teams to share experiences and learn from each other. This peer-to-peer learning enhanced rapid spread of solutions and motivated teams to improve their work. The facility quality improvement teams monitored the progress of change ideas over time.
On site mentoring and coaching in combination with learning sessions and peer-to-peer learning was successful in improving the quality of maternal and newborn care at the facility level. This improvement was achieved despite key challenges such as drug stock outs and insufficient human resources for health.
Working on birth preparedness in health facilities in Uganda
“Following the introduction of EQUIP, a change idea used at many health facilities was to employ ‘goal-oriented antenatal care’, whereby what was supposed to happen on individual antenatal care visits was shared with the mothers, and dates for future appointments were fixed. Because of this change idea, an increase in regular antenatal care attendance was observed. There are now special health education sessions during antenatal care, during which the birth preparedness of women is checked. Volunteers also check these preparations in the community. Due to the success of this change idea, women are now more actively preparing for birth.” (Rogers Mandu, EQUIP Uganda)
Working on increasing health facility delivery in Tanzania
“Health facility delivery was an improvement topic implemented by all health facility quality improvement teams. A common change idea was to encourage male involvement, particularly in birth preparedness planning. This was talked about during antenatal care visits and also in the community during volunteers’ visits. In areas where traditional birth attendants were influential, facility quality improvement teams cooperated with local authorities to educate [them] about the importance of delivering in a health facility. The response to this change idea was positive in most of the areas. Most facilities also made use of community volunteers to educate women on the importance of facility delivery at the community level; this was particularly important because the volunteers were trusted by the community.
The synergistic effect of the above change ideas brought tremendous results in the health facility delivery in rural Tanzania.” (Petro Arafumin, EQUIP Tanzania)
Reducing wound infections after caesarean sections in the district hospital
“In Tandahimba district hospital in 2011, wound infections were a big problem, with nearly three quarters of all mothers developing an infection after caesarean sections. Several measures had been taken by the hospital, including relocating the operating theatre to the new maternity wing, replacing worn-out mattresses and disposing of gauze suspected to be contaminated with fungi. But it was not until the implementation of EQUIP, through the systematic testing of change ideas and the measurement of follow up effects that a true impact was made. Prophylactic antibiotics and cleaning of the abdomen and genital area before surgery were introduced to all mothers, reducing the wound infection rate to around 10 percent.
We were conducting operations [caesarean sections] and later on we used to find almost three quarters of them had wound infections and some had septicaemia and others lost life. So after they [EQUIP] came here, we sat and talked to them, they gave us ways to fight against this problem and we have been practicing that and we continue to practice until now.” (Assistant Medical Officer, Tandahimba district hospital)
Experiences of Implementing Community-Level Quality Improvement
We used QI to engage community-level stakeholders in identifying problems and implementing local strategies to address health care seeking and household-level maternal and newborn care practices. Applying the collaborative approach to quality improvement, we worked in all 157 villages in Tandahimba District, Tanzania, and all 72 parishes in Mayuge District, Uganda.
In Tanzania, starting in November 2011, two volunteers from each village in Tandahimba formed a village-level quality improvement team. Ten government-employed extension workers supported volunteers in groups of 15–18 villages. A district mentor supported volunteers and extension workers alongside EQUIP staff.
In Uganda, we conducted two-day trainings of two volunteers from each village to form a parish quality improvement team from 2011 and 2012. Each parish has 6–15 villages. A total of 72 quality improvement teams were formed. The trained volunteers were often also members of pre-existing village health teams. We trained 30 community mentors to support the 72 community quality improvement teams.
During “learning sessions”, the community quality improvement teams learned the basic principles of quality improvement and how to use “plan-do-study-act” cycles. Essential health education in maternal and newborn health topics that could feasibly be addressed at the community level was also given during these sessions (see Box 1). Report cards of indicators linked to these topics, collected from EQUIP continuous survey data, were shared with volunteers at learning sessions to underline the need for action.
Shortly after learning sessions, community quality improvement teams were mentored and coached to support their understanding of improvement topics and to ensure that plausible change ideas related to the topics were created. In Tanzania and Uganda, monthly meetings were held to review learning session content and to allow the community quality improvement teams to share experiences on various issues related to implementation.
Community quality improvement teams were able to learn and apply quality improvement techniques, but ongoing mentoring and coaching was critical to their success. In Tandahimba, throughout the EQUIP intervention, the contribution that community quality improvement teams made to the increases in health facility births and birth preparedness was noted by village leaders and community members. Major challenges included limited literacy of quality improvement team members, dropout, resistance to change ideas among traditional birth attendants or in households, and some health facilities being unable to meet increased demand for services.
It was feasible to conduct quality improvement at the community level. Community quality improvement team members were accepted in their communities and the change ideas generated by them were able to respond to local contexts. Community quality improvement also helped to increase cooperation between communities and health facility staff. However, community-level quality improvement was heavily reliant on extensive mentoring and coaching, which was resource intensive.
Community quality improvement in Uganda: Birth preparedness and health facility delivery
“Initially, when we did orientation and training of the community quality improvement team members, we taught them the steps that we go through: the problem analysis and brainstorming and developing possible change ideas. To increase birth preparedness, some suggested that money should be saved to assist mothers in need to go to the nearest health facility at the time of delivery. This was the start of women’s savings groups initiated by community members. Mothers were informed about these saving groups during home visits and at a special meeting in one village. Now there are over 45 of these groups. They work together to save small amounts throughout each woman’s pregnancy.
Women have expressed their joy and happiness about how they have been assisted. There was one mother who was brought to the health centre to deliver, but the midwife wasn’t there. She didn’t have enough money to get to another health facility, so she used the money from the savings groups to go to a higher-level facility and delivered there safely. In general, many women are able to use the money from the savings groups at the time of birth. Before, if a woman could not prepare for birth, she was afraid to go to the health facility and would instead give birth at home. The women are now comfortably and confidently going to the facility to deliver.” (Rogers Mandu, EQUIP Uganda)
Community quality improvement in Tanzania: Improved collaboration between health facilities and communities
“In one cluster, a staff member from a nearby health facility rushed to be present for a community quality improvement meeting. She wanted to convey that there had been an increase in home births recently, and she wanted to encourage the community quality improvement team to continue with their efforts. Nowadays, even those working in the health facilities see the importance of community involvement and recognise that they can make a difference in maternal and newborn health. People in the community can support each other to make sure that there is no maternal or newborn death. Now health facility staff are cooperating with the community level and seeing it as essential in order to encourage good health-seeking behaviours.” (Albert Majura, EQUIP Tanzania)
Using Report Cards for Feedback and Action
Both quality improvement and health system strengthening programs need health data as the basis for decision-making, planning, and monitoring. High quality population data on health outcomes are rarely available at the local level. The continuous household and health facility surveys implemented by EQUIP produced high quality real-time, district-specific data, which we summarised every three-to-four months for district, health facility, and community quality improvement teams using ”report cards”. We used reports card to introduce new topics for quality improvement during the learning sessions and, at the district level, to discuss progress.
Report cards were specifically adapted for three different levels of care. The community report cards included very few indicators and used a simple graphical display for ease of being understood by community quality improvement teams (see Figure 7). Content of report cards was tailored to the interests of quality improvement teams and designed to trigger action. The health facility and district report cards used bar charts and run-charts, which also allowed the indication of trends over time or levels in different parts of the district (see Figure 8).
The use of the report cards was supported by the EQUIP team. The report cards were introduced during learning sessions and indicators were explained: how they were constructed; which questions were asked of respondents; and which limitations the indicators had.
This explanation led to discussions on the reliability of many common indicators. This process also built capacity to interpret indicators for monitoring of maternal and newborn health at the district level.
The report cards were met with high interest at the community, health facility, and district levels and were able to trigger attention for neglected areas of health care. Help with interpretation of indicators was important so that improvement teams understood what the data could tell them. The capacity to interpret population- and facility-based data improved over time but required considerable facilitation by project staff.
Results / Effect Evaluation
Our analysis included 27,380 households in the two countries and 4,408 interviews with mothers with a live births in the year prior to the survey (see Figure 9).
The results indicate an increase in live births where mothers received uterotonics within one minute after birth in both countries. In Tanzania, the increase over the project period in the proportion of women with a live birth in the year prior to the survey who received AMTSL was 26 percentage points higher (95% CI 25%–28%) in the intervention district compared to the comparison district, adjusted for baseline. In Uganda, the difference was less pronounced at 8% (95% CI 6%–9%). There was some evidence of an increase in preparation of clean birth kits for home deliveries in Tanzania, with the difference adjusted for baseline at 31% (95% CI 2%–60%) (Table 5)
In Tanzania, we saw weak evidence for an increase in the percentage of deliveries (difference-in-difference 7%; 95% CI -7%–21%). We found no evidence of a positive effect for any of our other primary indicators (immediate breastfeeding, or knowledge of danger signs), however the teams did not target these areas specifically.
In Tanzania our analysis also indicated weak evidence of improvement in early postnatal care (17%; 95%CI -8%–17%) and availability of key items for infection prevention (21% difference, 95% CI -4%–46%). In Uganda we found no evidence of a positive change for the improvement topics early postnatal care and early vaccination of newborns. Table 5 in the attached report presents the effects of EQUIP on utilization coverage, quality, and knowledge of danger signs.
The intervention district in Uganda (Mayuge district) has double the population size (400 000 people) than the intervention district in Tanzania (Tandahimba, 220 000 people). The Tanzanian comparison district of Newala reported 12 USD per capita for health expenditure, compared to 7 USD in the intervention district, Tandahimba, in 2013/2014. For Uganda, such data were not available, but availability of funds at the district level is limited. Availability of human resources was similar in both districts in Tanzania, but slightly higher in the intervention district than in the comparison district in Uganda (Table 1). There was little training in the area of maternal and newborn health in both districts in Tanzania, whereas several trainings were provided in the intervention area in Uganda, supported by other partners during the EQUIP implementation period. Availability of drugs and supplies were better overall in Tanzania than in Uganda.
In both countries, health planning was based on information from the health management information system (HMIS), whereas other data from surveys or studies were not used, except in the intervention district in Uganda. The primary health care infrastructure was similar, but the comparison district in Uganda had no district hospital. Referral systems were relatively underdeveloped in Uganda. See Table 6 in the attached report for a context information summary.
Discussion
We successfully implemented a comprehensive quality improvement project over 30 months covering two districts in two countries. We included all facilities providing reproductive health services - except six located on Remote Island in Uganda for economic reasons - and reached out to the majority of communities. The district managers, health providers and communities appreciated the intervention very much [19]. We observed a positive effect of a 26% and 8% increase of the intervention on the quality indicator of uterotonics within one minute of birth in Tanzania and Uganda, respectively. In both countries this was a result of changes within the facilities with strong support from the district health managers (supply side improvement, improved drug management), and more mothers coming to deliver in facilities (demand side improvement). We also observed a positive on facility delivery and availability of infection prevention items in facilities in Tanzania.
The positive effect of the intervention on AMTSL gives some hint that systemic approaches to quality improvement might be better able to yield results. In Malawi, a combined facility quality improvement and community mobilization project also indicated a positive effect [14].
Many quality improvement initiatives are evaluated through data produced by the quality improvement teams themselves, and only in the facilities where such improvement work was done. These often indicate impressive improvement [9, 20]. Maternal audits, a classical quality improvement intervention has often been evaluated using time series and case study approach [21], and few used a trial design [22, 23].
Evaluation without control group or only using data produced by the quality improvement teams, however, cannot provide robust evidence since it cannot account for general improvements in the health care system, or bias. Also, population-level effects (including in those who do not come to health facilities) cannot be estimated. In EQUIP, we aimed to provide high-quality evidence as to whether the intervention was able to produce the changes it aimed to make, by using independent population based household survey data, linked to health facility data, and carefully monitoring the context in both intervention and comparison districts.
Improvements were not observed for all improvement topics. For example, knowledge of danger signs amongst mothers with a recent birth, and immediate breastfeeding. However, these areas were not prioritised by the quality improvement teams over the period of the study. Teams were encouraged to choose their own improvement topics, rather than being bound by the pre-selected primary outcome indicators. One limitation of our approach was an inability to cover many priority areas within a short period of time.
Overall, we observed smaller improvements in Uganda than in Tanzania. Contextual factors are likely to be important for quality improvement. The population was double in size in Uganda compared to Tanzania. Unlike Tanzania, the district could not give any indication of the available budget, suggesting less financial opportunities in Uganda compared to Tanzania. Importantly District Health Teams in Uganda have very limited non-earmarked funds, while in Tanzania roughly one dollar per capita per year is available. Lack of such “fiscal space” may limit the potential of quality improvement. Also, availability of critical drugs and supplies was better in Tanzania than Uganda. As drugs and supplies are crucial not just to provide quality interventions, but also to keep health workers motivated, this could be a factor explaining the differences observed.
Potential Impact:
Impact of district health strengthening and health provider capacity building
We evaluated a comprehensive quality improvement approach and could indicate positive effects on the quality indicator of mothers having received uterotonics within 1 minute after birth. Our results also indicated some evidence of a positive effect on facility delivery and availability of infection prevention items as well as supervision in Tanzania. The positive effects observed provided some evidence that the innovative quality improvement approach applied in EQUIP can improve the quality of care within the existing resources of district health systems in low income countries.
Our approach neither included the introduction of any new technology nor financial support to human resources or drugs and supply. Our approach was based on the hypothesis that gains in implementation of essential interventions can be afforded by supporting the district health system to improve within their capacities. The approach is in line with ‘district health system strengthening’, which recently gained momentum again [16].
Our positive evidence of improvements afforded by quality improvements alone, without new investments, is in contrast to the common thinking that new technologies are needed to save more mothers and their babies. We indicate that important gains can be afforded without new technology but by working with the district managers, health providers, and communities. We were simply trying to support the implementation of interventions known to be effective and widely promoted, and thus to overcome the know-do gap. Our innovative approach to overcome the gap was successful and appreciated by the local teams.
Our results also underscore the importance of implementation research to evaluate whether interventions work or not in respective settings. To note, this goes against the common practice in research funding that overwhelmingly favors development of new technologies over implementation research; for example, the US National Institutes of Health spends 97% of its research funds on the former and 3% on the latter [24]. Yet the priority for reaching MDGs 4 and 5 is implementation research on scale-up issues in complex health systems [25, 26]. However, there are increasing calls for change [27].
Nonetheless, our results also indicate that if local recourses are too thin, and non-earmarked funding to support local quality improvement teams and bottom-up solution are too little, like in Uganda, improvements are more difficult to obtain.
We used the plausibility approach based on independent household and facility surveys to assess an intervention, which included the district as a whole with its management, health provision and consumer functions. As the funds were too small to implement the intervention and the assessment in several districts to enable a randomized design, the chosen ‘plausibility design’ was the most appropriate. Our study also indicates the usefulness using this type of evaluation to rigorously evaluate a health system intervention. The comparison between two different countries added to the strength of our evaluation. The results also underscore the need to invest in a thorough context analysis so that findings can be explained and interpreted within a certain background.
Our evaluation contrasts with the most common quality improvement evaluation approach, that quality improvements are evaluated by data produced by the teams themselves [20, 28] which has to be considered much less robust than our population-based design. However, we faced the limitation that our evaluation period covered only a few months of full implementation, which is why we might have missed changes due to the intervention.
Our evaluation also included an in-depth analysis of the mechanisms and processes in communities and health facilities. Thus our evaluation agrees with the calls to document mechanisms and contexts, while learning from incremental changes in Plan-Do-Study-Act cycles in which researchers and implementers collaborate [29]. Others even suggest integrating participatory action research into controlled trials [30].
Our process evaluation indicates that our intervention had particularly positive effects at the community level. We documented that the community felt empowered through the use of data and their own monitoring. Our intervention initiated a dialogue between the health facility and community, centered on joint efforts and joint evaluation. This could also be seen as a model for mutual accountability. We also documented a positive effect on the way the district managers perceived their task. In Tanzania, we saw improvement in supervision of lower level health facilities, and a change in the way the community was included in the yearly district planning.
The most successful change topic, consistent application of uterotonics, was the result of combined effects: a) more women delivering in facilities, b) health workers were more consistent in adhering to the recommended practice, and c) district managers assisting in the regular procurement of the necessary drug oxytocin, underscores the importance of systemic interventions to improve implementation, such as EQUIP.
Our program was less successful on a few other interventions where a similar effect was thought to be possible, such as for more consistent syphilis testing. Here the country-level supply constraints, such as irregular national purchasing and supply of screening tests, were hindering a more successful implementation. This result points to the important limitation of district quality improvement initiatives: they cannot address the national resource and coordination constraints, nor can they overcome local supply deficits – unless enabled through non-earmarked funding and policies allowing private procurement.
EQUIP also included an innovative capacity building approach by using regular mentoring and coaching rather than conventional in-service training. The World Health Report 2008, titled “Primary health care – Now more than ever”, questioned the traditional form of capacity building through training and planning and management toolkits. EQUIP indicates that a short reorientation of providers, as done during the learning sessions, on essential interventions coupled with joint brainstorming on how implementation can be improved, and mentoring and coaching during the implementation phase, can bring about improvements which are not regularly seen after short term trainings. In addition, our learning sessions were information driven, as we used the data on implementation deficits as provided by our continuous household and health facility surveys. This unique combination initiated successful implementation changes.
Thus our EQUIP approach can be seen as a model of change, where all levels of the health system - community, health provider and district health manager - work together. This work was supported by district-own data generated through the continuous health facility and household surveys and made available regularly in facilitated sessions.
Our QM approach had several positive changes on the district health system, in particular:
• The intervention encouraged local and innovative solution to problems and bottom-up knowledge and experience generation, as promoted in the World Health Report 2008 [17].
• The intervention was largely implemented as a systemic intervention addressing several levels of the health system. The systemic approach was appreciated and our results indicate that the most successful gains can be made when problems at different levels are addressed in a synergistic way.
• Our results indicated that improvements are possible through quality improvement approaches without concurrent additional training or supplies.
• We document the potential to overcome health-system constraints [31] by supporting the operationalization of known essential and effective interventions.
• The intervention improved mutual accountability for health care provision in communities and their health facilities
• Our 1-day learning sessions followed by mentoring and coaching might be an alternative to conventional in-service trainings to improve implementation of known interventions.
The positive effects were communicated through continuous policy exchange channels to policy makers and partners at the national level. Members engaged actively in the discussion of national quality improvement approaches and particularly highlighted the need to engage the community (Fatuma Manzi, keynote speaker, 3rd National Quality Improvement Forum, Dar-es-Salaam).
The film prepared in Uganda on EQUIP received large attention by policy makers in Uganda and NGOS, and the Ministry of Health plans to adopt the approach within another area.
Development to improve measurement of coverage of essential interventions
EQUIP is also contributing to the development for improved measurements of indicators of coverage along the continuum of care for mothers and their newborns. The manuscript by Ulrika Baker, titled “Implementation bottlenecks for effective coverage of maternal and newborn care in rural Tanzania”, which is currently under review by the Bulletin of the World Health Organization, provides an innovative idea of measuring the coverage of essential implementation, particularly in the neglected area of intrapartum care. We combine results from household surveys and health facility assessments from the same area to construct effective coverage as an indicator summarizing utilization of care (population utilizing the respective services) and readiness of the facilities to provide these services (availability of drugs and supplies) and implementation practices. Our analysis has indicated that effective coverage is very low, often below 25% at population level. We hope with this paper to contribute to a re-conceptualization of coverage, which has up to now mainly focused on ‘utilization’ of services (such as four times antenatal care or delivery in a health facility) without taking into consideration the large losses of effective coverage as health facilities are not equipped to provide the service and health workers might be insufficiently trained. Moreover, our approach to estimating effective coverage and identifying implementation bottlenecks provides a generic framework which may help to operationalise measurements and track progress towards universal health coverage, in all areas of health care and in any context.
Individual capacity building impact
EQUIP included the training of four PhD students, out of them one from Uganda and one from Tanzania. The PhD student in Tanzania in particular profited from being part of a larger multi-professional and multi-disciplinary team. She received support from several members of the team for her study design, write-up and presentation of results. Her award as being the best presenter on the 4th National Quality Improvement Forum in Tanzania is giving witness to her strength and also this excellent capacity building environment.
In Uganda, one member of the team finalized his MSc degree and used the EQUIP data for his thesis.
At the district level, several members of the district teams within the health sector and beyond were trained to be mentors and coaches – a rare and much needed skill set in this setting. In addition, health managers, health providers and community members increased their knowledge and skills in several areas: clinical expertise, use of data and quality improvement. More MSc students will use the data compiled by EQUIP for further analysis, and thus more capacity building will be built.
Wider social and economic implication
Besides the direct health impact of the EQUIP intervention (see results), EQUIP with its systemic approach of quality improvement can also be seen as a model for quality improvement of other health priorities. EQUIP can also be seen as a model for district health system strengthening which has lately gained support after being neglected many years [16].
EQUIP might have a wider social implication if it is further piloted or implemented by other partners. We see positive notes that our knowledge generated and the experience gained may have such an impact.
Quality improvement for better implementation of essential interventions is an investment into efficiencies and has a wider economic implication. Systems with insufficient, inefficient, or ineffective implementations are costly as they fail to do what they ought to do.
Effects on EU policy
EQUIP shows the potential and the limitations of quality improvement when applied in health services and communities in low-income settings. This has direct implications for EU policies, as the health sector is supported in low income countries, and also to high income countries in Europe, as quality improvement is increasingly used at different levels of the health system. Methods to involve communities along with health services to address issues of concern to both are much needed, not the least in the field of lifestyle and non-communicable diseases, as well as to improve health system responsiveness and equity. Here the methods developed in EQUIP offer clear, transferable lessons, which include aspects of task shifting to communities and community/patient groups, as well as for governance. There are also applicable lessons on design and methods to evaluate quality improvement interventions.
We would like to propose that a mechanism be found within the EC to facilitate learning from research projects, such as EQUIP, to other relevant parts of the commission, to facilitate cross-learning.
Dissemination and Exploitation of Results
EQUIP did include a model of continuous policy dialogue as an accompanying part of the intervention. This was enabled by regular communication with key stakeholders in the Ministry of Health in Tanzania and Uganda and active participation in steering committees and working groups. Members of the EQUIP team were included or invited to key hearings and discussions on quality improvement and district health strengthening.
We drafted information leaflets at three occasions during the project and had a web-site. We published the EQUIP protocol early on so that our overall approach was more widely known. We also had several occasions were preliminary findings were disseminated, in particular:
• The International forum for quality and safety in Health care (yearly international conference)
• The international conference of Health System Global in Beijing (2012) and Cape town (2014)
• The Global Maternal Health Conference in Arusha
• The Global maternal newborn Conference in Johannesburg
In addition we participated in the conference on “District health in Africa: ,Progress and Prospects 25 years after the Harare Declaration, Dakar, Senegal. This conference was in particular important to disseminate our experience also within a group of people engage into health planning, management and district health strengthening.
We had a close dialogue with our partners at the district level. We used platforms as available in the regions and districts and used every occasion to distribute and discuss results. Our final dissemination briefs facilitated an indepth discussion on explanation of findings at international, national and local (district) level. In addition, the EQUIP dissemination briefs were shared with our technical advisors and raised their interests, so that we believe results will more widely be discussed and might led to new projects.
Link to EQUIP resources: http://ki.se/en/phs/equip-expanded-quality-management-using-information-power-aiming-to-improve-maternal-newborn
