Improved performance of district health systems through implementing health center clinical and administrative standards in the Amhara region of Ethiopia

Background Standards represent benchmarks against which improvements can be measured. In 2016, the Federal Ministry of Health developed and endorsed a set of standards entitled the Ethiopian Health Center Reform Implementation Guidelines (EHCRIGs). This study aims to assess the effects of planned interventions on performance and quality of services in primary health care units (PHCUs). Methods A quasi-experimental pre-post study design was used to compare the performance of PHCUs against the ECHRIG standards before and after its implementation in 76 woredas of the Amhara region from July – December 2017. Pre and post-intervention validation measurements of performance improvement in 76 woredas were conducted. The data were entered and analyzed using Statistical Package for Social Sciences. Result For this study, ten sessions of two-day trainings were conducted for 1306 staff. Additionally, on-site mentoring and coaching was conducted for a period of three months. The average EHCRIG standards met before and after intervention totaled 59 and 66%, respectively. The scores showed a positive correlation coefficient (r = 0.74) with statistical significant paired sample t-test with t = − 7.15, df = 75, P < 0.000. In addition, the mean scores among performance tiers were 74, 66 and 65% for high, medium and low performing woredas respectively. The One-way Analysis of Variance (ANOVA) showed a borderline significant difference between groups with (F = 2.4, P < 0.09). Conclusions The implementation of the standards has garnered significant improvements in performance at the primary health care level. Therefore, continuing the initiated strengthening of health system performance against the standards and evaluation of the quality of primary health services is recommended. Electronic supplementary material The online version of this article (10.1186/s12913-019-3939-y) contains supplementary material, which is available to authorized users.


Background
The World Health Organization (WHO 2007) defines health systems as: "The sum total of all the organizations, institutions and resources whose primary purpose is to improve health." [1] These entities need to provide services that are responsive and financially fair while treating people decently. Globally, limited available resources and increasing healthcare expenditure with high rates of service demand have amplified the need for institutionalizing innovative performance standards, performance measurements and performance improvements. Reporting progresses have also become key elements of ensuring accountability through exercising planning, contracting, directing, implementing, monitoring and evaluating activities [2,3]. Like many developing countries, Ethiopia strives to strengthen its health systems through implementing several initiatives and interventions of WHO's recommended six building blocks, namely: Service Delivery; Health Workforce; Information; Medical Products, Vaccines, and Technologies; Financing; and Leadership and Governance [1].
In the last two decades, the Government of Ethiopia has made significant gains in improving the health status of its citizens. The life expectancy of its citizens improved from 45 years in 1990 to 64 years in 2014; maternal mortality ratio has declined from 1400 deaths per 100,000 live births to 351 in 2016 and under-five mortality rate has declined by 67%, to 68 deaths per 1000 births [4]. These results were achieved as a result of strong commitment from the Ethiopian government as well as technical and other resource support from development partners. Hence, the Ministry is in a favorable position to create demand and to expand access to effective interventions to improve maternal, newborn and child health outcomes.
Despite these gains, the government recognizes the need to maintain progress and tackle the high maternal and neonatal mortality rates. In addition, it works to narrow the variability among regions, districts (woredas) and primary health care facilities. As providing quality health services in an equitable manner is a major goal, developing clinical and administrative standards are a key element of health reform interventions.
Professional Standards Review Council (1974) as cited in Donabedian (1981) defines standards as: "Professionally developed expressions of the range of acceptable variation from a norm or criterion." In addition, in medical care, norms are numerical or statistical measures of observed performance, and criteria are predetermined elements against which aspects of the quality of medical service may be compared [5]. A practice standard is a statement that defines the performance expectations, structures, or processes that must be in place for an organization to provide safe and high-quality care, treatment, and services [5]. In 2016, the Federal Ministry of Health (FMOH) of Ethiopia developed and endorsed minimum clinical and administrative performance standards called the Ethiopian Health Center Reform Implementation Guidelines (EHCRIGs). These standards are expected to be adhered to by all primary health care units [6].
According to Armstrong (2009), performance management is defined as: "A systematic process for improving organizational performance by developing the performance of individuals and teams. It is a means of getting better results by understanding and managing performance within an agreed framework of planned goals, standards and competency requirements." [7] In addition, Weiss and Hartle (1997) as cited in Armstrong (2009) define, performance management as: "A process for establishing a shared understanding about what is to be achieved and how it is to be achieved and an approach to managing people that increases the probability of achieving success." [7] Furthermore, performance improvement and quality improvement are addressed using data collected from probing: "How can one ensure whether the changes bring the desired improvement?". Reporting progress deals with "Communicating results to broader audiences". Therefore, performance management involves a systematic and structured problem-solving approach and demands a conducive environment that encompasses transparency, visible leadership, customer focus, strategic alignment and culture of quality [3].
Through the Health Sector Transformation Plan (2015-2020), the Ethiopian government's Ministry of Health aims to significantly improve access and quality of universal health service coverage through implementing several sector-wide reforms, and innovative performance management and improvement interventions [4]. Hence, a range of health reforms were undertaken to cultivate a culture of working against standards and enhance team work among the health workforce in Ethiopia. The Federal Ministry of Health and Regional Health Bureaus believe that compliance with standards like EHCRIGs will contribute to improving the effectiveness and empowerment of health workforces and streamlining decision making [6]. Therefore, the initiative helps to ensure that health service managers carry out their roles and responsibilities through improving their effectiveness and assists performance review teams to guide primary health care facility staff to become more efficient in their roles.
Institutionalizing clinical and administrative guidelines USAID Transform: Primary Health Care is a five-year project (January 2017 -December 2021), funded by United States Agency for International Development (USAID) [8]. The project is a successor of an eight year project, Integrated Family Health Program (IFHP) (2008-2016) which had a very large geographic reach and improved the lives of millions of women, men and young people throughout Ethiopia through implementing interventions in 301 woredas located in Amhara; Oromia; Southern Nations, Nationalities and Peoples (SNNP) and Tigray regions as well as a few woredas in Benshangul Gumuz and Somali Regions [9]. Based on selected key maternal and child health indicators, 40 (13%), 95 (32%) and 165 (55%) of targeted districts were high, medium and low performing, respectively [8]. USAID Transform: Primary Health Care project provides technical and financial resource support towards the implementation of the Health Sector Transformation Strategic Plan (HSTPII) in four regional states of Ethiopia, namely; Amhara, Oromia, Southern Nations Nationalities and Peoples (SNNP), and Tigray. Moreover, within these regions, there are over 300 districts (woredas) receiving technical, financial and other resource support from the project [4,8].
The 'theory of change' in USAID transform: Primary health care project interventions As a pathway to the agreed upon global targets for 2030 and beyond, the USAID Transform: Primary Health Care project, was developed to contribute towards the successful achievement of the national goal of preventing child and maternal deaths (PCMD). In addition, its long-term outcomes (LTO) are increased utilization of quality family planning and reproductive health, maternal newborn health, child health and development, adolescent and youth health and development and nutrition (RMNCAH-N) services under a well-functioning primary health care system [4].
In order to achieve the above goal, the project hypothesized four major preconditions and assumptions: (1) Successful achievement in creating a conducive work environment, improving resource mobilization, allocation, and utilization; and working towards high performing woredas will improve management and performance of health systems. (2) Enhanced the technical competence of health providers and enhancement of their skills will lead to increased sustainable quality of services, which is managed by well-functioning primary health care level systems. (3) Reduced inappropriate health practices will improve household and community health practices and health-seeking behaviors. (4) Program learning institutionalized within the public health sector will enhance program learning to impact policy and programs related to preventing child and maternal deaths.
The theory of change is presented below using illustrations of the causal pathways ( Fig. 1).
To strengthen health systems, from July to December 2017, USAID Transform: Primary Health Care project implemented performance management and improvement interventions in 76 (42%) districts in the Amhara region. A range of measures were taken to ensure that the district's health system managers execute identified minimum standards effectively. These include; provision of training on the standards, self-assessment and validation of measurements, performance improvement, and reporting progress, coaching, observing, providing feedback and developing agreed action plans. The aim of this study was to document the effects of implementing clinical and administrative standards set in the Ethiopian Health Center Reform Guidelines on the performance of district (woreda) health systems. Therefore, this article highlights performance improvements using clinical and administrative standards as a result of USAID Transform: Primary Health Care project's pre and post-intervention measurements of the Ethiopian Health Center Reform Implementation in the Amhara region of Ethiopia.

Study site
Ethiopia is located in the horn of Africa. The land area is estimated to be about 1.1million square kilometers [10]. Amhara region has a land mass of 154,709 km 2 and Fig. 1 A diagrammatic presentation of the causal pathways [4]. The figure illustrates the theory of change with, Long term outcome, Long term outcomes, preconditions, and assumptions lies within the geographic coordinate 11.3494°N, 37. 9785°E . It has ten administrative zones and 181 districts (woredas). This study was conducted in purposively selected 76 (42%) districts, of which 10 were high performing, 23 were medium performing and 43 were low performing woredas (Fig. 2). USAID Transform: Primary Health Care project provides demand-driven technical, financial and resource support to strengthen the six building blocks of the health system. In addition, there are integrated service deliveries implemented in the areas of health systems strengthening, family planning and reproductive health, maternal newborn health, obstetric fistula, child health and development, expanded immunization program, nutrition, adolescent and youth health and development, malaria, gender, community-based health insurance and social and behavioral change communication.

Health tier system calibers
The Ethiopian health tier system has three levels of healthcare delivery systems, namely: primary, secondary and tertiary level of healthcare organizations. As depicted in the Fig. 3 below, level one is district (woreda) health systems comprised of primary hospitals which cover 60,000-100,000 people, health centers serving 15,000-25,000 people and their satellite health posts covering 3000-5000 people connected to each other through a referral system. According to the Ethiopian Service Standards [11] health center shall mean a health facility at the primary level of the healthcare system which provides promotive, preventive, curative and rehabilitative outpatient care including basic laboratory and pharmacy services with the capacity of 10 beds for emergency and delivery services. The primary hospitals, health centers and health posts form primary health care units (PHCUs). At level two are general hospitals covering a population of 1-1.5 million people, and at level three are specialized hospitals covering a population of 3.5-5 million people [12].

Performance tier
Based on routine health management information system reports, twelve maternal and child health-related service indicators were selected and collated. A group of experts which consisted of the Federal Ministry of Health, USAID and IFHP rated achievements in accordance with the indicators with scores out of 100%. The key indicators included; early institutional neonatal mortality rate, proportion of surviving infants vaccinated for Penta-3, proportion of people tested positive for malaria: all ages, proportion of people tested positive for malaria: < 5 yrs., proportion of children with moderate malnutrition, proportion of children with severe malnutrition, institutional maternal death rate, proportion of pregnant women that received antenatal care: at least four visits, proportion of births attended by a skilled health worker, early postnatal care coverage, proportion of pregnant women tested for HIV, and contraceptive acceptance rate. Following scoring based on these criteria, the top one-third are categorized as high performer woredas, the middle one-third as medium performing woredas and the lower one-third are categorized as low performing woredas [8].

Study design
A quasi-experimental pre-post design study was conducted among 76 districts of the Amhara region with technical and resources support being provided by

Performance management and improvement intervention
The Amhara Regional Health Bureau provided training of trainers and validation of team members trainings from 20th to 22nd July 2017 in Bahir Dar City, Ethiopia. The trainings were then cascaded to all ten zones of the region in August 2017 (Additional file 1, Fig. 4). In each district (woreda), twenty-five staff members of health offices and health centers were trained on the standards of EHCRIGs, systematic problem-solving tools i.e. fishbone analysis and the 'five why' techniques, provision of support supervision and observation and validation techniques. In addition, each district selected one lead person and four members in primary health care facilities to report progress and organize learning collaboratives (i.e. quarterly organized field visits and performance review meetings). Furthermore, USAID Transform: Primary Health Care project staff and experts of zone health departments received the necessary orientation in implementing the standards in selected districts. Validation team members were deployed from September 1-25, 2017, to take preintervention measurements. Two sessions of onsite coaching were conducted on a quarterly basis. During facility visits, the teams inquired about the achievement of each health facility and ask a series of questions to get a holistic picture of the processes adapted to achieve the set standards. Through the discussions with primary health care facility teams, the coaches identified gaps and assisted the teams to strategically analyze the problems. In addition, the coaches provided feedback on performances and gave technical updates and information on guidelines deemed helpful to improve skill gaps. Following that, both the coaches and primary health care performance review (management) teams developed an agreed action plan on identified, prioritized solutions. The coaches then submitted a copy of agreed measurements reports, identified gaps and developed action plans to district health offices and the zone health department. Six months after the intervention which took place from 1 to 20 , April 2018, the validation team collected post intervention measurements.

Sampling
All 76 USAID Transform: Primary Health Care project intervention woredas were purposively selected.

Data collection methods and data quality
Nationally developed and validated instruments and the Ethiopian Health Center Reform Implementation Guideline's tools were adapted for this study. The pre-post interventions data collection tools have ten chapters and 81 standards with composite validation indicators. The tools were used to assess the availability of essential resources and services. The checklist used at primary healthcare facilities consists of ten chapters:   [12]. Depicted the Ethiopian Health Tier System, Where Health Centres are located at the primary level health care category Improvement and Health Information System. Detailed description is presented in Table 1 below.
Data were collected using structured questionnaires. The questionnaires were prepared in Amharic, the local and official national language. In each facility, two to five health professionals were trained on the standards, measurements, performance improvements and reporting progress tools. In addition, the validation teams recruited from zone health departments, project and partner staff provided trainings on the standards, criteria, observation techniques, ethical consideration as well as providing technical support to facility members.

Data analysis
Standards measurement reports were submitted by trained health professionals working in health centers and districts (woreda) health offices. Data on average scores for district health offices were reviewed and data of district health offices measured on two occasions were included. For each district, data on performance across the ten chapters of EHCRIGs were entered in Microsoft Office Excel Spreadsheet (2010), then exported to Statistical Package for Social Science Research (SPSS IBM Version 20) for analysis [14]. The data collected from district health offices were cleaned and checked for consistencies then described using frequencies, proportion, and percentages and were presented using charts and tables. Based on the proportion of standards met, scores out of a hundred were used to categorize districts into three performance levels i.e. ideal best performing primary health care facility which met greater or equal to 80% of standards; mid performing primary health care facility for scores ranging from 60 to 79.9% and low performing primary health care facility for scores below 60% of the standards [6]. In addition, to determine the linear relationship between pre and post intervention scores, the Pearson Product-Moment Correlation technique was employed; then a paired t-test was employed to estimate the mean score difference in district health office performances between the first and the second measurement scores. To check for statistical difference in performance tier of targeted districts, the test used in this study was one-way analysis of variance (ANOVA). A statistically significant relationship was claimed at P < 0.05.

Ethical clearance
Ethical clearance was obtained from the Amhara National Regional State Health Bureau Research and Ethics Committee. Permission to use the data was sought and obtained from the USAID Transform: Primary Health Care project. In this study, no facility identifier information was collected. The summary of this study was synthesized and submitted to the regional health bureau on two occasions. Moreover, the district health system managers were encouraged to use the evidence for performance management and improvement.

Distribution of districts with primary health care units
This study analyzed the pre-post intervention validation measurement in EHCRIGs standards in 76 woredas of the Amhara region. As is indicated in the table below, within ten administrative zones of Amhara region, the number of districts that received direct support from USAID Transform: Primary Health Care project range from the least 2 to the highest 14. On average, the number of functional primary health care units were 5.8 ± (SD) 2.3 ( Table 2).

Socio-demographic background of trained health professionals
As depicted in the table above, the implementation of the EHCRIGs was started with the provision of training for 1306 health professionals. Among these, 207 (15.9%) were female. With regards to the professional mix of the trainees, the largest 395 (30.2%) were health information technicians; followed by diploma nurses 344 (26.4%) and 5 (0.4%) possessed a BSc in Midwifery (Table 3).

Changes in EHCRIG standard performances
The validation pre-post intervention measurement showed the proportion of met standards increased by 6% in all chapters. The majority of district health systems scored very close to the threshold (80%) in four chapters of the categorization of high performing primary health care facilities. During the intervention period, there were improvements in the average of standards met such as: (73 to 77%) in leadership and governance, (73 to 77%) in health center-health post linkage; (69 to 75%) in clean, and safe health facility and (68 to 75%) in quality improvement and health information systems chapters. Conversely, the performance scores in three chapters were close to 50%; namely: (41 to48%) in patient flow and service organization; (45 to 53%) in laboratory services; and (45 to 51%) in medical equipment management and biomedical engineering chapters (Fig. 4).

Changes in district health system performance
The district health systems improved their performance as a result of USAID Transform: Primary Health Care project's technical and resource support interventions. The majority of the districts scored medium at the second validation measurement with a mean score of 66% and a range between 36 to 90% compared to the first quarter measurement scores of 59% mean and range of 35 to 89%. The difference between the first and second measurements scores was 7% and was statistically significant (t = − 7.15, df = 75, P = 0.000). However, five districts showed a decline in performance scores met against standards. Table 4 Summarizes the district health system performance scores across the first and second validation scores. As presented, the result of descriptive statistics below (Table 5), the mean scores with Standard Deviations (SD) were 74.3 ± 10.7; 66.0 ± 12.0; and 66.6 ± 12.1% of EHCRIGs which were documented for high; medium and low performing districts (woredas), respectively. Resultts revealed that there were high performing districts with the minimum score of 60.0% and a maximum score of 90% and low performing districts with a minimum score of 36% and followed by 89% for maximum scores.
As the results show in the table below (Table 6), one-way ANOVA was employed to test mean differences of EHCRIGs scores and district (woreda) performance tiers. The results revealed that EHCRIGs mean scores have a borderline statistical significance relationship with district performance tiers (F(2, 73) = 2.4, P = 0.09).

EHCRIG performance categorization zones by chapters using scorecard
Based on the national criteria to categorize district health systems using EHCRIGs scores, districts scored greater or equal to 80, 60 to 79.9% and less than 60% were classified   (Table 7), and the overall performance scores ranged from 76% in Wag-Himira to 63% in the Oromia zone.

Discussion
This pre-post intervention survey highlighted the importance of performance improvement through implementing clinical and administrative standards to strengthen district health systems. The data were collected in 76 woredas within the Amhara region of Ethiopia. All sample primary health care facilities and district health systems submitted pre and post intervention measurements. The response rate was 100%. The high response rate could be due to an intensive involvement of the public health sector, development partners and other relevant stakeholders. In addition, the direct technical and resource support from USAID Transform: Primary Health Care project, zone health departments, and other development partners consisting of orientations on standards, performance measurements, performance improvement through problem-solving, and action plan development and organizing reporting on progress is likely to positively influence participation and response rates. Furthermore, the leadership of Amhara regional state's health bureau took the initiative to exercise the standards and use the performance measurement scores for recognition and rewards using semi-annual, result presentation workshops. The main purpose of the implementation of the health center reform standards was to improve performance and quality of care at the district health system level using ten chapters and 81 standards of the EHCRIGs [6]. Generally, the results of this study revealed a positive correlation, coefficient with statistically significant differences in mean performance scores between pre and post intervention stages. These significant performance improvements were achieved through the execution of several interventions within six months.
The project provided tailored technical, financial and other resource support which enabled the district health system managers to create a conducive work environment for health workforces. In addition, through the involvement of stakeholders, the project enhanced capacity in use of systematic problem-solving tools, developing agreed action plans, and ensured appropriate measures and actions were taken. This approach improved the culture of working in teams, created shared visions and understanding of the mission of organizations, helped in mobilizing and aligning the necessary resources as well as implementing and following up with monitoring the performance of district health systems.
With the support of the project, the technical competency of health care providers was enhanced through coaching and arranging several experiences sharing and learning collaborative workshops. Trained coaches and validation team members facilitated meetings and strived   [15]. Furthermore, the project strived to institutionalize social accountability and implemented health system strengthening initiatives so that the community would receive safe and quality health services at the primary health care level. In addition, the technical support of the project ensured the presence of guideline tools to reduce inappropriate health practices. Regarding program learning initiatives, USAID Transform: Primary Health Care project enhanced the capacity of district health system managers to facilitate review meetings and document case studies through the use of zone health department and project staff. Prior to support from the project, performance review meetings were not effective and efficient. Following project's efforts to change this, the number of review meeting participants ranged from 100 to 150 with attendants comprised of kebele managers, health extension workers as well as health center and district health office staff. Each health facility attending the meetings presents plans versus achievement using over 122 indicators. Moreover, the discussion sessions address a range of issues including staff training and development as well as transfer. Subsequent meetings are arranged at the end of each review meeting prior to forming action plans. Through USAID Transform: Primary Health Care project's guidance, the Ethiopian Primary Health Care Alliance for Quality (EPAQ) was adhered to ensuring that the number of participants was limited to 25 to 30 [16]. The participants include managers who are directors of the health center, board chair and health center-health post linkage focal person who are responsible for scanning, focusing, planning, organizing, aligning and mobilizing, implementing, monitoring and evaluating health system strengthening activities. Agenda of meetings consist of 5 chapters of woreda (district) Management Standards; 10 chapters of the EHCRIGs; 29 to 37 selected Key Performance Indicators (KPIs) and five change package achievements. In addition, experience sharing, and validation of reports are arranged among staff of lead and member facilities. Finally, action plans were developed on identified priority activities with a responsible person, time and resources allotted.
This new approach helps health managers to improve their performance against the standards within the district health system. Such purposeful review meetings and access to facilitative supervisors documented elsewhere were shown to improve achievements of health facilities in low and middle-income countries [17][18][19][20][21][22][23][24].
The majority of district health system performance management scores were significantly improved in four chapters, i.e. Leadership and Governance; Health Center-Health Post linkage; Clean, and Safe Health Facility; and Quality Improvement and Health Information Systems. This result could be due to the technical and financial support provided to successfully implement change packages [16,25]. Despite all the technical and on-site support, the scores for three chapters remained low i.e. Patient Flow and Service Organization, Laboratory Services and Biomedical Engineering standards. The low scores could be due to these chapters and standards demanding a high level of technical skills. In addition, lack of properly trained human resources in the primary health care facilities could negatively affect the performance of district health systems. Furthermore, to successfully meet these standards, primary health facilities should be fulfilling the structural requirement [26] such as adequate spaces for counseling services, triage and storing which may require a significant budget. The gains in these chapters after six months of interventions ranged from 4 to 7%. This could be due to that some standards demand recruitment of staff and renovation of health facilities which can be costly as well as limited technical skills available at primary health care facilities.

Limitations
This quasi experimental study had several limitations. The first limitation was that it only measured results after six months of interventions which means the long-term effects of the applied standards were not observed. In addition, the study did identify predictor variables. Finally, the opinions of staff on compliance with the standards were not included.

Conclusions
The implementation of performance management improvement through implementing the Ethiopian Health Center Reform Implementation Guideline Standards and its tools significantly improved the performance of district health systems in the Amhara region of Ethiopia. Scaling