Skip to main content

Protocol for the evaluation of a complex intervention aiming at increased utilisation of primary child health services in Ethiopia: a before and after study in intervention and comparison areas

Abstract

Background

By expanding primary health care services, Ethiopia has reduced under-five mor4tality. Utilisation of these services is still low, and concerted efforts are needed for continued improvements in newborn and child survival. “Optimizing the Health Extension Program” is a complex intervention based on a logic framework developed from an analysis of barriers to the utilisation of primary child health services. This intervention includes innovative components to engage the community, strengthen the capacity of primary health care workers, and reinforce the local ownership and accountability of the primary child health services. This paper presents a protocol for the process and outcome evaluation, using a pragmatic trial design including before-and-after assessments in both intervention and comparison areas across four Ethiopian regions. The study has an integrated research capacity building initiative, including ten Ph.D. students recruited from Ethiopian Regional Health Bureaus and universities.

Methods

Baseline and endline surveys 2 years apart include household, facility, health worker, and district health office modules in intervention and comparison areas across Amhara, Southern Nations Nationalities and Peoples, Oromia, and Tigray regions. The effectiveness of the intervention on the seeking and receiving of appropriate care will be estimated by difference-in-differences analysis, adjusting for clustering and for relevant confounders. The process evaluation follows the guidelines of the UK Medical Research Council. The implementation is monitored using data that we anticipate will be used to describe the fidelity, reach, dose, contextual factors and cost. The participating Ph.D. students plan to perform in-depth analyses on different topics including equity, referral, newborn care practices, quality-of-care, geographic differences, and other process evaluation components.

Discussion

This protocol describes an evaluation of a complex intervention that aims at increased utilisation of primary and child health services. This unique collaborative effort includes key stakeholders from the Ethiopian health system, the implementing non-governmental organisations and universities, and combines state-of-the art effectiveness estimates and process evaluation with capacity building. The lessons learned from the project will inform efforts to engage communities and increase utilisation of care for children in other parts of Ethiopia and beyond.

Trial registration

Current Controlled Trials ISRCTN12040912, retrospectively registered on 19 December, 2017.

Peer Review reports

Background

Ethiopia reached the Millennium Development Goal 4, reflected in a reduction of the under-five mortality from 205 deaths per 1000 live births in 1990 to 64 in 2014 [1]. Neonatal mortality also decreased from 55 to 28 deaths per 1000 live births in the same period. The expansion of primary care services, improvements in nutrition [2], and progress across other sectors of society have reportedly contributed to reaching this goal. An analysis based on the Global Burden of Disease study reported neonatal conditions, together with lower respiratory tract infection and diarrhoeal diseases, as the dominant causes of under-five death [3].

As part of the Health Extension Program, in 2003 the Ethiopian Government introduced a new cadre of primary care workers called Health Extension Workers (HEWs) to reinforce efforts to improve maternal, newborn and child health in Ethiopia [4]. This category of health workers is reportedly able to correctly manage multiple child illnesses through the integrated community case management (iCCM) activities that the government initiated in 2010 [5]. If appropriately trained and supported, this cadre can also treat severe infections of the newborn in a cost-effective way within the Community-Based Newborn Care (CBNC) program that has been running since 2014 [6]. Although nearly all the HEWs throughout the country have been trained, relatively few sick newborns have been identified and treated [7]. Since 2011 a volunteer cadre called the Women’s Development Army (WDA-also known as the Health Development Army or Women’s Development Group) is active in promoting the use of health services [8, 9]. There are two levels to the WDA leaders. The smaller unit is comprised of 6 women, with one serving as a leader. Five or six of these networks are combined to form a group. The approximately 30 women in one group are led by one of the network leaders. The WDA leaders work closely with HEWs to promote maternal and child health services.

In Ethiopia, the utilisation of maternal health services shows disparities between regions and social groups [10, 11], and skilled birth attendance, although increasing, has remained inadequate with increasing social inequity [12]. Insufficient number of sick children are taken to HEWs for treatment of common childhood diseases, and the expansion of the iCCM program has not created sufficient community demand for use of these services [13].

The low utilisation of child health care calls for concerted efforts to improve the equitable reach of high-quality services [14, 15]. This background formed the rationale for developing a complex intervention to engage the community, strengthen the capacity of primary care workers to provide high-quality services and reinforce the local ownership and accountability of the primary level maternal and child health services. It was hypothesized that, given the low service utilization, increased awareness and promotion of primary health services in combination with strengthened provision of iCCM and CBNC and improved ownership and accountability of these services would lead to increased service use, which ultimately would further reduce neonatal and under-five mortality.

Thus, the evaluation described in this protocol aims to assess whether the proportion of children under the age of 5 years with suspected pneumonia, diarrhoea, fever or neonatal sepsis, that seek and receive appropriate care has increased in intervention as compared to comparison areas. Further, we present a plan for whether the information on the fidelity, reach, and dose of the interventions support the plausibility of changes in the outcome, and provide insights into the pathways of effects, feasibility, and program cost-effectiveness. Findings of this evaluation will be used to improve existing platforms of service delivery in the country. It is anticipated that successful interventions will be scaled up nationally provided sufficient budget support is available.

Design and methods

Setting

The Federal Ministry of Health in collaboration with the non-governmental partners UNICEF, Last 10 Kilometres/John Snow, Inc., Save the Children, and Program for Appropriate Technology in Health (PATH) initiated the “Optimizing the Health Extension Program” project to increase the utilisation of primary child health services. This evaluation has a pragmatic trial design with purposefully selected 26 intervention and 26 comparison districts (woredas), with a total population of 8 million, across four regions of Ethiopia (Amhara, Southern Nations Nationalities and People, Oromia, and Tigray). Figure 1 shows the intervention and comparison areas within these four regions.

Fig. 1
figure 1

Invervention and comparsion areas for the evaluation of Optimzting the Halth Extention Program intevention, Ethiopia. Produced by the authors using ArcGIS 10

The intervention areas were selected by government and implementing partners for having a relatively low utilization of primary child health services. The Regional Health Bureaus in these regions, with the support from their local universities, selected the comparison districts to match the intervention districts. Selection was based on demographic and health criteria that included population size, number of primary health care units, burden of diseases, health service performance data, length of time since iCCM and CBNC program initiation, prior exposure to other similar programs, and absence of non-governmental organizations addressing demand generation. The intervention, which started in 2016, has an intended duration of 2.5 years and is based on an analysis of barriers to the utilisation of newborn, child and maternal health services. The planned evaluation follows a plausibility approach [16]. It includes analysis of difference in differences of outcomes and a process evaluation of the intervention in line with the UK Medical Research Council’s guidelines [17].

The baseline and endline surveys, as well as the process evaluation are implemented by the London School of Hygiene & Tropical Medicine (LSHTM) and Ethiopian Public Health Institute (EPHI) along with representatives from Gondar, Jimma, Mekelle and Hawassa Universities. A steering committee comprising representatives from each of the universities, implementing partners, Ethiopian Public Health Institute and Federal Ministry of Health was established to meet quarterly. The committee provides advice on the evaluation of the project and assists in resolving issues encountered during the course of the evaluation. Given that the Optimising the Health Extension Program was a community and health system level intervention, a data monitoring committee was not deemed necessary.

Barrier analysis

Key Ethiopian governmental and non-governmental stakeholders in the field of maternal, newborn and child health services met in 2016 for a facilitated workshop where the perceived demand- and supply-side barriers to CBNC and iCCM service utilisation were identified. The demand-side barriers included perceived lack of knowledge of diseases and danger signs [18], and lack of awareness of what primary level services could offer. Further, it was suggested that families often have a preference for traditional healers and home remedies [19] and that the higher availability of services offered by private providers was also appreciated in the households [20]. In a qualitative analysis of barriers to care-seeking for common childhood infectious diseases, the trust in the primary care services was low [21]. The barrier analysis showed a lack of community awareness of the curative, as well as preventive services provided by the HEW and that the quality of care on the primary level was perceived to be low [22]. Also, there was a felt need to strengthen the HEWs in supporting pregnant women in birth preparedness, referral to midwives, and institutional delivery [23]. All the above listed barriers resulted in under-use of maternal, newborn and child health services. Other demand side barriers included delay in seeking care due to the need to obtain husbands’ permission and financial support, the perceived cost and real cost of travel, particularly due to costs associated with referral of severely ill children to a further facility and repeated travel due to health post closure [21]. Supply-side barriers included frequent stock-outs of medicines and other necessary supplies, service interruption and inconsistent operating hours at health posts [24]. Further, it showed that HEWs had poor skills and confidence [25], especially when managing and treating newborns [26], lack of local government ownership and lack of accountability for both the CBNC and iCCM programs, and inconsistent supervision and monitoring. This barrier analysis formed the basis of a logic framework for a complex intervention that postulated that community engagement would increase care seeking for ill children, capacity building would improve availability of quality of CBNC and iCCM services and district level ownership and accountability would improve integration of these services into the district level planning and budgeting (Table 1). Together these three strategies would lead to an increased utilization of CBNC and iCCM services.

Table 1 Logic framework for the Optimizing Health Extension Program intervention in selected districts of Ethiopia

The intervention and underlying assumptions

The package of interventions to be implemented across 26 districts includes three interlinked strategies with possible synergies: [1] community engagement activities that aim at increasing the awareness of newborn and child diseases, the recognition and acceptance of the care provided on the primary level, and the formulation of action plans at the local level [2]; capacity building of HEWs and WDA leaders such as gap filling training, supportive supervision and mentorship to improve iCCM and CBNC services, and [3] strengthening the local government’s ownership and accountability of the primary newborn and child health services by advocating for the sustained integration of CBNC and iCCM into the planning, budgeting, monitoring, management and support systems of the district and sub-district level (Table 1). Assumptions made by the implementers to achieve the success of the Optimizing the Health Extension Program included support from local stakeholders, traditional and religious leaders, governmental health sector and supply chain partners.

Implementation

The Ethiopian Government, in collaboration with PATH and UNICEF (through sub-contractors Save the Children and Last 10 Kilometres) implement the intervention. These organizations have quarterly meetings to harmonize the intervention activities across the 26 intervention districts.

Trained professionals from the implementing organizations and the public sector facilitate the community engagement activities (Table 1). To raise community awareness of iCCM and CBNC services, implementers organise health post open house sessions to introduce available services and conduct workshops with schoolteachers and religious leaders. Community engagement will also be supported by behaviour change communication materials (brochures, posters, and banners). Educational films developed by implementers will be screened in health facilities and similarly radio messages and dramas will be developed and broadcast in implementation districts. Health professionals within the district health services facilitate the capacity building activities. For example, midwives from health centres provide training to HEWs, while HEWs provide training for WDA leaders. Where missing, implementers will provide registration books, iCCM and CBNC treatment algorithm booklets and backpacks to carry these items for community level service provision. Facilitators from the implementing partners lead the ownership and accountability activities. District ownership efforts include advocacy workshops at district and sub-district levels and support to the annual district health planning sessions.

The implementers plan to achieve a high reach of the different innovations across all intervention districts. Thus, the ambition is that the community meetings, school engagement activities, training of WDA leaders and HEWs, as well as district ownership efforts should reach all areas and relevant stakeholders.

Process evaluation, including mechanisms of impact, and contextual factors

The process evaluation is guided by the UK Medical Research Council framework for complex interventions [17]. A graphical representation of the process evaluation is provided in Fig. 2.

Fig. 2
figure 2

Framework for the process evaluation of the Optimizing the Health Extension Program intervention

The implementing partners will prospectively collect data on the implementation of the interventions. The innovations within the three strategies will continually be harmonized across the implementing partners. Some variations between implementers and geographies was intended due to the different contexts of the districts. Such variations can be captured through the process evaluation. Implementers will also make efforts to harmonize the corresponding data sources. These databases will include information on each performed activity, including information on the innovation, the facilitator(s), the recipients, the health-system level and place, and timing of each activity. The implementing partners will also provide information on training and support to deliver the interventions and any changes made in some parts of the intervention districts or across all districts. This information will be aggregated to describe the fidelity (whether an intervention was delivered as intended), reach (how much of the intended audience was exposed to the intervention), and where feasible the dose (how much of the intervention was received by the intended audience) and adaptations (originally unintended changes made to an intervention across all or in selected parts of the study areas). In the endline survey, the participants (mothers of children below the age of 5 years, WDA leaders, HEWs, woreda health office representatives) will be asked about their contact with the interventions to measure reach. The baseline and endline surveys include questions on the interactions between the households and both WDA members and HEWs to measure dose (Table 2). Overall, the implementation process data will allow for analyses of whether the Optimizing the Health Extension Program innovations were implemented according to plans and associated with better awareness and acceptance of CBNC and iCCM services, provision of improved quality of services, and strengthened ownership and accountability of CBNC and iCCM services, respectively. In addition, a qualitative study will be conducted to explore views and experiences of program planners, managers and implementers, identifying what components worked well and what was not successful in the delivery of Optimizing the Health Extension Program interventions.

Table 2 Baseline and endline survey questionnaires for the Optimizing the Health Extension Program intervention evaluation

Effectiveness of the optimizing the health extension program

The baseline and endline surveys in intervention and comparison areas will include modules for household, facility preparedness to provide child health services, the health worker, a quality of care assessment, the WDA leader and woreda contextual factors. As part of the endline survey the HEWs will be interviewed, using the questionnaire-based Context Assessment for Community Health (COACH) tool [27]. This validated tool measures eight aspects of the context in which the HEWs work.

Outcomes

The primary outcome is care seeking (at health posts, health centres, hospitals and clinics) for an illness in under-fives. Secondary outcomes include appropriate treatment for diarrhoeal diseases (oral rehydration therapy, zinc tablets), probable pneumonia (antibiotics), fever and malaria, and neonatal sepsis (antibiotics); improved knowledge towards childhood illnesses and treatment services among caregivers of under-5 children; improved attitudes or perceptions towards the iCCM and CBNC services at health posts; improved iCCM and CBNC program ownership by the public health sector (including inclusion of iCCM and CBNC indicators in their planning and budget allocation); and, improved availability of quality iCCM services provided by the HEWs.

The sample size for baseline and endline surveys is based on the requirement that these surveys should have adequate power to measure changes in a fixed number of percentage points between intervention and comparison areas from start to end of the study. For the household survey, the sample size was powered for the main outcome of care seeking for any illness in the 2 weeks prior to the survey (Table 3). Sampling 30 households in the 100 selected enumeration areas would yield 3000 households per group. The Ethiopia Demographic Heath Survey (DHS) data has shown that the rate of children under-five to households surveyed was 0.65. Based on this assumption, a sample survey of 3000 households per group would be expected to achieve a sample size of 1747 children below the age of 5 years. The evaluation of the integrated management of childhood illness in Tanzania reported 50% of under-five children to have had an illness in the 2 weeks prior to the survey [28]. We assumed a more conservative 30%. Based on the calculations of a sample size of 3000 households per group (6000 in total) with 90% completeness and a design effect of 1.3, we would have 80% power to detect differences of 10–20 percentage points across the range of child health indicators as statistically significant at the 5% level.

Table 3 Sample size for before vs. after comparison of sick child care seeking

For the survey to assess the quality care provided by HEWs, with the assumption that each of the sampled enumeration areas will be served by one health post, the survey will include 100 intervention and 100 comparison area health posts for the quality of care assessment, where in each health post, the HEWs’ assessment of four sick children mobilized to come to the health post will be observed, followed by a re-examination of the child by a health officer. This will yield a sample size of 400 children in each group. A total of 800 children, with a design effect of 1.4, will have 80% power to detect a minimum of 15 percentage point changes in the correct classification of iCCM illnesses between intervention and comparison area HEWs at baseline and endline as statistically significant at the 5% level (Table 4).

Table 4 Sample size for before vs. after comparison of sick children aged 2–59 months correctly managed

The questionnaire tools for baseline and endline surveys were adapted from the team’s own, and others’ previous work on the Integrated Management of Childhood Illness [28], iCCM [29], and CBNC [30]. The questionnaire instrument included modules regarding household, health posts and health centres, HEWs, health centre staff, WDA members, an observation and re-examination of the HEW assessing sick children, and district contextual factors (Table 2) [see additional files 1–9].

The baseline survey took ten weeks and was completed in February 2017 in intervention and comparison areas to assess the situation before the intervention. A two-stage stratified cluster sampling was applied in these two surveys using lists of enumeration areas of the 52 intervention and comparison districts from the latest (2007) Ethiopian census as the sampling frame. In the first stage, a list of all enumeration areas of the study districts were based on the 2007 Ethiopian Housing and Population Census. Two hundred enumeration areas were selected from 52 districts with probability proportional to size. Each enumeration area formed one cluster, and these clusters constituted the primary sampling unit. In the second stage, a systematic random sampling technique was applied to select 30 households in each cluster. All women aged 13 to 49 years who lived in the selected households were included, in order to identify women who had a live birth in the 12 months prior to the survey to assess care seeking in the neonatal period. Furthermore, children under the age of 5 years were included to assess care seeking for any illness in the 2 weeks prior to the survey. For every cluster, the WDA leader serving the cluster was interviewed. The health post and the HEWs serving the selected cluster, the health post’s referral health centre, and staff, and the district health office providing support to the selected facility were approached with survey modules. All study participants were sampled without replacement. Up to three visits were made to each participant to maximize their inclusion into the study.

To evaluate the quality of assessment and care provided to sick children at the health posts, an observation of a sick child consultation with an HEW and a re-examination by a child health officer was performed. Given that very few sick children are brought to the health post each day, data collectors mobilized the community to bring sick children on the day of the survey to ensure the required sample size is met.

Data collection teams and supervisors were trained over the course of ten days. They were not provided information on whether a district was an intervention or comparison area. Data were collected on tablets and encrypted data were regularly sent from the field to the Ethiopian Public Health Institute’s central server. The data manager then decrypted data and rigorous quality checks were conducted with feedback to the field teams. Data cleaning involved checking for errors, completeness and consistency. The data manager also ensured that all standards for data security, curation and access were met.

The endline survey is being conducted 2 years after the baseline survey, following the same procedures.

The effectiveness assessment will be based on a plausibility design [16], analysing difference-in-differences of the primary and secondary outcomes [31]. Data on primary and secondary outcomes will be analysed from baseline and endline household, health post, and health centre surveys in intervention and comparison districts, with adjustment for the cluster sampling and relevant confounding factors. The assessment will use blinded analysis. The code identifying the intervention and comparison areas will be revealed after the analysis and interpretations are completed.

Ten Ph.D. students from four Ethiopian universities, including candidates from the Regional Health Bureaus in the study provinces, and from the Ethiopian Public Health Institute, are engaged in the evaluation. PhD students were involved in the conduct of the surveys, participating in the training of data collectors and serving as regional survey coordinators. They have chosen topics for in-depth sub-studies linked to the evaluation, including equity in the utilisation of services; spatial analyses of care utilisation; quality of care provided by the HEWs; the role of WDA leaders in promoting the use of services; newborn care practices; the referral of sick children; and focused studies of care utilisation for diseases of the newborn and diarrhoeal diseases. Most of the students plan to use baseline and endline surveys for quantitative data and to perform qualitative studies within their chosen topics.

Findings from the effectiveness study, process evaluation and PhD research will be published as scientific articles and reports. No professional writers will be used. Publications arising from this evaluation will follow the recommendations from the International Committee of Medical Journal Editors [32].

Trial status

The interventions were running up to the end of 2018 and endline survey started immediately thereafter. Data for process evaluation collection and analysis will take place until September 2019. Data analysis for the effectiveness study will be performed from mid of 2019 onwards.

Discussion

This protocol describes an evaluation of a complex intervention that aims at an increased utilisation of primary child and maternal health services in Ethiopia. The interventions are based on a logic framework developed from an analysis of barriers to the utilisation of primary maternal and child health services. The program includes innovative components to engage the community, strengthen the capacity of primary care workers, and reinforce the local ownership and accountability of the primary maternal and child health services. This evaluation combines state-of-the-art effectiveness and process evaluation with capacity building and training of Ethiopian Ph.D. students from universities in the four study regions of Ethiopia, and is linked to regional health bureaus. We will assess the effectiveness of the intervention to increase the seeking of appropriate care by difference-in-differences analysis based on baseline and endline surveys. The process evaluation follows the guidelines of the UK Medical Research Council.

Barriers to health care utilisation

An analysis of barriers to the utilisation of child and maternal health primary services was the point of departure for the development of a logic framework of the interventions. Representatives of the Ethiopian health system and non-governmental organizations active in maternal, newborn and child health reviewed findings form a qualitative barrier analysis study carried out by local researchers and a literature review conducted by UNICEF. The findings were in line with recently published literature from Ethiopia that had addressed these questions.

Outcomes

The primary outcome of this intervention is care seeking for the major causes of death in children below the age of 5 years in Ethiopia: diarrhoeal diseases, pneumonia, and malaria [3]. Care seeking for suspected pneumonia was especially low in Ethiopia in a study comparing different countries in sub-Saharan Africa [33]. When measurements are challenging, as is the case for assessing antibiotic treatment for children with pneumonia, we will ensure findings are interpreted with caution [34]. A study in Eastern Ethiopia stressed the importance of having previous knowledge and experience of rehydration therapy and an established contact with the health post to seek care when the child suffers from diarrhoea [35].

Three intervention strategies

The complex intervention includes innovations and activities to engage the community, build the capacity of the HEWs and leaders of the WDA, and foster ownership and accountability of the CBNC and iCCM programs by local health authorities.

Different community engagement interventions are increasingly getting attention for maternal, newborn, and child health. The evidence is mounting that such interventions may be effective to improve utilisation of services and get improved health outcomes [36]. Ethiopian studies showed that community engagement activities could expand the use and quality of perinatal care [37] and increase the coverage of postnatal care [38]. Community support to iCCM activities and community health workers is needed for sustained health benefits coupled with a focus on strengthening and enabling the public health system [39].

Although demand-side interventions may increase utilisation, this does not necessarily lead to better child and maternal outcomes, for which good quality, timely care is essential [40]. Training of primary health workers in CBNC and iCCM programs is a prerequisite, and Ethiopian experiences indicate that the knowledge gained may be satisfactorily retained, although refresher training is advisable for sustained effect [25, 41]. Supportive supervision and performance review meetings may be effective strategies to improve the quality of services provided by HEWs [42]. A review paper from several low-income countries reported that high-quality supervision focusing on supportive approaches, community monitoring, and problem-solving may be an effective solution [43].

In a review of data from facilities in eight mainly sub-Saharan African countries, the facility infrastructure was poorly associated with the observed quality of maternal, newborn, and child health care provided [44]. We have such infrastructure data from baseline and endline surveys that are essential to describe the setting, although it may not shed light on the intervention’s effect on the quality of services provided by the HEWs. In a meta-review of almost 100 systematic reviews of interventions to improve the quality of care, the facilitators and barriers were in the domains of information, patient-population engagement, leadership, regulations and standards, organizational capacity, models of care, communication, and satisfaction [45]. The COACH questionnaire tool that is planned to be part of the endline assessment with HEWs covers such domains and may potentially provide explanations as to whether the interventions have improved the quality of care provided or not [27]. Data on the inner as well as the organizational context offer opportunities to analyse the potential influence of these factors on the process and outcomes of the intervention.

As described above, the CBNC and iCCM programs implemented within the Ethiopian primary care context have proven to be effective in reducing mortality. For further quality improvements and sustained effects, the ownership and accountability for these programs on all levels of the health system must be considered [46]. Efforts to increase accountability for maternal and child health services might be more successful if many stakeholders get involved [47].

While the interventions rolled out in Ethiopia will be tailored to address the critical bottlenecks identified and therefore may be country-specific, the approaches to address some of the demand and service challenges may apply to other countries. As more countries begin to adopt iCCM, CBNC, and other community-based services to extend child and maternal health care beyond health facilities, lessons learned from Ethiopia may prove useful for other settings. Lessons learned from this evaluation can serve to adopt approaches in countries already rolling out iCCM and CBNC or feed into policy and implementation strategies for new countries planning to develop community-based platforms of service delivery.

Conclusion

This protocol describes the evaluation of a complex intervention in four regions of Ethiopia that aims at increasing utilisation of primary care services for mothers and children. The intervention is based on an analysis of barriers to the utilisation of services and includes innovations to engage communities, train and support primary care workers, and promote ownership and accountability of CBNC and iCCM programs. Representatives of the health system, non-governmental organisations, and universities work together to learn, evaluate, and strengthen university capacity for health systems and implementation research, which will enhance sustainable work relationship between universities and Regional Health Bureaus. The process and outcome evaluations will inform the possible scale-up of efforts to increase primary health care utilisation for sick children in Ethiopia and similar settings.

Availability of data and materials

A data sharing committee has been established comprised of each of the universities and Ethiopian Public Health Institute. Any requests will be reviewed by this committee and if requests are granted data will be shared without any identifiers. Request for data can be made to Yemisrach B. Okwaraji (Yemisrach.Okwaraji@lshtm.ac.uk) or Desta Wolassa (desta.wolassa@lshtm.ac.uk).

Abbreviations

CBNC:

Community-Based Newborn Care

COACH:

Context Assessment for Community Health

DHS:

Demographic Heath Survey

EPHI:

Ethiopian Public Health Institure

HEW:

Health Extension Worker

iCCM:

Integrated Community Case Management

LSHTM:

London School of Hygiene & Tropical Medicine

PATH:

Program for Appropriate Technology in Health

WDA:

Women’s Development Army

References

  1. Ruducha J, Mann C, Singh NS, Gemebo TD, Tessema NS, Baschieri A, et al. How Ethiopia achieved millennium development goal 4 through multisectoral interventions: a countdown to 2015 case study. Lancet Glob Health. 2017 Nov;5(11):e1142–51.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Corsi DJ, Neuman M, Finlay JE, Subramanian S. Demographic and health surveys: a profile. Int J Epidemiol. 2012 Dec;41(6):1602–13.

    Article  PubMed  Google Scholar 

  3. Deribew A, Tessema GA, Deribe K, Melaku YA, Lakew Y, Amare AT, et al. Trends, causes, and risk factors of mortality among children under 5 in Ethiopia, 1990-2013: findings from the global burden of disease study 2013. Popul Health Metrics. 2016;14(1):42.

    Article  Google Scholar 

  4. Yitayal M, Berhane Y, Worku A, Kebede Y. Health extension program factors, frequency of household visits and being model households, improved utilization of basic health services in Ethiopia. BMC Health Serv Res. 2014;14(1):156.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Miller NP, Amouzou A, Tafesse M, Hazel E, Legesse H, Degefie T, et al. Integrated community case management of childhood illness in Ethiopia: implementation strength and quality of care. Am J Trop Med Hyg. 2014;91(2):424–34.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Mathewos B, Owen H, Sitrin D, Cousens S, Degefie T, Wall S, et al. Community-Based Interventions for Newborns in Ethiopia (COMBINE): Cost-effectiveness analysis. Health Policy Plan. 2017;32(suppl_1):i21–32.

    Article  PubMed  Google Scholar 

  7. Degefie Hailegebriel T, Mulligan B, Cousens S, Mathewos B, Wall S, Bekele A, et al. Effect on neonatal mortality of newborn infection Management at Health Posts When Referral is not Possible: a cluster-randomized trial in rural Ethiopia. Glob Health Sci Pract. 2017;5(2):202–16.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Girmaye M, Berhan Y. Skilled antenatal care service utilization and its association with the characteristics of Women's health development team in Yeky District, south-West Ethiopia: a multilevel analysis. Ethiop J Health Sci. 2016;26(4):369–80.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Jackson R, Tesfay FH, Gebrehiwot TG, Godefay H. Factors that hinder or enable maternal health strategies to reduce delays in rural and pastoralist areas in Ethiopia. Tropical Med Int Health. 2017;22(2):148–60.

    Article  Google Scholar 

  10. Mezmur M, Navaneetham K, Letamo G, Bariagaber H. Socioeconomic inequalities in the uptake of maternal healthcare services in Ethiopia. BMC Health Serv Res. 2017;17(1):367.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Bobo FT, Yesuf EA, Woldie M. Inequities in utilization of reproductive and maternal health services in Ethiopia. Int J Equity Health. 2017;16(1):105.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Tesfaye B, Mathewos T, Kebede M. Skilled delivery inequality in Ethiopia: to what extent are the poorest and uneducated mothers benefiting? Int J Equity Health. 2017;16(1):82.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Amouzou A, Hazel E, Shaw B, Miller NP, Tafesse M, Mekonnen Y, et al. Effects of the integrated community case Management of Childhood Illness Strategy on child mortality in Ethiopia: a cluster randomized trial. Am J Trop Med Hyg. 2016;94(3):596–604.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Ayalneh AA, Fetene DM, Lee TJ. Inequalities in health care utilization for common childhood illnesses in Ethiopia: evidence from the 2011 Ethiopian demographic and health survey. Int J Equity Health. 2017;16(1):67.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Skaftun EK, Ali M, Norheim OF. Understanding inequalities in child health in Ethiopia: health achievements are improving in the period 2000-2011. PLoS One. 2014;9(8):e106460.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Habicht JP, Victora CG, Vaughan JP. Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. Int J Epidemiol. 1999;28(1):10–8.

    Article  CAS  PubMed  Google Scholar 

  17. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350(mar19 6):h1258.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Nigatu SG, Worku AG, Dadi AF. Level of mother’s knowledge about neonatal danger signs and associated factors in north west of Ethiopia: a community based study. BMC Res Notes. 2015;8(1):309.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Degefie T, Amare Y, Mulligan B. Local understandings of care during delivery and postnatal period to inform home based package of newborn care interventions in rural Ethiopia: a qualitative study. BMC Int Health Hum Rights. 2014;14(1):17.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Tefera W, Tesfaye H, Bekele A, Kayessa E, Waltensperger KZ, Marsh DR. Factors influencing the low utilization of curative child health services in Shebedino District, Sidama zone, Ethiopia. Ethiop Med J. 2014;52(Suppl 3):109–17.

    PubMed  Google Scholar 

  21. Shaw B, Amouzou A, Miller NP, Tafesse M, Bryce J, Surkan PJ. Access to integrated community case management of childhood illnesses services in rural Ethiopia: a qualitative study of the perspectives and experiences of caregivers. Health Policy Plan. 2016;31(5):656–66.

    Article  PubMed  Google Scholar 

  22. King R, Jackson R, Dietsch E, Hailemariam A. Barriers and facilitators to accessing skilled birth attendants in Afar region, Ethiopia. Midwifery. 2015;31(5):540–6.

    Article  PubMed  Google Scholar 

  23. Medhanyie A, Spigt M, Kifle Y, Schaay N, Sanders D, Blanco R, et al. The role of health extension workers in improving utilization of maternal health services in rural areas in Ethiopia: a cross sectional study. BMC Health Serv Res. 2012;12(1):352.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Tilahun H, Fekadu B, Abdisa H, Canavan M, Linnander E, Bradley EH, et al. Ethiopia's health extension workers use of work time on duty: time and motion study. Health Policy Plan. 2017;32(3):320–8.

    PubMed  Google Scholar 

  25. Gobezayehu AG, Mohammed H, Dynes MM, Desta BF, Barry D, Aklilu Y, et al. Knowledge and skills retention among frontline health workers: Community maternal and newborn health training in rural Ethiopia. J Midwifery Womens Health. 2014;59(s1):S21–31.

    Article  PubMed  Google Scholar 

  26. Berhe AK, Tinsae F, Gebreegziabher G. Knowledge and practice of immediate newborn care among health care providers in eastern zone public health facilities, Tigray, Ethiopia, 2016. BMC Pediatr. 2017;17(1):157.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Bergström A, Skeen S, Duc DM, Blandon EZ, Estabrooks C, Gustavsson P, et al. Health system context and implementation of evidence-based practices—development and validation of the context assessment for community health (COACH) tool for low- and middle-income settings. Implement Sci. 2015;10(1):120.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Armstrong Schellenberg JRM, Adam T, Mshinda H, Masanja H, Kabadi G, Mukasa O, et al. Effectiveness and cost of facility-based integrated Management of Childhood Illness (IMCI) in Tanzania. Lancet. 2004;364(9445):1583–94.

    Article  PubMed  Google Scholar 

  29. Marsh DR, Nefdt R, Hazel E. Introduction to a special supplement: delivering integrated community case management to treat childhood illness at scale in Ethiopia. Ethiop Med J. 2014;52(Suppl 3):1–6.

    PubMed  Google Scholar 

  30. Berhanu D. Community Based Newborn Care. In: Baseline report summary, Ethiopia. London: IDEAS, London School of Hygiene & Tropical Medicine; 2014. https://ideas.lshtm.ac.uk/wcpcontent/uploads/2017/08/CBNCBaselineReport_18nov15_WEB.pdf.

    Google Scholar 

  31. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA. 2014;312(22):2401–2.

    Article  CAS  PubMed  Google Scholar 

  32. Uniform requirements for manuscripts submitted to biomedical journals. International Committee of Medical Journal Editors, vol. 277: JAMA; 1997. p. 927–34.

  33. Noordam AC, Carvajal-Velez L, Sharkey AB, Young M, Cals JWL. Care seeking behaviour for children with suspected pneumonia in countries in sub-Saharan Africa with high pneumonia mortality. PLoS One. 2015;10(2):e0117919.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  34. Hazir T, Begum K, Arifeen El S, Khan AM, Huque MH, Kazmi N, et al. Measuring coverage in MNCH: a prospective validation study in Pakistan and Bangladesh on measuring correct treatment of childhood pneumonia. PLoS Med. 2013;10(5):e1001422.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Mengistie B, Berhane Y, Worku A. Predictors of Oral rehydration therapy use among under-five children with diarrhea in eastern Ethiopia: a community based case control study. BMC Public Health. 2012 Nov 24;12(1):1029.

    Article  PubMed  PubMed Central  Google Scholar 

  36. World Health Organization, (null) IIFIE. An evidence map of social, behavioural and community engagement interventions for reproductive, maternal, newborn and child health. Geneva: World Health Organization; 2017.

    Google Scholar 

  37. Barry D, Frew AH, Mohammed H, Desta BF, Tadesse L, Aklilu Y, et al. The effect of community maternal and newborn health family meetings on type of birth attendant and completeness of maternal and newborn care received during birth and the early postnatal period in rural Ethiopia. J Midwifery Womens Health. 2014;59(Suppl 1):S44–54.

  38. Tesfaye S, Barry D, Gobezayehu AG, Frew AH, Stover KE, Tessema H, et al. Improving coverage of postnatal care in rural Ethiopia using a community-based, collaborative quality improvement approach. J Midwifery Womens Health. 2014;59(Suppl 1):S55–64.

  39. Strachan C, Wharton-Smith A, Sinyangwe C, Mubiru D, Ssekitooleko J, Meier J, et al. Integrated community case management of malaria, pneumonia and diarrhoea across three African countries: a qualitative study exploring lessons learnt and implications for further scale up. J Glob Health. 2014;4(2):020404.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hurst TE, Semrau K, Patna M, Gawande A, Hirschhorn LR. Demand-side interventions for maternal care: evidence of more use, not better outcomes. BMC Pregnancy Childbirth. 2015;15(1):297.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Dynes M, Buffington ST, Carpenter M, Handley A, Kelley M, Tadesse L, et al. Strengthening maternal and newborn health in rural Ethiopia: early results from frontline health worker community maternal and newborn health training. Midwifery. 2013;29(3):251–9.

    Article  PubMed  Google Scholar 

  42. Miller NP, Amouzou A, Hazel E, Legesse H, Degefie T, Tafesse M, et al. Assessment of the impact of quality improvement interventions on the quality of sick child care provided by health extension Workers in Ethiopia. J Glob Health. 2016 Dec;6(2):020404.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Hill Z, Dumbaugh M, Benton L, Källander K, Strachan D, Asbroek Ten A, et al. Supervising community health workers in low-income countries - a review of impact and implementation issues. Glob Health Action 2014;7(1):24085.

  44. Leslie HH, Sun Z, Kruk ME. Association between infrastructure and observed quality of care in 4 healthcare services: a cross-sectional study of 4,300 facilities in 8 countries. PLoS Med. 2017;14(12):e1002464.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Nair M, Yoshida S, Lambrechts T, Boschi-Pinto C, Bose K, Mason EM, et al. Facilitators and barriers to quality of care in maternal, newborn and child health: a global situational analysis through metareview. BMJ Open. 2014;4(5):e004749.

    Article  PubMed  PubMed Central  Google Scholar 

  46. George A, Rodríguez DC, Rasanathan K, Brandes N, Bennett S. iCCM policy analysis: strategic contributions to understanding its character, design and scale up in sub-Saharan Africa. Health Policy Plan. 2015;30(Suppl 2):ii3–ii11.

    Article  PubMed  Google Scholar 

  47. Martin Hilber A, Blake C, Bohle LF, Bandali S, Agbon E, Hulton L. Strengthening accountability for improved maternal and newborn health: a mapping of studies in sub-Saharan Africa. Int J Gynaecol Obstet. 2016;135(3):345–57.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Lindsay Mangham-Jefferies who drafted the initial protocol and questionnaires for the baseline survey.

Funding

This project is funded by Bill & Melinda Gates Foundation (OPP1132551). The funder has no role in the study design, collection, management, analysis and interpretation of data.

Author information

Authors and Affiliations

Authors

Contributions

JS is the principal investigator and, in collaboration with EL, was responsible for the design and development of the Dagu project. DB prepared the first draft of the manuscript with contributions from YO and LP. AB, NA, BB, KA, WB, AM, MA, MY and FB read and commented on the manuscript and approved the final version.

Ethics declarations

Ethics approval and consent to participate

Data collectors obtained written informed consent from all interviewees at baseline. For women between the ages of 13–17 years to be included in the household survey, assent was obtained from a parent or guardian. The same process will be followed at endline survey. The Ethiopian Public Health Institute Ethical Review Board and the London School of Hygiene & Tropical Medicine Ethical Review Board approved the evaluation (LSHTM Ethics Ref 16117 and EPHI Ethics Ref 613/52). Any deviations to the approved protocol will be reported to the Ethical Review Boards of both countries.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Berhanu, D., Okwaraji, Y.B., Belayneh, A.B. et al. Protocol for the evaluation of a complex intervention aiming at increased utilisation of primary child health services in Ethiopia: a before and after study in intervention and comparison areas. BMC Health Serv Res 20, 339 (2020). https://doi.org/10.1186/s12913-020-05151-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-020-05151-3

Keywords