Using the Implementation Research Logic Model to design and implement community-based management of possible serious bacterial infection during COVID-19 pandemic in Ethiopia
BMC Health Services Research volume 22, Article number: 1515 (2022)
Community-based treatment of possible serious bacterial infection (PSBI) in young infants, when referral to a hospital is not possible, can result in high treatment coverage and low case fatality. However, in Ethiopia, the coverage of PSBI treatment remains low, worsened by COVID-19. To understand the challenges of delivery of PSBI treatment and design and test adaptative strategies to mitigate the impact of COVID-19 on neonatal mortality, we did implementation research (IR) employing Implementation Research Logic Model (IRLM). In this paper, we describe IRLM application experiences in designing, implementing, and evaluating strategies to improve community-based treatment of PSBI during the COVID-19 pandemic in Ethiopia.
This IR was conducted between November 2020-April 2022 at Dembecha and Lume woredas of Amhara and Oromia regions, respectively. We employed narrative reviews, formative assessment and facilitated stakeholder engagement to develop the PSBI treatment IRLM to identify barriers, understand the conceptual linkages among determinants, choose implementation strategies, elicit mechanisms, and link to implementation outcomes. In addition, we used the IRLM to structure the capture of emerging implementation challenges and resulting strategy adaptations throughout implementation.
This IR identified COVID-19 and multiple pre-existing contextual factors. We designed and implemented implementation strategies to address these challenges. These adaptive strategies were implemented with sufficient strength to maintain the delivery of PSBI services and improve mothers’ care-seeking behavior for their sick young infants.
The IRLM offers us a clear process and path to prioritize implementation challenges, choose strategies informed by mechanisms of action, and where the adaptive implementation of community-based management of PSBI would lead to high-implementation fidelity and change in mother behavior to seek care for their sick young infants. The IRLM was also an effective tool for stakeholder engagement, easily explained and used to structure discussion and decision-making during co-design meetings.
The use of the IRLM helps us to specify the conceptual links between the implementation challenges, strategies, mechanisms of action, and outcomes to explore the complex community-based management of PSBI during complex contexts to improve high-fidelity implementation and integration of PSBI treatment in the primary healthcare delivery systems through active engagement of stakeholders.
Possible serious bacterial infection (PSBI) was identified as a leading cause of mortality in young infants (0–59 days of age) in Sub-Saharan Africa, contributing to 37% of the 2.1 million neonatal deaths . Population-based surveillance data of young infants found a prevalence of 13.1% of one or more signs of infection , with an incidence of sepsis estimated to be 6.2% and a case fatality rate of 14.1%  in sub-Saharan Africa (as high up to 24% in low and middle-income countries [4,5,6]). The World Health Organization (WHO) recommends a simplified regimen (injectable and oral antibiotics) for community management of PSBI when referral to a hospital is not possible . Evidence from Ethiopia, Malawi, Nigeria, Pakistan, and India [8,9,10,11,12,13,14,15,16] revealed that the simplified regimen scale-up for PSBI treatment results in high coverage and low case fatality.
Ethiopia introduced community-based newborn care (CBNC) in its flagship Health Extension Program (HEP) to improve access to and use of appropriate treatment for neonatal sepsis when a referral was not possible in 2012 . However, the coverage of sick newborn care is low due to multiple factors: an overall decline in the performance of the HEP observed in recent years , low Health Extension Workers’ (HEWs) competency and motivation [9, 19, 20], erratic supply of essential drugs [19, 20], suboptimal supportive and referral links [9, 19, 20], communities’ misperceptions about newborn illnesses [21, 22], and socio-cultural beliefs and mothers’ limited decision-making power [19, 21,22,23]. In addition, there is a concern from program managers that the Coronavirus (COVID-19) pandemic has stressed the already weak healthcare system in Ethiopia, exacerbating the already fragile newborn and child health services.
There is limited evidence of the level of impact of the COVID-19 pandemic on care-seeking for neonatal sepsis programs in Ethiopia. Implementation research (IR) was conducted between November 2020-April 2022 in Dembecha and Lume districts of Amhara and Oromia regions, respectively. The research was carried out by JSI Research & Training Institute Inc./The Last Ten Kilometers (L10K) Project, funded by the Bill & Melinda Gates Foundation. The aim was to understand the demand- and supply-side challenges of delivery of PSBI treatment during the COVID-19 pandemic and to develop and test adaptative strategies to mitigate the impact of COVID-19 and other barriers on community-based management of PSBI implementation and uptake. Implementation research is the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, to improve the adoption, high-fidelity implementation, and maintenance of quality, effective, evidence-based practices in health care services . The Implementation Research Logic Model (IRLM) combines a number of commonly used IR frameworks to help design or evaluate the implementation of evidence-based interventions by identifying determinants and selection strategies through mapping mechanisms and linking those implementation and effectiveness outcomes . There is also emerging evidence from the United States that IRLM is an effective tool for stakeholder engagement critical for success in implementation work , but not in Ethiopia or similar settings.
In this paper, we describe the use of the IRLM for stakeholder engagement and incorporate formative research to inform the design, implementation, and evaluation of strategies to improve community-based management of neonatal bacterial infection during the COVID-19 pandemic. The utilization of the IRLM in Ethiopia represents important insights into using this tool and IR to inform adaptive implementation, engaging key stakeholders, and generating generalizable knowledge to inform the broader, context-adapted implementation of PSBI treatment.
This implementation research was conducted between November 2020-April 2022 in Dembecha and Lume woredas of Amhara and Oromia regions (Fig. 1) Dembecha is in the West Gojjam Zone of the Amhara region. The woreda town is 349 km north of Addis Ababa and 215 Kilometers South of Bahir Dar, the regional capital. It has 31 rural kebeles with poor electricity and road access. Based on the 2007 national census conducted by the Central Statistical Agency of Ethiopia , the woreda has a total population of 129,260, with 13.9% urban inhabitants. Thirty-one health posts (HPs) and six health centers provide primary health care services in Dembecha. Lume is located in East Shewa Zone of the Oromia region in the Great Rift Valley. The woreda capital, Modjo, is located 64 km east of Addis Ababa and 19 km east of Adama, the zonal capital. The 2007 national census  reported a total district population of 117,080, 33.06% of which are urban dwellers. The woreda has 35 rural kebeles with better access than Dembecha to health facilities, electricity, and roadways. 35 HPs and seven health centers provide primary health services.
We employed narrative review, formative assessment, and facilitated stakeholder engagement to develop the management of PSBI IRLM. The work was designed to articulate determinants, choose implementation strategies, and link these through mechanisms to targeted implementation outcomes. In addition, during the implementation phase, we captured ongoing implementation challenges and adaptations of strategies using a tracker. We applied the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) . We also identified metrics for implementation outcomes and process evaluation to measure the implementation of the strategies.
In April 2021, we conducted a narrative review of existing literature to quickly identify the overview of previously described barriers and facilitators of community-based management of PSBI implementation in Ethiopia. One author (GT) searched Google Scholar and PubMed for publications in English between 2012 to 2021, starting after the initiation of CBNC services in Ethiopia. Search terms used included: possible serious bacterial infection, community-based management of newborn illnesses, neonatal sepsis, uptake, utilization, barriers, facilitators, determinants, and Ethiopia. The title or abstract of the article was downloaded. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram (Fig. 2) was used to select articles for this review. A narrative synthesis was used to extract themes of the barriers and facilitators to community-based management of PSBI implementation in Ethiopia. Eleven papers were included in the narrative analysis (five qualitative [19,20,21,22,23], four mixed [29,30,31,32], and two quantitative design [9, 33] studies) (see Additional file 1 for details of the characteristics of reviewed articles).
Guided by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance)  framework and Consolidated Framework for Implementation Research (CFIR) , we conducted concurrent quantitative and explorative qualitative formative research in April–May 2021 in the two woredas. The goals were to understand the current implementation of community-based management of PSBI and implementation challenges to establish benchmarks, identify context-specific implementation gaps, and inform potential strategies to bridge these gaps.
We conducted a cross-sectional population-based household survey of 4,242 mothers who gave live birth 2–14 months before data collection and resided in the two woredas. Survey captured information including household and sociodemographic characteristics, experiences related to using maternal and newborn health services across the continuum of care, care-seeking for sick infants and children, and Knowledge and risk perception of COVID-19. A structured questionnaire translated into local languages (Amharic and Oromiffa) was used to capture the data using a web-based mHealth platform (SurveyCTO) using smartphones (Additional file 2). We also conducted surveys of 66 HPs and captured self-administered interviews with 79 HEWs available on the day of the visit. Surveys were based on an adapted tool (Additional file 3) from the WHO Service Availability and Readiness Assessment (SARA) . They were designed to capture the availability of drugs, equipment, and knowledge and skills of HEWs.
We conducted 34 in-depth interviews of purposively selected respondents, including program managers (6), development partners (4), direct service providers (16), and community volunteers (8). The interview guide was designed using CFIR to understand what factors facilitate or hinder community-based management of PSBI implementation, including characteristics of the health system, intervention, individual, inner, and outer settings.
We engaged stakeholders throughout the research process as a keystone of our implementation research in the selection, prioritization, and operationalization of tailored and appropriate implementation strategies to improve the adoption, implementation, and sustainment of community-based management of PSBI. For this purpose, we followed a rigorous and thoughtful development process to systematically plan for the implementation strategies by facilitating a national stakeholder consultation which expanded on adapted expert recommendations for implementing change (ERIC) protocol  employing a modified Delphi approach and woreda-level co-creation.
Consultations with stakeholders
Following the formative assessment and exploration of potential strategies from the literature, we developed an initial IRLM. Then, we conducted a national co-designing workshop on May 11–12, 2021. The workshop involved the Ministry of Health (MOH), National Child Health Technical Working Group, including program managers and development partners; it also included the National RMNCH-N Research Advisory Council Child Health and Immunization group, including their program managers, academics, researchers, and development partners. The workshop also included development partners active in neonatal care in Ethiopia. The workshop’s primary aim was to share the assessment findings and design tailored solutions to address implementation challenges.
On the first day, the research team presented an overview of implementation research and formative assessment findings, followed by a discussion on the barriers to community-based management of PSBI implementation. On the second day, the research team presented the draft IRLM, including the identified determinants and preliminary implementation strategies from the literature. Next, the team facilitated a group process of identification and prioritization of implementation strategies. The research team then populated the determinants and strategies of the IRLM raised through an iterative discussion with stakeholders and used the matrix to understand the conceptual links between determinants, implementation strategies, mechanisms, and outcomes.
Woreda-level contextualization and micro-planning
In the last week of May 2021 and the first week of June 2021, we conducted a woreda-level co-creation workshop with regional health bureaus, zonal health departments, woreda health offices, facility-level managers, service providers, kebele managers, and HEWs. Through the co-creation workshop, L10K and participants contextualized and validated the nationally co-designed implementation strategies, identified additional local contextual community-based management of PSBI implementation challenges that had not been captured during the national co-design phase, designed relevant strategies, and developed micro-plans to improve PSBI treatment implementation. All HPs and health centers in the two implementation woredas developed micro-plans. (See Additional file 4 for the detailed implementation strategies designed during the workshop).
Iterative learning process and capturing ongoing challenges and adaptations of strategies
During program implementation, the project provided technical support to woredas to implement and monitor the adapted strategies. Throughout project implementation and process evaluation, we systematically captured ongoing implementation challenges and adaptation of the implementation strategies using the Framework for FRAME-IS . We facilitated local communities of practice (CoP), learning visits, supportive supervision, and performance reviews to monitor the high fidelity of implementation strategies. In addition, we participated in monthly community-based management of PSBI CoP calls to share examples of work, including the IRLM, mapping strategies, standardized processes, approaches, methods, and metrics with the cross-country teams—including the teams in Kenya and India. The implementation strategies and IRLM were updated throughout the implementation process.
In October 2021, we conducted a process evaluation to measure the implementation fidelity of the adaptive implementation strategies to understand ‘what works and does not work’ by exploring and capturing ongoing implementation challenges and adaptations of strategies.
Pathways of community-based management of PSBI adaptive implementation
Based on the literature findings, formative assessment, and stakeholder engagement, we developed the pre-implementation IRLM for two different contexts—Dembecha and Lume woredas (Fig. 2). We also recognized that while the evidence-based intervention (PSBI treatment when referral was not possible) was the same, contextual factors differed between the two areas. Therefore, the IRLM allowing for two different contexts was used.
Determinants of Implementation
Multiple pre-existing challenges were identified (Table 1), with some differences and similarities across the two woredas.
From the narrative review, we identified the complexity of implementing the community-based management of PSBI intervention as a challenge. Our respondents also reported that the management of PSBI interventions was too complex for HEWs to perform, including identifying signs and symptoms of infection, assessing and classifying cases among newborns, and administering gentamicin injections. HEWs also perceived that the management of PSBI was too complex for them, given their current knowledge and skills. This barrier identified the need to address numerous factors along the care-seeking and caregiving continuum.
Both the formative assessment and narrative review identified the following as major barriers to the management of PSBI program in most HPs: inconsistent supply of essential drugs, poor logistics management systems, shortage of HEWs, the decline of the HEP performance in recent years, and a weak support system. We also identified differences in the magnitude of some factors, such as gentamicin stockout between Lume and Dembecha woredas— 69% of HPs in Lume and 52% of HPs in Dembecha for gentamicin 20 mg/ml on the day of data collection.
COVID-19 pandemic posed an unprecedented challenge to delivering PSBI treatment in the first 2–3 months since the first COVID-19 case in Ethiopia in March 2020. The management of the PSBI program was interrupted due to staff panic and the closure of facilities. More women in Lume compared to Dembecha reported COVID-19 as a barrier to seeking care for their sick children. Maternal fear of contracting COVID-19, infecting their fetus during ANC or their child during either delivery or while seeking postnatal care, was higher in Lume than Dembecha.
The formative assessment identified a high prevalence of newborn illness in rural communities and low uptake of integrated community case management (iCCM) of childhood illnesses and newborn care, particularly for sick young infants (SYI). A higher proportion of mothers who resided in Dembecha (11%) compared to Lume woredas (5%) reported symptoms of severe neonatal infection in their infants. Communities in Dembecha demonstrated lower care-seeking behavior compared to Lume woreda, with only 52% of mothers receiving antibiotics for neonatal sepsis there compared to 70% of mothers in Lume. Most women sought care for their SYI at health centers and bypassed the HPs. Additionally, the following pre-existing factors were identified as external barriers to the delivery of PSBI treatment: phase-out of partner support, low community care-seeking behavior, negative community attitudes toward services for neonatal illness and PSBI, lack of community trust in HEWs’ capacity to manage PSBI, and low functionality of organized female community volunteers called Women Development Armies (WDAs).
Characteristics of individuals
The following characteristics were identified as barriers to HEW management of PSBI implementation: lack of competency and confidence in their PSBI knowledge and skills; lack of training, mentoring, and coaching; low motivation, attendance, and engagement regarding PSBI care implementation; competing workload and engagement in non-health activities; and fear of contracting COVID-19. In Dembecha, HEW knowledge of all PSBI signs was lower than in Lume, with 33% and 41%, respectively. Likewise, a higher number of HEWs untrained in the basics of PSBI were identified in Dembecha compared to Lume.
Process: The formative assessment, as well as the stakeholder consultation, identified the following implementation process barriers to PSBI care: poor integration of management of PSBI services in the woreda health system, weak coordination, ownership, and stakeholder engagement, and a weak culture of learning.
Previous implementation research on the management of PSBI when referral was not feasible conducted in Ethiopia, Malawi, Nigeria, Bangladesh, Pakistan, and India identified several strategies critical to supporting effective implementation at the individual, community, facility, and systems levels. At the individual and community level, these included: training health care workers, in-field technical support, performance reviews , and community engagement [8,9,10,11,12,13,14,15,16, 38]. At the facility and systems levels, strategies included: strengthening the identification and assessment of young infants for illness, improving supply provision, strengthening supply chain management [8, 38], standardization of treatment protocols at the health center and HP levels , policy dialogue, presence of state-level sensitization workshop, and establishment of a technical support unit.
During the stakeholder engagement, based on the formative assessment and narrative review findings, we grouped major management of PSBI implementation challenges that existed before COVID-19 and were exacerbated by the pandemic into the following categories: 1) low competency, confidence, and motivation of HEWs; 2) weak primary health care system; 3) suboptimal community engagement; 4) other systemic factors (i.e., HEW workload and engagement in non-health activities); and 5) weak coordination and integration. We also presented results from the literature review and formative assessment. In consultation with stakeholders, we adopted and adapted a number of the strategies from the literature and developed new ones, including reflecting on COVID-19 -related challenges to enhance the adoption, acceptability, and high-fidelity implementation of community-based management of PSBI interventions, with the aim of further integration and scale up.
As presented in Table 2 below, we identified implementation strategies at various levels. At the individual level, we identified on-site coaching and training of HEWs by PSBI/iCCM-trained focal mentors from health centers as strategies. At the facility- and systems level, we identified the following strategies: coordinated referrals between HPs and health centers; the establishment of a technical support unit to facilitate implementation across the area; strengthening of the supply chain, support system, and links; integration of management of PSBI implementation in the district health system work stream: and integration of COVID-19 and routine services.
Using the IRLM, we described the mechanisms of action that the proposed strategies would operate to influence the desired outcomes. For example, building the organization's capacity for better integration, coordination, and stakeholder engagement would lead to strong district-level operational management and high-fidelity intervention delivery during/after study to maintain strategies. Adoption of the strategies, training and supportive supervision increased skills and interest in delivering PSBI treatment effectively and with higher fidelity. It also strengthened referral systems, addressed community knowledge and fears while increasing fidelity, effectiveness, and adoption (Fig. 3).
At the start of the project, we identified implementation outcomes, proximal impacts of the strategies, and their mechanisms using RE-AIM  and Proctor et al.’s taxonomy of implementation outcomes . Previous studies [10, 14] defined metrics as Reach: number of newborns identified and visited by community health workers, percentage of treatment coverage, and percentage of referral refusal; Effectiveness: recovery, and case fatality rate; Fidelity: percentage of patient adherence to antibiotic treatment, percentage of patients who received follow-up care, percentage of classification errors, and percentage of patients who received correct antibiotic; Acceptability: number of parents/family who refused referral but accepted community-based PSBI treatment, barriers and facilitators of acceptability and care-seeking, percentage of PSBI cases who accepted the referral, and rate of adherence to outpatient treatment. We developed a matrix of indicators and data sources for adaptive implementation of PSBI treatment during COVID-19 in Ethiopia (Table 3). This table includes our metrics: reach, effectiveness, adoption, feasibility, fidelity, acceptability, and maintenance.
The implementation outcomes from the RE-AIM process evaluation are presented below. Most HPs conducted different community dialogues with kebele leaders, WDAs, and kebele-level multi-sectoral meetings to create demand for PSBI treatment. The iCCM trained personnel from the under-five clinic of health centers, health center heads, and woreda child health officers were assigned to serve as a technical support unit and provided on-site training and coaching to HEWs through supportive supervision; facilitated performance reviews; coordinated referrals and provided feedback to HEWs; identified implementation challenges; and supported community mobilization. The program monitoring data showed that almost all HPs received support from their catchment PHCUs at least once from April to September 2021. Out of 107 HEWs available, 87 were trained or mentored on-site on iCCM in the last six months. The data from the HP assessment show that 77 HEWs (72%) from 42 HPs reported participation in at least one PHCU level PRCMM in the last six months (Table 4).
From April-September 2021, HEWs made 5,465 home visits, identified 645 newborn infants, and treated 292 sick young infants. This was an increase of 49% from 149 cases during September 2020-March 2021. Accordingly, fifty-one (80%) HPs treated at least one sick young infant in the previous six months, significantly improving from 36% in the last six months.
Adaptations and modifications of strategies
During implementation, we tracked ongoing implementation challenges and adaptive strategies. We used the IRLM framework to capture these emerging or persistent challenges and adapted strategies to articulate the mechanisms of action and link them to outcomes (Fig. 3).
During the implementation, adaptive implementation strategies were also incorporated, including facilitating the redistribution of gentamicin 20 mg/ml, intensifying the support and monitoring visits, and increasing offsite PSBI treatment training of HEWs and their supervisors at HPs in Dembecha. Due to the civil war in the northern part of the country, a regional emergency decree was issued on October 31, 2021, halting the regular services of public institutions. This subsequent shift in attention and resources away from the management of the PSBI program emerged as a challenge in Dembecha. We provided one round of supervision when the national and regional situations improved. The district health office conducted monitoring visits, performance reviews, and corrective measures to reestablish prior support for community-based management of PSBI services and strengthen their implementation.
In the Lume district, NGO support for maternal and child health was identified as a facilitator for the management of PSBI service during implementation. In addition, the MOH introduced a digital version of the PSBI/iCCM module into the electronic Community Health Information System (eCHIS) in Lume in August 2021. The eCHIS was developed to replace the paper-based information system, a family folder containing household information, registration for incoming and leaving household members (member registration), data regarding service provision, and referral services rendered. The eCHIS incorporated job aids to support service provision, improve quality of care, and improve data use for decision-making. As such, digitization of the PSBI/iCCM treatment module aided HEWs as a decision support tool. It also facilitated digital referral, identification, and registration of newborns and SYI in the app during household visits, leading to improved delivery of community-based management of PSBI services (Table 5).
We successfully employed implementation research frameworks in designing, implementing, and evaluating the adaptive implementation of PSBI treatment during COVID-19 in Ethiopia. The RE-AIM and CFIR frameworks were used to design the COVID-19 pandemic that needed to be addressed to improve the reach, adoption, fidelity, and uptake of community-based management of PSBI services. In addition, we populated an initial IRLM. We used the framework to meaningfully engage stakeholders to systematically develop, identify, categorize, and prioritize implementation strategies to address identified and newly emerged barriers. We found that using an increasingly used framework, which combines CFIR and RE-AIM using concept mapping, effectively identifies strategies by integrating existing evidence and stakeholder engagement and expertise.
The learnings demonstrated the unprecedented challenges of the COVID-19 pandemic, and the pre-existing health system-level barriers exacerbated the fragile community-based management of PSBI in Ethiopia. Previous studies revealed significant service disruptions in outpatient care and child vaccinations ; a decline in clinic attendance and hospital admissions of children [43, 44]; and delayed health-seeking for child health services [45, 46] during the pandemic indicating fear of COVID-19 might intensify delays and reduce access to newborn care. In addition to COVID-19, the following pre-existing factors proved to be the most formidable challenges in the management of PSBI implementation. To address these challenges, we identified the following as key mitigating implementation strategies: the integration of PSBI treatment in the district health system work stream, the integration of COVID-19 services with primary health care services, continuous on-site coaching and training of HEWs, the creation of a robust support system from health centers and woreda health offices, the use of participatory design and implementation methods, the strengthening of the supply chain system, and the presence of community-level SBCC activities. These approaches 1) effectively engaged community volunteers, HEWs, and the primary health care system; 2) recovered services interrupted due to COVID-19; 3) improved community awareness about the iCCM/PSBI treatment availability; and 4) integrated PSBI treatment with the PHCU and woreda workstream for sustained implementation and scaled up.
Using implementation research methods to inform the evidence synthesis using narrative review and formative assessment helped us better identify the challenges and provide an initial understanding of the conceptual links between determinants, implementation strategies, mechanisms, and outcomes. The use of concept mapping is essential in informing the choice of implementation strategies and is associated with more effective implementation [26, 37]. Stakeholder engagement and co-creation have been identified as key steps in designing and implementing complex interventions such as treatment for PSBI [26, 47]. We also found that these strategies helped us gain insights and clarity to develop the PSBI treatment IRLM. Furthermore, our iterative learning and deliberate capturing of ongoing challenges and strategies modifications were critical to understanding and conceptualizing the links between determinants, implementation strategies, mechanisms, and outcomes.
Throughout our implementation research, we employed IRLM and expanded the adapted ERIC protocol and concept mapping technique to engage stakeholders to gain insights and clarity for developing the PSBI IRLM. This allowed for the systematic planning and application of the community-based management of PSBI implementation strategies that consider determinants, mechanisms, and outcomes. Consequently, we bridged implementation gaps by selecting strategies that improved the implementation process, increased adoption, and sustained effective intervention uptake . Current IR theories and prior research lacked consideration of ground-level stakeholder knowledge and preferences. As a result, implementation strategy development and selection challenges surface, which affect the adoption and application of evidence-based interventions [26, 37].
The IRLM enabled the development of adaptive implementation strategies based on an in-depth understanding of the barriers and facilitators. The application of the framework also allowed us to identify implementation challenges, strategies, and mechanisms of action for adaptive implementation of PSBI treatment to increase implementation fidelity, reach, and adoption. Furthermore, these adaptive implementation strategies elucidated changes in the care-seeking behavior of mothers for their sick young infants. IRLM helped guide the measurement of the activities, intermediate outputs, and outcomes and synthesizing lessons from implementation.
During implementation, we tracked ongoing implementation challenges and adaptive strategies, using the IRLM framework to conceptualize the mechanisms of action and outcomes. We also employed this framework to document the implementation learnings and share them with the broader community. This allowed us to examine what happened during the implementation and accurately report the hypothesized relationships we observed. For instance, the digital version of the iCCM module, which is a mobile health (mHealth) solution for HEWs that incorporates job aids— to counsel mothers on essential newborn care and educate them on danger signs of PSBI, assess newborns for possible signs of infection, diagnosis, treat and counsel mothers to adhere to treatment protocols—improved the delivery of PSBI treatment. Specifically, it facilitated identifying and registering newborns and sick young infants and aided clinical and referral decisions and adherence to PSBI treatment protocols. This is in contrast to the responses from the data in Dembecha, where the iCCM module was not incorporated into eCHIS and where few or none of the above advantages from eCHIS were reported. At the same time, close monitoring and timely corrective measures to address emerging contexts like the effect of civil war are critical.
We continued using this framework to share examples of our work in the cross-country community of practice with the community-based management of PSBI Kenya and India teams to cross-map implementation strategies, adaptations, mechanisms of action, and implementation outcomes. We also explored efficient ways to capture contextual factors to build on our IRLM to inform the management of PSBI work at scale. This has informed our continued work across country projects to identify areas that could benefit from standardized processes, approaches, methods, and metrics.
Despite our efforts of exploring all possible challenges, co-designing and implementing strategies with active engagement of stakeholders, capturing ongoing challenges, and adapting strategies throughout the implementation period, we might not include all possible barriers affecting the community-based management of the PSBI program in the model. In addition, some of the determinants of this study may have been affected by social desirability and recall biases which would, in turn, affect the logical links between each component of the IRLM. Though the IRLM provides formats for a standard IR study that include multiple contexts and comparative implementation, it doesn’t explicitly show how to synthesize and report implementation outcomes across various contexts to compare variations in implementation processes and influences across contexts for understanding what works for whom, and how .
In conclusion, we found that the IRLM used in this research helped us organize the formative work and facilitate stakeholder engagement to specify the conceptual links between the implementation challenges, strategies, mechanisms of action, and outcomes to design and strengthen the management of the PSBI program under complex contextual factors, including COVID-19, as well as an existing health system and community barriers. The capturing of ongoing implementation challenges and document modifications of strategies during implementation allowed us to update the IRLM to provide better generalizable knowledge on adapting and implementing management of PSBI in similar and different settings.
The participatory design and implementation of adaptive COVID-19 strategies effectively maintained the quality delivery of PSBI treatment during the pandemic. These approaches effectively engage community volunteers, HEWs, and the primary health care system and better integrate PSBI treatment for sustained implementation and scale up. Continuous support and feedback systems, continuous supply of essential drugs, and strategies to alleviate HEWs’ workload to deliver quality services should be prioritized.
Community-based newborn care
Consolidated Framework for Implementation Research
Electronic Community Health Information System
Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies
Expert Recommendations for Implementation Change
Health Extension Program
Health Extension Worker
Integrated community case management of childhood illnesses and newborn care
Implementation Research Logic Model
Korea International Cooperation Agency
Last Ten Kilometers Project
Ministry of Health
Primary Health care Unit
Performance review and clinical mentoring meeting
Possible serious bacterial infection
Reach, Effectiveness, Adoption, Implementation, and Maintenance
State of Emergency
Sick young infant
United Nations Children's Fund
United States Agency for International Development
Women Development Armies
Ahmed I, Ali SM, Amenga-Etego S, Ariff S, Bahl R, Baqui AH, et al. Population-based rates, timing, and causes of maternal deaths, stillbirths, and neonatal deaths in south Asia and sub-Saharan Africa: a multi-country prospective cohort study. Lancet Glob Health. 2018;6(12):e1297–308.
Puri D, Nisar YB, Tshefu A, Longombe AL, Esamai F, Marete I, et al. Prevalence of clinical signs of possible serious bacterial infection and mortality associated with them from population-based surveillance of young infants from birth to 2 months of age. PLoS One. 2021;16(2):e0247457.
Seale AC, Blencowe H, Manu AA, Nair H, Bahl R, Qazi SA, et al. Estimates of possible severe bacterial infection in neonates in sub-Saharan Africa, south Asia, and Latin America for 2012: a systematic review and meta-analysis. Lancet Infect Dis. 2014;14(8):731–41.
Oza S, Lawn JE, Hogan DR, Mathers C, Cousens SN. Neonatal cause-of-death estimates for the early and late neonatal periods for 194 countries: 2000–2013. Bull World Health Organ. 2014;93:19–28.
Osrin D, Vergnano S, Costello A. Serious bacterial infections in newborn infants in developing countries. Curr Opin Infect Dis. 2004;17(3):217–24.
Celik IH, Hanna M, Canpolat FE, Mohan P. Diagnosis of neonatal sepsis: the past, present and future. Pediatr Res. 2022;91(2):337–50. https://doi.org/10.1038/s41390-021-01696-z.
WHO. Guideline: managing possible serious bacterial infection in young infants when referral is not feasible. World Health Organization; 2015.
Berhane M, Girma T, Tesfaye W, Jibat N, Abera M, Abrahim S, et al. Implementation research on management of sick young infants with possible serious bacterial infection when referral is not possible in Jimma Zone, Ethiopia: Challenges and solutions. PLoS One. 2021;16(8):e0255210. https://doi.org/10.1371/journal.pone.0255210.
Leul A, Hailu T, Abraham L, Bayray A, Terefe W, Godefay H, et al. Innovative approach for potential scale-up to jump-start simplified management of sick young infants with possible serious bacterial infection when a referral is not feasible: Findings from implementation research. PLoS One. 2021;16(2):e0244192.
Goyal N, Rongsen-Chandola T, Sood M, Sinha B, Kumar A, Qazi SA, et al. Management of possible serious bacterial infection in young infants closer to home when referral is not feasible: Lessons from implementation research in Himachal Pradesh, India. PLoS One. 2020;15(12):e0243724. https://doi.org/10.1371/journal.pone.0243724.
Awasthi S, Kesarwani N, Verma RK, Agarwal GG, Tewari LS, Mishra RK, et al. Identification and management of young infants with possible serious bacterial infection where referral was not feasible in rural Lucknow district of Uttar Pradesh, India: An implementation research. PLoS One. 2020;15(6):e0234212.
Roy S, Patil R, Apte A, Thibe K, Dhongade A, Pawar B, et al. Feasibility of implementation of simplified management of young infants with possible serious bacterial infection when referral is not feasible in tribal areas of Pune district, Maharashtra, India. PLoS One. 2020;15(8):e0236355.
Guenther T, Mopiwa G, Nsona H, Qazi S, Makuluni R, Fundani CB, et al. Feasibility of implementing the World Health Organization case management guideline for possible serious bacterial infection among young infants in Ntcheu district, Malawi. PloS One. 2020;15(4):e0229248.
Wammanda RD, Adamu SA, Joshua HD, Nisar YB, Qazi SA, Aboubaker S, et al. Implementation of the WHO guideline on treatment of young infants with signs of possible serious bacterial infection when hospital referral is not feasible in rural Zaria, Nigeria: Challenges and solutions. PLoS One. 2020;15(3):e0228718.
Ariff S, Soofi SB, Suhag Z, Chanar S, Bhura M, Ahmed I, et al. Innovation to Increase Treatment Coverage of Possible Serious Bacterial Infections in Young Infants When a Referral is Not Feasible Implemented in the Existing Health System in Sindh, Pakistan. Preprint. 2021.
Ayede AI, Ashubu OO, Fowobaje KR, Aboubaker S, Nisar YB, Qazi SA, et al. Management of possible serious bacterial infection in young infants where referral is not possible in the context of existing health system structure in Ibadan, south-west Nigeria. PLoS One. 2021;16(3):e0248720.
MOH. National Strategy for Newborn and Child Survival in Ethiopia: 2016-2020. Addis Ababa: Ministry of Health, Ethiopia; 2015.
MOH. Realizing Universal Health Coverage through Primary Health Care: A Roadmap for Optimizing the Ethiopian Health Extension Program 2020–2035. Addis Ababa: Ministry of Health, Ethiopia; 2020.
Tareke KG, Lemu YK, Feyissa GT. Exploration of facilitators of and barriers to the community-based service utilization for newborn possible serious bacterial infection management in Debre Libanos District, Ethiopia: descriptive qualitative study. BMC Pediatr. 2020;20(1):1–14.
Gebretsadik A, Melaku N, Haji Y. Community Acceptance and Utilization of Maternal and Community-Based Neonatal Care Services Provided by Health Extension Workers in Rural Sidama Zone: Barriers and Enablers: A Qualitative Study. Pediatric Health Med Therapeutics. 2020;11:203–17. https://doi.org/10.2147/phmt.S254409.
Tareke KG, Lemu YK, Yidenekal SA, Feyissa GT. Community’s perception, experiences and health seeking behavior towards newborn illnesses in Debre Libanos District, North Shoa, Oromia, Ethiopia: Qualitative study. PLoS One. 2020;15(1):e0227542. https://doi.org/10.1371/journal.pone.0227542.
Asfaha MD, Comeau DL, Spangler SA, Spratt BL, Alamineh L, Gobezayehu AG, et al. Neonatal care and community-level treatment seeking for possible severe bacterial infection (PSBI) in Amhara, Ethiopia. BMC Health Serv Res. 2020;20:1–12.
Onarheim KH, Moland KM, Molla M, Miljeteig I. ‘I wanted to go, but they said wait’: Mothers’ bargaining power and strategies in care-seeking for ill newborns in Ethiopia. PLoS One. 2020;15(6):e0233594. https://doi.org/10.1371/journal.pone.0233594.
Eccles MP, Mittman BS. Welcome to Implementation Science. Implementation Science. 2006;1(1):1. https://doi.org/10.1186/1748-5908-1-1.
Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implementation Science. 2020;15(1):84. https://doi.org/10.1186/s13012-020-01041-8.
Knapp AA, Carroll AJ, Mohanty N, Fu E, Powell BJ, Hamilton A, et al. A stakeholder-driven method for selecting implementation strategies: a case example of pediatric hypertension clinical practice guideline implementation. Implementation Science Communications. 2022;3(1):25. https://doi.org/10.1186/s43058-022-00276-4.
CSA. 2007 population and housing census of Ethiopia. Addis Ababa: Central Statistical Agency (CSA); 2007.
Miller CJ, Barnett ML, Baumann AA, Gutner CA, Wiltsey-Stirman S. The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare. Implement Sci. 2021;16(1):1–12.
Kahsay AB, Luel A, Hailu T, Mulugeta A. Referral practices on community based management of sick young infants with Possible Serious Bacterial Infection in Tigrai, Ethiopia: the caregivers’ perspectives. East African Journal of Health Sciences. 2019;1(2):128–37.
Abdella YM, Bekele A, Mathewos B, Tadesse Y, Wall S, Gardner H, et al. Do Caretakers of Sick Young Infants with Possible Serious Bacterial Infection Adhere to Referrals from Health Posts to Health Centers. Ethiop Med J. 2019;(3):215–22.
Gebremedhin T, Daka DW, Alemayehu YK, Yitbarek K, Debie A. Process evaluation of the community-based newborn care program implementation in Geze Gofa district, south Ethiopia: a case study evaluation design. BMC Pregnancy Childbirth. 2019;19(1):492. https://doi.org/10.1186/s12884-019-2616-9.
Mathewos B, Musema Y, Bekele A, Yeshanew B, Tadele G, Teferi E, et al. Community-based newborn care in Ethiopia: implementation strength and lessons learned. Ethiop Med J. 2019;(3):269–79.
Gebremedhin T, Atnafu A, Dellie E. Community-based newborn care utilisation and associated factors in Geze Gofa rural district, South Ethiopia: a community-based cross-sectional study. BMJ Open. 2020;10(8):e037792. https://doi.org/10.1136/bmjopen-2020-037792.
Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50.
WHO. Service availability and readiness assessment (SARA): an annual monitoring system for service delivery: reference manual. World Health Organ. 2015:1–172.
Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci. 2015;10(1):109. https://doi.org/10.1186/s13012-015-0295-0.
Rahman AE, Herrera S, Rubayet S, Banik G, Hasan R, Ahsan Z, et al. Managing possible serious bacterial infection of young infants where referral is not possible: Lessons from the early implementation experience in Kushtia District learning laboratory, Bangladesh. PLoS One. 2020;15(5):e0232675. https://doi.org/10.1371/journal.pone.0232675.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(2):65–76.
Tiruneh GT, Karim AM, Yihun BY, Birhanu BG, Amaha A, Mathiwos B, et al. Effectiveness of Supervision on the Consistency of Neonatal Sepsis Management Skills of Health Extension Workers. Ethiop Med J. 2019;(3):223–29.
Keitel K, D’Acremont V. Electronic clinical decision algorithms for the integrated primary care management of febrile children in low-resource settings: review of existing tools. Clin Microbiol Infect. 2018;24(8):845–55. https://doi.org/10.1016/j.cmi.2018.04.014.
Shapira G, Ahmed T, Drouard SHP, Amor Fernandez P, Kandpal E, Nzelu C, et al. Disruptions in maternal and child health service utilization during COVID-19: analysis from eight sub-Saharan African countries. Health Policy Plan. 2021;36(7):1140–51. https://doi.org/10.1093/heapol/czab064.
Abdul-Mumin A, Cotache-Condor C, Bimpong KA, Grimm A, Kpiniong MJ, Yakubu RC, et al. Decrease in Admissions and Change in the Diagnostic Landscape in a Newborn Care Unit in Northern Ghana During the COVID-19 Pandemic. Front Pediatr. 2021;9:642508. https://doi.org/10.3389/fped.2021.642508.
Jensen C, McKerrow NH. Child health services during a COVID-19 outbreak in KwaZulu-Natal Province, South Africa. South African Medical Journal. 2021;111(2):114–9. https://doi.org/10.7196/SAMJ.2021.v111i2.15243.
Estifanos AS, Kazmi K, Morris SK. Could COVID-19 Reverse the Modest Gains Made in Newborn Health in Ethiopia? Matern Child Health J. 2021;25(6):849–54.
Gebreegziabher SB, Marrye SS, Kumssa TH, Merga KH, Feleke AK, Dare DJ, et al. Assessment of maternal and child health care services performance in the context of COVID-19 pandemic in Addis Ababa, Ethiopia: evidence from routine service data. Reprod Health. 2022;19(1):42. https://doi.org/10.1186/s12978-022-01353-6.
Peters DH, Bhuiya A, Ghaffar A, Bennett S, Mahmood SS, Edward A, et al. Engaging stakeholders in implementation research: tools, approaches, and lessons learned from application. Health Res Policy Syst. 2017;15(2):104.
Kim B, Sullivan JL, Ritchie MJ, Connolly SL, Drummond KL, Miller CJ, et al. Comparing variations in implementation processes and influences across multiple sites: what works, for whom, and how? Psychiatry Res. 2020;283:112520.
We thank the Bill & Melinda Gates Foundation for funding this implementation research. The implementation of this survey would not have been possible without the support of Ethiopia’s Ministry of Health, Amhara and Oromia Regional Health Bureaus, National Child Health Technical Working Group, National RMNCH-N Research Advisory Council Child Health and Immunization group, and other development partners (Transform Primary Health Care, Fenot project, UNICEF, KOICA, and USAID). We acknowledge the interviewers and the supervisors for their hard work, dedication, and for accomplishing the fieldwork on schedule. Finally, we take this opportunity to extend our gratitude to all study participants who took their time to respond to the survey questionnaires and provide us with invaluable information. Alexandra Kamberos is well-acknowledged for editing the manuscript.
The article write-up and publication fee were supported by the Bill & Melinda Gates Foundation, Investment Number INV-024241. JSI Research & Training Institute, Inc. has helped us in the form of salaries for authors [GT]. However, none of the funders played a role in the study design, data collection, analysis, decision to publish, or manuscript preparation.
Ethics approval and consent to participate
Ethical clearance was obtained from the Ethiopian Public Health Association Research Ethics Review Committee (Reference #: EPHA/OG/166/21 dated April 16, 2021). Written permission to undertake the study was sought from the zonal health departments and institutions. Informed verbal consent was obtained from all the participants and voluntary participation were ensured. All study participants were informed about the study’s purpose, benefits, hazards, and their right to opt-out or not respond to questions. If the respondent was under 18 years old, their assent was sought, and consent was sought from an adult responsible (either from her husband or a guardian). The authors confirm that all methods were carried out in accordance with the relevant guidelines and regulations.
Consent for publication
The authors declare that they have no competing interests. The author [GT] works for JSI Research & Training Institute, Inc., a commercial company. We declare that this commercial affiliation does not alter our adherence to BMC policies on sharing data and materials. One of the authors of this manuscript (HM) works for the Gates Foundation. We would like to declare that we do not have any conflict of interest with the Gates Foundation-paid staff in preparing this manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tiruneh, G.T., Nigatu, T.G., Magge, H. et al. Using the Implementation Research Logic Model to design and implement community-based management of possible serious bacterial infection during COVID-19 pandemic in Ethiopia. BMC Health Serv Res 22, 1515 (2022). https://doi.org/10.1186/s12913-022-08945-9