- Research
- Open access
- Published:
Evaluating the implementation of the Pediatric Acute Care Education (PACE) program in northwestern Tanzania: a mixed-methods study guided by normalization process theory
BMC Health Services Research volume 24, Article number: 1066 (2024)
Abstract
Background
In low- and middle-income countries (LMICs), such as Tanzania, the competency of healthcare providers critically influences the quality of pediatric care. To address this issue, we introduced Pediatric Acute Care Education (PACE), an adaptive learning program to enhance provider competency in Tanzania’s guidelines for managing seriously ill children. Adaptive learning is a promising alternative to current in-service education, yet optimal implementation strategies in LMIC settings are unknown.
Objectives
(1) To evaluate the initial PACE implementation in Mwanza, Tanzania, using the construct of normalization process theory (NPT); (2) To provide insights into its feasibility, acceptability, and scalability potential.
Methods
Mixed-methods study involving healthcare providers at three facilities. Quantitative data was collected using the Normalization MeAsure Development (NoMAD) questionnaire, while qualitative data was gathered through in-depth interviews (IDIs) and focus groups discussions (FGDs).
Results
Eighty-two healthcare providers completed the NoMAD survey. Additionally, 24 senior providers participated in IDIs, and 79 junior providers participated in FGDs. Coherence and cognitive participation were high, demonstrating that PACE is well understood and resonates with existing healthcare goals. Providers expressed a willingness to integrate PACE into their practices, distinguishing it from existing educational methods. However, challenges related to resources and infrastructure, particularly those affecting collective action, were noted. Early indicators point toward the potential for long-term sustainability of the PACE, but assessment of reflexive monitoring was limited due to the study’s focus on PACE’s initial implementation.
Conclusion
This study offers vital insights into the feasibility and acceptability of implementing PACE in a Tanzanian context. While PACE aligns well with healthcare objectives, addressing resource and infrastructure challenges as well as conducting a longer-term study to assess reflexive monitoring is crucial for its successful implementation. Furthermore, the study underscores the value of the NPT as a framework for guiding implementation processes, with broader implications for implementation science and pediatric acute care in LMICs.
Contributions to the literature
Introduces PACE: This study uniquely evaluated the PACE program in a low-resource setting, offering initial evidence on its implementation and potential impact on pediatric care.
Utilizes the NPT framework: By employing a NPT framework, this research provides a novel methodological example of how to assess the incorporation of e-learning in LMIC clinical settings.
Informs Implementation Strategies: These findings contribute to the design of effective e-learning strategies for healthcare education in LMICs, suggesting practical steps for broader application.
Expands Local Capacity: Demonstrates how PACE can build local healthcare capacity, informing ongoing efforts to sustainably improve pediatric care through education in similar environments.
Background
Context and importance of the study
Pediatric in-service education for healthcare providers in Low- and Middle-income countries (LMICs) often lacks reach, effectiveness, and sustainability, contributing to millions of child deaths annually [1, 2]. Pneumonia, birth asphyxia, dehydration, malaria, malnutrition, and anemia cause over 4 million child deaths annually, with half occurring in sub-Saharan Africa and thousands in Tanzania [3, 4]. The Tanzanian government aims to reduce neonatal mortality from 20/100,000 to the Sustainable Development Goals (SDGs) target of 12/100,000 by 2030 [5].
Brief review of the literature
Provider knowledge and skills competency are crucial for care quality in LMICs [2, 6]. However, conventional in-service education methods are often inadequate and unsustainable [6]. These methods do not adapt to individual providers’ knowledge or schedules, target minimal competency, and lack long-term refresher learning, limiting their effectiveness [7,8,9,10].
Adaptive learning can address these limitations by customizing the timing and sequence of combined e-learning and in-person skills training, creating individualized pathways that reinforce learning and enhance skills competency. This approach helps mitigate manpower and resource shortages in LMICs and represents a strategic innovation in knowledge dissemination.
The World Health Organization (WHO) emphasizes the importance of e-learning solutions for healthcare workers globally [11]. Adaptive learning, with its capacity to adjust to individual needs, holds significant promise for enhancing training efficiency. However, formal studies on adaptive learning in LMIC contexts are scarce. Establishing best practices in e-learning and adaptive methodologies will enhance the dissemination of evidence-based interventions and improve clinical practice and patient outcomes.
To address current educational limitations for healthcare workers in LMICs, we developed the Pediatric Acute Care Education (PACE) program [12, 13]. This adaptive e-learning program offers 340 learning objectives across 10 assignments, covering newborn and pediatric care guidelines for management of seriously ill children. The PACE program’s implementation strategy includes an adaptive e-learning platform optimized for mobile phones, a steering committee, a full-time PACE coordinator, and an escalating nudge strategy to encourage participation.
Study aims and objectives
The primary aim of this research is to assess the preliminary implementation of the PACE intervention across two types of pediatric acute care facilities: zonal hospitals and health centers. The study has two principal objectives: (1) To evaluate the initial PACE implementation in Mwanza, Tanzania, using the constructs of Normalization Process Theory (NPT); (2) To provide insights into its feasibility, acceptability, and scalability potential.
Methods
Study design
This study employed a mixed methods approach to evaluate the implementation of the PACE program in three healthcare settings in northwestern Tanzania, nested within a larger pilot implementation of PACE within eight health facilities of the Pediatric Association of Tanzania’s Clinical Learning Network. The study utilized NPT as a framework, combining quantitative and qualitative methods. Quantitatively, a tailored NoMAD survey instrument evaluates the integration of PACE into routine clinical practice. Qualitatively, in-depth interviews and focus group discussions enrich the data.
Theoretical framework
NPT has been described as a sociological toolkit for helping us understand the dynamics of implementing, embedding, and integrating new technology or a complex intervention into routine practice [14]. NPT provides a conceptual framework for understanding and evaluating the processes (implementation) by which new health technologies and other complex interventions are routinely operationalized in everyday work (embedding) and sustained in practice (integration) [15,16,17,18,19,20]. The theory is organized around four main constructs, each of which has its own subconstructs [15]. These constructs collectively offer insights into the feasibility, acceptability, and scalability of an intervention or innovation (Fig. 1). Each of these constructs and subconstructs offers a unique lens through which the feasibility, acceptability, and scalability of a new practice can be evaluated, thereby aiding in its effective implementation.
Study setting
The study was conducted between August 2022 and July 2023 at three healthcare facilities in Mwanza, Tanzania. The Bugando Medical Centre (BMC), an urban zonal referral and teaching hospital, sees about 7,000 births per year and 6,550 pediatric admissions per year for children aged 1 month to 5 years; the urban Makongoro Health Centre, handles approximately 359 births per year but refers newborn and pediatric admissions to the nearby regional or zonal hospital; and the rural Igoma Health Centre sees about 3,850 births per year and 959 pediatric admissions per year for children aged 1 month to 5 years.
Providers
Eligibility criteria
Providers included in the study were required to have a minimum command of English and be actively providing pediatric care to sick patients at least part-time. Eligible providers encompassed a wide range of professional cadres, reflecting the diversity of healthcare providers in Tanzania. These included specialists (medical officers with 3 additional years of specialization), medical officers (5 years of education and 1-year internship), nursing officers (4 years of education and 1-year internship), assistant medical officers (clinical officers with 2 additional years of clinical training), assistant nursing officers (3 years of education), clinical officers (3 years of education), clinical assistants (2 years of education), enrolled nurses (2 years of education), and medical attendants (1 year of education). In addition to providers, senior facility staff with administrative roles who supervise PACE providers, such as ward matrons, medical officers-in-charge, and nursing officers-in-charge, were eligible to participate. The bulk of the care is provided by junior medical officers and nurses, who have limited training and experience caring for children with severe illnesses
Recruitment process
Healthcare providers were informed about the study through their facility leaders, and individuals who responded to the survey were not necessarily the same as those who participated in the focus groups or in-depth interviews.
Data collection tools
NoMAD questionnaire
The NoMAD is a 23-item questionnaire based on the NPT that was designed to assess the social processes influencing the integration of complex interventions [18, 21]. It includes 3 general items and 20 related to specific NPT constructs (4 Coherence, 7 Collective Action, 4 Cognitive Participation, 5 Reflexive Monitoring). The general items were scored on a scale of 0-100, and the NPT construct items were modified to include a five-point Likert scale (1-Strongly Agree, 5-Strongly Disagree) and additional options for respondents to indicate whether a question was not relevant to their role, stage, or intervention itself. The NPT subconstruct survey items are listed in Table 1, and the complete survey is provided in the Supplementary Materials.
In-depth interviews (IDIs) and focus group discussions (FGDs)
Interview guides were developed based on previous experience with similar data collection tools. The training and pretesting of the tools were conducted by the study investigators.
Data collection process
NoMAD survey
All PACE participants were invited via WhatsApp to complete the NoMAD survey directly in REDCap, 30 days post-intervention or upon completion of the PACE course.
Focus group discussions and in-depth interviews
We employed a purposeful sampling strategy for the qualitative components, selecting senior healthcare providers for in-depth interviews (IDIs) and junior providers for focus group discussions (FGDs). This approach ensured junior providers felt comfortable speaking openly, avoiding inhibition from senior participants in focus groups, and facilitated methodological triangulation to enhance the credibility and validity of the findings. Data was triangulated using three different types: methodological triangulation with IDIs and FGDs, investigator triangulation with different research assistants collecting data, and data triangulation using data from IDIs, FGDs and NoMAD surveys. Data collection began with a series of field visits, guided by NPT constructs, and included IDIs and FGDs. FGDs, segregated by sex but including a mix of cadres from each health facility, enriched the diversity of perspectives. The iterative nature of our methodology allowed for continuous refinement of our theoretical framework, methodologies, and sampling strategies, informed by emerging data. Consequently, the guides for both the IDIs and FGDs were dynamically modified to reflect the evolving study themes. All sessions, including IDIs and FGDs, were conducted in Kiswahili at the providers’ work premises, adding contextual depth. The IDI and FGD interview guides were originally developed in English, translated into Kiswahili (the national language), and then back translated into English to ensure that the meaning was retained. Both IDIs and FGDs were meticulously audio-recorded, transcribed verbatim, and then translated into English for analysis. Back-translation was employed to ensure validity.
Data analysis
Quantitative analysis
Descriptive statistics are reported as frequencies and percentages or medians and interquartile ranges, with comparisons via Fisher’s exact test or the Mann‒Whitney U test as appropriate. Analyses were conducted using Stata 17.0 (Stata Corp, College Station, TX, USA).
Qualitative analysis
The analysis process, conducted concurrently with data collection, was instrumental in achieving theoretical saturation, marked by the cessation of new information from ongoing IDIs and FGDs. To ensure the validity and depth of our findings, we implemented member checking and investigator triangulation, with two independent investigators coding and interpreting the data using NVivo 2020 software (QSR International Pty Ltd., Sydney, Australia). This software facilitated a hybrid coding approach in which blended deductive and inductive methods were used for comprehensive thematic content analysis. Contextual insights from the IDIs and FGDs were key to interpreting the findings, with representative quotations included to illustrate the identified themes. Data triangulation was achieved using diverse data sources, and the research team’s expertise further enhanced the rigor and reflexivity of the analysis.
Summary of feasibility, acceptability and scalability
We used the Proctors definition of implementation outcomes and mapped the NoMAD survey results to NPT subconstructs using the definition of May et al. [22, 23].
Feasibility is concerned with the practical aspects of implementing a new intervention, including resource allocation, training, and ease of integration into existing work. In the NPT, this aligns closely with the construct of “collective action,” which refers to the operational work that people do to enact a set of practices. To assess feasibility, we interpreted our responses as follows: “Sufficient training is provided to enable staff to use PACE” (collective action, skill set workability); “Sufficient resources are available to support PACE”; “Management adequately supports PACE” (collective action, contextual integration); and “I can easily integrate PACE into my existing work” (collective action, interactional workability).
Acceptability refers to the extent to which the new intervention is agreeable or satisfactory among its users. To assess acceptability, we interpreted our responses as follows: “Staff in this organization have a shared understanding of the purpose of PACE” (coherence: communal specification); “I believe that participating in PACE is a legitimate part of my role” (cognitive participation, legitimation); “The staff agree that PACE is worthwhile” (reflexive monitoring, communal appraisal); and “I value the effects PACE has had on my work” (reflexive monitoring, individual appraisal). In addition, we compared scores between zonal hospitals and health centers.
Scalability involves the ability to expand the intervention to other settings while maintaining its effectiveness. To assess scalability, we interpreted our responses as “I will continue to support PACE” (cognitive participation, activation); “Work is assigned to those with skills appropriate for PACE” (collective action, skill set workability); “feedback about PACE can be used to improve it in the future”; and “I can modify how I work with PACE” (reflexive monitoring, reconfiguration).
Ethical considerations
All the providers provided informed consent, and the study was approved by the Institutional Review Board of the Tanzania National Institute of Medical Research (NIMR/HO/R.8a/Vol. IX/3990), Stanford University (60379), the ethics committee of the Catholic University of Health and Allied Science (no ID number given), and the Mwanza Regional Medical Officer (Ref. No. AG.52/290/01A/115).
Techniques to enhance trustworthiness
Techniques to enhance trustworthiness included a purposeful sampling strategy, meticulous data collection in Kiswahili with back-translation, and the use of methodological, investigator, and data triangulation [24]. The analysis process was iterative and concurrent with data collection, employing hybrid coding and member checking to ensure systematic, explicit, and reproducible findings.
Reporting guidelines
This study adheres to the STROBE and SRQR reporting guidelines for comprehensive and explicit reporting of observational and qualitative studies, respectively [25, 26].
Results
Provider demographics
Eighty-two of the 272 eligible healthcare providers from the three facilities completed the NoMAD survey, resulting in a 30% response rate. Of the 82 respondents, 59 were from zonal hospitals and 23 from health centers (Table 2). The median ages were 27 and 29 years for zonal hospital and health center staff, respectively. The gender distribution was similar in both settings, with 39% female in the zonal hospital group and 43.5% in the health centers.
There were significant differences in cadre distribution: zonal hospitals had more medical staff (47.5% vs. 8.7%) and nurses (42.4% vs. 30.4%), while health centers had more clinical officers (30.4% vs. 0%). Clinical experience also varied, with a median of 1 year at zonal hospitals and 4 years at health centers (p = 0.004). Previous participation in newborn or pediatric in-service education (e.g., Helping Babies Breathe, Helping Children Survive) was similar across the facilities, ranging from 71% to 73%. Job satisfaction scores did not significantly differ between the two groups.
A total of seventy-nine healthcare providers participated in IDIs or FGDs. Twenty-four senior providers completed IDIs, 18 from the zonal hospital and 6 from health centers., 13 FGDs with an average of 4 junior providers per group were conducted to achieve thematic saturation, including 39 participants from zonal hospitals and 16 from health centers. The represented cadres included medical officers (26, 32.9%), nurses (19, 24.1%), interns (16, 20.3%), clinical officers (12, 15.2%), assistant medical officers (3, 3.8%), and medical attendants (3, 3.8%). Clinical experience among participants ranged from 1 to 20 years. Compared to the NoMAD survey, participants in IDIs and FGDs included a higher proportion of medical officers (including interns) and clinical officers, but a lower proportion of nursing officers and other cadres.
NoMAD survey results
General items
Familiarity and general satisfaction with PACE were high, with median scores of 89 and 91, respectively, and both showed moderate, balanced variability (interquartile ranges of 76-100 and 75-100, respectively) (Table 3, Fig. 1). Optimism for the future use of PACE was highest, with a median score of 99 and narrow variability (87-100), indicating a strong skew towards higher scores. No significant differences were observed between the zonal hospitals and health centers.
NPT constructs
Coherence
Providers reported understanding how to work together and plan the activities to put PACE and its components into practice. Strong agreement on the value of PACE is indicated by the median score for “Internalization" (1, “strongly agree,” IQR [1, 2]) (Table 3, Fig. 1). Agreement on PACE’s purpose and its differentiation from existing work is indicated by the median scores for “Communal Specification” (2, “agree,” IQR [1, 2]), “Differentiation” (2, “agree,” IQR [1, 4]) and “Individual Specification” (2, “agree,” IQR [2, 4]), respectively. No significant differences were observed between the zonal hospitals and health centers.
Cognitive participation
Providers reported understanding how to work together to create networks of participation and communities of practice around PACE and its components. Strong agreement for ongoing PACE support, PACE participation and leadership, and PACE integration into work is indicated by the median scores for “Activation” (1, “strongly agree,” IQR [1, 1]), “Enrollment” (1, “strongly agree,” IQR [1, 1]), “Initiation” (1, “strongly agree,” IQR [1, 1]), and “Legitimation” (1, “strongly agree,” IQR [1, 2]), respectively. Narrow IQRs highlight the homogeneous support among providers. No significant differences were observed between the zonal hospitals and health centers.
Collective action
Providers reported understanding how to work together to enact PACE and its components, with greater certainty of not disrupting working relationships in the zonal hospital compared to health centers. Strong agreement that the work required by PACE is manageable, has sufficient training and resources, and receives strong organizational support is indicated by the median scores for “Interactional Workability” (1, “strongly agree,” IQR [1, 1]), “Skill-set Workability” (1, “strongly agree,” IQR [1, 2]) and “Contextual Integration” (2, “agree,” IQR [2, 2]) (Table 3, Fig. 1). Agreement that PACE does not disrupt working relationships is indicated by the median score for “Relational Integration” (4, “disagree,” IQR [4, 5]). Zonal hospital providers had significantly less variability that PACE would not disrupt working relationships (relational integration)compared to health centers(IQR [4, 5] vs [3, 5] p=0.02).
Reflexive monitoring
Providers reported understanding how to work together to evaluate the benefits of PACE and its components. Strong agreement on how people individually assess the value of PACE is indicated by the median score for “Individual Appraisal” (1, “strongly agree,” IQR [1, 1]). Agreement on how people access information to assess the value of PACE, how to value PACE collectively, and work adjustments needed for PACE is indicated by the median scores for “Systematization” (2, “agree,” IQR [2, 2]), “Communication Appraisal” (2, “agree,” IQR [2, 2]), and “Reconfiguration” (2, “agree,” IQR [2, 2]) (Table 3, Fig. 1). No significant differences were observed between the zonal hospitals and health centers.
IDI and focus group results
Coherence themes
Providers value PACE for its detailed guidance on specific pediatric cases, such as difficulty breathing, which was not covered in their basic training (Table 4). PACE is seen as a tool for empowering providers to reduce child mortality and improve service quality, aligning with facility goals. Providers believe that PACE has enhanced their understanding and management of seriously ill children. They find that PACE is consistent with Tanzanian and WHO guidelines and useful both in their work and in training medical students.
Cognitive participation themes
Providers were introduced to PACE by colleagues and supervisors, prompting them to enroll (Table 4). They mainly use PACE individually but also share modules to spread knowledge. PACE is seen as empowering providers to enhance their pediatric care. Despite busy schedules, providers are committed to PACE training.
Collective action themes
PACE's digital format allows for individual study and facilitates group discussions (Table 4). Initially, providers engaged with PACE for personal benefit but later saw the value in sharing knowledge. Providers value the practical application of PACE knowledge in patient care. Challenges like inadequate supplies and a lack of electricity hinder PACE implementation, but the availability of tools and support from PACE management facilitates implementation.
Reflexive monitoring themes
Providers find PACE valuable for educating junior doctors, simplifying complex topics, and boosting confidence (Table 4). They believe that PACE has enriched their knowledge and confidence in pediatric care. A notable challenge is the inaccessibility of learned material for future reference, hindering providers’ ability to refresh their knowledge.
Summary of feasibility, acceptability, and scalability
Overall, data from NoMAD survey responses indicated that PACE is generally feasible across healthcare settings, with providers either agreeing or strongly agreeing that people do the work required by interventions and their components (interactional workability median 1 “strongly agree” [1, 2]) or that the work of interventions and their components is supported by host organizations (contextual integration median 2 “agree” [1, 2]).
Furthermore, NoMAD survey responses indicated that PACE is also generally acceptable among healthcare providers. Providers collectively agreed about the purpose of PACE and its components (communal specification median 2 “agree” [1, 2]), agreed that PACE and its components are the right thing to do and should be part of their work (legitimation median 1 “strongly agree” [1, 2]), and collectively and individually agreed that PACE is worthwhile (communal appraisal median 2 “agree” [1, 2]; individual appraisal median 1 “strongly agree” [1, 2]).
Lastly, NoMAD survey responses indicated that PACE appears to be scalable, with some variability in its adaptability and skill-set alignment. Providers strongly agreed that they would continue to support PACE and its components (activation median 1 “strongly agree” [1, 1]), that they could modify their work in response to their appraisal of PACE, and that feedback could be used to improve it in the future (reconfiguration median 1 “strongly agree” [1, 2]). Providers agreed or were neutral about the work of PACE and its components being appropriately allocated to people (skill-set workability median 3 “neutral” [2, 4]), indicating that additional work is needed to identify the correct providers to participate in PACE or that additional support needs to be allocated to those providers to complete PACE.
Discussion
This mixed-methods pilot study explored the feasibility, acceptability, and scalability of the PACE intervention among healthcare providers in Mwanza, Tanzania, using the NPT framework. The study demonstrated that PACE is generally well understood, aligns with existing healthcare goals, and is feasible to providers. There was strong acceptance and understanding that PACE should become part of normal work. Challenges to scalability lie in ensuring adequate resource and infrastructure support. Qualitative data from IDIs and FGDs enriched the findings by providing detailed insights that supported and contrasted with the NoMAD survey results, highlighting both the strengths and challenges of implementing PACE in a resource-limited setting.
Interpretation of findings
The study demonstrated that PACE is feasible. It is generally well understood by healthcare providers and aligns with existing healthcare goals. Providers found PACE to be practical in enhancing their ability to manage pediatric cases, particularly those not adequately covered in their basic training. For instance, providers appreciated the detailed guidance PACE offers for managing conditions like difficulty breathing, which they found invaluable. This alignment with healthcare objectives, such as focusing on improving service quality of newborn and child acute care to reduce child mortality underscores PACE’s potential for integration into routine clinical practice. The fact that PACE aligns with both Tanzanian and WHO guidelines further reinforces its relevance and applicability in the local healthcare context.
Provider training programs that focus on improving specific clinical performance objectives tend to yield better outcomes compared to those that cover broad topics. Targeted training programs, such as those designed to enhance specific clinical skills, have been shown to significantly improve the competency and confidence of healthcare providers. For instance, a systematic review we conducted in 2010 demonstrated that provider education programs in LMICs that focused on the needs and resources of the local healthcare environment had greater effectiveness [8]. Similarly, a study by Bluestone et al. (2013) found that focused training in neonatal resuscitation improved the performance of healthcare providers in emergency situations, as evidenced by increased neonatal survival rates [27]. In contrast, broad-topic training programs, while valuable for general knowledge enhancement, often lack the specificity needed to address critical clinical skills gaps effectively. As highlighted by Frenk et al. (2010), broad educational approaches may not adequately prepare providers for the complex, high-stakes situations they encounter in practice [28]. Therefore, training programs with a clear focus on enhancing specific clinical skills are generally more effective in improving clinical performance and patient outcomes.
The study demonstrated that PACE is acceptable. There is strong acceptance and understanding among providers that PACE should become part of their normal work. This cognitive participation reflects a high level of engagement and willingness to incorporate PACE into daily routines. Providers recognized the value of PACE in improving their knowledge and skills, with many noting that the program had significantly enhanced their understanding and management of seriously ill children. They also found PACE useful in training medical students and junior doctors, indicating its potential for broader educational impact. This widespread acceptance and integration into daily work routines suggest that PACE is viewed not just as an additional resource but as a vital component of their professional development.
When individuals perceive that a new activity should become part of their normal work, it is often associated with increased usage and integration into their daily routines [29, 30]. This concept, known as cognitive participation, reflects a high level of engagement and commitment, which positively influences the adoption and sustained use of new practices. For instance, a study by May et al. (2009) on Normalization Process Theory highlighted that when healthcare providers viewed new clinical practices as integral to their work, they were more likely to implement them consistently [31]. Similarly, if people recognize the value of a new activity, there is substantial evidence that this recognition leads to increased usage and behavior change. Michie et al. (2011) found that perceived usefulness and perceived ease of use are significant predictors of the intention to use and actual usage of new interventions [32]. Furthermore, Rogers’ Diffusion of Innovations theory (2003) emphasizes that when individuals see clear benefits and value in a new practice, they are more likely to adopt it, leading to a transformation in their behavior and routines [33]. These findings collectively suggest that cognitive acceptance and perceived value are critical drivers of the successful implementation and sustained usage of new activities in various contexts.
However, challenges to scalability remain, particularly in ensuring adequate resource and infrastructure support. While the program itself is well-received, practical barriers such as adequate time to complete adaptive e-learning or participate in skills practice sessions and health system internet support hinder its full implementation. In addition, providers reported difficulties in accessing necessary equipment and managing cases during power outages, which directly impact their ability to apply PACE training effectively and cement long-term knowledge and skills. These challenges highlight the need for systemic improvements in resource allocation and infrastructure to support the sustainable and effective integration of PACE into the healthcare system. Without addressing these critical barriers, the scalability of PACE may be limited, preventing it from reaching its full potential impact.
Three strategies would address these challenges: 1) Strengthening Digital Infrastructure, 2) Flexible Scheduling and Time Management, and 3) Provision of Essential Equipment and Resources.
Strengthening digital infrastructure
Investing in robust digital infrastructure is crucial for the successful implementation of e-learning programs. Ensuring reliable internet connectivity and access to digital devices can significantly enhance the feasibility of adaptive learning modules. UNICEF’s conducted a review of digital learning programs in low-resource settings that highlights the positive impact of improved digital infrastructure [34]. Additionally, providing technical support and maintenance can prevent disruptions and ensure the smooth operation of online learning platforms (Aranda-Jan et al., 2014).
Flexible scheduling and time management
Allowing healthcare providers flexible scheduling to complete adaptive e-learning modules and participate in skills practice sessions can mitigate time-related barriers. Research by Yardley et al. (2012) demonstrates that flexible learning schedules increase participation and completion rates in professional development programs. Implementing self-paced learning options and modular training formats can help healthcare providers integrate training into their busy schedules without compromising clinical duties.
Provision of essential equipment and resources
Ensuring the availability of necessary medical equipment and resources is essential for the practical application of training programs. Partnerships with governmental and non-governmental organizations can facilitate the procurement and distribution of essential tools. A study by Bertram et al. (2018) suggests that strategic resource allocation and collaborative efforts can address equipment shortages and improve healthcare delivery. Additionally, creating contingency plans for managing power outages, such as providing backup power solutions, can enhance the reliability of training programs in resource-limited settings.
Qualitative data from focus groups and interviews enriched the findings by providing detailed insights that both supported and contrasted with the NoMAD survey results. These qualitative insights highlighted the strengths of PACE, such as its alignment with Tanzanian guidelines and its educational value, while also revealing challenges like resource constraints. Providers shared specific examples of how PACE had positively impacted their clinical practice, such as improving their ability to manage emergencies and enhancing their confidence in providing care. However, they also pointed out the difficulties in sustaining PACE’s benefits without adequate support and clinical resources to translate this knowledge into improved care delivery. This mixed-methods approach offered a comprehensive understanding of the implementation process, emphasizing the importance of addressing both the strengths and weaknesses of PACE in a resource-limited setting. The contrast between the high satisfaction reported in surveys and the practical challenges discussed in interviews underscores the need for a mixed methods approach when implementing new complex interventions in such environments.
Implications for implementation science and pediatric acute care
This study highlights the utility of the NPT as a conceptual framework for understanding the complexities involved in implementing adaptive learning interventions in LMICs. The findings provide valuable insights into the various factors that influence the implementation of adaptive learning, which can be applied to other healthcare interventions.
For pediatric acute care, the strong agreement among healthcare providers on the benefits of PACE for managing specific pediatric cases suggests that the program could significantly enhance provider proficiency and improve patient outcomes. Given the often time-sensitive nature of pediatric acute care, where timely and effective interventions such as oxygen therapy, intravenous fluids, and anti-microbial therapy can have a significant impact on patient outcomes, the effective and efficient training provided by PACE could lead to improved patient outcomes. Additionally, the consistency of these findings across various implementation contexts points to the scalability of the program, indicating its potential to be effectively expanded to other healthcare settings.
Limitations
The study has several limitations. The small sample size limits the generalizability of the findings, and the low response rate of 30% may introduce response bias. Additionally, the study's short duration did not allow for a comprehensive assessment of all NPT constructs, particularly reflexive monitoring. The reliance on self-reported data may also introduce social desirability bias. Our mixed methods approach, and methodological triangulation enhance the robustness of the findings despite these limitations.
Recommendations for future research
Future research should focus on longitudinal studies to assess the long-term sustainability and impact of PACE on provider proficiency, patient outcomes, and the quality of care. More rigorous qualitative research designs, such as detailed case studies and ethnographic studies, could provide a deeper understanding of the challenges and opportunities associated with implementing PACE. Additionally, research should explore the scalability of PACE, assessing how the program can be adapted for different healthcare settings and evaluating the resource implications of scaling up the intervention.
Conclusions
This study offers valuable insights into the feasibility, acceptability, and scalability of implementing PACE in a Tanzanian context. While PACE aligns well with healthcare objectives, addressing resource and infrastructure challenges is crucial for its effective and sustainable implementation. The study underscores the value of the NPT as a framework for guiding implementation processes, with broader implications for implementation science and pediatric acute care in LMICs. Future researchers can apply these insights by ensuring alignment with facility goals, engaging stakeholders early, planning for long-term evaluations, addressing resource challenges proactively, and considering the specific context and available resources when assessing scalability.
Availability of data and materials
Deidentified participant data from this study are available upon reasonable request. Interested researchers may obtain the data by contacting the corresponding author, Dr. Peter Meaney, at meaneypa@stanford.edu. Access to the data will be granted following approval by an independent review committee established to evaluate the scientific validity and ethical justification of the proposed use of the tool. Please note that only the deidentified participant data are available, and no additional supporting information, such as study protocols or statistical analysis plans, will be provided. This process ensures that the data are used responsibly and in accordance with ethical research standards.
Data availability
Deidentified participant data from this study are available upon reasonable request. Interested researchers may obtain the data by contacting the corresponding author, Dr. Peter Meaney, at meaneypa@stanford.edu. Access to the data will be granted following approval by an independent review committee, established to evaluate the scientific validity and ethical justification of the proposed use. Please note that only the deidentified participant data is available, and no additional supporting information, such as study protocols or statistical analysis plans, will be provided. This process ensures that the data is used responsibly and in accordance with ethical research standards.
Abbreviations
- BMC:
-
Bugando Medical Centre
- e-learning:
-
Electronic Learning
- FGD:
-
Focus Group Discussion
- IDI:
-
In-Depth Interview
- LMIC:
-
Low- or -Middle-Income Country
- NPT:
-
normalization process theory
- NoMAD:
-
normalization measure development
- PACE:
-
Pediatric Acute Care Education
References
Kruk ME, Gage AD, Joseph NT, Danaei G, García-Saisó S, Salomon JA. Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries. Lancet. 2018;392(10160):2203–12.
Lewis TP, Roder-DeWan S, Malata A, Ndiaye Y, Kruk ME. Clinical performance among recent graduates in nine low- and middle-income countries. Trop Med Int Health. 2019;24(5):620–35.
Perin J, Mulick A, Yeung D, Villavicencio F, Lopez G, Strong KL, et al. Global, regional, and national causes of under-5 mortality in 2000–19: an updated systematic analysis with implications for the sustainable development goals. Lancet Child Adolesc Health. 2022;6(2):106–15.
Sharrow D, Hug L, Lee S, Liu Y, You D. Levels & Trends in child mortality: report 2021, estimates developed by the United Nations Inter-agency Group for Child Mortality Estimation. New York: United Nations Children’s Fund; 2021.
UNICEF Data. United Republic of Tanzania (TZA) - demographics, health & infant mortality. Available from: https://data.unicef.org/country/tza/. Cited 2023 Jun 11.
Di Giorgio L, Evans DK, Lindelow M, Nguyen SN, Svensson J, Wane W, et al. Analysis of clinical knowledge, absenteeism and availability of resources for maternal and child health: a cross-sectional quality of care study in 10 African countries. BMJ Glob Health. 2020;5(12):e003377.
Meaney PA, Sutton RM, Tsima B, Steenhoff AP, Shilkofski N, Boulet JR, et al. Training hospital providers in basic CPR skills in Botswana: acquisition, retention and impact of novel training techniques. Resuscitation. 2012;83(12):1484–90.
Meaney PA, Topjian AA, Chandler HK, Botha M, Soar J, Berg RA, et al. Resuscitation training in developing countries: a systematic review. Resuscitation. 2010;81(11):1462–72.
Bloom BS. The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. Educ Res. 1984;13(6):4–16.
Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D, Rowe AK. A systematic review of the effectiveness of strategies to improve health care provider performance in low- and middle-income countries: methods and descriptive results. PLoS ONE. 2019;14(5):e0217617.
WHO. Recommendations on digital interventions for health system strengthening. World Health Organization: Geneva; 2019. Licence: CC BY-NC-SA 3.0 IGO.
Meaney PA, Hokororo A, Masenge T, Mwanga J, Kalabamu FS, Berg M, et al. Development of pediatric acute care education (PACE): an adaptive electronic learning (e-learning) environment for healthcare providers in Tanzania. Digit Health. 2023;1(9):20552076231180470.
Meaney P, Hokororo A, Ndosi H, Dahlen A, Jacob T, Mwanga JR, et al. Feasibility of an Adaptive E-Learning Environment to Improve Provider Proficiency in Essential and Sick Newborn Care in Mwanza, Tanzania. medRxiv; 2023. p. 2023.07.11.23292406. Available from: https://www.medrxiv.org/content/10.1101/2023.07.11.23292406v1. Cited 2023 Aug 30.
Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18(1):135.
May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54.
May C. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res. 2006;6(1):86.
May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4(1):29.
May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13(1):80.
Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8(1):63.
McEvoy R, Ballini L, Maltoni S, O’Donnell CA, Mair FS, MacFarlane A. A qualitative systematic review of studies using the normalization process theory to research implementation processes. Implement Sci. 2014;9(1):2.
Finch TL, Rapley T, Girling M, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: measure development based on normalization process theory (NoMAD): study protocol. Implement Sci. 2013;8(1):43.
May CR, Albers B, Bracher M, Finch TL, Gilbert A, Girling M, et al. Translational framework for implementation evaluation and research: a normalisation process theory coding manual for qualitative research and instrument development. Implement Sci. 2022;17(1):19.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38(2):65–76.
Greenhalgh T, Taylor R. How to read a paper: papers that go beyond numbers (qualitative research). BMJ. 1997;315(7110):740–3.
von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–8.
O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med J Assoc Am Med Coll. 2014;89(9):1245–51.
Bluestone J, Johnson P, Fullerton J, Carr C, Alderman J, BonTempo J. Effective in-service training design and delivery: evidence from an integrative literature review. Hum Resour Health. 2013;11(1):51.
Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376(9756):1923–58.
Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet Lond Engl. 2003;362(9391):1225–30.
Eccles MP, Grimshaw JM, MacLennan G, Bonetti D, Glidewell L, Pitts NB, et al. Explaining clinical behaviors using multiple theoretical models. Implement Sci. 2012;7(1):99.
May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, et al. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 2007;7(1):148.
Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci IS. 2011;23(6):42.
Rogers EM, Singhal A, Quinlan MM. Diffusion of innovations. In: An integrated approach to communication theory and research. London: Routledge; 2014. p. 432–48.
Effectiveness of digital learning solutions to improve educational outcomes | UNICEF. Available from: https://www.unicef.org/documents/effectiveness-digital-learning-solutions-improve-educational-outcomes. Cited 2024 Aug 7.
Lawrence CE, Dunkel L, McEver M, Israel T, Taylor R, Chiriboga G, et al. A REDCap-based model for electronic consent (eConsent): oving toward a more personalized consent. J Clin Transl Sci. 2020;4(4):345–53.
Acknowledgements
We are grateful to healthcare providers from Bugando Medical Centre, Makongoro and Igoma Health Centres in Mwanza City Tanzania, who participated in the study for their time, cooperation, and invaluable feedback. The authors would like to thank the Pediatric Association of Tanzania; the Tanzanian Ministry of Health, Regional and Council Health Management Teams for participating in stakeholder meetings. We thank the Catholic University of Health and Allied Sciences, London School of Hygiene and Tropical Medicine, Paediatric Association of Tanzania, and Area 9 for practical support.
REDCap database
Study data were collected and managed using REDCap electronic data capture tools hosted at Stanford University.71,72 Research Electronic Data Capture (REDCap) is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trials for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources. The Stanford REDCap platform (http://redcap.stanford.edu) was developed and operated by the Stanford Medicine Research IT team. The REDCap platform services at Stanford are subsidized by a) the Stanford School of Medicine Research Office and b) the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grant UL1 TR001085. Area9 Rhapsode™ meets the requirements for full GDPR compliance, including encryption, data security, and ‘forget me’.
Use of ChatGPT
We employed ChatGPT in two ways: 1) to edit and revise manuscripts for clarity to ensure adherence to writing standards (such as STROBE and SRQR) and to format manuscripts, tables, and figures; and 2) to conduct preliminary data screening/monitoring using deidentified data exported from our REDCap databases. All the statistical results and conclusions in this manuscript were confirmed by a biostatistician or member of the author group.
Patient and public involvement
This research was performed without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient-relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.
Funding
1. This study was funded by the Laerdal Foundation for Acute Medicine, Stanford University School of Medicine Maternal and Child Health Research Institute, Stanford Center for Innovation in Global Health, and the Stanford University School of Medicine Division of Pediatric Critical Care Medicine.
2. Funding sources had no role in the project design; data collection, analysis, or interpretation; reporting; or decision to submit results for publication.
3. Stanford CTSA award number UL1 TR001085 from the NIH/NCRR.
Author information
Authors and Affiliations
Contributions
R.M., A.H., H.N., and P.A.M. made substantial contributions to the conception and design of the study, as well as to the acquisition, analysis, and interpretation of data. T.M., N.C., N.P.M., and A.A. were involved in the conception and design of the study and contributed significantly to the interpretation of the data. F.S.K., M.B., and Z.S. played key roles in the conception and design of the study and contributed to data analysis. D.T., R.P.M., B.R., and C.M. were primarily responsible for data analysis and interpretation. E.D. contributed to the interpretation of data. All authors were involved in drafting the manuscript or revising it critically for important intellectual content. Each author has approved the final version to be published and agrees to be accountable for their contributions. They have also committed to ensuring that questions related to the accuracy or integrity of any part of the work, including parts in which they were not personally involved, are appropriately investigated, resolved, and documented in the literature. In accordance with the guidelines of BMC journals, we have collaborated with local colleagues where the research was conducted. They have been included as co-authors as they meet all the authorship criteria mentioned above. Contributors who did not meet all criteria for authorship are acknowledged separately. Changes to the authorship list post-submission, including any changes in the order of authors, the deletion or addition of authors, or changes in the corresponding author, will be managed according to the journal’s policy. Such changes will be finalized only upon agreement by all authors and completion of the required change of authorship form.
Author’s information
1. Joseph R. Mwanga, B.A., MSc., PhD: Senior Lecturer and Qualitative Researcher at the Catholic University of Health and Allied Sciences, Tanzania. Focuses on pediatric care and public health initiatives.
2. Adolfine Hokororo, M.D., MMed, MSc: Chief of Quality Improvement at CUHAS and Clinical Epidemiologist. Brings extensive experience in clinical quality leadership, with significant influence on healthcare policies in Tanzania through her work with NIH and USAID.
3. Hanston Ndosi, M.D.: PACE Program Manager at the Catholic University of Health and Allied Sciences, Tanzania. A crucial role is played in healthcare program management and implementation.
4. Theopista Masenge, M.D.: Vice Chair of the Pediatric Association of Tanzania and Pediatric GI Specialist. Leads implementation programs in child health and holds an MBA.
5. Florence S Kalabamu, M.D.: Pediatric Specialist at Hubert Kairuki Memorial University. Research has focused on health law and ethics.
6. Daniel Tawfik, M.D.: Assistant Professor of Pediatrics (Critical Care) at Stanford University School of Medicine. We are practicing at Stanford Medicine Children's Health, specializing in Pediatric Critical Care Medicine, and is a member of the MCHRI.
7. Rishi P Mediratta, M.D., MSc, MA: Clinical Associate Professor in Pediatrics at the Stanford University School of Medicine. A Pediatric Hospitalist at Lucile Packard Children’s Hospital and Faculty Fellow at CIGH. His work includes contributions to pediatrics and public health research in Ethiopia and teaching about the implications of COVID-19.
8. Boris Rozenfeld, M.D.: Learning Architect for Healthcare Education at Area9 Lyceum. Leads a team of medical learning engineers with a strong background in CME and educational technology.
9. Marc Berg, M.D.: Clinical Professor in Pediatrics - Critical Care at Stanford University School of Medicine. Her research interests included CPR performance and pediatric defibrillation science through simulation. Medical Director of the Revive Initiative for Pediatric Resuscitation Excellence.
10. Zack Haines Smith, M.D., M.P.H.: Pediatric Critical Care Physician at Kaiser Permanente, Oakland, CA.
11. Neema Chami, M.D.: Neonatologist at Bugando Medical Center, Pediatric Faculty at CUHAS, Subject Matter Expert and Learning Engineer for PACE, and Key Member of the Pediatric Association of Tanzania.
12. Namala P Mkopi, M.D.: Tanzania’s first Pediatric Critical Care Physician, leading Pediatric Critical Care at Muhimbili National Hospital and Subject Matter Expert and Learning Engineer for PACE.
13. Castory Mwanga, M.D.: Senior Pediatrician and Leader at Simiyu Designated District Hospital, Subject Matter Expert and Learning Engineer for PACE, focusing on pediatric healthcare services.
14. Enock Diocles, M.D.: Nurse Educator at Mwanza College of Allied Sciences, Researcher, Clinician, and Founding Member of the Tanzanian Pediatric Nursing Association.
15. Ambrose Agweyu, M.D., MSc.: Kenyan Pediatrician and Professor of Epidemiology at the London School of Hygiene and Tropical Medicine. Influential in adapting WHO guidelines for Kenya and leading clinical trials for childhood pneumonia treatment.
16. Peter Andrew Meaney, M.D., MPH: Clinical Professor at Stanford University School of Medicine specializing in Pediatric Critical Care Medicine. His research focuses on provider education, implementation science, and quality care in resource-limited settings. He is an Associate Program Director for the T-32 Pediatric Subspecialty Global Health Fellowship and a Global Health Faculty Fellow at CIGH.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The Institutional Review Board of the Tanzania National Institute of Medical Research (NIMR/HO/R.8a/Vol. IX/3990), Stanford University (60379), the ethics committee of the Catholic University of Health and Allied Science (no ID number given), and the Mwanza Regional Medical Officer (Ref. No. AG.52/290/01A/115) approved the study protocol, including consent. The data collection procedures were completed in compliance with the guidelines of the Health Insurance Portability and Accountability Act (HIPAA) to ensure subject confidentiality. Informed electronic consent was obtained through REDCap from all providers who participated in the PACE [35]. All providers who provided consent were included. All the surveys and questionnaires were entered directly by providers into REDCap. This study is reported according to the Consolidated Standards of Reporting Trials (CONSORT) 2010 extension to randomized pilot and feasibility trials.
Consent for publication
All individual persons’ data included in this study are entirely unidentifiable, and there are no details on individuals reported within the manuscript. Therefore, consent for publication is not applicable to this study. If there were any identifiable details, images, or videos relating to individual persons, consent would be obtained from those persons or, in the case of children, their parent or legal guardian as noted in this section accordingly.
Competing interests
BR and MB are compensated by Area 9 Lyceum as Senior Learning Architect and Medical Director, respectively.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Mwanga, J.R., Hokororo, A., Ndosi, H. et al. Evaluating the implementation of the Pediatric Acute Care Education (PACE) program in northwestern Tanzania: a mixed-methods study guided by normalization process theory. BMC Health Serv Res 24, 1066 (2024). https://doi.org/10.1186/s12913-024-11554-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12913-024-11554-3