Simulation-based low-dose, high-frequency plus mobile mentoring versus traditional group-based trainings among health workers in Nigeria; a cluster randomized controlled trial

Background: The aim of this study was to compare health workers knowledge and skills competencies between those trained using the onsite simulation-based, low-dose, high frequency (LDHF) training plus mobile (m) mentoring and the ones trained using the traditional offsite, group-based training (TRAD) approach in Kogi and Ebonyi states, Nigeria, over a 12-month period. Methods: A prospective cluster randomized controlled trial was conducted by enrolling 299 health workers in 60 health facilities in Kogi and Ebonyi states, randomized to either LDHF/m-mentoring (intervention, n=30 facilities) or traditional group-based training (TRAD, n=30 facilities) control arm. These health workers in both arms received basic emergency obstetric and newborn care training with simulated practice using anatomic models and role-plays. The control arm was trained offsite while the intervention arm was trained onsite where they worked. Mentorship was done through telephone calls and reminder text messages. The multiple choice questions and objective structured clinical examinations mean scores were compared; p-value <0.05 was considered statistically significant. Qualitative data were collected and content analysis was done. Results: The mean knowledge scores between the two arms at months 3 and 12 post-training were equally high; no statistically significant differences. Both arms showed improvements in composite scores for assessed BEmONC clinical skills from around 30% at baseline to 75% and above at endline (p <0.05). Overall, the observed improvement and retention of skills was higher in intervention arm compared to the control arm at 12 months post-training, (p<0.05). Some LDHF/m-mentoring approach trainees reported that mentors’ support improved their acquisition and maintenance of knowledge and skills, which may have led to reductions in maternal and newborn deaths in their facilities. Conclusion: The LDHF/m-mentoring intervention is more effective than TRAD approach in improcquisition and retention. Health care managers should have the option to select the LDHF/m-mentoring learning approach, depending on their country’s priorities or context, as it ensures health workers remain in their place of work during training events thus less disruption to service delivery. This study aimed to compare the knowledge and skills competencies of health workers in improving maternal and newborn day of birth care after the LDHF/m-mentoring versus the TRAD training approaches in Ebonyi and Kogi states, Nigeria. The primary outcomes were increase in knowledge, clinical skills, and retention of clinical competency at 3- and 12-months post-training. The secondary outcomes measured were facilitators of and barriers to the LDHF/m-mentoring training approach at individual and institutional levels. Our hypothesis was that LDHF/m-mentoring results in better knowledge and skills outcomes compared to the TRAD approach. shift frequency training approach,

As there is limited quantity and quality of skilled birth attendants in Nigeria, evidence-based approach is needed to train health workers to improve competencies and maternal/newborn outcomes [1,2]. Doctors, nurses and community health extension workers (CHEWs) are commonly trained on day of birth care. Considering the high maternal and newborn mortality and morbidity figures in Nigeria, there is need to train birth attendants on high-impact interventions using competency-based approaches to improve the quality of maternal and newborn care and reduce maternal and perinatal/neonatal mortality [3--6]. It's expected that all health workers have the necessary level of competency to perform life-saving procedures if appropriately trained on basic emergency obstetric and newborn care (BEmONC) functions, including newborn resuscitation and management of bleeding after birth.
Health workers in Nigeria are usually trained using traditional lecture-based, offsite approach in which a limited number of persons from any health facility can be trained at one time. This approach to training is known to be expensive [7][8][9][10]. Maternal and newborn health outcomes have remained suboptimal despite continuous training of health workers [8]. Health workers absenteeism from work to attend offsite training is also a concern for health facility managers [9,10]. Moreover, since only a few health workers are trained each time through the current offsite workshop approach, the quality of care may not be improved to the desired level [3]. Health workers also find it difficult to practice new or updated skills because the entire team members have not been trained on the relevant skills and approaches. In addition, step-down training by those that have been trained most often does not happen as intended due to lack of will and management support among other factors. Health worker turnover and attrition negatively affects the quality of care provided if those leaving are the only ones who received training.
Similar studies done in Ghana and Uganda (7,8) showed promising results with the onsite simulationbased low-dose, high-frequency (LDHF) plus mobile (m)-mentoring training approach. In these studies health workers competencies were better with the LDHF+mMentoring approach compared with the TRAD approach although limited skills were assessed such as newborn resuscitation and postpartum haemorrhage (PPH). The present study however assessed other skills such as active management of labour, management of eclampsia, etc and utilized a more rigorous methodology in order to improve the strength of evidence. There is limited evidence from developing countries including Nigeria on facilitators, barriers, and effectiveness of simulation-based LDHF/m-mentoring learning approaches in improving maternal and newborn health, including day of birth care. In the few studies where the LDHF/m-mentoring approach has been tested, there sample sizes have been limited and methodological approaches have not been standardized [2]. The studies should be adequately powered and use rigorous designs to answer the research questions [2].
The LDHF approach uses in-service learning updates to deliver information based on local needs during short, structured, onsite, interactive learning activities that involve the entire team and are spaced over time to optimize learning. It also involves brief, ongoing activities (e.g., skills practice, team drills, games, and quality improvement activities) at the work place to sustain learning and support clinical decision-making [11]. The approach also uses mobile phone mentorship calls and reminder text messages to the trainees to reinforce gains made from training and resolve emerging issues at the workplaces.
As described by the authors in a previous study [11], the principles of the LDHF approach include:

1.
Competency-focused learning activities concentrates on what providers "need to know"-eliminating what is "nice to know."

2.
Simulation-and case-based learning focuses on skills practice, problem-solving, roleplay, and other interactive exercises. Dosing and frequency depend on topic, extent of the learning gaps, and learner characteristics.

3.
Appropriately spaced, brief periods of learning deliver targeted information in 1 day or over several days.

4.
Team-focused training ensures that all providers have updated clinical practice and can work together to implement improvements in care.

5.
Facility-based training decreases absenteeism, improves teamwork, addresses onsite barriers, and promotes changes to provider performance.

6.
Ongoing practice and quality improvement activities reinforce learning and transfer to clinical practice.

7.
Facility-based peer staff, coaches others as they practice or engage in interactive exercises after learning to increase compliance and improve performance and outcomes.

Study Objectives
This study aimed to compare the knowledge and skills competencies of health workers in improving maternal and newborn day of birth care after the LDHF/m-mentoring versus the TRAD training approaches in Ebonyi and Kogi states, Nigeria. The primary outcomes were increase in knowledge, clinical skills, and retention of clinical competency at 3-and 12-months post-training. The secondary outcomes measured were facilitators of and barriers to the LDHF/m-mentoring training approach at individual and institutional levels.
Our hypothesis was that LDHF/m-mentoring results in better knowledge and skills outcomes compared to the TRAD approach.

Study setting
The study was conducted in Ebonyi

Study design
This was a prospective cluster randomized controlled trial. It was a mixed-method study. Informed consent and ethical approval were obtained. The study was conducted between October 2016 and November 2017 in sixty health facilities that were randomly selected and assigned to either the intervention arm or the control arm. The authors have attempted a brief description of the methodology. The training approaches, eligibility criteria and randomization have been described in the published study protocol [11].
Training approach 1 and data collection: Simulation based LDHF/m-mentoring training of

participants -group 1 or intervention group
The training for the LDHF arm was for the entire team of service providers available at the health facility, but only those who met the study inclusion criteria were assessed. The training was divided into two "low-dose" training courses of 4 days each, with additional time for assessment as needed and was conducted at the health facilities using the basic emergency obstetrics and newborn care (BEmONC) package. The BEmONC package include training on (i) administering parenteral antibiotics, (ii) administering uterogenic drugs for active management of the third stage of labour and prevention of postpartum haemorrhage, (iii) use of parenteral anticonvulsants for the management of preeclampsia/eclampsia, (iv) manual removal of placenta, (v) removal of retained products (e.g. manual vacuum extraction, dilatation, and curettage), (vi) performing assisted vaginal delivery (vii) performing basic neonatal resuscitation [12]. This was for an initial 4days with emphasis on normal uncomplicated cases and repeated after 1 month to emphasize more complicated skills (figure 1  The scores at each assessment were collected using validated tools and recorded in real-time on android devises and sent to a central server after verification. Qualitative data were collected through six focus group discussions (FGD) comprising 8-10 participants per group purposively selected from LDHF arm at 12 months. The participants were greeted and invited via telephone and given full study information including the aims of the study. All the invited respondents gave consent to participate in the study. The FGDs were used to collect data on experiences and satisfaction of trainees with the LDHF/m-mentoring training approach. The Oral consent was obtained. The participants/trainees took pre-training assessments consisting of MCQs and OSCEs through use of manikins to assess their baseline knowledge and skills respectively.
The assessments tested their knowledge and skills on conduct of normal delivery, AMTSL, neonatal resuscitation, case management of PEE and management of PPH (e.g. manual removal of placenta, internal bimanual uterine compression and compression of the abdominal aorta). At the end of the eight-day training, the participants had an immediate post-training assessment which included MCQs and OSCEs. As described for the intervention arm, the questions answered correctly and procedure done competently were scored out of a total of 100%. These were recorded in real-time on android devises and sent to a central server after verification. The assessments results were compared.
Trainees' satisfaction survey was also conducted.
Both study arms were trained and assessed by senior clinicians (mainly obstetricians, pediatricians and midwives) whose knowledge and skills were standardized. The tools were pre-tested among 25 health workers from health facilities that were not part of the study. Before being used, checklists for the assessed clinical skills were modified in accordance with the Nigeria context by the research team and data collectors. At the time of the training, MCSP did not support any interventions in these facilities that are likely to cause contamination. Other quality improvement interventions only happened at the non-study facilities at the time of the study.
The assessors were blinded to the groups the participants were assigned to as they conducted the assessments.

Eligibility criteria
The sixty health facilities were selected from a sampling frame of the 120 MCSP-supported health facilities in the two states located in three geopolitical zones where the project support. They represented all three levels of the healthcare system in Nigeria (Primary health care or PHC, secondary, tertiary). The health workers were drawn from among those working in labor and delivery sections of the participating health facilities in the two states. In addition, the health workers had spent at least six months in the health facility working in either maternal or newborn care section.

Randomization
The unit of randomization in this study was the health facility. The 60 facilities were matched based on locations and level in the health care system, then divided into nine strata taking into consideration the three geopolitical zones as well as the three levels of the health care system in Nigeria. Thereafter, each stratum was randomized to either intervention or control arm using randomly permuted blocks in a ratio of 1:1 so as to achieve balance in geographical location and types of health facilities in the two study arms. Randomization was done by a study team member.
Those assessing outcomes were blinded to the training methodology used for the health facilities.
Since there were at times less than three skilled birth attendants in some health facilities, all health workers employed in the maternity or newborn units in such facilities who met the inclusion criteria were selected from the randomized health care facilities. [5] Sample size The details on the computation of the sample size, including the assumptions are provided in the study protocol manuscript. [5]. Briefly, the required number of participants was computed as the average number of health workers to be included from each of the 60 facilities (30 per arm) using a test of two proportions. The percent of competent health workers was estimated at 50% in the control group. Study power was set at 80% to reject a null hypothesis that the proportions of competent health workers are equal in the two study arms against the alternative hypothesis of 20 percentage point difference in proportions of competent health workers between the two study arms -effect size.
The significance level was set at 0.05. Sample size computation was done using PASS statistical software. The correlation of health workers competency in each health facility was assumed to be 0.05 given that some facility-level factors are shared by the health workers working together and influence how they perform certain tasks. We assumed there are about four health workers selected per facility, thus 240 health workers sampled across the two groups. An adjusted sample size of 300 participants (150 per study arm) was arrived at after factoring in potential 20% drop-out during follow up period after baseline. More health workers were recruited from Kogi State since it had a larger population than Ebonyi. Study participants recruited from each facility included the maternity unit head, wherever possible, and two others to ensure that the other team members received the necessary support to practice.

Data analysis
The  In regards to the qualitative data, transcripts in Microsoft Word were imported into ATLAS.ti software (version 8.0) for content analysis. The codebook was developed by the two (2) qualitative researchers, the data analyst and a second coder, using a priori codes developed from the research aims and questions, and the interview guides adopted from previous studies. In order to enhance validity, we adopted a 1st tier triangulation (of researchers) and ensured a well-documented audit trail of materials and processes. To ensure reliability, refutational analysis, constant data comparison, comprehensive data use was done. Reliability checks were performed by each coder independently re-coding documents already coded by the other.

Results
Two hundred and ninety-nine (299) participants completed the study out of the original 323 providers who were randomized to different study arms; LDHF arm=172; TRAD arm=127 (Table 1 and Figure 3).
Both arms had similar socio-demographic characteristics, but statistical significant difference between the arms was found in Median Time to Training (minutes) and Cadre/Job (Table 1). Table 2 shows that the TRAD arm had better knowledge test scores compared to the LDHF/mmentoring arm at baseline and immediately after the training intervention in some thematic areas, such as AMTSL, essential newborn care, and neonatal resuscitation (p <0.05). However, at 3 and 12 months after training assessment, both arms were equal in knowledge acquisition and retention; no statistically significant differences were noted.
Health workers in both arms showed improvement in overall pass rates in clinical skills competency, improving from around 30% at baseline to 75% and above at end line; difference-in-differences were statistically significant (p <0.05); TRAD (from 27.4% to 74.8%) and LDHF/m-mentoring (from 30.1% to 81.1%). Overall, the observed improvement and retention of BEmONC skills was higher in LDHF/mmentoring study arm participants (81.1%) compared to the TRAD arm participants (74.8%) at 12 months post-training (p<0.05). The LDHF/m-mentoring study arm showed better skills assessment results ( Figure 4). There was a dip in both arms at the three-month assessment. (Figure 5).

Mentoring approach
The respondents reported reduction in maternal and neonatal morbidity and mortality were common theme across states and respondents. In Ebonyi state, a respondent said since after the simulationbased training commenced, they have not had any maternal or neonatal deaths in their facility. In Kogi state, a participant stated a reduced mortality from pre-ecclampsia and ecclampsia because of improved skills among the peer practice coordinators.

Theme 4: Barriers to LDHF/m-Mentoring approach
The barriers mentioned included lack of funding/equipment/supplies and incessant strikes actions.
The cost of obtaining or providing training and guidance from the master mentors was one of the most important barriers identified across the two states. In both Ebonyi and Kogi states lack of finance for the various costs involved in the LDHR/m-Mentoring-related activities was mentioned more frequently and consistently than any other barrier. Different work schedules prevented some trainees from attending some practice sessions and unavailability of equipment hindered some from translating what they learnt into practice.
The research subjects said that:

Discussion
In this cluster randomized control trial, the knowledge assessment scores increase from baseline to end line at 12 months among the study arms. The acquisition and retention of BEmONC skills was different at 12 months with the LDHF/m-mentoring arm demonstrating better performance than the TRAD arm. This study contributes to the much needed evidence base for LDHF/m-mentoring training approach in LMICs, adding on the finding of related studies done in Uganda [7] and Ghana [8]. As part of the intervention package, LDHF/m-mentoring study arm participants received weekly SMS reminders and quizzes along with continuous skills practice on anatomical models, which helped, reinforce knowledge on specific clinical areas over time. In contrast, the TRAD arm participants, which represented the status-quo, did not have the extra support, which might explain the differences in skills performance at 12 months. Furthermore, information gathered qualitatively suggests that a key benefit of m-mentoring is the ability of trainees to call their mentors at the point of care to address complications such as placenta praevia or birth asphyxia. This close mentor-mentee relationship might also have helped the mentees to develop confidence in these crucial skills, which demonstrates a great potential of the LDHF/m-mentoring approach.
Overall, the LDHF arm had better skills performance at all the assessment points compared to the TRAD arm. The qualitative data also support this improvement in knowledge and skills related to maternal and newborn care. Of note, there was an observed drop in skills competency at three months in both arms from the immediate post-training assessment. Some of the possible explanation for the drop in skills competency is that it took time following intense training, before a practice was established as routine. Another possible cause for the drop may have been that there was a health workers' strike that was ongoing just before the post-training assessments in the two states. Despite these factors, the LDHF/m-mentoring arm still performed better in all competencies at 3-and 12months post-training. It's encouraging to note that this observed better performance was in competences that are very crucial for the survival of the mother and the baby on the day of birth; these include, AMTSL, manual removal of placenta, neonatal resuscitation, loading dose of magnesium sulphate, and pre-eclampsia/eclampsia. The continued simulation practice may have helped in building and retaining these competencies.
Studies on BEmONC in-service training have demonstrated the value of simulation and continuous practice in improving knowledge and skills competency [2,7,8]. This has led to adaptation of various approaches to determine which ones led to the greatest returns on investment [3--10]. Our findings are consistent with those of previous studies that reported that TRAD offsite training may improve the trainees' competencies, but knowledge and skills may not be transferred to other co-workers at the facility, nor translated into practice or performance [13][14]. On the other hand, methods that involve repetitive, mentor-supported learning and regularly coordinated practice sessions, such as the onsite LDHF/m-mentoring training, resulted in better skills acquisition and retention over time [1,[7][8][11][12][13][14][15][16][17].
This claim has been supported by other well-designed studies that showed that onsite LDHF/mmentoring training approach, coupled with an efficient mentorship program, resulted in improved provider preparedness to deal with complications [7][8]15].
Findings from the Ghana study that measured patient-level outcomes showed that the LDHF/mmentoring training approach was associated with a sustained decrease in facility-based newborn mortality and intrapartum stillbirths [7]. The limitations acknowledged by Gomez and colleagues in the Ghana study, such as the lack of a randomized control trial design and lack of concealment/information bias, have been addressed in our study using a cluster randomized controlled trial in a similar low-resource setting. Although our study did not measure patient-level outcomes, we assessed a crucial component of the day of birth, namely the competency of the service providers. Our study was adequately powered to produce generalizable conclusions on the comparative effectiveness of the LDHF/m-mentoring and TRAD approaches in the study setting and possibly in similar environments especially in LMICs. As reported by other studies, to be effective, training programs should be conducted as close as possible to the workplace and be competencybased [16][17][18].
The present study also adds to the growing yet still insufficient body of evidence in LMICs supporting the use of simulation, specifically for training in the management of obstetrics and newborn emergencies [19][20][21]. Skills that are not used very frequently in management of obstetric and newborn complications are likely to be non-existent or lost due to a lack of use, and therefore the need to practice often. The appropriate dose or frequency of practice in order to reach competency level depends on many factors, which include the personal attributes of service health worker among other things. Simulation-based trainings provide participants with the opportunity to repeat practice sessions in a safe environment where they will not harm patients [7,8,18,21]. Using the workplace to practice skills is beneficial. The availability of the anatomical models for continuous onsite simulation practice sessions also helps to organize emergency drills to test the health systems response to life-threatening complications.
Another important finding is that some of the facilitators for the LDHF/m-mentoring approach are related to successful outcomes that were reported as health workers improved their skills, which motivated the participants to practice more and seek support from master mentors. Inadequate mentorship should be addressed to improve quality of BEmONC [22]. On the other hand, some of the barriers to this approach included lack of funding, logistics, medical equipment and supplies, . Labor strikes by health workers also limited peer practice sessions. To function maximally, health workers need to improve their competency and they need an enabling environment, including availability of supplies, equipment, appropriate guidelines and policies, and timely remuneration [23, 24]. Our study findings corroborated those of previous studies in LMICs on the effectiveness of the LDHF training approach compared to the offsite traditional training approach to build health workers capacity [7,8].
This study had some limitations. These include some operational-level logistic challenges in ensuring that the mentors always had the airtime in a timely manner in order to maintain seamless communication with the mentees. This was mitigated by the mentors being very understanding by using their resources as they awaited support provided through the study. Another limitation was the health workers' strike which occurred around three months post-training, which potentially affected the findings negatively at that assessment period. Finally, although the study team appealed to MOH not to transfer the study participants to other non-study facilities for duration of the study, some staff were transferred. However, the proportion was very low; they were not included in the analysis.

Conclusion
This is one of the few studies from a middle-income country that has used a pragmatic design-a cluster randomized controlled trial and qualitative methods-to generate much needed information on learning outcomes of two approaches to health worker capacity building. Our study findings corroborated those of similar studies to support the proposition to a shift from traditional offsite group-based training to low-dose, high-frequency and onsite training as an effective way to build the capacity of maternal and newborn care providers. This shift can be explored in other clinical settings to improve the quality of care. The findings suggest that a low-dose, high frequency and simulationand practice-based training approach, including mobile mentoring is likely to result in better skills competency outcomes and higher skills retention among health workers. This study has shown promising results that can address health worker absenteeism due to frequent offsite trainings, which are a major concern for health facility managers in LMICs. Training staff onsite without taking them away from service delivery points is a strong reason for adoption of the LDHF/m-mentoring approach.
The findings are expected to provide health facility managers and other decision makers with valuable information to guide resource allocation for maximum benefit and effectiveness in building the capacity of frontline health workers. The authors recommend that capacity building among healthcare workers, to the extent possible, should utilize the onsite LDHF/m-mentoring learning approach. Further evidence is needed to determine the value of m-mentoring in influencing learning outcomes and the cost-effectiveness of both training approaches as these were not explored in our study.

Consent for publication
Not applicable

Availability of data and material
The datasets generated and/or analyzed during the current study are available in the figshare repository   Theme 2: Trainees' satisfaction with successful outcomes following the simulation-based LDHF/m-Mentoring approach

A.
Successful outcome on maternal survival refers to numerical gains achieved following training in service delivery pertaining to reduced maternal mortality/better maternal survival following trainings and application of skills.

B.
Successful outcome on newborn/neonatal survival refers to successes gained as numerical improvement neonatal/infant survival, reduced neonatal/infant mortality following trainings and application of skills.  Training approach 2 and data collection: Traditional off-site training of participants -group 2 or control group Figure 3 Consort flowchart of enrolment of study participants and data analysis