Skip to main content

Evaluation of COVID-19 ECHO training program for healthcare workers in India - A Mixed-Method Study

Abstract

Background

The Coronavirus Disease 2019 (COVID-19) has severely challenged healthcare delivery systems worldwide. Healthcare Workers were unable to assess and manage the cases due to limited knowledge of treating the virus and inadequate infrastructure. Digital interventions played a crucial role in the training of healthcare workers to get through the pandemic. Project Extension for Community Healthcare Outcomes (ECHO) initiated the COVID-ECHO telementoring program for strengthening the knowledge and skills of healthcare workers. The study aimed at assessing the effects of the ECHO telementoring model in the capacity building of healthcare workers in the context of COVID-19 in India.

Method

We adopted a mixed-method approach with a parallel combination design. A quantitative survey was used to measure changes in the knowledge and self-efficacy among doctors and nurses. In-depth Interviews were used for qualitative exploration of perceptions and experiences of all the study participants. Student t-test and ANOVA were used to assess significant differences between mean scores across participant characteristics for different themes. Statistical significance was set at p < 0.05. In-depth Interviews were analyzed using Framework Analysis. The evaluation followed the first five levels of Moore’s model.

Results

The results highlighted the strengthening of knowledge and skills of healthcare workers in the assessment and management of COVID-19 after the ECHO training. Learning and performance ratings were high as 96% reported an increase in knowledge and 98% were able to apply it in their clinical practices. The key challenges identified were technical issues like internet connectivity and lack of interaction due to limited visual connection. The hybrid sessions, use of video camera, feedback mechanism, and inclusion of Continuing Medical Education were recommended by participants to improve the model.

Conclusions

The findings of this study are an important addition to the pre-existing literature supporting the replicability of the ECHO model in the upskilling of healthcare professionals working in underserved and remote areas, not only in the context of COVID-19 but also in other public health domains. To enhance the effectiveness of this ECHO model, the study findings may be used to refine the model and improve the areas of concern.

Peer Review reports

Background

The novel coronavirus disease (COVID-19) outbreak due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was first reported in Wuhan, China, on 31 December 2019. Due to an alarming rise in the spread, World Health Organization (WHO) officially declared COVID-19 as a pandemic on 11 March 2020 [1]. The abrupt rise in COVID-19 cases affected the healthcare system of many countries which were unable to cope with the rising number of cases and limited knowledge of treating the novel virus. This was true especially in low- and middle- income countries (LMICs) like India where the health system is overburdened and has inadequate infrastructure [2, 3]. Previous studies suggest that the disease outbreak significantly elevated the psychological stress levels of healthcare workers (HCWs) due to rapidly changing medical information, overwhelming infection control procedures, dealing with logistic challenges, and associated stigma that leads to an increase in disease transmission and lower level of patient care [3,4,5]. Education intervention at the organization level can play a pivotal role in overcoming the adverse psychosocial impact caused by COVID-19 [6,7,8].

The training and capacity building of healthcare workers (HCWs) has been identified as one of the important strategies under the National Patient Safety Implementation Framework (NPSIF) 2018–2025 as well as in the Regional Strategy for Patient Safety (RSPS) in the WHO south-east Asia Region (2016–2025) [9]. Learning through continuous medical education and intensive training will increase the knowledge and skills to cope with public health emergencies [10]. The Government of India launched a special digital platform iGOT for training and capacity building of healthcare staff including doctors, nurses, paramedics, hygiene workers, and technicians thus stepping up efforts to fight the coronavirus pandemic [11].

Digital interventions can provide many opportunities for strengthening health systems and can be vital resources in the current public health emergency [2]. The applications of digital technology for training and capacity building during public health emergencies are well known. Digitalization of services provides an excellent forum for national and international experts to give advice and support to inexperienced HCWs through online training and mentoring [12, 13].

The Extension for Community Healthcare Outcomes (ECHO) is a model that provides informative training and case discussion by medical experts via telementoring, in urban and rural settings, connecting through the web-based videoconferencing platform. It uses a “hub and spoke” model, in which the ‘hub’ is the central physical location from which a specialist team hosts the clinic and the ‘spokes’ are HCWs who connect remotely from their workplace [14, 15]. In February 2020, Project ECHO took the initiative to implement the COVID-ECHO model for training and mentoring HCWs in India. The telementoring COVID-ECHO program is aimed at building capacity and strengthening knowledge of HCWs.

The present study aimed at assessing the effects of the ECHO telementoring model in the capacity building of HCWs in the context of COVID-19 in India. Specific objectives of the study were to:

  1. 1)

    Understand the barriers and facilitators in using the ECHO telementoring model for COVID-19.

  2. 2)

    Examine the healthcare providers’ knowledge, competence, performance, and satisfaction associated with the use of the COVID-ECHO telementoring program.

  3. 3)

    Explore the potential of the ECHO telementoring model for the expansion of clinical services beyond COVID-19.

Methods

Study design

The study involves a mixed-method approach using a parallel combination design [16], where quantitative and qualitative data were collected in the same phase. We adapted this approach to maximize the validity, reliability, and confirmability of the findings.

Study setting and participants

The study was conducted at four ECHO hubs located in different states of India - Punjab, Assam, Kerala, and Tamil Nadu from January 2021 till April 2021. Inclusion criteria for selection of hubs was based on the minimum number of sessions (ten training sessions) conducted and the minimum number of participants (500 participants) trained. The study participants for survey data were doctors and nurses (trainees) who had received training. For qualitative data, we included other stakeholders involved in the implementation of the training program i.e. hub-leaders and trainers.

Sampling method and recruitment

A non-probability, purposive sampling was used to select the universe of the trainees. This was followed by simple random sampling for the quantitative survey. Doctors and nurses who attended at least three or more sessions of this program and agreed to participate in the study were approached for the survey.

For qualitative inquiry, we used In-depth Interviews (IDIs). Purposive sampling with maximum variation was used to generate different perspectives from all stakeholders. Interviews were conducted until data saturation was achieved.

Data collection and study tools

The data was collected through telephonic (cell phone) interviews to avoid exposure/risk to both data collectors and participants because of the COVID-19 situation. A data collection team that had expertise in telephonic interviews was selected and trained by the research team in a two-day training program to collect the data.

For the quantitative study, a structured survey tool was developed for this study. This questionnaire was designed keeping Moore’s Expanded Outcomes Framework for assessing learners and evaluating instructional activities. This tool was developed by adapting instruments used by previous researchers to assess similar ECHO educational projects in low- and middle-income countries [17,18,19]. We used a 5-point Likert scale to assess the participant ratings of knowledge and self-efficacy after attending the COVID-ECHO program. This was designed in the CS Pro software (version 7.5) – an offline electronic data capture system (EDC) and data were collected using CS Pro smartphone application.

Qualitative data was led by two researchers (NM and SL) who worked closely with the field data collectors. Separate discussion guides were developed for conducting IDIs with different stakeholders. The interviews were audio-recorded after taking participant's consent, each of which lasted for 20-30 minutes. Audio recordings were translated and transcribed verbatim by a professional transcriptionist.

Study size

To estimate the sample number for the survey, we assumed a 10% improvement in knowledge, learning, and skills; anticipated 8% precision and response rate 60% with a design effect factor of 1.5. The sample estimated was 135 participants from each hub (state). The final sample size for this study was 540 participants. These were selected from 4 hubs that consented to participate in the study.

A total of 28 IDIs were conducted with trainees as well as other stakeholders (hub-leaders, trainers).

Data analysis

Discrete variables are reported as frequency, proportions, and continuous variables as mean± SD. The mean scores were calculated using Likert 5 scale for the different themes. Student t-test and ANOVA were used to assess significant differences between mean scores across participant characteristics for different themes. Subgroups analysis was performed for different subgroups (doctors and nurses). The regression diagnostic and residual plot were used for testing violation of assumptions. Polynomial Linear Regression was performed to identify the factors associated with favourable learning and competence, performance, and satisfaction scores. Age and years of experience variables were used as the polynomial variables in the regression model. All statistical analysis was done using STATA Version 16 software (StataCorp. 2019. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC). All tests were two-sided, and statistical significance was set at p < 0.05.

The qualitative data was analysed by a framework analysis approach [20]. To familiarize with the data two researchers (NM and SL) reviewed the transcripts, coded them independently, and identified initial themes. Data coding was done using both deductive and inductive approaches. A third trained researcher (AM) helped develop consensus around themes and addressed disagreements between the two initial coders. After reconciliation, 90% agreement with the codes was achieved and the final codebook was revised. Clusters of linked codes were grouped into categories, emergent themes, and verbatim quotes. Atlas ti software version 8 was used for the data analysis.

Data quality assurance

To check the data credibility in quantitative data, 10% of the data was analyzed by two researchers (KS and RP). The researchers conducted a preliminary descriptive analysis of the 10% data and checked for inconsistencies and missing data. To ensure the accuracy of qualitative data, we reviewed the audio recordings and compared them with transcriptions, and field notes of data collectors.

Results

The quantitative and qualitative findings were arranged with Moore’s first five levels of evaluation. In addition, the qualitative findings explored challenges faced during the training as well as understanding the potential and scope of the viability of the ECHO program.

Moore’s evaluation

Moore’s Model of Outcomes Assessment framework for continuing medical education (CME) focusses on the first five outcome levels - participation, satisfaction, learning, competence, and performance [21].

Moore’s level 1 - participation

Table 1 presents the participants’ demographic characteristics. The survey included 523 participants, out of which 306 (58.5%) were doctors and 217 (41.5%) were nurses. Most of the participants were female (66%), and were nurses (92%). The overall mean age of the participants was 36.1±10.1 years. Among doctors and nurses the mean age was 36.8±10.5 years and 35.2±9.37 years, respectively.

Table 1 Participants demographic characteristics

Most of the participant’s (41%) practiced in primary care facilities that included primary health care centre and sub-centres. The doctors (47%) in our study were from tertiary care facilities whereas half of the nurses (49%) were from primary care facilities. One-third of the doctors (39%) were having a MBBS degree while one-fourth of nurses (26%) were having a diploma nursing degree.

Moore’s level 2 – satisfaction

Seven questions assessed participants’ satisfaction with the COVID-ECHO telementoring program. Most participants (97%) were satisfied with the ECHO model (Figure 1), with participants either ‘agreeing’ or ‘strongly agreeing’ with the majority of satisfaction items. They felt that the sessions were easy to understand and have met their expectations in terms of information shared and duration of session. The t-test and ANOVA results showed that the dentists (doctors with BDS degree) were more likely to be satisfied (32.9±3.8) with the session compared to doctors with MBBS degree (30.2±3.3) or MD degree (31.3±3.4; p=0.002). The doctors who were practicing in private facilities showed greater satisfaction levels (32.9±3.0) when compared to doctors practicing at primary (29.8±3.5), secondary (31.4±3.3), tertiary (31.0±3.3; p=0.002) facilities (Table 2).

Fig. 1
figure 1

Participant's average learning, competence, satisfaction, and performance associated with the use of COVID-ECHO program

Table 2 T-test and ANOVA summary—Doctors

The qualitative findings complement the quantitative findings and suggest that participants were satisfied with the model. The findings indicate that the sessions were interactive and participants’ experience with trainers was satisfactory.

“The model was very good, it was very practical, and it met my expectations” (Doctor, ID 5.20)

“It was a good interaction in the sessions. we got to meet people sitting thousands of kilometres away, (and with) doctors across the globe” (Trainer, ID 17.3)

Participants also noted the advantages of the virtual learning platform.

“It's [medical profession] dynamic, things keep changing, New protocols or new guidelines come, we get to learn and do things now and then. So, this platform helps us to gain knowledge faster and easily” (Doctor, ID 21.16)

“We need to go someplace to learn [for physical sessions]. For example, if I need to learn something. I need to travel someplace. Once we got it virtually, there is no need to travel, and it is a time-saving factor” (Doctor. ID 22.10)

Moore’s level 3 - learning

The learning and competence among the participants were assessed together in the quantitative survey. Around 96% of participants reported improvement in knowledge and clinical skills for assessing and recognizing COVID-19 symptoms after attending the program (Figure 1). Participants reported increased confidence in advising and managing patients with COVID-19 after attending these sessions

The t-test and ANOVA results showed that the dentists (doctors with BDS degree) (57.2 ±4.8) had a higher mean score on the knowledge scale as compared to doctors with MBBS (52.2±5.6) or MD degrees (53.7±5.5; p=0.001). Doctors practicing in private health facilities which include personal clinics reported higher mean score on the knowledge scale (54.6 ±6.0) when compared with doctors practicing in primary (51.3±5.5), secondary (53.9±5.9), and tertiary (53.7±5.5; p=0.006) facilities (Table 2). Among the nurses, there was no significant difference in the mean scores on the knowledge scale related to their qualification, practice site, and practice location (Table 3). The findings from the qualitative data are in tune with the quantitative findings.

“I liked the fact that when we started off, we had no knowledge about the COVID. But through these sessions, we got a lot of knowledge” (Nurses, ID 14.3)

Table 3 T-test and ANOVA summary – Nurses

A few study participants mentioned improved task performance and skill development.

“On this ECHO platform, we trained them on how to wear PPE (Personal Protective Equipment) kits, how to wash hands, how to protect healthcare workers” (Trainer, ID 15.5)

“So, dealing with dementia cases is also very new to us. It was helpful interacting with people from all over the country with experts from NIMHANS (National Institute of Mental Health and Neurosciences)” (Doctor, ID 5.27)

Moore’s level 4 - competence

The qualitative findings show that the training helped participants gain confidence in their practice.

“Some of the people have sent their messages to me that after listening to you on this particular topic I could manage with this thing” (Trainer, ID 3.4)

“It was really helpful in gaining enough confidence to write on the prescription of psychiatric drugs” (Doctor, ID 5.14)

Moore’s level 5 - performance

Participants were also asked to rate their self-reported performance on a 5-point Likert scale. Majority of participants (above 95%) have reported that they have implemented the learning in their clinical practice and disseminated the knowledge with colleagues (Figure 1).

The dentists (doctors with BDS degree) showed a higher self-reported performance mean score (18.9±2.0) compared to doctors with MBBS degree (17.3±1.9) or a MD degree (18.0±2.0; p=<0.001). The doctors practicing in urban areas had a higher performance mean score (17.9±2.0) compared to doctors in rural areas (17.3±2.0; p=0.007) (Table 2). There were no statistically significant differences in performance mean scores among nurses related to their qualification, practice site, and practice location (Table 3). The participants reported that they had applied these knowledge and skills in treating their patients. Many of the trainees spoke of gaining knowledge and felt that they were more efficient in handling patients after joining the training program.

“I used it [learning] in ICU, and in the wards on my patients” (Doctor, ID 12.10) “As far as I feel, my patients benefitted too.” (Nurse, ID 13.18)

Participants also felt that there was a positive change in their behaviour towards patient care after attending the training program.

“My empathy level has increased that I can see in my interaction with the patients. Earlier I was a little bit robotic I have now become more human. Now I am looking into the psychosocial aspects also an equally important part, it is having a tremendous impact in recovery of the patient.” (Hub-leader, ID 4.13)

Regression analysis: After adjusting for demographic factors i.e. age, gender, qualification, caste, years of experience, practice site, and polynomial terms of age and years of experience in current job, the dentists (doctors with BDS degree) had a significantly higher learning score (Coef. 3.97 95% CI :0.89,7.04; p=0.012) compared to doctors with MBBS degree. The doctors practicing in secondary (Coef. 2.76 95% CI :0.37,5.14; p=0.024) and tertiary (Coef. 2.11 95% CI :0.41,3.80; p=0.015) care settings had significantly higher learning scores as compared to primary care settings. The doctors practicing in secondary care settings (Coef. 1.53 95% CI :0.08,2.98; p= 0.03) were more satisfied as compared to doctors in primary care settings. The dentists had a significantly higher self-reported performance score (Coef. 1.28 95% CI :0.19,2.37; p=0.021) as compared to MBBS doctors (Additional file 3, Appendix 1).

Challenges during COVID-ECHO training

Five themes (limited interaction, difficulty to follow up, time constraint, technical issues, and need for physical training) were identified as challenges faced by the participants during these training. A summary of challenges is mentioned in (Table 4).

Table 4 Summary of challenges

Viability of ECHO program

The majority of participants felt that the training program is effective, and most of the trainees were in the favor of attending future ECHO sessions.

“I liked their manner of explaining very much, their method of telling. That is why I would certainly want to attend their next session” (Nurse, ID 13.23)

“So, beyond COVID-19 and cancer there is the scope of expanding this ECHO session [training in other clinical services]” (Trainer, ID 10.24)

Based on their experience, participants also recommended strategies to overcome the challenges and improve the model. We summarized the recommendations under the given themes in (Table 5).

Table 5 Recommended strategies to improve model

Discussion

Our study findings reveal that participants of the COVID-ECHO telementoring program were satisfied with the virtual ECHO platform, the sessions as well as the trainers conducting the sessions. Participants reported improvement in their knowledge and confidence in COVID-19 assessment and management after attending the sessions. These findings are similar to other studies that have used Project ECHO for building capacity in chronic pain [22], hypertension [23], and pain management in dementia [24]. Participants also expressed that the COVID- ECHO sessions improved their clinical practice and benefitted their patients. The results of this study indicate the ability of the COVID-ECHO program to reach out to remote populations for building capacity in evidence-based COVID-19 protocols. This was made possible through telementoring and an interactive system that integrates telecommunications into the planning, design, and delivery of training programs.

Interprofessional collaborative practice is considered an ideal method of improving staff satisfaction while benefitting the community [25, 26]. Participants perceived that virtual training was more relevant than physical training for collaboration because of the advantages of a larger and more diverse audience. The discussions in the COVID sessions were centered around expert’s opinions and global best practices. Our quantitative and qualitative results are consistent with the findings of previous assessments of Project ECHO [27,28,29].

Our study findings regarding providers’ level of satisfaction towards the telementoring program, our data is aligned with previous reports by Zhou et al. [29]. The review by Zhou examined thirteen studies that measured satisfaction levels in primary care providers. In all the thirteen studies conducted in North America, the participants indicated a high level of satisfaction and performance with the ECHO telementoring model [27, 29]. Our findings regarding participants’ improvement in their interaction as well as practice with patients after attending the ECHO sessions are in accordance with other studies which report improved relationships with patients after attending the ECHO training [30, 31]. The mean ratings for the learning, competence, satisfaction, and performance in this study were found to be consistently above 4 on the 5-point scale. This finding is similar to the other studies that have also assessed the effectiveness of the ECHO telementoring model in North America and Ireland from 2000 to 2018 [14, 19].

Participants also described some of the limitations of the training such as administrative issues like inconvenient schedule of sessions, and technical issues like internet connectivity, and video conferencing. Technical issues, in particular internet connectivity and bandwidth, have been identified as problematic by other studies evaluating online training [24, 32]. By their very nature, virtual learning environments and platforms like ECHO are subject to technical issues, such as security, network, and low bandwidth. Hence, the strategies for addressing them on a priority basis will help build more confidence in such models. A further challenge in the training as stated by few participants was a lack of interaction due to limited visual connection and face-to-face interactivity with the trainees. A few participants believe skill development cannot be achieved through online training while some other participants do not see these issues as major problems in future ECHO clinics.

To improve the model few suggestions were put forward by the participants. One of them was to have ECHO sessions not only in the clinical domain but also in other areas like training human resources and strengthening community awareness in the area of COVID-19. The potential of ECHO sessions to go beyond the clinical domain to cover vulnerable populations has been found effective in a study conducted in the USA [33]. To improve active participation and effective interaction, participants emphasized the use of visual interactions in addition to audio ones. To strengthen the model, a hybrid session of remote learning coupled with face-to-face training would be useful. Participants also felt the inclusion of certification and continuing medical education credits will expand the training program.

The COVID-19 pandemic has shown how lack of knowledge among HCWs has been a major barrier for managing outbreaks. Training programs like ECHO has the potential to reach and strengthen the healthcare workforce in collaboration with public health universities and government institutions [34, 35]. The results of this study are an important addition to the pre-existing literature supporting the replicability of the Project ECHO model in the upskilling of health professionals and capacity building of healthcare providers working in underserved and remote areas. The findings of this study indicate that the ECHO tele-mentoring model has strong potential to be replicated in low-middle-income settings like India. The current training was an excellent example of collaboration of ECHO India with academic medical centres, departments of health in several states, and non-profit organizations. Such models are critical and serve as disruptive innovators for complex problems plaguing the delivery of specialized care to underserved populations [34].

Strengths and limitations

This study is one of the first to use a mixed-method approach to assess an online model for building the capacity of healthcare providers in the context of the COVID-19 in India. Despite current COVID wave constraints, we were able to collect good quality data from different stakeholders across different states in India. The limitations of the study include the smaller number of qualitative interviews with nurses due to constraints posed by the second wave of COVID. We were unable to validate the quantitative survey tool due to resource constraints. The retrospective design of the study involved data collection after 6 months of the COVID- ECHO training and there could be a potential recall bias by study participants.

Conclusion

Digital health innovations are rapidly changing the delivery of healthcare services globally. The results of our study support that the ECHO telementoring model has the potential to bring this transformation through building capacities in HCWs working across different geographies. This model can bridge the knowledge asymmetry and help build capacity in healthcare providers working in inaccessible communities. The findings of this study may be used to improve the model. This will help expand and enhance the effectiveness of the ECHO model.

Availability of data and materials

All data generated or analysed during this study are included in this published article [as supplementary information files].

Abbreviations

COVID-19:

Coronavirus disease 2019

SARS-CoV-2:

Severe Acute Respiratory Syndrome Coronavirus 2

WHO:

World Health Organization

HCWs:

Healthcare Workers

LMICs:

Low- and Middle-Income Countries

NPSIF:

National Patient Safety Implementation Framework

RSPS:

Regional Strategy for Patient Safety

ECHO:

Extension for Community Healthcare Outcomes

IDIs:

In-depth interviews

EDC:

Electronic Data Capture

IEC:

Institutional Ethics Committee

PIS:

Participant Information Sheet

JSPH:

Jodhpur School of Public Health

CPR:

Cardio-Pulmonary Resuscitation

OPD:

Outpatient Department

CME:

Continuing Medical Education

References

  1. Timeline of WHO’s response to COVID-19 .  Available from: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/interactive-timeline#! [Cited 2021 Feb 4].

  2. Moradian N, Ochs HD, Sedikies C, Hamblin MR, Camargo CA, Martinez JA, et al. The urgent need for integrated science to fight COVID-19 pandemic and beyond. J Transl Med . 2020;18(1):1–7. Available from: https://doi.org/10.1186/s12967-020-02364-2

  3. Elhadi M, Msherghi A, Alkeelani M, Zorgani A, Zaid A, Alsuyihili A, et al. Assessment of healthcare workers’ levels of preparedness and awareness regarding Covid-19 infection in low-resource settings. Am J Trop Med Hyg. 2020;103(2):828–33.

    Article  Google Scholar 

  4. Tan BYQ, Chew NWS, Lee GKH, Jing M, Goh Y, Yeo LLL, et al. Psychological Impact of the COVID-19 Pandemic on Health Care Workers in Singapore. Ann Intern Med. 2020;173(4):317–20.

    Article  Google Scholar 

  5. Tan ESLSY, Koh; PSSLL, Sim; SWWSMmFK, Chong WETP-N. Perceived stress and associated factors among healthcare workers in a primary healthcare setting: the Psychological Readiness and Occupational Training Enhancement during COVID-19 Time (PROTECT) study. SMJ,Singapore Med J. 2020;(Dec 2020):1–25. Available from: https://doi.org/10.11622/smedj.2020163.

  6. Batra K, Singh TP, Sharma M, Batra R, Schvaneveldt N. Investigating the Psychological Impact of COVID-19 among Healthcare Workers: A Meta-Analysis. Int J Environ Res Public Heal Artic. Available from: www.crd.york.ac.uk/PROSPERO/.

  7. Cui PP, Wang PP, Wang K, Ping Z, Wang P, Chen C. Post-traumatic growth and influencing factors among frontline nurses fighting against COVID-19. Occup Environ Med . 2021 Feb 1;78(2):129–35. Available from: https://oem.bmj.com/content/78/2/129. [Cited 2022 Mar 4].

  8. Heath C, Sommerfield A, Von Ungern-Sternberg BS. Resilience strategies to manage psychological distress among healthcare workers during the COVID-19 pandemic: a narrative review. Anaesthesia. 2020;75(10):1364–71.

    CAS  Article  Google Scholar 

  9. Lahariya C, Gupta S, Kumar G, De Graeve H, Parkash I, Das JK. Patient safety in graduate curricula and training needs of health workforce in India: A mixed-methods study. Indian J Public Health. 2020;64(3):277–84.

    Article  Google Scholar 

  10. Liu Q, Luo D, Haase JE, Guo Q, Wang XQ, Liu S, et al. The experiences of health-care providers during the COVID-19 crisis in China: a qualitative study. Lancet Glob Heal . 2020;8(6):e790–8. Available from: https://doi.org/10.1016/S2214-109X(20)30204-7.

  11. Ministry of Personnel, Public Grievance & Pensions, Government of India. iGOT (Intergrated Govt. Online Training) courses on DIKSHA platform on COVID-19 pandemic [Internet]. 2020. Available from: https://www.mohfw.gov.in/pdf/iGOTCOVID19Circular.pdf.

  12. Demeke HB, Pao LZ, Clark H, Romero L, Neri A, Shah R, et al. Telehealth Practice Among Health Centers During the COVID-19 Pandemic — United States, July 11–17, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(50):1902–5.

    CAS  Article  Google Scholar 

  13. Sultan Mahmood, Khaled Hasan, Michelle Colder Carras AL. Global Preparedness Against COVID-19: We Must Leverage the Power of Digital Health. JMIR Public Health and Surveillance; Available from: http://publichealth.jmir.org.

  14. De Witt JB, Brazil K, Passmore P, Buchanan H, Maxwell D, McIlfatrick SJ, et al. Evaluation of the impact of telementoring using ECHO© technology on healthcare professionals’ knowledge and self-efficacy in assessing and managing pain for people with advanced dementia nearing the end of life. BMC Health Serv Res. 2018;18(1):1–12.

    Article  Google Scholar 

  15. Gleason LJ, Beiting KJ, Walker J, Shervani S, Graupner J, Mittal K, et al. Using Telementoring to Share Best Practices on COVID-19 in Post-Acute and Long-Term Care Facilities. J Am Geriatr Soc. 2020;68(11):E58–60.

    Article  Google Scholar 

  16. USAID. Technical Note: Conducting Mixed-Method Evaluations | U.S. Agency for International Development [Internet]. Version 1, 2013 [cited 2022 Jul 7]. Available from: https://www.usaid.gov/evaluation/technical-note/mixed-method.

  17. Temsah MH, Al Huzaimi A, Alrabiaah A, Alamro N, Al-Sohime F, Al-Eyadhy A, et al. Changes in healthcare workers’ knowledge, attitudes, practices, and stress during the COVID-19 pandemic. Medicine (Baltimore). 2021 ;100(18):e25825. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8104280. [Cited 2022 Apr 13].

  18. De Witt Jansen B, Brazil K, Passmore P, Buchanan H, Maxwell D, McIlfatrick SJ, et al. Evaluation of the impact of telementoring using ECHO© technology on healthcare professionals’ knowledge and self-efficacy in assessing and managing pain for people with advanced dementia nearing the end of life. BMC Health Serv Res. 2018 Apr 2;18(1). Available from: https://pubmed.ncbi.nlm.nih.gov/29606132/ [Cited 2022 Apr 13].

  19. Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The Impact of Project ECHO on Participant and Patient Outcomes: A Systematic Review. Acad Med. 2016 Oct 1;91(10):1439–61. Available from: https://pubmed.ncbi.nlm.nih.gov/27489018/. [Cited 2022 Apr 13].

  20. Hackett A, Strickland K. Using the framework approach to analyse qualitative data: a worked example. Nurse Res. 2019;26(2):8–13.

    Article  Google Scholar 

  21. Mehrotra K, Chand P, Bandawar M, Rao Sagi M, Kaur S, G A, et al. Effectiveness of NIMHANS ECHO blended tele-mentoring model on Integrated Mental Health and Addiction for counsellors in rural and underserved districts of Chhattisgarh, India. Asian J Psychiatr. 2018;36(July):123–7. Available from: https://doi.org/10.1016/j.ajp.2018.07.010.

  22. Katzman JG, Comerci G, Boyle JF, Duhigg D, Shelley B, Olivas C, et al. Innovative Telementoring for Pain Management: Project ECHO Pain. J Contin Educ Health Prof. 2014 Jan 1;34(1):68–75. Available from:https://doi.org/10.1002/chp.21210. [Cited 2021 Jul 29].

  23. C M, T H, A D, K B, S B, B P, et al. Using an established telehealth model to train urban primary care providers on hypertension management. J Clin Hypertens (Greenwich). 2012 Jan;14(1):45–50. Available from: https://pubmed.ncbi.nlm.nih.gov/22235823/. [Cited 2021 Jul 29].

  24. De Witt Jansen B, Brazil K, Passmore P, Buchanan H, Maxwell D, McIlfatrick SJ, et al. Evaluation of the impact of telementoring using ECHO© technology on healthcare professionals’ knowledge and self-efficacy in assessing and managing pain for people with advanced dementia nearing the end of life. BMC Heal Serv Res 2018 181. 2018 Apr 2;18(1):1–12. Available from: https://doi.org/10.1186/s12913-018-3032-y. [Cited 2021 Jul 29].

  25. Creative Inc M, Design C. Framework for Action on Interprofessional Education & Collaborative Practice Health Professions Networks Nursing & Midwifery Human Resources for Health. 2010; Available from: http://www.who.int/hrh/nursing_midwifery/en/. [Cited 2021 Jul 29].

  26. Core Competencies for Interprofessional Collaborative Practice: 2016 Update. 2016.

  27. K M, P C, M B, M RS, S K, A G, et al. Effectiveness of NIMHANS ECHO blended tele-mentoring model on Integrated Mental Health and Addiction for counsellors in rural and underserved districts of Chhattisgarh, India. Asian J Psychiatr. 2018 Aug 1;36:123–7. Available from: https://pubmed.ncbi.nlm.nih.gov/30086513/.

  28. S S, A A, E S, L M, J A, A C. Building Provincial Mental Health Capacity in Primary Care: An Evaluation of a Project ECHO Mental Health Program. Acad Psychiatry. 2018 Aug 1;42(4):451–7. Available from: https://pubmed.ncbi.nlm.nih.gov/28593537/. [cited 2021 Jul 29].

  29. C Z, A C, E S, P K, S S. The Impact of Project ECHO on Participant and Patient Outcomes: A Systematic Review. Acad Med. 2016 Oct 1;91(10):1439–61. Available from: https://pubmed.ncbi.nlm.nih.gov/27489018/. [Cited 2021 Jul 29].

  30. Carlin L, Zhao J, Dubin R, Taenzer P, Sidrak H, Furlan A. Project ECHO telementoring intervention for managing chronic pain in primary care: Insights from a qualitative study. Pain Med (United States). 2018;19(6):1140–6.

    Article  Google Scholar 

  31. Furlan AD, Zhao J, Voth J, Hassan S, Dubin R, Stinson JN, et al. Evaluation of an innovative tele-education intervention in chronic pain management for primary care clinicians practicing in underserved areas. J Telemed Telecare. 2019;25(8):484–92.

    Article  Google Scholar 

  32. White C, McIlfatrick S, Dunwoody L, Watson M. Supporting and improving community health services—a prospective evaluation of ECHO technology in community palliative care nursing teams. BMJ Support Palliat Care. 2019 Jun 1;9(2):202–8. Available from: https://spcare.bmj.com/content/9/2/202. [Cited 2021 Jul 29].

  33. Thies KM, Gonzalez M, Porto A, Ashley KL, Korman S, Lamb M. Project ECHO COVID-19: Vulnerable Populations and Telehealth Early in the Pandemic. J Prim Care Community Heal. 2021;12:1–6.

  34. Baru. Health systems preparedness during COVID-19 pandemic: China and India. Indian J Public Health. 2020;64(6):96. Available from: https://www.ijph.in/article.asp?issn=0019-557X;year=2020;volume=64;issue=6;spage=96;epage=98;aulast=Baru. [Cited 2021 Jul 29].

  35. Gunther Eysenbach, Mohammed Ikmal, Sudarshan Paudel, Ya-Wen Chiu, Chinenye Nwoke, Sharuk Khan, Mohammad Amin Bahrami CB, Bhagavathula AS. Knowledge and Perceptions of COVID-19 Among Health Care Workers: Cross-Sectional Study. JMIR Public Health and Surveillance; Available from: http://publichealth.jmir.org.

Download references

Acknowledgments

The authors wish to thank all the healthcare workers who kindly participated in this study giving their time, experience, and insights. We are grateful to Dr. Anmol Rawat for his technical support in the implementation of the study. We also thank Dr. Sourabh Chakraborty (Professor, JSPH), Mr. Swapnil Gupta, and the JSPH data collection team for their contribution to the collection of good quality data in a short time.

Funding

The study was funded by Extension for Community Healthcare Outcomes (ECHO) India.

Author information

Authors and Affiliations

Authors

Contributions

R.M. contributed to the conception, design, implementation of the protocol, and significant inputs for data analysis. N.M. and S.L. contributed to qualitative data collection. A.M., N.M., and S.L. contributed to the analysis and validation of qualitative data. R.P. and K.S. contributed to the analysis and validation of quantitative data. N.M. wrote the manuscript and S.L., R.P., A.M., R.M., K.R. reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Nivedita Mishra.

Ethics declarations

Ethics approval and consent to participate

Approval for this study was obtained from the Institutional Ethics Committee (IEC) of the Jodhpur School of Public Health (ref: 10032/IRB/20-21, dated 8th February 2021) and all methods were performed in accordance with the relevant guidelines and regulations. A written Participant Information Sheet (PIS) and informed consent form was provided to the participants. The verbal informed consent was taken from all the participants and the process of verbal informed consent was approved by the ethics committee (Institutional Ethics Committee (IEC) of the Jodhpur School of Public Health). Data confidentiality was maintained by providing Unique Identification (ID) codes.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Panda, R., Mishra, N., Lahoti, S. et al. Evaluation of COVID-19 ECHO training program for healthcare workers in India - A Mixed-Method Study. BMC Health Serv Res 22, 883 (2022). https://doi.org/10.1186/s12913-022-08288-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-08288-5

Keywords

  • COVID-19
  • ECHO Telementoring
  • Capacity building
  • Healthcare workers
  • Mixed-method study