The COVID-19 pandemic triggered unprecedented expansion of outpatient telemedicine in the United States in all types of health systems, including safety-net health systems. These systems generally serve low-income, racially/ethnically/linguistically diverse patients, many of whom face barriers to digital health access. These patients’ perspectives are vital to inform ongoing, equitable implementation efforts.
Twenty-five semi-structured interviews exploring a theoretical framework of technology acceptability were conducted from March through July 2020. Participants had preferred languages of English, Spanish, or Cantonese and were recruited from three clinics (general medicine, obstetrics, and pulmonary) within the San Francisco Health Network. Both deductive and inductive coding were performed. In a secondary analysis, qualitative data were merged with survey data to relate perspectives to demographic factors and technology access/use.
Participants were diverse with respect to language (52% non-English-speaking), age (range 23-71), race/ethnicity (24% Asian, 20% Black, 44% Hispanic/Latinx, 12% White), & smartphone use (80% daily, 20% weekly or less). All but 2 had a recent telemedicine visit (83% telephone). Qualitative results revealed that most participants felt telemedicine visits fulfilled their medical needs, were convenient, and were satisfied with their telemedicine care. However, most still preferred in-person visits, expressing concern that tele-visits relied on patients’ abilities to access telemedicine, as well as monitor and manage their own health without in-person physical evaluation.
High satisfaction with telemedicine can co-exist with patient-expressed hesitations surrounding the perceived effectiveness, self-efficacy, and digital access barriers associated with a new model of care. More research is needed to guide how healthcare systems and clinicians make decisions and communicate about visit modalities to support high-quality care that responds to patients’ needs and circumstances.
The Coronavirus Disease 2019 (COVID-19) pandemic triggered unprecedented expansion of outpatient telemedicine encounters throughout all medical settings in the United States (U.S.) [1, 2]. In contrast to before the pandemic, when telemedicine was being used primarily in specialized patient populations (e.g. heart failure) or to increase access to care for rural patients, in August 2020, telemedicine encounters (including telephone and video visits) accounted for 50-60% of all primary care delivered in two large delivery systems in San Francisco—including the study site for this research . Given this widespread experience with and normalization of telemedicine in all settings in the United States, almost all healthcare settings are preparing to continue telemedicine into the future [4,5,6,7].
Telemedicine is defined by the U.S. Health Resources and Services Administration as “ … the use of telecommunications and information technologies to support long distance clinical health care...”  For the purposes of this paper, we use “telemedicine” to refer to scheduled telephone and video encounters for outpatient clinical care. Although most U.S. health systems have strived to provide both telephone and video encounters, different delivery systems and patient populations have experienced the transition to telemedicine encounters differently. Specifically, safety-net healthcare delivery settings have relied almost exclusively on telephone visits , while medical centers serving primarily privately-insured and/or higher-income patients have had higher uptake of video visits . This difference has potential to increase structural and individual health inequity, especially given that in the United States, insurance payments for telephone visits will likely be lower than video visits following the pandemic and therefore may no longer be provided as an option . While there are clear organizational and infrastructure differences between the safety-net and other medical settings that influence these patterns, visit modality is likely also influenced by historical, social and individual determinants. For example, a safety-net healthcare delivery system may offer video visits, but uptake will still be limited if the patients they serve face barriers to digital device/connectivity access [12,13,14,15], consider telemedicine visits inferior in quality to in-person visits, and/or have higher trust or preference for in-person visits and communication with providers .
Although two recent studies have demonstrated that many safety-net patients are open and interested in telemedicine visits, specifically video visits [12, 17], there is a need to contextualize and add depth to these findings to ensure that ongoing implementation efforts are sensitive to patients’ needs and context. It is especially vital that we highlight the experiences and perspectives of patients with lower incomes and who face structural barriers to technology use as to avoid designing and building systems that increase inequity. In order to fill this need, we conducted semi-structured interviews with safety-net patients sampled for a wide range of ages, language preferences, technology access, and clinical settings.
We performed 25 semi-structured interviews between March and July 2020. We used a theoretical framework of acceptability to guide interviews and explore whether participants’ considered telemedicine (including telephone and video) visits appropriate . This framework guided us in exploring domains of acceptability, such as: affective attitude (how an individual feels about an intervention), burden (the perceived amount of effort that is required to participate in the intervention), perceived effectiveness (the extent to which the intervention is perceived as likely to achieve its purpose), and self-efficacy (the participant’s confidence that they can perform the behavior(s) required to participant in the intervention) .
In order to obtain a range of experiences, English-, Spanish-, and Cantonese-speaking patients who were scheduled for telemedicine visits were purposively sampled from the patient panels of three clinics—general medicine, obstetrics, and pulmonary--within the San Francisco Health Network, which is the public healthcare delivery system in the city and county of San Francisco, California, serving almost exclusively Medicaid and other uninsured/publicly-insured patients. We chose these three languages because together they account for the preferred language of 90% of patients receiving care in this health system. 105 qualifying participants were identified, 63 participants were contacted, and 25 participated in the study.
Participants were interviewed by bilingual study staff (including BZ, EK, FG) in their preferred language over the telephone. All interviews were recorded. Interviews included discussion of participants’ experiences with recent telemedicine visits, as described above. We asked the two participants who had not experienced a telephone or video visit about their openness to participate in this type of visit and their anticipated preferences. Participants were reimbursed for their time up to $40. We used a combination of bilingual study staff (BZ, FG) and professional translation services to translate and transcribe transcripts into English.
For our quantitative data collection, trained research assistants (two of whom were bilingual) conducted surveys with all participants before their interviews. Surveys included demographic information (age, preferred language, gender, self-identified race and ethnicity, education level, health literacy), as well as their access to and use of digital devices, the Internet, and smartphone applications (including video chat applications). Participants from the primary care and pulmonary clinics, all of whom had experienced either a telephone or video visit in the two weeks preceding their interview, were also asked to rate their visit and answer whether all of their medical needs were met.
First, we summarized the quantitative survey data among all participants. Next, we conducted a thematic analysis using deductive coding according to the theoretical framework that guided the semi-structured interviews , combined with inductive coding to allow for novel ideas to emerge. Dedoose software was used for coding. We underwent a group consensus process to ensure intercoder consistency. More specifically, the same interview transcripts were coded by all members of the coding team (CRL, FG, JJ, BZ), after which we held multiple group meetings to discuss and review differing coding applications until consensus was achieved. MTN, CRL, and FG independently looked for and jointly discussed common themes across all transcripts. All survey items were descriptively summarized using Excel software. In a secondary analysis, we used the quantitative survey data to supplement our qualitative results by reviewing excerpts stratified by clinic, preferred language, gender, age categories (20-49 years old vs. 51-80 years old), health literacy (high health literacy vs. all others), and Internet use (several times per day vs. once per day or less).
Participants represented a broad range of spoken language, age (range 23-71 years), self-identified race/ethnicity, & technology use (Table 1).
All but two had experienced a recent telemedicine visit with a clinician. Out of the 23 telemedicine visits, 19 were telephone visits and 4 were video visits.
Our findings centered around four major, cross-cutting themes, which mapped to the constructs of affective attitude, burden, perceived effectiveness, and self-efficacy from the acceptability framework used in our coding (Fig. 1). While we also reviewed coded excerpts stratified by patients’ preferred language, gender, age, health literacy, and Internet access/use, we found no substantial differences between these groups.
Theme 1: participants were satisfied with telemedicine as a convenient, valuable option for patients
Almost all participants felt satisfied with their telemedicine care and felt their needs were fulfilled. Participants were accepting of telephone and video visits as comfortable, convenient, and valuable options for patients. For example, the majority of participants expressed positivity about saving travel time and not having to wait for visits. They also recognized that having the option for telemedicine would make accessing medical care easier for people who work, take care of children, or have limited mobility. Representative quotes are summarized in Table 2.
Theme 2: participants had concerns about accessibility issues related to Internet/data access and technology experience
Patients were aware of their own accessibility barriers, such as limited access to Internet and cellular data, as well as potential access barriers for certain groups of patients such as older people (Table 2).
Theme 3: participants expressed visit type preferences (particularly for in-person visits) that were driven by anxiety related to monitoring and managing their own health without undergoing a physical evaluation by a trained medical professional
Despite their acceptance of telemedicine visits, participants generally favored a) in-person care more than video visits (Table 3) and b) video visits more than telephone visits. Despite most not having experience with video visits, many participants felt that video visits would be better than telephone visits because it would more closely resemble an in-person interaction in which the doctor could see them.
In addition, the preference for in-person visits was driven by anxiety and lack of confidence in monitoring and managing their own health without physical evaluation by a trained medical professional (Table 3). Participants expressed that in-person visits provided a sense of reassurance that telemedicine visits did not. Part of this reassurance stemmed from physical evaluation with vital sign measurements and/or physical examination by a trained medical professional. Most participants felt that in-person visits were particularly more appropriate than telemedicine for evaluating severe pain:
“What I’m thinking is like if it’s just a concern about the question or understanding something about the question, maybe still okay with the [telemedicine] visit, but if it’s something that you’re really in pain that you want show to your doctor, my suggestion is it’s much better if you can go to the visit face-to-face instead of the video call.”
“If it’s like a 4/10 pain then maybe it’s not needed to go in person. But if anything’s more severe or I really want the doctor to look at it I will still need to go.”
Obstetrics patients, in particular, more often expressed anxiety regarding not being physically evaluated:
“Yes, [I] was left with a feeling like, ‘Okay. She says everything’s okay. I answered to the best of my ability. I just hope everything’s okay,’ because that’s the only way I could ease my own mind, is to just hope that everything’s okay after that.”
Theme 4: participants felt more comfortable with and expressed preference for telemedicine visits with providers they already knew
Finally, across the interviews, most participants expressed more reassurance and comfort with telemedicine if the visit was with a doctor or other clinician they already had a relationship with.
“it’s better if I see them in person first if I never met them. Because I don’t know them that well. But for someone who I seen for a long time like my personal doctor I don’t need to go in person … For new doctors I haven’t seen before my first choice would be in-person then second video, lastly would be phone.”
“I’m glad I didn’t have to talk to anyone else. … That was my own personal doctor because I asked her about that first before she said about the Zoom and I was making sure like, “Is it going to be with you” and she said, ‘It would be me.’”
“I want to visit the doctor, the doctor I know already. If it is a new doctor, I prefer to see regular new doctor [in person]. Next time, the same doctor. I don’t want new doctors every video, every phone, and – yes, I want it to be like make sure the patient is comfortable.”
To our knowledge, our study was the first to examine in-depth qualitative perceptions of acceptability of telemedicine among multi-lingual safety-net patients during the COVID-19 pandemic. Few telemedicine-focused studies have purposefully sampled participants speaking multiple languages, which is critical given the needs of healthcare systems to deliver telemedicine care seamlessly to patients who face structural barriers to healthcare.
We found that participants had high satisfaction with their current telemedicine encounters (which were primarily telephone visits) but generally preferred in-person visits and the associated medical support and physical examination. Our qualitative coding allowed a deeper exploration of patient perceptions and mapped themes to an existing theory. Our analysis provides a more nuanced understanding of telemedicine acceptability: how high satisfaction with telemedicine can co-exist with patient-expressed hesitations about perceived effectiveness, self-efficacy, and digital access barriers.
Our findings align with previous research that found high rates of satisfaction with telemedicine in other settings [17, 19, 20], as well as preferences for in-person relationships with clinicians . They build upon prior literature that has demonstrated high initial interest in and perceived effectiveness of video visits and other digital healthcare modalities among racially/ethnically and/or linguistically diverse patients, including Spanish-speaking patients from our same healthcare delivery system [12, 21, 22].
Moving forward, research is needed to guide how healthcare systems and clinicians make decisions and communicate about visit modality with the overarching goal of utilizing multiple visit modalities to support personalized, high-quality care that responds to patients’ needs and circumstances. This includes understanding not only how to best utilize telemedicine versus in-person visits, but also how to best utilize telephone versus video visits. We need to better understand how to tailor our use of visit modalities and communicate this new standard of care to patients, making explicit that virtual care is not a “substitute” for in-person care but rather a valuable care option for patients who want more access to visits to supplement ongoing in-person relationships. This is especially important while caring for communities that have experienced exploitation and discrimination. Beyond clear communication to patients about when and why telemedicine might make sense for specific encounters, we also need to understand and uphold patient preferences for care. Many of our participants expressed the desire to have telemedicine visits with a clinician they already had an established relationship with. This should motivate health systems to prioritize continuity in their telemedicine implementation efforts. Overall, it is especially important to recognize that each patient has unique needs and some may need more time to consider how new care modalities help facilitate their health and wellness. Finally, we need to support digital access and training for patients, given our findings that patients may prefer video visits over telephone visits and yet there are known barriers to devices, data/Internet, and skills that impede interested patients from using digital modalities of care [14, 23].
An important limiting factor to our work is that interviews were conducted during the COVID-19 pandemic, which likely influenced how participants felt about and experienced telemedicine while also broadening almost all participants’ experiences trying out telemedicine for the first time. Several participants expressed relief at having telemedicine as an option for safety reasons, and it is unclear how their perspectives may change once the pandemic ends. Other limitations of this work include a single delivery system and small sample size within each clinic type. It is also unclear whether patients had any other experiences with telemedicine in other healthcare settings. Future studies that sample more video visit experiences are still needed to directly compare experiences with telephone visits versus video visits. Finally, our secondary analysis examining excerpted codes by patient factors, such as language and digital skills/experience, was exploratory in nature and should be repeated in future work engaging diverse patient populations in other settings.
The widespread expansion of telemedicine has and will continue to be experienced differently by different patient populations. The option of telemedicine visits has the potential for decreasing access barriers for patients who work, are caretakers, have limited mobility, or limited access to transportation. This potential may not be realized if implementation is not sensitive to patient perspectives and needs. Telemedicine implementation efforts in the safety-net should prioritize clear two-way communication regarding visit modality selection that will not only fulfill clinicians’ standards for appropriate care but will also ensure that patients feel reassured and cared for in a high-quality and safe manner.
After the COVID-19 pandemic has ended, we must continue to support patients’ needs and preferences, recognizing that interest in, openness to, and satisfaction with telemedicine can co-exist with concerns and hesitations surrounding a new model of care.
Availability of data and materials
The datasets generated during and/or analysed during the current study are not publicly available due to participants’ privacy concerns but are available from the corresponding author on reasonable request.
Zulman DM, Verghese A. Virtual Care, Telemedicine Visits, and Real Connection in the Era of COVID-19: Unforeseen Opportunity in the Face of Adversity. JAMA. 2021;325(5):437–8. https://doi.org/10.1001/jama.2020.27304.
Eberly LA, Kallan MJ, Julien HM, et al. Patient Characteristics Associated With Telemedicine Access for Primary and Specialty Ambulatory Care During the COVID-19 Pandemic. JAMA Netw Open. 2020;3(12):e2031640. https://doi.org/10.1001/jamanetworkopen.2020.31640.
Westby A, Nissly T, Gieseker R, Timmins K, Justesen K. Achieving Equity in Telehealth: “Centering at the Margins” in Access, Provision, and Reimbursement. J Am Board Fam Med. 2021;34(Supplement):S29–32. https://doi.org/10.3122/jabfm.2021.S1.200280.
Khoong EC, Butler BA, Mesina O, et al. Patient interest in and barriers to telemedicine video visits in a multilingual urban safety-net system. J Am Med Inform Assoc. https://doi.org/10.1093/jamia/ocaa234.
Nouri S, Khoong EC, Lyles CR, Karliner L. Addressing Equity in Telemedicine for Chronic Disease Management During the Covid-19 Pandemic. NEJM Catalyst Innov Care Deliver. 2020;4. Published online May. https://catalyst.nejm.org/doi/full/10.1056/CAT.20.0123. Accessed 14 Dec 2020.
Lyles CR, Allen JY, Poole D, Tieu L, Kanter MH, Garrido T. “I Want to Keep the Personal Relationship With My Doctor”: Understanding Barriers to Portal Use among African Americans and Latinos. J Med Internet Res. 2016;18(10):e263. https://doi.org/10.2196/jmir.5910.
Volcy J, Smith W, Mills K, et al. Assessment of Patient and Provider Satisfaction With the Change to Telehealth From In-Person Visits at an Academic Safety Net Institution During the COVID-19 Pandemic. J Am Board Fam Med. 2021;34(Supplement):S71–6. https://doi.org/10.3122/jabfm.2021.S1.200393.
Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res. 2017;17(1):88. https://doi.org/10.1186/s12913-017-2031-8.
Powell RE, Stone D, Hollander JE. Patient and Health System Experience With Implementation of an Enterprise-Wide Telehealth Scheduled Video Visit Program: Mixed-Methods Study. JMIR Med Inform. 2018;6(1):e10. https://doi.org/10.2196/medinform.8479.
Casillas A, Perez-Aguilar G, Abhat A, et al. Su salud a la mano (your health at hand): patient perceptions about a bilingual patient portal in the Los Angeles safety net. J Am Med Inform Assoc. 2019;26(12):1525–35. https://doi.org/10.1093/jamia/ocz115.
Phenicie R, Acosta Wright R, Holzberg J. Patient Satisfaction with Telehealth During COVID-19: Experience in a Rural County on the United States–Mexico Border. Telemed e-Health. 2021;27(8):859–65. https://doi.org/10.1089/tmj.2021.0111.
This research was presented as an original research poster at the Society for General Internal Medicine Annual Meeting on April 21, 2021.
Research reported in this publication was supported by salary support from the U.S. Health Resources and Services Administration under Award Number NRSA T32HP19025 (Michelle-Linh T. Nguyen), National Heart Lung and Blood Institute of the NIH under Award Number K12HL138046 (Elaine C. Khoong), the National Center for Advancing Translational Sciences of the NIH under Award Number KL2TR001870 (Elaine C. Khoong), the National Cancer Institute of the NIH under Award Number K24CA212294 (Urmimala Sarkar), the Patrick and Catherine Weldon Donaghue Foundation (Courtney R. Lyles & Malini A. Nijagal), and a UCSF/Genentech Mid-Career Development Award (Courtney R. Lyles). None of the listed funding agencies were involved in the design, implementation, or analysis of this study. The content is solely the responsibility of the authors and do not necessarily represent the official views of the NIH or other funders.
Authors and Affiliations
Medicine, Division of General Internal Medicine, University of California San Francisco, 1001 Potrero Avenue, San Francisco, CA, 94110, USA
Michelle-Linh T. Nguyen, Faviola Garcia, Billy Zeng, Elaine C. Khoong, Urmimala Sarkar & Courtney R. Lyles
Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, CA, USA
Faviola Garcia, Elaine C. Khoong, Urmimala Sarkar & Courtney R. Lyles
Obstetrics & Gynecology, University of California San Francisco, San Francisco, CA, USA
Jennifer Juarez & Malini A. Nijagal
Medicine, Division of Pulmonary and Critical Care, University of California San Francisco, San Francisco, CA, USA
M-LN led data analysis and manuscript drafting. FG performed interviews, assisted with translation and transcription, collected data, and assisted with data analysis. JJ performed interviews and assisted with analysis. BZ performed interviews, assisted with translation and transcription, and assisted with data analysis. EK helped conceive of and design the analysis, performed interviews, assisted with data analysis, and assisted with writing the manuscript. MN helped conceive of and design the analysis, recruited participants, and assisted with writing the manuscript. US helped conceive of and design the analysis and assisted with writing the manuscript. GS helped conceive of and design the analysis and recruited participants. CL conceived of and designed the analysis, contributed data analysis tools, guided data analysis, and assisted with writing the manuscript. All authors have read and approve of this manuscript.
Verbal consent was obtained from all participants because the study procedures presented no more than minimal risk of harm. This project was categorized as quality improvement and determined to be exempt from Institutional Review Board review by Western Institutional Review Board (#1-1261001-1).
Consent for publication
Michelle-Linh T. Nguyen owns shares of the company, Doximity, which provides telemedicine services. She purchased these shares after data analysis for this study had ended.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Nguyen, ML.T., Garcia, F., Juarez, J. et al. Satisfaction can co-exist with hesitation: qualitative analysis of acceptability of telemedicine among multi-lingual patients in a safety-net healthcare system during the COVID-19 pandemic.
BMC Health Serv Res22, 195 (2022). https://doi.org/10.1186/s12913-022-07547-9