Skip to main content

The impact of the COVID-19 pandemic on rates and predictors of missed hospital appointments in multiple outpatient clinics of The Royal Hospital, Sultanate of Oman: a retrospective study

Abstract

Background

The global outbreak of the COVID-19 pandemic resulted in significant changes in the delivery of health care services such as attendance of scheduled outpatient hospital appointments. This study aimed to evaluate the impact of COVID-19 on the rate and predictors of missed hospital appointment in the Sultanate of Oman.

Methods

A retrospective single-centre analysis was conducted to determine the effect of COVID-19 on missed hospital appointments at various clinics at The Royal Hospital (tertiary referral hospital) in Muscat, Sultanate of Oman. The study population included scheduled face-to-face and virtual appointments between January 2019 and March 2021. Logistic regression models were used with interaction terms (post COVID-19) to assess changes in the predictors of missed appointments.

Results

A total of 34, 3149 scheduled appointments was analysed (320,049 face-to-face and 23,100 virtual). The rate of missed face-to-face hospital appointments increased from 16.9% pre to 23.8% post start of COVID-19, particularly in early pandemic (40.5%). Missed hospital appointments were more frequent (32.2%) in virtual clinics (post COVID-19). Increases in missed face-to-face appointments varied by clinic (Paediatrics from 19.3% pre to 28.2% post; Surgery from 12.5% to 25.5%; Obstetrics & Gynaecology from 8.4% to 8.5%). A surge in the frequency of missed appointments was seen during national lockdowns for face-to-face and virtual appointments. Most predictors of missed appointments did not demonstrate any appreciable changes in effect (i.e., interaction term not statistically significant). Distance of patient residence to the hospital revealed no discernible changes in the relative effect pre and post COVID-19 for both face-to-face and virtual clinic appointments.

Conclusion

The rate of missed visits in most clinics was directly impacted by COVID-19. The case mix of patients who missed their appointments did not change. Virtual appointments, introduced after start of the pandemic, also had substantial rates of missed appointments and cannot be viewed as the single approach that can overcome the problem of missing hospital appointments.

Peer Review reports

Background

The outbreak of the SARS-CoV-2 virus, coronavirus (COVID-19), was reported in the end of 2019 in Wuhan province, China, and was declared a Public Health Emergency of International Concern (PHEIC) by the World Health Organisation in January 2020 [1,2,3]. Since then, the virus has spread all over the world and as of August 2021, 209 million people have been infected with COVID-19, leading to 4.3 million deaths [4, 5]. Prevention measures have been taken by governments around the world to reduce the spread of the virus and the number of infections, for example, social distancing, wearing masks, washing hands, national lockdowns, and restrictions to travel have been implemented in order to reduce the spread of COVID-19 and decrease the number of new cases to prevent overburden and the collapse of our healthcare systems [6,7,8,9].

The first COVID-19 positive cases in Oman were reported on 24thof February 2020 for two Omani passengers coming back from Iran [10]. The number of new cases increased thereafter, reaching two peaks in July 2020 and April 2021 [11, 12]. Different measures were taken in Oman to reduce the spread of the virus. Schools, colleges, along with mosques and other public places such as beaches, playgrounds, airports and boarders were closed and national lockdowns were enforced [13,14,15]. The government reduced the number of employees attending their place of work in both public and private sectors to a minimum and remaining staff were required to work from home [16]. The Supplement provides further details on key dates and national lockdown periods, as well as total number of confirmed COVID-19 cases and death during the study period (Supplementary paragraph 1). In Oman, total number of confirmed COVID-19 cases was 6,370 and death was 30 by May of 2020 [10].

The spread of the virus interrupted the treatments and follow up and hospital admissions for non-COVID-19 patients, with admission declining [17,18,19]. The British Medical Association report stated that there was an increase in missed outpatient appointments and a decrease in elective hospital admission during April, May and June of 2020 in England [20] and a noticed reduction by 25% in A&E visits during the COVID-19 lockdown periods was reported in England [21]. Previous work by the research team found that between 2014–2019 the overall rate of missed appointment at The Royal Hospital was 22.3%, ranging from 14% to 30.3% by clinic. The study found that age, sex, service cost, waiting time, appointment day and the distance a patient lived from the hospital were important predictors for missed appointment [22]. To our knowledge, no such study has been conducted to investigate the impact of COVID-19 on the rate of missed hospital outpatient appointment in Oman.

The aims of this study were (i) to evaluate the impact of COVID-19 on the rate of missed hospital appointments in The Royal Hospital in the Oman capital city Muscat, (ii) to evaluate the impact of the restrictions measures taken by the government to reduce the rate of infection on face-to-face and virtual appointment attendance and (iii) to compare the impact of COVID-19 on the factors associated with missed appointment pre and post pandemic, providing knowledge to the health authorities.

Methods

Study design

A single-centred retrospective case–control design was implemented to assess outpatient attendance for multiple clinics at The Royal Hospital, Sultanate of Oman. All scheduled appointments from 1st of January 2019 to 31st of March 2021 were included.

Setting

The Royal Hospital is the largest tertiary hospital, with 50 specialty and sub-specialty clinics located in the capital city Muscat of the Sultanate of Oman. The hospital provides both inpatient and outpatient medical services with 630-hospital bed capacity and 500 daily-referred patients [23]. The Royal Hospital is the main hospital in Oman providing specialised medical services that cannot be found anywhere else in the country, for example the National Cardiology Centre and the National Oncology Centre of Oman reside here. This means that the majority of treatment delivered by these subspecialty clinics is the only place where patients can access required medical services. The hospital remained open during the pandemic period with maximum staffing to provide service to patents with urgent medial needs and to patients admitted with COVID-19. Non-urgent medical services/procedures and outpatients’ appointments were cancelled and/or rescheduled to reduce patient contact and the spread of COVID-19 whilst reducing the burden on hospital staff and facilities.

Data collection

The electronic health records (EHRs) stored in The Royal Hospital health information management system (ALSHIFA system, Version 3 +) was used to extract scheduled outpatient clinics appointment data.

Study sample

There were 343,149 scheduled appointments included in the final study sample divided into 320,049 face-to-face and 23,100 virtual clinic appointments. Supplementary Fig. 1 shows the exclusion process for the face-to-face appointments. A total of 2,121 observations was excluded from the initial dataset abstracted from the hospital health information management system (ALSHIFA) due to duplicated records, missing data for age, governorate, and marital status; outlier age for the Paediatric clinic (age > 18 years) and the Obstetrics and Gynaecology clinic (age < 13 years); appointments recorded on a Saturday (not a working day) and unidentified gender.

The face-to-face sample was divided into three datasets, pre COVID-19 (N = 178,849), between (N = 11,516) and post start of COVID-19 (N = 129,684). All scheduled appointment from the 1st of January 2019 to the first reported case of COVID-19 (24th February 2020) was grouped as pre COVID-19. The observations from the date when the first cases were reported until the date when the government started to implement restriction and measures to contain and reduce the spread of the virus (1stof April 2020) were grouped as between COVID-19 and the observations after the implementation of the first restriction measure were grouped as post COVID-19. This process is in line with similar published studies [24, 25].

The virtual clinic appointments were introduced post COVID-19 from 16th of April 2020. The face-to-face scheduled appointments for pre and post COVID-19 periods were used in the analysis with a total of 308,533 observations. These observations were further divided into one dataset including appointments scheduled within the Paediatric clinic only (N = 79,444), another dataset including Obstetrics and Gynaecology appointment only (N = 51,850) and an overall dataset including scheduled appointments from all remaining outpatient clinics (N= 177,239). Our previous work observed large variability in the rate of missed appointments by clinic [22]. Therefore, the top five clinics with highest number of scheduled appointments during the study period and higher rate of missed appointments compared to other clinics were identified and five subsets of data created, including: Diabetes and Endocrine clinic (N = 42,238), Surgery clinic (N = 22,781), Oncology clinic (N = 20,850), Urology clinic (N = 11,620), and Gastroenterology clinic (N= 7,946). The detailed sampling procedure is documented here [22].

Measures

A missed appointment was defined as a patient who had a scheduled appointment but did not attend the appointment without contacting the hospital to cancel or rebook the appointment. This is recorded in the system as failed to attend, in line with other research studies [26]. Our previous study showed that older patients (65 +), living further away from the hospital, that waited for more than 60 days, and patients paying visit and registration fees were more likely to miss their hospital appointments [22]. The same predictors were adopted in the analysis for this study.

Statistical analysis

Patient demographic characteristics were summarised for the overall population, by appointment status and stratified by the time period pre and post COVID-19 (face-to-face and virtual clinic appointments). Logistic regression models were used to estimate odd ratios (OR) and 95% confidence intervals (CI) for a binary outcome of hospital appointment status (attended and missed). According to many studies, logistic regression is the most widely used technique for modelling a binary outcome [27]. Interaction term analysis was performed for the binary outcome (pre and post COVDI-19 and post COVID-19 face-to-face and post COVID-19 virtual clinic appointments). Models were fitted separately for face-to-face appointments for the overall dataset and stratified by clinic, for the Paediatric clinic, for the Obstetrics and Gynaecology clinic, and for each of the top five clinics with highest number of scheduled appointments. For the virtual clinic appointment one overall model was fitted. The R statistical program (version 3.6.2) was used for the statistical analyses [28]. All methods were carried out in accordance with relevant guidelines and regulations. The study was approved by the Study and Research Centre, Ministry of Health, Sultanate of Oman on 2 May 2019 (proposal ID: MoH/CSR/20/24172).

Results

There were 343,149 scheduled outpatient appointments included in the analysis, divided into 320,049 face-to-face appointments for the time period between 1st of January 2019 and 31st of March2021 and 23,100 virtual clinic appointment for the time period between 16th of April 2020 and 31st of March 2021. A total of 63,257 face-to-face appointments (19.8%) was missed. For face-to-face appointments included in the study, 116,859 (36.5%) appointments were for male patients and 72,780 (22.7%) were for patients aged between 31 and 40 years old (Table 1). Additional baseline characteristics of the study population can be found in Supplementary Tables 1 and 2.

Table 1 Characteristics of the study population for each unique patient, for all scheduled appointments, stratified by attended or missed face-to-face appointments

The face-to-face pre and post COVID-19 analysis showed that the number of scheduled appointments declined by 27.5% from pre to post COVID-19 (Table 2). However, the rate of missed hospital appointments rose from 16.9% pre to 23.8% post COVID-19 (OR of 1.72 [95% CI 1.68–1.76] comparing post to pre COVID-19). When looking at the weekly rates of missed appointments overall, an increase in the rate of missed hospital appointments was observed when the first lockdown was implemented from 10 April 2020 to 21 June 2020, raising from 19.2% to 40.5%. Similarly, an increase in the rate of missed appointments was seen in the second and third lockdown periods (Fig. 1). Supplementary Table 3 provides details about the distribution and frequency of appointments during lockdown periods.

Table 2 Case mix and rate of missed face-to-face appointments pre and post COVID-19 for all outpatient clinics at The Royal Hospital
Fig. 1
figure 1

Weekly rate of missed face-to-face hospital appointments over calendar time at The Royal Hospital outpatient clinics. C: The first confirmed case of COVID-19 in Oman (24–2-2020). L1: Lockdown period 1 (from 1/4/2020 to 29/5/2020). L2: Lockdown period 2 (from 13/6/2020 to 3/7/2020). L3: Lockdown period 3 (from 25/7/2020 to 8/8/2020). Lockdown periods included the restriction of movement between governorates, except for emergency situations and patients with a hospital appointment. Lockdown period 3 included a public holiday with a reduction in scheduled appointments for three days (2–3-4/8/2020). The drop shown in August 2019 (Pre COVID-19) correlated to weeks public holiday (with no scheduled appointments). The increase shown in January 2020 (Pre COVID-19) correlated to the three days of mourning following the death of Sultan Qaboos bin Said (two days of scheduled appointments only). For data underlying the plot see Supplementary Table 3

The rates of missed appointments increased post COVID-19 in most clinics. In the Paediatric clinic, it increased from 19.3% pre to 28.2% post COVID-19 and in the Surgery clinic from 12.5% to 25.5%. The only exception was the Obstetrics & Gynaecology clinic in which rates remained similar (8.4% to 8.5%). More details are given in Supplementary Fig. 2.

Table 3 shows the ORs for predictors and the change in their effect pre and post COVID-19 (interaction analyses). Most predictors of missed appointments showed no substantive changes in effect after COVID-19 compared to before (i.e., interaction term not statistically significant), except for waiting time and appointment day. The effect of waiting time increased pre to post COVID-19 for all waiting time groups from 1.38 (95% CI 1.31 to 1.46) pre COVID-19 to 2.47 (95% CI 2.33 to 2.62) post COVID-19 for waiting time between 91–120 days. This effect was greater for patients with a waiting time > 120 days, from 1.46 (95% CI 1.39 to 1.53) pre COVID-19 to 2.68 (95% CI 2.56 to 2.80) post COVID-19. In contrast, the day of the scheduled appointment had opposite effects pre and post COVID-19. For example, appointments scheduled on Mondays were less likely to be missed pre COVID-19 [OR 0.94 (95% CI 0.89 to 0.99)] but missed post COVID-19 [OR 1.24 (95% CI 1.17 to 1.31)] for all clinics except Obstetrics & Gynaecology and Paediatric clinics; a similar pattern was observed for appointments scheduled on Thursdays. The effect of age and sex on missed appointment was proportionally similar pre and post COVID-19. For patients aged 41–50 the adjusted OR was 1.04 (95% CI 0.99 to 1.09) pre COVID-19 and 0.95 (95% CI 0.90–1.00) post COVID-19; interaction term was not statistically significant (adjusted OR 0.96 (95% CI 0.89–1.02)).

Table 3 Predictors of missed face-to-face hospital appointments at The Royal Hospital outpatient clinics (excluding Obstetrics & Gynaecology and Paediatric clinics) pre and post COVID-19

Distance of patient residence to the hospital also showed no major changes in relative effect pre and post COVID-19 (Supplementary Table 4). For Dhofar governorate (800–1120 km from Royal hospital), the adjusted OR was 1.15 (95% CI 0.90 to 1.48) pre COVID-19 and 1.01 (95% CI 0.75 to 1.35) post COVID-19; interaction term was not statistically significant (OR 0.97 (95% CI 0.66 to 1.41)). Supplementary Tables 511 shows the results of logistic regression analysis pre and post COVID-19 in the top five clinics, Obstetrics & Gynaecology, and Paediatric clinics.

There were 23,100 virtual clinics appointments from the start of the service in 16th of April 2020 to the 31st of March 2021 scheduled for 11,437 patients. A total of 7,447 appointments (32.2%) was missed. For all virtual appointments, 15,341 (66.4%) appointments were for female patients and 5,326 (23.1%) were for patients aged between 31 and 40 years old (Table 4). Additional baseline can be found in Supplementary Tables 12 and 13.

Table 4 Characteristics of the study population for each unique patient, for all scheduled appointments, stratified by attended or missed for virtual clinics appointments

When looking at the weekly rates of missed virtual clinic appointments overall, an increase in the rate of missed appointments was observed in all lockdown periods (Fig. 2). See supplementary Table 3 for more details about the distribution and frequency of virtual clinic appointments during lockdown periods.

Fig. 2
figure 2

Weekly rate of missed virtual clinic appointments over calendar time at The Royal Hospital. The first recorded virtual clinic appointment in (16–04-2020), with just one scheduled appointment. L1: Lockdown period 1 (from 1/4/2020 to 29/5/2020). L2: Lockdown period 2 (from 13/6/2020 to 3/7/2020). L3: Lockdown period 3 (from 25/7/2020 to 8/8/2020). Lockdown periods included the restriction of movement between governorates, except for emergency situations and patients with a hospital appointment. Lockdown period 3 included a public holiday with a reduction in scheduled appointments for three days (2–3-4/8/2020). For data underlying the plot see Supplementary Table 3

Table 5 shows the ORs for predictors and change in their effect post COVID-19 face-to-face and virtual appointments (interaction analyses). Distance of patient residence to the hospital also showed no major changes in relative effect post COVID-19 face-to-face and post COVID-19 virtual appointments (Supplementary Table 14).

Table 5 Predictors of missed hospital appointments post COVID-19 at The Royal Hospital outpatient clinics (excluding Obstetrics & Gynaecology and Paediatric clinics) for face-to-face and virtual appointments

Discussion

This study found a substantive decrease in the number of scheduled appointments after start of the COVID-19 pandemic in Oman while the rate of missed appointments increased substantially. This increase in missed appointments rates occurred in almost all clinics except for the Obstetrics & Gynaecology clinic that had comparable rates pre and post COVID-19. Rates of missed virtual appointments, introduced after the start of the pandemic, were also high.

The overall increase in rate of missed appointments observed in our study could be linked to the restrictions enforced in Oman to control the spread of the virus. The increases observed were associated with the time when restriction of movement was implemented by the government and the highest weekly increase was noticed in the same week the first national lockdown was enforced across Oman. Different studies had reported an increase in the rate of missed hospital appointment during COVID-19 lockdowns [29,30,31]. The study also showed a decline in the overall number of scheduled appointments within the outpatient clinics during the COVID-19 pandemic, which is consistent with reports from other published studies [32, 33]. This can be explained by the measures taken by the hospital to protect patients and medical staffs by cancelling non urgent appointments and reducing the number of newly booked appointments, only offering them to patients with critical needs [34].

Furthermore, the decline in the scheduled appointments and increase in the rate of missed appointments observed during the COVID-19 pandemic could be related to the patients fear to be infected with the virus causing them to skip their hospital appointments without notifying the hospital. All announcements regarding the rate of COVID-19 infection in Oman, including new case rates, COVID-19 related hospital admissions and COVID-19 related deaths were published and broadcasted in a weekly news conference by the authorities (COVID-19 Supreme Committee) [35]. Patients decisions to attend their hospital appointments may have been influenced by these announcements, including an increased fear of acquiring COVID-19 from attending the hospital, which could also explain the increase in the rate of missed appointments noticed during the post COVID-19. Similar reports from different studies documented the same reasons for the increase in missed hospital appointments [35,36,37]. Our previous work looking at the reasons for missed hospital appointments at The Royal Hospital between March and April of 2021 and showed that some patients did not attend because they were afraid of getting infected with the virus [38]. As this study was conducted almost a year after the first cases of COVID-19 were reported in Oman, it indicates this fear may still be prevalent in the post COVID-19 period.

Variability in the missed appointment rate was found in the top outpatient clinics. The nature of the service and severity of illness may explain the variability in rate of missed appointment by clinic. Similar published studies have reported variability in the rate of missed appointment within different clinics due to the difference in type of disease and the required medical services, especially during the COVID-19 pandemic [39,40,41,42]. The rate of missed appointment in the Oncology clinic and Obstetrics and Gynaecology were low compared to other clinics, where as some clinics such as Paediatric and Surgery, there was a large increase in missed appointment rates.

The Royal Hospital is the only health facility providing oncology treatment in Oman and no such service is available elsewhere, suggesting the rate of missed appointments for this clinic is less because patients have no other location to receive vital treatment. The Obstetrics and Gynaecology clinic in the Royal Hospital is advanced compared to other Obstetrics and Gynaecology clinics in other hospitals. It is staffed with more consultants and specialties trained medical staff. As a result, patients may prefer to have their antenatal, postnatal and other gynaecological care within the Royal Hospital. Published papers found that patients would miss less appointments if they feel that they receive a good quality services [43, 44]. Unlike oncology services, patients can seek medical service for Urology and Gastroenterology clinics in private hospitals. As a result, more patients may miss these appointments because they can find the same service in private sector. In addition, given the age of the patients and likely in the early onset of disease, paediatric patients may have missed their appointments during the pandemic as they may be more able to wait for a follow-up appointment once restrictions lifted, whereas older patients in later stages of disease and increased severity may not be able to wait for treatment.

When comparing the characteristics of patient’s pre and post COVID-19 there were no significant changes. It was expected that more vulnerable patients (e.g., the elderly and or patients of poor health) would be more likely to miss their appointments post COVID-19 compared to pre COVID-19 because they were more at high risk of complications if they caught COVID-19. However, case mix analysis showed no shift in characteristics of patients’ pre and post COVID-19. This indicates factors related to patients’ characteristics may have more influence in their attendance behaviour than external factors such as COVID-19 pandemic [45]. This finding is different from findings in other published studies, which reported that older patients were more likely to miss their appointments post compared to pre COVID-19 [46, 47]. Indeed, these studies suggested that older patients were more likely to have multiple health problems and therefore, more likely to fear being infected with the virus which might lead to worsen their medical conditions. Our finding may be explained by different cultural practice in Oman where elderly patients typically live with their families, meaning they have better support for getting to and attending their hospital appointments.

Our study also showed that longer waiting time was a strong predictor for missed hospital appointment. Patients with longer waiting time tended to miss more compared to patients with short waiting time. This finding is supported by other studies that showed similar results [48, 49]. This could be explained by the fact that patients with longer waiting time might seek health services in private hospitals or other government hospitals since a patient can schedule additional appointments within other hospitals without notifying The Royal Hospital. We found that the effect of appointment day was different pre and post COVID-19. Appointments scheduled in the beginning of the week (Monday) and end of the week (Thursday) were less likely to be missed pre COVID-19 compared to appointments scheduled on other days of the week. In contrast other studies found appointments at the beginning and end of the working week were more likely to be missed [50,51,52]. Patients have to travel quite some distance for appointments in the capital which may be challenging, particularly if they lack transportation. These patients may share transportation with family or friends who are traveling to the capital for a working week and therefore less likely to miss an appointment at the beginning the week. On the other hand, there was a shift in the effect of weekday on missed hospital appointment rate post COVID-19. Appointments scheduled on Monday and Thursday were more likely to be missed compared to other days of the week. The change post COVID-19 is expected given the restrictions of travel to and from the capital and the enforcement for employees to work from home.

In March 2020 The Royal Hospital introduced the idea of “virtual clinics” [53]. Patients with follow up outpatient appointments for some clinics were given the option to attend their appointment via a phone call instead of face-to-face. Several published studies have shown that there was an increase in the usage of telehealth/telemedicine technology during the COVID-19 pandemic compared to the pre pandemic period. The usage of such technology helped the health care system to continue providing medical service during lockdowns and COVID-19 restriction implemented by governments to reduce the spread of the virus [54,55,56,57,58,59]. It could be expected that the possibility of virtual clinics and choice for patients would have reduced the rate of missed appointments for face-to-face clinics. However, our findings of substantially raised rates of missed virtual appointments in the Royal Hospital suggest that virtual appointments may not be the single solution to reducing missed appointments. The higher rate of missed hospital appointment for the virtual clinics appointments (32.2%) compared to face-to-face appointment pre and post COVID-19 (16.9 and 23.8 respectively) can be related to many factors. Virtual clinics were used for the first time during COVID-19 as an alternate method of delivering normal outpatient care. Also, there was no preliminary planning for the introduction of such a service, and patients might not have received the proper instructions. According to other published research, the virtual clinic operation caused confusion and resulted in a high proportion of patients missing their virtual appointments [60, 61]. There may a number of other reasons patients missed their hospital call. For instance, some patients may not be within the network's service region, may only have limited coverage, or may not have had a fully charged phone to receive a call. Technical challenges associated with using such an approach have been documented [62, 63]. Patients' perceptions of and satisfaction with the usage of virtual clinics may also be unfavourable, which may result in them opting to miss future virtual appointments. Indeed, some studies have indicated that this technique of appointments has a low patient satisfaction rating, which results in patients missing the appointment and future appointments [64, 65]. We did not have data regarding virtual clinic appointments for the pre COVID-19 period since this service was only implemented post COVID-19 so we cannot compare or interpret how the implementation of virtual appointments and their attendance rate was affected by the COVID-19 pandemic.

Limitations

This study has several limitations. First, it is based on a single tertiary referral hospital in the capital city of Muscat, Sultanate of Oman. Hence, the results may be not widely generalisable to other referral hospitals and primary health care settings. Although the aim of this study was to investigate the rates and drivers of missed appointment by clinic and how these may have changed during the COVID-19 pandemic to inform decision makers in the Ministry of Health, future work may need to investigate the impact of missed appointments on patient outcomes. In addition, our study sample included data that covered a period of two years, one-year pre and one year post COVID-19, where other studies looked at the effect of COVID-19 during a short period of time (three month before and three months after). This allowed for a large sample size compared to similar studies. Our research team performed analyses of the results using logistic regression and interaction terms to estimate the association of patient characteristics with missing scheduled appointments compared to similar studies that only reported overall counts and percentages. Although logistic regression was reported to be the most widely utilised approach for binary outcomes, other studies used a variety of statistical models and machine learning models such as Decision Tree, Neural Network and Random Forest [27]. Finally, we used multiple appointments per patients as it was justified by the finding from previous study that the rate of missed appointment across groups of patients with different number of previous appointment was similar. However, other might consider using a model of repeated measure or single appointment per patient [45].

Conclusions

There was a direct impact of COVID-19 on missed appointments in most clinics. There was an increase in the rate of missed face-to-face and virtual clinics appointments post COVID-19 and a decline in the number of scheduled appointments as well. Substantial increase in the rate of missed appointments was observed during lockdowns period for all clinics. The case mix of patients who missed did not change. Virtual appointments, introduced after start of the pandemic, also had a substantial rate of missed appointments, and cannot be viewed as the single approach to overcome hospital attendance issues. Given the high rates during lockdowns and the ineffectiveness of recently implemented virtual clinics, the results of this study highlight the critical need for strong contingency plans in healthcare systems to ensure uninterrupted access to critical medical services during a crisis.

Availability of data and materials

The data that support the findings of this study are available from Ministry of Health, Sultanate of Oman, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. The corresponding author should be contacted for the process to request data access.

Abbreviations

ALSHIFA:

Health information management system

EHRs:

Electronic health records

COVID-19:

Coronavirus severe acute respiratory syndrome

WhatsApp:

“WhatsApp is a free, multiplatform messaging app that lets you make video and voice calls, send text messages, and more — all with just a Wi-Fi connection

References

  1. WHO. 2019-nCoV outbreak is an emergency of international concern. WHO. https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/news/news/2020/01/2019-ncov-outbreak-is-an-emergency-of-international-concern. Published 2020. Accessed 20 Aug 2021.

  2. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet. 2020;395(10223):470–3. https://doi.org/10.1016/S0140-6736(20)30185-9.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  3. Sparks MA, South AM, Badley AD, et al. Severe acute respiratory syndrome coronavirus 2, COVID-19, and the renin-angiotensin system: Pressing needs and best research practices. Hypertension. 2020;76(5):1350–67. https://doi.org/10.1161/HYPERTENSIONAHA.120.15948.

    Article  PubMed  CAS  Google Scholar 

  4. WHO. Weekly epidemiological update on COVID-19 - 17 August 2021. WHO. https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19---17-august-2021. Published 2021. Accessed 20 Aug 2021.

  5. WHO. WHO Coronavirus (COVID-19) Dashboard. WHO. Published 2021. Accessed 2 Aug 2021. https://covid19.who.int/.

  6. Wilder-Smith A, Bar-Yam Y, Fisher D. Lockdown to contain COVID-19 is a window of opportunity to prevent the second wave. J Travel Med. 2021;27(5):1–3. https://doi.org/10.1093/JTM/TAAA091.

    Article  Google Scholar 

  7. Teslya A, Pham TM, Godijk NG, Kretzschmar ME, Bootsma MCJ, Rozhnova G. Impact of self-imposed prevention measures and short-term government-imposed social distancing on mitigating and delaying a COVID-19 epidemic: a modelling study. PLoS Med. 2020;17(7):1–21. https://doi.org/10.1371/journal.pmed.1003166.

    Article  CAS  Google Scholar 

  8. Kenyon C. Flattening-the-curve associated with reduced COVID-19 case fatality rates- an ecological analysis of 65 countries. J Infect. 2020;81(1):e98–9. https://doi.org/10.1016/j.jinf.2020.04.007.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  9. Preiser W, Van Zyl G, Dramowski A. COVID-19: Getting ahead of the epidemic curve by early implementation of social distancing. South African Med J. 2020;110(4):258. https://doi.org/10.7196/SAMJ.2020.V110I4.14720.

    Article  CAS  Google Scholar 

  10. Khamis F, Al Rashidi B, Al-Zakwani I, Al Wahaibi AH, Al Awaidy ST. Epidemiology of COVID-19 infection in Oman: Analysis of the first 1304 cases. Oman Med J. 2020;35(3):1–4. https://doi.org/10.5001/omj.2020.60.

    Article  CAS  Google Scholar 

  11. Al-Rawahi B, Prakash KP, Al-Wahaibi A, et al. Epidemiological characteristics of pandemic coronavirus disease (Covid-19) in Oman. Sultan Qaboos Univ Med J. 2021;21(2):e195–202. https://doi.org/10.18295/squmj.2021.21.02.007.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Times of Oman. COVID-19 infections in Oman spike to record high in April. https://timesofoman.com/article/100563-covid-19-infections-in-oman-spike-to-record-high-in-april. Published 24 Apr 2021.

  13. Mohamed AHHM, Abdel Fattah FAM, Bashir MIA, Alhajri M, Khanan A, Abbas Z. Investigating the acceptance of distance learning amongst Omani students: a case study from Oman. Glob Knowl , Mem Commun. https://doi.org/10.1108/GKMC-02-2021-0021. Published online 2021

  14. Alandijany TA, Faizo AA, Azhar EI. Coronavirus disease of 2019 (COVID-19) in the Gulf Cooperation Council (GCC) countries: Current status and management practices. J Infect Public Health. 2020;13(6):839–42. https://doi.org/10.1016/j.jiph.2020.05.020.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Alabdulkarim N, Alsultan F, Bashir S. Gulf Countries Responding to COVID-19. Dubai Med J. 2020;3(2):58–60. https://doi.org/10.1159/000508662.

    Article  Google Scholar 

  16. Oman News Agency. Reduce number of employees in workplace: Manpower.https://www.omanobserver.om/article/14246/Main/reduce-number-of-employees-in-workplace-manpower. Published 1 Apr 2020.

  17. Chang AY, Cullen MR, Harrington RA, Barry M. The impact of novel coronavirus COVID-19 on noncommunicable disease patients and health systems: a review. J Intern Med. 2021;289(4):450–62. https://doi.org/10.1111/joim.13184.

    Article  PubMed  CAS  Google Scholar 

  18. Birkmeyer JD, Barnato A, Birkmeyer N, Bessler R, Skinner J. The impact of the COVID-19 pandemic on hospital admissions in the United States. Health Aff. 2020;39(11):2010–7. https://doi.org/10.1377/hlthaff.2020.00980.

    Article  Google Scholar 

  19. Bhatt AS, Moscone A, McElrath EE, et al. Fewer hospitalizations for acute cardiovascular conditions during the COVID-19 Pandemic. J Am Coll Cardiol. 2020;76(3):280–8. https://doi.org/10.1016/j.jacc.2020.05.038.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  20. British Medical Association. The Hidden Impact of COVID-19 on Patient Care in the NHS in England. 2020. https://www.bma.org.uk/media/2841/the-hidden-impact-of-covid_web-pdf.pdf?utm_source=TheKing%27sFundnewsletters(mainaccount)&utm_medium=email&utm_campaign=11677797_NEWSL_HMP2020-07-17&dm_i=21A8,6YAN9,W9BSTE,RZKML,1.

    Google Scholar 

  21. Thornton J. Covid-19: A&E visits in England fall by 25% in week after lockdown. BMJ. 2020;369(m1401). https://doi.org/10.1136/bmj.m1401.

  22. Alawadhi A, Palin V, Van Staa T. Prevalence and factors associated with missed hospital appointments: a retrospective review of multiple clinics at Royal Hospital, Sultanate of Oman. BMJ Open. 2021;11(e046596):1–9. https://doi.org/10.1136/bmjopen-2020-046596.

    Article  Google Scholar 

  23. Ministry of Health. Hospital Services - Ministry of Health. https://www.moh.gov.om/en/web/royal-hospital/hospital-services. Accessed 25 Sept 2021.

  24. Wosik J, Clowse MEB, Overton R, Adagarla B. Impact of the COVID-19 pandemic on patterns of outpatient cardiovascular care. Am Heart J. 2021;231:1–5. https://doi.org/10.1016/j.ahj.2020.10.074.

    Article  PubMed  CAS  Google Scholar 

  25. Abebe W, Worku A, Moges T, et al. Trends of follow-up clinic visits and admissions three-months before and during COVID-19 pandemic at Tikur Anbessa specialized hospital, Addis Ababa, Ethiopia: an interrupted time series analysis. BMC Health Serv Res. 2021;21(1):1–10. https://doi.org/10.1186/s12913-021-06730-8.

    Article  Google Scholar 

  26. Berliner A, Id S, Id TK, et al. It ’ s how you say it : Systematic A / B testing of digital messaging cut hospital no-show rates. PLoS One. 2020;15(6):e0234817.

    Article  Google Scholar 

  27. Carreras-García D, Delgado-Gómez D, Llorente-Fernández F, Arribas-Gil A. Patient no-show prediction: a systematic literature review. Entropy. 2020;22(6):675.

    Article  PubMed  PubMed Central  Google Scholar 

  28. R: A language and environment for statistical computing. R Found Stat Comput. https://www.r-project.org/. Published online 2022.

  29. Al Khalili S, Al Maani A, Al Wahaibi A, et al. Challenges and Opportunities for Public Health Service in Oman From the COVID-19 Pandemic: Learning Lessons for a Better Future. Front Public Heal. 2021;9. https://doi.org/10.3389/fpubh.2021.770946

  30. Praliaud R, Greigert H, Samson M, et al. Impact of the COVID-19 lockdown on the management and control of patients with GCA. Ann Rheum Dis. 2021;80(6):1–2. https://doi.org/10.1136/annrheumdis-2020-218360.

    Article  Google Scholar 

  31. Balasopoulou A, Κokkinos P, Pagoulatos D, et al. Symposium Recent advances and challenges in the management of retinoblastoma Globe - saving Treatments. BMC Ophthalmol. 2017;17(1):1. https://doi.org/10.4103/ijo.IJO.

    Article  Google Scholar 

  32. Chatterji P, Li Y. Effects of the COVID-19 Pandemic on Outpatient Providers in the United States. Med Care. 2021;59(1):58–61. https://doi.org/10.1097/MLR.0000000000001448.

    Article  PubMed  Google Scholar 

  33. Sulaiman K, Al Riyami M, Al Riyami A, Al Maashani S, Al Saidi J. Short-Term Impact of the COVID-19 Pandemic on Cardiology Service in Oman. Hear Views. 2020;21(3):149–50 HEARTVIEWS.HEARTVIEWS_147_20.

    Google Scholar 

  34. Oman Observer. Royal hospital to stop admitting non emergency cases.https://www.omanobserver.om/article/14822/11/royal-hospital-to-stop-admitting-non-emergency-cases. Published 16 Mar 2020.

  35. Oman News Agency. Nighttime Lockdown to Begin from 10pm: Covid-19 Supreme Committee.https://omannews.gov.om/NewsDescription/ArtMID/392/ArticleID/36377/Nighttime-Lockdown-to-Begin-from-10pm-Covid-19-Supreme-Committee. Published 29 Jul 2021.

  36. Dimelow J, Lowe D, Rogers SN. Balancing patients’ fears of recurrence and fears of COVID-19 when considering their preference for review consultations. Eur Arch Otorhinolaryngol. 2021;278(11):4441–8. https://doi.org/10.1007/s00405-021-06662-3.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Lai THT, Lee M, Au AKH, Tang HHY, Tang EWH, Li KKW. The use of short message service (SMS) to reduce outpatient attendance in ophthalmic clinics during the coronavirus pandemic. Int Ophthalmol. 2021;41:613–20. https://doi.org/10.1007/s10792-020-01616-w.

    Article  PubMed  Google Scholar 

  38. Alawadhi A, Palin V, van Staa T. Investigating the reasons for missing an outpatient appointment in Royal Hospital, Sultanate of Oman: Perspectives of patients and medical staff in a survey. Heal Sci Reports. 2022;5(1):1–10. https://doi.org/10.1002/hsr2.470.

    Article  Google Scholar 

  39. Hsieh YP, Yen CF, Wu CF, Wang PW. Nonattendance at scheduled appointments in outpatient clinics due to covid-19 and related factors in taiwan: A health belief model approach. Int J Environ Res Public Health. 2021;18(9). https://doi.org/10.3390/ijerph18094445.

  40. Kim Y, Ahn E, Lee S, et al. Changing Patterns of Medical Visits and Factors Associated with No-show in Patients with Rheumatoid Arthritis during COVID-19 Pandemic. J Korean Med Sci. 2020;35(48):1–8. https://doi.org/10.3346/jkms.2020.35.e423.

    Article  CAS  Google Scholar 

  41. Shayganfard M, Mahdavi F, Haghighi M, Bahmani DS, Brand S. Health anxiety predicts postponing or cancelling routine medical health care appointments among women in perinatal stage during the covid-19 lockdown. Int J Environ Res Public Health. 2020;17(21):1–13. https://doi.org/10.3390/ijerph17218272.

    Article  CAS  Google Scholar 

  42. Sankaranarayanan R, Hartshorne-Evans N, Redmond-Lyon S, et al. The impact of COVID-19 on the management of heart failure: a United Kingdom patient questionnaire study. ESC Hear Fail. 2021;8(2):1324–32. https://doi.org/10.1002/ehf2.13209.

    Article  Google Scholar 

  43. Graham TAD, Ali S, Avdagovska M, Ballermann M. Effects of a web-based patient portal on patient satisfaction and missed appointment rates: Survey study. J Med Internet Res. 2020;22(5):1–8. https://doi.org/10.2196/17955.

    Article  Google Scholar 

  44. Mannava P, Durrant K, Fisher J, Chersich M, Luchters S. Attitudes and behaviours of maternal health care providers in interactions with clients: a systematic review. Global Health. 2015;11(1):1–17. https://doi.org/10.1186/s12992-015-0117-9.

    Article  Google Scholar 

  45. Borges A, Carvalho M, Maia M, Guimarães M, Carneiro D. Predicting and explaining absenteeism risk in hospital patients before and during COVID-19. Socioecon Plann Sci. 2023;87(PB):101549. https://doi.org/10.1016/j.seps.2023.101549.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Brant AR, Pershing S, Hess O, et al. The impact of COVID-19 on missed ophthalmology clinic visits. Clin Ophthalmol. 2021;15:4645–57. https://doi.org/10.2147/OPTH.S341739.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  47. Yeung S, Wong S, Zhang D, et al. Impact of COVID-19 on loneliness, mental health, and health service utilisation : a prospective cohort study of older adults with multimorbidity in primary care. Br J Gen Pract. 2020;70(700):e817–24. https://doi.org/10.3399/bjgp20X713021.

    Article  Google Scholar 

  48. Suk MY, Kim B, Lee SG, You CH, Kim TH. Evaluation of patient no-shows in a tertiary hospital: Focusing on modes of appointment-making and type of appointment. Int J Environ Res Public Health. 2021;18(6):1–15. https://doi.org/10.3390/ijerph18063288.

    Article  Google Scholar 

  49. Lee S, Groß SE, Pfaff H, Dresen A. Waiting time, communication quality, and patient satisfaction: An analysis of moderating influences on the relationship between perceived waiting time and the satisfaction of breast cancer patients during their inpatient stay. Patient Educ Couns. 2020;103(4):819–25. https://doi.org/10.1016/j.pec.2019.11.018.

    Article  PubMed  Google Scholar 

  50. Ellis DA, Jenkins R. Weekday Affects Attendance Rate for Medical Appointments: Large-Scale Data Analysis and Implications. PLoS One. 2012;7(12). https://doi.org/10.1371/journal.pone.0051365.

  51. Peng Y, Erdem E, Shi J, Masek C, Woodbridge P. Large-scale assessment of missed opportunity risks in a complex hospital setting. Informatics Heal Soc Care. 2016;41(2):112–27. https://doi.org/10.3109/17538157.2014.965303.

    Article  Google Scholar 

  52. Triemstra JD, Lowery L. Prevalence, Predictors, and the Financial Impact of Missed Appointments in an Academic Adolescent Clinic. Cureus. 2018;10(11). https://doi.org/10.7759/cureus.3613.

  53. Oman Observer. The Royal Hospital launched virtual clinic.https://www.omanobserver.om/article/13782/Main/royal-hospital-launches-virtual-clinic. Published 22 Apr 2020.

  54. Alharbi KG, Aldosari MN, Alhassan AM, Alshallal KA, Altamimi AM, Altulaihi BA. Patient satisfaction with virtual clinic during Coronavirus disease (COVID-19) pandemic in primary healthcare, Riyadh. Saudi Arabia J Fam Community Med. 2021;28(2):48–54. https://doi.org/10.4103/jfcm.JFCM_353_20.

    Article  Google Scholar 

  55. Hanumunthadu D, Adan K, Tinkler K, Balaskas K, Hamilton R, Nicholson L. Outcomes following implementation of a high-volume medical retina virtual clinic utilising a diagnostic hub during COVID-19. Eye. 2022;36:627–33. https://doi.org/10.1038/s41433-021-01510-4.

    Article  PubMed  CAS  Google Scholar 

  56. Triana AJ, Gusdorf RE, Shah KP, Horst SN. Technology literacy as a barrier to telehealth during COVID-19. Telemed J E Heal. 2020;26(9):1118–9. https://doi.org/10.1089/tmj.2020.0155.

    Article  CAS  Google Scholar 

  57. Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. 2020;4:1–9. https://doi.org/10.21203/rs.3.rs-23906/v1.

  58. Galiero R, Pafundi PC, Nevola R, et al. The importance of telemedicine during COVID-19 Pandemic: a focus on diabetic retinopathy. J Diabetes Res. 2020;2020. https://doi.org/10.1155/2020/9036847.

  59. Alsaffar H, Almamari W, ALFutaisi A. Telemedicine in the Era of COVID-19 and Beyond: A new horizon. Sultan Qaboos Univ Med J. 2020;20(4):e277-e279. https://doi.org/10.18295/squmj.2020.20.04.001

  60. Altaf MZ, Altaf LZ, Farahat FM, AlThaqafi AO. Patients’ Attendance to The virtual compared to the in-person complimentary outpatient clinics at a tertiary care hospital in western saudi arabia: a comparative cross-sectional study. World Fam Med J /Middle East J Fam Med. 2021;19(12). https://doi.org/10.5742/mewfm.2021.94170.

  61. Smith N, Newton C, Barbacuta D, Tseng OLI. An unexpected transition to virtual care: family medicine residents’ experience during the COVID-19 pandemic. BMC Prim Care. 2022;23(1):1–9. https://doi.org/10.1186/s12875-022-01728-5.

    Article  CAS  Google Scholar 

  62. Dainty KN, Seaton MB, Estacio A, et al. Virtual Specialist Care during the COVID-19 Pandemic: A Multi-Method Study of Patient Experience (Preprint). JMIR Med Informatics. 2022;10:1–10. https://doi.org/10.2196/37196.

    Article  Google Scholar 

  63. Saad M, Chan S, Nguyen L, Srivastava S, Appireddy R. Patient perceptions of the benefits and barriers of virtual postnatal care: a qualitative study. BMC Pregnancy Childbirth. 2021;21(1):1–10. https://doi.org/10.1186/s12884-021-03999-9.

    Article  Google Scholar 

  64. Chan S, Oriordan A, Appireddy R. Exploring the determinants and experiences of senior stroke patients with virtual care. Can J Neurol Sci. 2021;48(1):87–93. https://doi.org/10.1017/cjn.2020.162.

    Article  PubMed  Google Scholar 

  65. Kieran R, Murphy C, Maher E, Buchalter J, Sukor S, Alken S. A permanent legacy of the pandemic? Outcomes of and staff views on the introduction of virtual clinics to an Irish oncology service. Ir J Med Sci. 2022;(0123456789). https://doi.org/10.1007/s11845-021-02892-w.

Download references

Acknowledgements

The researchers express their appreciation to the Centre for Research and Study in the Ministry of Health, Sultanate of Oman the approval of this study and the Directorate General of Information Technology for providing the required data.

Funding

This study was funded by the Ministry of Higher Education, Scientific Research and Innovation, Sultanate of Oman. Grant/Award Number: PGE24720.

Author information

Authors and Affiliations

Authors

Contributions

AA drafted the ethics application, analysed, interpreted the EHR data and drafted the manuscript. VP oversaw the statistical analyses and reviewed the manuscript. TvS reviewed the ethics application, supervised AA and reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ahmed Alawadhi.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Research and Ethical Review & Approval Committee (RERAC) in the Centre of Studies and Research, Ministry of Health, Sultanate of Oman (P.O. Box: 393, Postal Code: 100; info@mohcsr.gov.om) on 9/2/2021—ID: MoH/CSR/20/24172. The RERAC waived the need for participants’ informed consent in this retrospective study because the data used were fully anonymised and the study was fully observational without any study-related changes in treatments.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary Figure 1. 

Study sample selection for the face-to-face appointments. Supplementary Table 1. Additional baseline characteristics for the study population for each unique patient for all scheduled appointments and stratified by attended and missed face-to-face appointments. Supplementary Table 2.  Numbers of scheduled appointments by outpatients clinics at The Royal Hospital stratified by attended and missed face-to-face appointments. Supplementary Table 3. Distribution and frequency of face-to-face and virtual clinic appointments during study period. Supplementary Figure 2. Rate of missed face-to-face appointments in the top seven clinics pre and post COVID-19. Supplementary Table 4. Effects of distance between patient’s residence and The Royal Hospital on missed face-to-face appointments pre and post COVID-19. Supplementary Table 5. Predictors of missed hospital face-to-face appointment for the Obstetrics & Gynaecology clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 6. Predictors of missed hospital face-to-face appointments for the Paediatrics clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 7. Predictors of missed hospital face-to-face appointments for the Diabetics and Endocrine clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 8. Predictors of missed hospital face-to-face appointments for the Surgery clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 9. Predictors of missed hospital face-to-face appointments for the Oncology clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 10. Predictors of missed hospital face-to-face appointments for the Urology clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 11. Predictors of missed hospital face-to-face appointments for the Gastroenterology clinic at The Royal Hospital pre and post COVID-19. Supplementary Table 12. Characteristics of virtual clinic appointments for each unique patient for all appointments stratified by attended and missed appointments. Supplementary Table 13. Distribution of the virtual clinic appointments scheduled within different medical speciality at The Royal Hospital stratified by attended and missed appointments. Supplementary Table 14. Effects of distance between patient’s residence and Royal Hospital on missed appointments post COVID-19 face to-face and post COVID-19 virtual clinics.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alawadhi, A., Palin, V. & van Staa, T. The impact of the COVID-19 pandemic on rates and predictors of missed hospital appointments in multiple outpatient clinics of The Royal Hospital, Sultanate of Oman: a retrospective study. BMC Health Serv Res 23, 1438 (2023). https://doi.org/10.1186/s12913-023-10395-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-023-10395-w

Keywords