Skip to main content

Minimising population health loss in times of scarce surgical capacity: a modelling study for surgical procedures performed in nonacademic hospitals



The burden of the COVID-19 pandemic resulted in a reduction of available health care capacity for regular care. To guide prioritisation of semielective surgery in times of scarcity, we previously developed a decision model to quantify the expected health loss due to delay of surgery, in an academic hospital setting. The aim of this study is to validate our decision model in a nonacademic setting and include additional elective surgical procedures.


In this study, we used the previously published three-state cohort state-transition model, to evaluate the health effects of surgery postponement for 28 surgical procedures commonly performed in nonacademic hospitals. Scientific literature and national registries yielded nearly all input parameters, except for the quality of life (QoL) estimates which were obtained from experts using the Delphi method. Two expert panels, one from a single nonacademic hospital and one from different nonacademic hospitals in the Netherlands, were invited to estimate QoL weights. We compared estimated model results (disability adjusted life years (DALY)/month of surgical delay) based on the QoL estimates from the two panels by calculating the mean difference and the correlation between the ranks of the different surgical procedures. The eventual model was based on the combined QoL estimates from both panels.


Pacemaker implantation was associated with the most DALY/month of surgical delay (0.054 DALY/month, 95% CI: 0.025–0.103) and hemithyreoidectomy with the least DALY/month (0.006 DALY/month, 95% CI: 0.002–0.009). The overall mean difference of QoL estimates between the two panels was 0.005 (95% CI -0.014–0.004). The correlation between ranks was 0.983 (p < 0.001).


Our study provides an overview of incurred health loss due to surgical delay for surgeries frequently performed in nonacademic hospitals. The quality of life estimates currently used in our model are robust and validate towards a different group of experts. These results enrich our earlier published results on academic surgeries and contribute to prioritising a more complete set of surgeries.

Peer Review reports


Since the first identification of a novel coronavirus in December 2019, the rapid global spread of COVID-19 has challenged health care systems worldwide [1]. COVID-19 outcompeted regular care since critical care capacity and operation room (OR) resources were allocated to COVID-19 patients [2,3,4]. Consequently, the available surgical capacity was reduced and in the Netherlands this resulted in 75–90% fewer surgeries performed at the beginning of the COVID-19 crisis compared with previous years [4]. Likewise, the surgical capacity during the third COVID-19 wave (February until May 2021) was reduced by 21% [5]. The lack of optimal care for oncological, orthopaedic, ophthalmological, and cardiovascular patients resulted in higher mortality and morbidity rates, accompanied by a reduced quality of life [6,7,8,9,10,11]. Thus far, the “collateral damage” of COVID-19 continues to accumulate.

During the pandemic, health care experts have been facing dilemmas as to which patients to prioritise. In addition, the shortage of OR capacity has resulted in a significant backlog of elective or semielective surgeries [12, 13]. This gives rise to the debate on how to deal with the short- and long-term consequences of scarce surgical resources. As a multidisciplinary group of experts within the fields of psychology, ethics, medicine, and epidemiology, we developed a decision model to provide transparent and objective information to physicians in the triage process [14]. This model quantifies the consequences of delay for 43 surgeries in an academic setting. One of the input parameters used in this model is quality of life (QoL). Since these QoL weights were partly estimated by an expert panel and could therefore be disputed, a follow-up study was conducted with another expert panel to validate these QoL estimates [15]. The ranking of surgeries was remarkably unresponsive to which panel estimates were used.

Since our original decision model only simulated academic semielective surgical interventions, it does not apply to nonacademic hospital settings [14]. The aim of this study is to validate our decision model in a nonacademic setting and include additional elective surgical procedures. This extension is especially of added value since the current prioritisation dilemmas mostly involve this type of surgery.


Markov model

We used a previously developed decision model to assess the health effects of surgery postponement of surgeries frequently performed in nonacademic hospitals. This model is a three-state cohort state-transition model, often called a Markov model, and was originally used to assess the health effects of surgery postponement in the academic setting. A detailed model description can be found in our previously published work [14].

The three health states considered in this model were the preoperative, postoperative, and dead states (see Additional file 1). The entire cohort started in the preoperative state, followed by a transition to the postoperative or dead state. The scenarios of surgical delay were modelled with intervals of ten weeks, starting from two weeks up to one year. Definitive cancellation of surgery was modelled as patients remaining in the preoperative health state for their remaining lifetime until they became 100 years old. The model required seven input parameters: 1) survival rate presurgery, 2) survival rate postsurgery, 3) QoL presurgery, 4) QoL postsurgery, 5) mean age of patients undergoing the surgery, 6) time until no effect of treatment can be expected on survival or 7) time until no effect of treatment can be expected on QoL (see Additional file 1). The model outcomes were years of life lost (YLL) and disability adjusted life years (DALY), which is similar to the outcome used in our study on surgeries performed in the academic setting. A detailed description of the methods used to calculate model outcome can be found in our previous work [14]. DALYs were preferred as the leading urgency measure since they incorporate QoL in contrast to YLL, and disability weights were used to calculate health loss. With respect to the utilitarian ethical perspective, priority should be given to patients with the most DALYs per unit of time since this approach will minimise total health loss for society. Health outcomes were calculated based on the expected outcome at 52 weeks compared with the outcome at two weeks of surgical delay. The difference yields the health loss per 50 weeks. This measure is converted to health loss per month delay. This urgency measure yields the ranking of the surgical procedures, where procedures with a high DALY/month are ranked higher.

Surgery selection

A list of the 100 most frequently performed surgeries in nonacademic hospitals in the Netherlands in 2019 was obtained from the Dutch Hospital Data (DHD) national registry [16]. Subsequently, acute, paediatric, and obstetric surgeries were excluded by a senior physician (RBdJ). Acute was defined as interventions that had to be performed within 72 h after hospital admission. Surgeries performed at the outpatient clinic were also excluded, assuming the outpatient clinic capacity was less affected by COVID-19 compared with the reduction in operating room capacity.

Literature data

The input parameters mean age and survival pre- and postsurgery were obtained from the National Heart Registry (NHR), Dutch Hospital Data, and Netherlands Cancer Registry (NCR) [16,17,18]. Registry data were complemented with input from the scientific literature. This extensive literature search also generated relevant data on the two input parameters of ‘time until no effect’ on survival and QoL. In cases of insufficient or missing data, physicians specialized in this specific surgery were consulted to give their considered opinion. All input parameters were classified as class I (Randomized Controlled Trials (RCT) or systematic reviews of RCTs), class IIa (Prospective observational studies, before-after studies), class IIb (Retrospective observational studies, expert panels for utilities, national registries), and class III (expert opinion).

Quality of life collection

To estimate the QoL input parameters, a two-round Delphi study was conducted. The preparation phase consisted of participant recruitment and establishing the method for QoL collection. Medical experts from surgical and nonsurgical specialties were asked to participate in an online panel (Value Based Operation Room Triage team collaborators, see Additional file 3). The primary expert panel consisted of 15 medical experts from a general teaching hospital affiliated with our academic centre (Maasstad Hospital Rotterdam), named ‘Maasstad experts’. Through snowball sampling, another 11 medical experts from different nonacademic hospitals in the Netherlands were recruited, called ‘Other experts’. They were asked to participate in a comparable Delphi study to validate the results of the primary expert panel.

QoL data were collected in accordance with the methods described by Stouthard et al., whereby health states are rated according to a given scale [19]. First, case descriptions were established for every surgery (n = 28) and illustrated a preoperative and postoperative health state. These descriptions were established after consultation with an external senior physician. For procedures in accordance with our previous study on academic surgeries, the descriptions were adjusted to the nonacademic setting. For both expert panels, the first round of the Delphi study started in August 2020. Due to COVID-19 restrictions, we used an online web-based Delphi method tool named Welphi [20]. The medical experts were asked to rate the QoL for each health state on a calibrated visual analogue scale (VAS). This scale ranged from 0 (being the worst health state imaginable, similar to death) to 100 (being the best health state imaginable). Available QoL estimates from the WHO Global Burden of Disease were presented to the participants as reference points (see Additional file 3) [21].

Additionally, they were asked to write a short commentary with their arguments for the given scores. These comments, accompanied by the median score and interquartile range of the entire expert group, were displayed during the second round in October 2020. Each individual was able to adjust his or her QoL valuations based on this additional information, approaching group consensus. A detailed overview of the QoL collection method is presented in our earlier publications [14, 15].


The published decision model was used for our analysis. This decision model was developed in the open-source software R [22] and based on coding recommendations for decision analysis in health care [23,24,25]. The full model code is available on GitHub via the following link: Utilitarian-distribution-of-OR-capacity-during-COVID-19. The decision model includes a probabilistic sensitivity analysis that is used to reflect parameter uncertainty in the model results [26].

A Bland–Altman analysis was performed to analyse the agreement between the QoL estimates from the ‘Maasstad experts’ and the ‘Other experts’ [27, 28]. The pre- and postoperative QoL estimates were analysed separately. Lower and upper levels of agreement were calculated. Furthermore, the overall mean difference in QoL estimates between the expert panels was calculated by fitting a linear mixed effects model with a random intercept for surgical procedures. Thereafter, the effect of using other QoL estimates on the DALY/month estimates and ultimate ranking of surgical procedures was evaluated using Spearman’s rank correlation; the QoL estimates from the ‘Maasstad experts’ were used as input parameters in the decision model, which yielded the original ranking of surgical procedures and DALY/month. Then, the new QoL estimates retrieved from the other expert panel were utilised as input. These new DALY/month estimates accompanied by the corresponding ranking, were compared with the results of the primary data only. Analyses were conducted using R software [22]. The lmer function in the lme4 package was used to fit the linear mixed effects model [29].


In total, 28 surgeries were selected and evaluated with the decision model. This selection comprised procedures from the following surgical specialties: general (n = 7), cardiothoracic (n = 5), vascular (n = 5), orthopaedic (n = 3), gynaecological (n = 2), urological (n = 2), ear, nose and throat (n = 1), endocrine (n = 1), neurosurgical (n = 1), and ophthalmological (n = 1) (see Additional file 2, Table A2.1). Both pre- and postoperative survival were mostly based on class III evidence (13 out of 28, Table 1). For eight surgical procedures, survival was preoperatively calculated through the treatment effect found in the literature (class IIb evidence). Time to no effect on QoL was based on class I evidence. Time to no effect on survival was assumed for eight surgical procedures, all based on class II evidence. For all 28 procedures, the QoL weights were estimated by the expert panels (see Table 1). All parameters were varied in sensitivity analyses (Fig. 1).

Table 1 List of frequently used abbreviations in the manuscript
Fig. 1
figure 1

Final distribution of the seven input parameters used during probabilistic sensitivity analysis (PSA). y-axis = surgeries; x-axis = parameters. See Table 3 for the abbreviations used

QoL estimates

Fifteen physicians from Maasstad Hospital participated, 6 of whom were from surgical specialities. The panel of 11 ‘Other experts’ that validated the primary QoL estimates had approximately the same ratio of surgical to nonsurgical specialities as the 15 ‘Maasstad experts’ (see Table 2). The QoL weights estimated by the two panels were very similar (see Fig. 2). The overall mean difference between these two panels was 0.005 (95% CI -0.014–0.004). The Bland–Altman mean bias and lower and upper levels of agreement were -0.009 (-0.101–0.083) for the preoperative estimates, and -0.001 (-0.033–0.031) for the postoperative estimates (see Additional file 4).

Table 2 Characteristics of the type and class of evidence incorporated into the decision model
Fig. 2
figure 2

Ranking established using QoL estimates from ‘Maasstad experts’ (top) and ‘Other experts’ (below). The average DALYs and YLLs per month of surgical delay are displayed for the surgeries frequently performed in nonacademic hospitals. The estimates (gray bars) and 95% confidence intervals (black lines) are shown. The 95% confidence interval for ‘carotid endarterectomy’ can be explained by the high uncertainty in the ‘time to no effect’ parameter. See Table 3 for the abbreviations used

Table 3 Characteristics of the two expert panels which estimated the QoL weights


Since the QoL estimates were comparable between the two panels, the definitive ranking of surgical procedures was relatively unresponsive to the estimates used. Spearman’s correlation coefficient between the primary ranking and the ranking based on QoL estimates of the ‘Other experts’ was rho = 0.983 (p < 0.001) (see Fig. 3). The largest difference in DALY/month was found for ‘instable AP, CABG’. Using the original QoL estimates from ‘Maasstad experts’, the DALY/month was 0.023 (95% CI 0.015–0.034), whereas using the QoL estimates from the ‘Other experts’ yielded 0.030 (95% CI 0.022–0.040) DALY/month. This decrease in DALY/month resulted in a shift from place seven to place five (see Fig. 2).

Fig. 3
figure 3

Correlation between rankings based on QoL estimates from ‘Maasstad experts’ (x-axis) versus ‘Other experts’ (y-axis). Rho = Spearman correlation coefficient

The DALYs resulting from one month of surgical delay ranged from 0.006 (95% CI 0.002–0.009) for performing a hemithyreoidectomy in patients with thyroid adenoma to 0.054 (95% CI 0.025–0.103) for pacemaker implantation (see Fig. 4 for an overview of all 28 surgeries). This ranking was established using the QoL estimates of both expert panels and considered the definitive ranking.

Fig. 4
figure 4

Final ranking of n = 28 surgeries frequently performed in nonacademic hospitals. This ranking incorporates QoL estimates of both expert panels. The average DALYs and YLLs per month of surgical delay are displayed. The estimates (gray bars) and 95% confidence intervals (black lines) are shown. See Table 3 for the abbreviations used

Eventually, all currently available (n = 84) were integrated into an online tool where users can select different surgeries and compare the surgical outcome applicable to their hospital setting. This tool is accessible via OR triage decision tool ( Figure 5 presents an overview of all surgeries currently incorporated in our model. As shown in this figure, the ranking based on YLL/month, which incorporates survival only, is comparable to the ranking of DALY/month, where QoL is also taken into account.

Fig. 5
figure 5

Ranking of n = 84 surgeries (performed in nonacademic and academic hospitals) incorporated into our model. The average DALYs and YLLs per month of surgical delay are displayed. The estimates (colored bars) and 95% confidence intervals (black lines) are shown. The red bars represent academic surgeries and the blue bars represent nonacademic surgeries. NA = nonacademic. See Table 3 for the abbreviations used


In this study, we estimated the health consequences of surgical delay for 28 surgeries frequently performed in a nonacademic setting. This is an extension of our previously published work and contributes to a better overview of the entire mix of surgeries performed in both academic and nonacademic hospitals. The surgical intervention associated with the most DALYs/month due to surgical delay, was a pacemaker implantation. Hemithyreoidectomy was the procedure with the least health loss per month delay. Furthermore, we showed that the QoL estimates used as input parameters for the model are robust and validate towards different groups of experts.

The results of this study further bolster our previous findings on surgeries performed in academic hospitals. By extending the model to a nonacademic setting, we included 16 new procedures and elective surgical interventions that mostly intended to avert QoL loss. This extension is especially of added value for clinical practice since the current worldwide backlog of surgeries mainly consists of elective procedures [5, 30,31,32]. Moreover, the current differentiation in hospital setting enables health care professionals to obtain an estimation of the expected health loss, which is applicable to their hospital setting. Whenever the model outcome does not correspond to their setting, it is crucial to take the patient’s health status into consideration. For fragile and comorbid patients, it could be argued that the likelihood that they will benefit from surgery is lower than that for relatively healthy patients [33]. This argument, however, is inherently related to the controversy over using utilities to allocate resources in general [34].

Our analysis showed that the ranking of surgeries frequently performed in nonacademic hospitals is primarily driven by survival and not QoL. The ranking based on YLL/month was comparable to the definitive ranking of DALY/month. This accords with our earlier results describing semielective surgeries [14]. In the current study, we added elective procedures, which are nonemergency surgeries for conditions that are by definition not an immediate threat to life [35, 36]. Since the impact of postponing these elective surgeries on YLL is relatively low compared with semielective surgeries, nonemergency surgical interventions (i.e., tonsillectomy) ended up lower in the ranking. However, it should be noted that the delay of these surgeries could still lead to unintended physical consequences, worsened quality of life and subsequently societal costs through decreased work productivity for patients [11, 37,38,39].

Other studies have emphasised the resource allocation dilemmas that occurred during the COVID-19 pandemic and subsequently suggested several strategies to counter them [40,41,42,43]. These strategies arise from different ethical principles that could all provide guidance but come with different practical implications. Our decision model is consistent with the utilitarian perspective: striving for the greatest good for the greatest number of people. The decision for this ethical principle was made deliberately, since utilitarianism was most preferred by health care professionals in the context of a pandemic [41, 44,45,46].

In the literature, the agreement on prioritisation between surgeons is low, especially between different specialties [47, 48]. A major advantage of our model is that it objectifies the consequences of prioritisation decisions, which otherwise remain implicit. This property helps to structure the discussion on prioritisation between health care professionals of different specialities and policymakers, as they are more objectively informed. With this model, we integrate the best available scientific evidence and merge this with validated expert opinions. The model outcome (DALY/month delay) enables direct comparisons of surgeries by providing a common outcome measure to quantify the impact of surgery postponement across disciplines. The current extension facilitates intraregional arrangements on prioritisation as a mix of surgeries from different settings is modelled.

The issues of prioritisation and rationing are both ethically and politically sensitive. The idea of prioritising surgical patients based on our decision model might lead to a feeling of discomfort for physicians. The urgency ranking should therefore be considered as an additional information tool to make a deliberate and concise decision. On the whole, the model outcome could be of great value to health care professionals worldwide during the COVID-19 pandemic. The model input could be adjusted to country or region-specific data, which makes the output applicable to a specific setting. Furthermore, we also believe that value-driven resource allocation will be useful after the pandemic. Other causes such as budgetary constraints or personnel shortages, will ultimately lead to scarce resources, thereby forcing physicians once more to prioritise patients.

Strengths and limitations

The limitations of the decision model we used have previously been published and are again applicable to the current study [14]. In summary, we made several assumptions in our model, thereby simplifying the complex process of prioritising surgical patients and disregarding the potential risks of surgery itself. Moreover, the validity and generalisability of the model outcome are highly dependent on the quality of the input parameters, which might have affected the results. In particular, the expected survival without surgery comes with substantial uncertainty. For the current study, it could be argued that the QoL estimates obtained from 26 physicians might not generally be applicable to all patients diagnosed with the included diseases. To conclude, a major limitation to using the model results in practice remains the limited number of surgeries that were modelled.

This study strengthens the usefulness of our previously published work by describing surgeries in the nonacademic setting. Further research should focus on incorporating patient characteristics that are known to influence perioperative or postoperative surgical outcomes, e.g., age and comorbidities. By this means, the model outcome can become much more refined, thereby further approaching the real-life clinical situation. Furthermore, major stakeholders (e.g., physicians, patients) are approached to evaluate any ethical or contextual considerations that should be considered for successful implementation of our model. Moreover, metrics from capacity management (OR time, complications, length of stay) are currently fed into the model. This could make the model helpful for capacity management, and it will provide insights into what combination of surgeries would yield the least health loss given a certain capacity constraint. Last, it is of utmost importance that the model’s validity is preserved by integrating the best available evidence and ensuring that it remains up to date.


We extended our model to surgical procedures in a nonacademic setting and validated the QoL estimates used in that extension. This extended model now incorporates 84 (semi)elective surgeries, including surgeries frequently performed in academic and nonacademic hospitals. We provide an objective framework as a starting point in the discussion on how to prioritise surgical patients, acknowledging the methodological and ethical assumptions made. Our model could help to optimise the utilisation of available surgical capacity in a transparent way in times of scarce surgical capacity and/or when countries are dealing with an extensive backlog of (semi)elective surgeries.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files.


  1. Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet (London, England). 2020;395(10223):470–3.

    Article  CAS  PubMed  Google Scholar 

  2. Fowler S, Zahir SF, Manning W, Kearney A, Sturgess D. Effect of the COVID-19 pandemic first wave and public policy on elective and emergency surgery provision in Southern Queensland. ANZ J Surg. 2021;91(3):249–54.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Iacobucci G. Covid-19: all non-urgent elective surgery is suspended for at least three months in England. BMJ. 2020;368: m1106.

    Article  PubMed  Google Scholar 

  4. NZa. NZa-analyse van de gevolgen van de coronacrisis voor de reguliere zorg - 20 april 2020. 2020. Accessed on January 19 2022. [Available from:].

  5. NZa. Gevolgen van de coronacrisis voor verwijzingen naar de medisch specialistische zorg en inzichten uit Zorgbeeld.: Nederlandse Zorgautoriteit; 2021. Accessed on January 19 2022. [Available from:].

  6. Lai AG, Pasea L, Banerjee A, Denaxas S, Katsoulis M, Chang WH, et al. Estimating excess mortality in people with cancer and multimorbidity in the COVID-19 emergency. medRxiv. 2020:2020.05.27.20083287.

  7. Sud A, Jones ME, Broggio J, Loveday C, Torr B, Garrett A, et al. Collateral damage: the impact on outcomes from cancer surgery of the COVID-19 pandemic. Ann Oncol. 2020;31(8):1065–74.

    Article  CAS  PubMed  Google Scholar 

  8. Byrnes ME, Brown CS, De Roo AC, Corriere MA, Romano MA, Fukuhara S, et al. Elective surgical delays due to COVID-19: the patient lived experience. Med Care. 2021;59(4):288–94.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Lamoureux EL, Fenwick E, Pesudovs K, Tan D. The impact of cataract surgery on quality of life. Curr Opin Ophthalmol. 2011;22(1):19–27.

    Article  PubMed  Google Scholar 

  10. Hanna TP, King WD, Thibodeau S, Jalink M, Paulin GA, Harvey-Jones E, et al. Mortality due to cancer treatment delay: systematic review and meta-analysis. BMJ. 2020;371: m4087.

    Article  PubMed  Google Scholar 

  11. The LR. Too long to wait: the impact of COVID-19 on elective surgery. The Lancet Rheumatology. 2021;3(2): e83.

    Article  Google Scholar 

  12. NZa. Oplegger Monitor Toegankelijkheid van Zorg; gevolgen van Covid-19 - 26 augustus 2021. 2021. Accessed on January 19 2022. [Available from:].

  13. Carr A, Smith JA, Camaradou J, Prieto-Alhambra D. Growing backlog of planned surgery due to covid-19. BMJ. 2021;372: n339.

    Article  PubMed  Google Scholar 

  14. Gravesteijn B, Krijkamp E, Busschbach J, Geleijnse G, Helmrich IR, Bruinsma S, et al. Minimizing population health loss in times of scarce surgical capacity during the Coronavirus Disease 2019 crisis and beyond: a modeling study. Value Health. 2021;24(5):648–57.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Gravesteijn B, van Hof K, Krijkamp E, Asselman F, Leemans R, Van der Horst H, et al. Minimizing population health loss due to scarcity in OR capacity: a validation study. Submitted to BMC Medical Research Methodology. 2021.

  16. DHD. Dutch Hospital Data. Collected on 28th of October 2020.

  17. Integrated Cancer Center Netherlands. Cancers 2022. Accessed on January 19 2022. [Available from:].

  18. Dutch Heart Registration. 2022. Accessed on January 19 2022. [Available from:].

  19. Stouthard MEA, Essink-Bot M-L, Bonsel GJ, on behalf of the DDWG. Disability weights for diseases: A modified protocol and results for a Western European region. European Journal of Public Health. 2000;10(1):24–30.

  20. Welphi. Welphi Application Internet 2021 [Available from:].

  21. Global Burden of Disease Collaborative Network. Global Burden of Disease Study 2016 (GBD 2016) Disability weights Seattle, WA: Institute for Health Metrics and Evaluation; 2017.

  22. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013.

  23. Alarid-Escudero F, Krijkamp E, Enns E, Hunink M, Pechlivanoglou P, Jalal H. An Introductory Tutorial on Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example2021.

  24. Alarid-Escudero F, Krijkamp E, Enns E, Yang A, Hunink M, Pechlivanoglou P, et al. A Tutorial on Time-Dependent Cohort State-Transition Models in R using a Cost-Effectiveness Analysis Example2021.

  25. Alarid-Escudero F, Krijkamp EM, Pechlivanoglou P, Jalal H, Kao SZ, Yang A, et al. A need for change! a coding framework for improving transparency in decision modeling. Pharmacoeconomics. 2019;37(11):1329–39.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM modeling good research practices task force working group-6. Med Decis Making. 2012;32(5):722–32.

    Article  PubMed  Google Scholar 

  27. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10.

    Article  CAS  PubMed  Google Scholar 

  28. Giavarina D. Understanding Bland Altman analysis. Biochem Med (Zagreb). 2015;25(2):141–51.

    Article  PubMed  Google Scholar 

  29. Bates D, Machler M, Bolker B, Walker SC. Fitting linear mixed-effects models using lme4. J Stat Softw. 2014;67:1–48.

    Google Scholar 

  30. Dobbs TD, Gibson JAG, Fowler AJ, Abbott TE, Shahid T, Torabi F, et al. Surgical activity in England and Wales during the COVID-19 pandemic: a nationwide observational cohort study. Br J Anaesth. 2021;127(2):196–204.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Truche P, Campos LN, Marrazzo EB, Rangel AG, Bernardino R, Bowder AN, et al. Association between government policy and delays in emergent and elective surgical care during the COVID-19 pandemic in Brazil: a modeling study. The Lancet Regional Health - Americas. 2021;3: 100056.

    Article  PubMed  Google Scholar 

  32. Moletta L, Pierobon ES, Capovilla G, Costantini M, Salvador R, Merigliano S, et al. International guidelines and recommendations for surgery during Covid-19 pandemic: a systematic review. Int J Surg. 2020;79:180–8.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Nord E, Enge AU, Gundersen V. QALYs: is the value of treatment proportional to the size of the health gain? Health Econ. 2010;19(5):596–607.

    Article  PubMed  Google Scholar 

  34. Analyses C-A-Y-E. A systematic literature review. Health Aff. 2021;40(9):1402–10.

    Google Scholar 

  35. National Confidential Enquiry into Patient Outcome and Death. The NCEPOD Classification of Intervention: NCEPOD; 2004 [Available from:

  36. American College of Surgeons. COVID-19: guidance for triage of non-emergent surgical procedures 2020, Accessed on January 19 2022 [Available from:].

  37. Sud A, Jones ME, Broggio J, Loveday C, Torr B, Garrett A, et al. Collateral damage: the impact on outcomes from cancer surgery of the COVID-19 pandemic. Ann Oncol. 2020;31(8):1065–74.

    Article  CAS  PubMed  Google Scholar 

  38. Gimbel HV, Dardzhikova AA. Consequences of waiting for cataract surgery. Curr Opin Ophthalmol. 2011;22(1):28–30.

    Article  PubMed  Google Scholar 

  39. COVIDSurg Collaborative. Elective surgery cancellations due to the COVID-19 pandemic: global predictive modelling to inform surgical recovery plans. Br J Surg. 2020;107(11):1440–9.

  40. Hempel S, Burke R, Hochman M, Thompson G, Brothers A, Shin J, et al. Allocation of scarce resources in a pandemic: rapid systematic review update of strategies for policymakers. J Clin Epidemiol. 2021;139:255–63.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Emanuel EJ, Persad G, Upshur R, Thome B, Parker M, Glickman A, et al. Fair allocation of scarce medical resources in the time of Covid-19. N Engl J Med. 2020;382(21):2049–55.

    Article  PubMed  Google Scholar 

  42. Rawlings A, Brandt L, Ferreres A, Asbun H, Shadduck P. Ethical considerations for allocation of scarce resources and alterations in surgical care during a pandemic. Surg Endosc. 2021;35(5):2217–22.

    Article  PubMed  Google Scholar 

  43. Rovers MM, Wijn SRW, Grutters JPC, Metsemakers SJ, Vermeulen RJ, van der Pennen R, et al. Development of a decision analytical framework to prioritise operating room capacity: lessons learnt from an empirical example on delayed elective surgeries during the COVID-19 pandemic in a hospital in the Netherlands. BMJ Open. 2022;12(4): e054110.

    Article  PubMed  Google Scholar 

  44. Biddison LD, Berkowitz KA, Courtney B, De Jong CM, Devereaux AV, Kissoon N, et al. Ethical considerations: care of the critically ill and injured during pandemics and disasters: CHEST consensus statement. Chest. 2014;146(4 Suppl):e145S – e155.

    Article  PubMed  Google Scholar 

  45. Bayer R. Ethical Considerations for Decision Making Regarding Allocation of Mechanical Ventilators during a Severe Influenza Pandemic or Other Public Health Emergency. 2011.

  46. Thompson AK, Faith K, Gibson JL, Upshur REG. Pandemic influenza preparedness: an ethical framework to guide decision-making. BMC Med Ethics. 2006;7:E12-E.

    Article  Google Scholar 

  47. MacCormick AD, Parry BR. Judgment analysis of surgeons’ prioritization of patients for elective general surgery. Med Decis Making. 2006;26(3):255–64.

    Article  PubMed  Google Scholar 

  48. Jacklin R, Sevdalis N, Harries C, Darzi A, Vincent C. Judgment analysis: a method for quantitative evaluation of trainee surgeons’ judgments of surgical risk. The American Journal of Surgery. 2008;195(2):183–8.

    Article  PubMed  Google Scholar 

Download references


We would like to thank Patrick Bindels (professor in General Practice of Erasmus Medical Centre) for his help on the case description development.

Collaboration group: Value Based Operation Room Triage team collaborators

Arend Arends, Brigitte Haberkorn, Charles van Rossem, Gabrielle H. van Ramshorst, Han de Graaff, Harm Sleeboom, Jonne Postema, Josien Terwisscha van Scheltinga, Linda Valk-Kleibreuker, Marco Hoedt,

Martin Baartmans, Mike Nieboer, Miriam Faes, Nieke Oversier, Niels Schep, Onno Schuitema, Patricia MacLean, Patrick Schouwenberg, Paul ten Koppel, Sjaak Pouwels, Suze Raaff, Taco Nieboer, Tietse van Dorp, Willem Maarten Bosman

Value Based Operation Room Triage team collaborators

Arend Arends7, Brigitte Haberkorn8,Charles van Rossem9 , Gabrielle H. van Ramshorst10, Han de Graaff11, Harm Sleeboom12, Jonne Postema11, Josien Terwisscha van Scheltinga13, Linda Valk-Kleibreuker14, Marco Hoedt15, Martin Baartmans16, Mike Nieboer17, Miriam Faes18, Nieke Oversier11, Niels Schep9, Onno Schuitema19, Patricia MacLean20, Patrick Schouwenberg21, Paul ten Koppel22, Pieter Bakx4, Sjaak Pouwels23, Suze Raaff24,Taco Nieboer24, Tietse van Dorp25, Willem Maarten Bosman15

1. Department of Geriatrics, Maasstad Hospital, Rotterdam, the Netherlands

2. Department of Internal Medicine, Maasstad Hospital, Rotterdam, the Netherlands

3. Department of Surgery, Maasstad Hospital, Rotterdam, the Netherlands

4. Department of Surgery, Ghent University Hospital, Ghent, Belgium

5. Department of Anaesthesiology, Maasstad Hospital, Rotterdam, the Netherlands

6. Department of Internal Medicine, HAGA hospital, The Hague, the Netherlands

7. Department of Gynaecology, Maasstad Hospital, Rotterdam, the Netherlands

8. Department of Rehabilitation Medicine, Maasstad Hospital, Rotterdam, the Netherlands

9. Department of Surgery, Albert Schweitzer Hospital, Dordrecht, The Netherlands

10. Department of Paediatrics, Maasstad Hospital, Rotterdam, the Netherlands

11. Department of Orthopedic Surgery, Albert Schweitzer Hospital, Dordrecht, The Netherlands

12. Department of Geriatrics, Amphia Hospital, Breda, The Netherlands

13. Department of Plastic Surgery, Maasstad Hospital, Rotterdam, the Netherlands

14. Department of Rheumatology, Maasstad Hospital, Rotterdam, the Netherlands

15. Department of Intensive Care Medicine, Maasstad Hospital, Rotterdam, the Netherlands

16. Department of Otorhinolaryngology, Maasstad Hospital, Rotterdam, the Netherlands

17. Department of Intensive Care Medicine, Elisabeth-Tweesteden Hospital, Tilburg, the Netherlands

18. General Practitioner, Dordrecht, the Netherlands

19. Retired neuroradiologist, Oegstgeest, the Netherlands


Eline Krijkamp was supported by the Society for Medical Decision Making fellowship through a grant (GBMF7853) by the Gordon and Betty Moore Foundation, Palo Alto, CA, United States. Support for this study was solely from institutional and/or departmental sources.

Author information

Authors and Affiliations




KH, BG and, EK collected the data and conceived and designed the analysis. BG and EK designed the model. PB, PL, and VB collaborators assisted with acquisition of the data. AA performed the analysis, interpreted the data and drafted the manuscript with input from all authors. JB, HL, and RB designed and supervised the study. All authors provided critical feedback and helped shape the research, analysis and manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Anouk M. I. A. van Alphen.

Ethics declarations

Ethics approval and consent to participate

This study was designed and executed in accordance with the ethical guidelines as stated in the Helsinki Declaration and the GDPR act. Informed consent was obtained from all panel members to use and store their data for this study. No approval from an institutional review board was required because our panel members were not subject to procedures or were required to follow rules of behaviour. Therefore, this study is not subject to the Medical Research Involving Human Subjects Act (WMO).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van Alphen, A.M.I.A., van Hof, K.S., Gravesteijn, B.Y. et al. Minimising population health loss in times of scarce surgical capacity: a modelling study for surgical procedures performed in nonacademic hospitals. BMC Health Serv Res 22, 1456 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: