Skip to main content

Cost-effectiveness of transcatheter aortic valve implantation in patients with severe symptomatic aortic stenosis of intermediate surgical risk in Singapore

Abstract

Objective

The objective was to assess the cost-effectiveness of transcatheter aortic valve implantation (TAVI) in patients with severe aortic stenosis with intermediate surgical risk in Singapore.

Methods

A de novo Markov model with three health states – stroke with long-term sequelae, no stroke, and death – was developed and simulated using Monte Carlo simulations with 10,000 iterations over a five-year time horizon from the Singapore healthcare system perspective. A 3% annual discount rate for costs and outcomes and monthly cycle lengths were used. By applying the longest available published clinical evidence, simulated patients received either TAVI or surgical aortic valve replacement (SAVR) and were at risk of adverse events (AEs) such as moderate-to-severe paravalvular aortic regurgitation (PAR).

Results

When five-year PARTNER 2A data was applied, base-case analyses showed that the incremental cost-effectiveness ratio (ICER) for TAVI compared to SAVR was US$315,760 per quality-adjusted life year (QALY) gained. The high ICER was due to high incremental implantation and procedure costs of TAVI compared to SAVR, and marginal improvement of 0.10 QALYs as simulated mortality of TAVI exceeded SAVR at 3.75 years post-implantation. One-way sensitivity analysis showed that the ICERs were most sensitive to cost of PAR, utility values of SAVR patients, and cost of TAVI and SAVR implants and procedures. When disutilities for AEs were additionally applied, the ICER decreased to US$300,070 per QALY gained. TAVI was dominated by SAVR when the time horizon increased to 20 years. Clinical outcomes projected from one-year PARTNER S3i data further reduced the ICER to US$86,337 per QALY gained for TAVI, assuming early all-cause mortality benefits from TAVI continued to persist. This assumption was undermined when longer term data showed that TAVI’s early mortality benefits diminished at five years.

Limitations and conclusion

TAVI is unlikely to be cost-effective in intermediate surgical-risk patients compared to SAVR in Singapore.

Peer Review reports

Introduction

Aortic stenosis refers to the narrowing of the aortic valve, which obstructs blood outflow from the left ventricle to the aorta [1].In Singapore, the estimated prevalence of aortic stenosis increased from 3% in adults older than 75 years to about 8% in those above 85 years [2]. Patients with symptomatic severe aortic stenosis who did not undergo aortic valve replacement (AVR) experienced high mortality [3, 4]. About 44% of patients with symptomatic severe aortic stenosis died within a median follow-up of 14.5 months [5].Transcatheter aortic valve implantation or replacement (TAVI or TAVR) is a minimally invasive procedure that replaces the stenosed valve with a catheter-deployed bioprosthetic valve, often considered as an alternative to conventional surgical aortic valve replacements (SAVR) [2, 6]. The expansion of regulatory approval for lower surgical risk groups has led to its increased uptake, catalysed by its less invasive nature, and improved technology, patient selection and operator learning curve over time [7,8,9]. Published evidence in intermediate surgical risk patients suggested that clinical effectiveness outcomes for TAVI were comparable to SAVR in randomised controlled trials (RCTs) and a propensity-score matched study (PSM). The study designs of published evidence varied with respect to the length of follow-up and the generation of TAVI device i.e. RCTs had longer follow-up with an earlier generation TAVI device while the PSM had shorter follow-up with later generation TAVI [10,11,12,13,14]. Unlike the published evidence for patients with aortic stenosis who were deemed to be inoperable or at unacceptably high surgical risk, the equivocal results observed in the intermediate surgical risk group with varied study types were of notable concern. These differences in clinical evidence could confer different levels of confidence, particularly when informing funding policy decisions.

From the healthcare system perspective, both clinical and economic evidence would need to be considered when making funding policy decisions [8]. Cost-effectiveness studies may be conducted to estimate and compare the costs and effects of different interventions for treatment of aortic stenosis. Although some jurisdictions have recommended funding for TAVI use in intermediate risk patients, there is considerable uncertainty which persists from global variations in device and procedure costs, limited long-term clinical data and heterogeneous economic evidence. Analyses reported to date are predominantly based on two-year trial outcomes, and assumptions about extrapolated long-term outcomes can have substantial impacts on estimates of cost-effectiveness [15,16,17,18,19,20]. The aim of this study is to assess the cost-effectiveness of TAVI compared with SAVR in patients with intermediate surgical risk from the Singaporean healthcare system perspective using local cost inputs and clinical evidence from the most up-to-date clinical trials with published time-to-event data.

Methods

Patients and intervention

A three-state Markov model was developed to evaluate the cost-effectiveness of TAVI compared with SAVR in patients with severe AS of intermediate surgical risk. Clinical inputs were derived from the PARTNER 2A randomized trial [10, 11], where the intervention considered was TAVI (balloon-expandable SAPIEN XT heart-valve system, Edwards Lifesciences) and the comparator was SAVR. The most appropriate comparator was SAVR for treatment in patients at intermediate surgical risk in local clinical practice, in line with published clinical guidelines, published pivotal RCTs, and inputs from local clinician experts [10,11,12, 21, 22]. Standard medical therapy and rapidly-deployed aortic valve replacement were not used as comparators in the model as the former was used mostly for inoperable patients with severe AS, while the latter was rarely used in local clinical practice.

The mean age of the trial population was 81 years; 54% were male and the average Society of Thoracic Surgeons Predicted Risk of Mortality risk score was 5.8% (where a score ranging from 4 to 8% is deemed to be intermediate risk). About 77% of patients had New York Heart Association (NYHA) Class III or IV heart failure and 66% to 69% had coronary artery disease. In the model patients were assumed to enter the model at 80 years old with clinical characteristics (age, stage of heart failure according to NYHA classification and presence of coronary artery disease) similar to key TAVI trials [10, 12, 23, 24]. To ensure the applicability of trial population characteristics to the local population, local clinician experts were consulted and this assumption was considered clinically valid.

Model structure and key specifications

The microsimulation model comprised three health states: “stroke”, which represented disabling or major stroke, a serious complication with long-term sequelae; “no stroke”; and “death”, an absorbing state (Fig. 1). To ensure its face validity, it was developed in consultation with local clinical experts and referenced published economic models of TAVI which were identified via a systematic search of the literature [15,16,17,18,19,20, 25].

At the start of the simulation, patients received the index procedure of either TAVI or SAVR. A monthly cycle length was chosen to capture relevant changes in the health states. Within the first month after the procedure, patients who survived would transition to the “stroke” health state if they experienced disabling or major stroke, or transition to the “no stroke” health state if they did not. Upon transition to the “stroke” state, patients could either stay in it or move to the “death” state if they die. Patients without stroke remained in the “no stroke” state unless they experienced an episode of major stroke or died. All simulated patients were also at risk of other clinically relevant AEs – myocardial infarction (MI), major vascular complications, life-threatening or disabling major bleeding, endocarditis, new permanent pacemaker implantation, transient ischaemic attack (TIA), acute kidney injury (AKI), atrial fibrillation (AF), moderate to severe PAR, and rehospitalisation after procedure. AEs incurred per-cycle costs and disutilities.

A five-year time horizon was used in the base case to capture the costs and effectiveness outcomes, as it corresponds to the longest follow-up from PARTNER 2A, thus avoiding the need for extrapolation. This time horizon was considered appropriate given the expected life expectancy of 83.9 years in Singapore, and the approach was aligned with the Agency of Care Effectiveness Methods and Process Guide [26, 27]. A longer time horizon of 20 years was explored in a scenario analysis. Using point estimate five-year all-cause mortality data from PARTNER 2A, mortality rates between two to five years were estimated from two year published KM data assuming a linear increase over time. The effectiveness outcome used in the model is QALY. The ICERs from pairwise comparisons of incremental costs and QALYs of TAVI and SAVR were estimated. Future costs and effectiveness outcomes were discounted at 3% per annum [26]. All analyses were conducted from the Singapore healthcare system perspective, and all costs were presented in 2020 United States (US) dollars (US$1 = S$1.32) [28]. Only direct medical costs were included. Monte Carlo simulation with 10,000 iterations (seed number set at 1) was used to run the analysis using TreeAge Pro 2019, R2.1 (TreeAge Software, Williamstown, MA).

Model inputs and model validation

Clinical outcomes that informed the base case analysis are summarised in Table 1. These included all-cause mortality, incidences of major stroke and other AEs or complications. Clinically relevant outcomes considered in the model were defined by the Valve Academic Research Consortium (VARC 2/3), consistent with endpoints reported in pivotal trials [10,11,12, 29, 30]. These inputs were informed by the TAVI and SAVR transfemoral-access intention-to-treat populations in PARTNER 2A trial for intermediate surgical risk patients [10, 11]. The PARTNER 2A trial was used as the base case as it was the only RCT for intermediate surgical risk group that provided the most comprehensive Kaplan–Meier (KM) curves for all-cause mortality, and AEs such as stroke (disabling and non-disabling) and moderate to severe PAR. A meta-analysis of the RCTs was not conducted as the results from different RCTs were analysed using different analytical methods. For instance, PARTNER trials used the frequentist approach for endpoint comparisons while the SURTAVI trial performed Bayesian analyses [12]. Without access to patient-level data, pooled analysis of trial data was not deemed feasible. Also, selective reporting of hazard ratios (HRs) meant that a meta-analysis would not accurately represent the collective conclusions of the trials. Where possible, the model had considered various uncertainties associated with the inputs used, and included sensitivity and scenario analyses as needed.

Table 1 Summary of clinical outcomes used in the base case analysis (based on PARTNER 2A trial) [10]

Transitions between health states were determined by all-cause mortality and the incidence of major stroke. All probabilities of clinical outcomes were based on KM estimates at the specific time points. Where clinical outcomes (e.g. all-cause mortality and major stroke) were presented as KM curves, survival data were extracted using WebPlotDigitizer, a web-based digitizer programme [31]. For KM probabilities that were only available at specific time points (e.g. 30 days, 1 year, 2 years, 3 years and 5 years after the index procedure) in the studies, the monthly incidence rate of clinical events was assumed to be constant between each time point. The monthly transition probability of each clinical outcome was calculated using the formula 1- S(t-1)/S(t), where S(t) was the survival probability of the event at time t.

Patients with stroke were assumed to be at a higher risk of death relative to those without stroke. Due to the lack of data in patients with intermediate surgical risk, the elevated all-cause mortality risk for patients in the “stroke” health state was derived from PARTNER 1B and 1A analyses [32] on patients who were inoperable or had high surgical risk respectively. To obtain lower mortality rates for the “no stroke” health state, all-cause mortality was divided by the weighted HRs of death of patients with and without stroke at each time point (see Table 2).

Table 2 Hazard ratios of all-cause mortality in patients with versus without stroke (cycle 1 to 15)

The risks of other AEs were assumed to be the same in the “stroke” and “no stroke” health states. HRs for mortality in the “stroke” health state were assumed to be the same regardless of the type of procedure received. Following a stroke event, patients who enter the “stroke” health state were at an increased risk of death compared with their age-specific counterparts who have not experienced a stroke up until 15 months; beyond 15 months, it was assumed that there was no difference in the hazard of death between patients in the “stroke” and “no stroke” health states and patients in the “stroke” health state would revert to background population mortality.. This assumption was corroborated with local clinician experts, other published CEAs, and the Markov traces used in model validation (see Additional materials file 1) [19, 20, 33].

The impact of AEs on mortality was assumed to have been accounted for in all-cause mortality. Markov traces of all-cause mortality generated for model validation showed that the simulated KM curves were very similar to that observed in the PARTNER 2A trial (see Additional materials file 1).

Resources and costs

Consistent with the healthcare system perspective, only direct medical costs were considered in the model. These included charges of index procedures of TAVI or SAVR, follow-up visits and treatment for AEs. After the index hospitalisation, patients incurred costs for outpatient follow-up care, which was set at one cardiologist visit per year. It was assumed that patients who experienced AEs would incur a one-time treatment cost. Charges of a follow-up visit every three months were only assigned in the “stroke” health state.

Table 3 showed the charges applied in the model extracted using relevant Australian Classification of Health Interventions (ACHI) codes and International Classification of Diseases (ICD)-10 codes (details in Additional materials file 2). The aggregated mean charges of respective index procedures and treatment of AEs were sourced from inpatient episodes admitted between 2016 and 2018 from the Singapore Ministry of Health (MOH) Casemix and Subvention System, using their respective ACHI codes for TAVI, SAVR, and permanent pacemaker implantation, and ICD-10 codes for other AEs. Codes were determined based on World Health Organization ICD-10 online version and published literature, which were then validated by a clinical coder. Charges of outpatient follow-up care after index procedure were based on estimates from the survey of local experts. The charge of a neurologist outpatient visit was obtained from MOH subvention data. Charges for outpatient follow-up applied in the model were converted to monthly costs for the analysis. As all costs used were assumed to be in 2020 Singapore dollars and converted to 2020 US dollars using the 2020 exchange rate [28], inflation was not considered.

Table 3 Cost parameters and corresponding values

Health state utilities

Table 4 showed the EQ-5D utilities from the PARTNER 2A trial used in the base case [34]. In the model, patients started with the first-month utility in the first cycle. To capture the gradual change in patient’s quality of life, utility was assumed to change linearly over time. For example, the change from Month-1 utility to the Month-12 utility occurred in equal increments or decrements. As the trial did not report utility data beyond 24 months, it was assumed that there was no difference in utilities between TAVI and SAVR patients beyond 24 months, and the last utility value applied in the TAVI group was assigned to both groups in subsequent cycles. As the impact of AEs on quality of life was assumed to be already incorporated in the utility values reported in the trials, no additional utility decrements associated with AEs were applied in the base case analysis to avoid double-counting.

Table 4 Utility values and utility decrements

Sensitivity analyses

One-way sensitivity analysis (OWSA) was conducted over the range of predefined values for specific model parameters (i.e. 95% confidence interval or ± 20% for all variables except discount rates which varied from 0 to 5%). The parameters included in the OWSAs were discount rate, implant costs, costs of procedure episode, costs for treatment of AEs and health state utilities. The impact on the ICER was presented in a Tornado diagram.

A multivariate probabilistic sensitivity analysis was performed, using 5,000 s-order Monte Carlo simulations (10,000 first-order simulation trials). Uncertainty in model inputs were explored by randomly sampling the parameters from assigned distributions (see details in Additional materials file 3). All-cause mortality and incidence of AEs in the first month and utility values were modelled as beta distribution, whereas costs of AE treatments were assumed to follow gamma distributions. A cost-effectiveness acceptability curve was generated to present the probability of each strategy being cost-effective at willingness-to-pay (WTP) thresholds.

Scenario analyses

To test the robustness of the model results to data and methodological assumptions, four scenario analyses were examined – Scenario 1: Clinical outcomes using SAPIEN 3 valve (PARTNER S3i PSM); Scenario 2: Clinical outcomes using CoreValve® System (SURTAVI), Scenario 3: Lifetime horizon of 20 years applied on base case setting; Scenario 4: Incorporating disutilities associated with AEs in base case. More details on the scenarios are in Additional materials file 4.

Results

Base case analysis

Table 5 showed that the base case ICER for TAVI at intermediate surgical risk was US$315,760 per QALY gained. This was based on average total discounted costs incurred per patient of US$70,959 for TAVI and US$39,492 for SAVR, and patients receiving TAVI experienced 0.10 more discounted QALYs than those receiving SAVR.

Table 5 Base case and probabilistic sensitivity analysis results

Sensitivity analyses

Results from deterministic OWSA showed that the ICERs were most sensitive to the cost of PAR, cost of TAVI implant, cost of SAVR procedure and implant, cost of TAVI procedure, and the utility value of SAVR patients at Month-12 (see Fig. 2).

The probabilistic sensitivity analysis results showed that the mean probabilistic ICER was US$319,241 per QALY gained, which was similar to US$315,760 per QALY gained in the deterministic base case results (see Table 5). Only when the threshold exceeded approximately US$321,970 (S$425,000) per QALY gained did TAVI become the more cost-effective option (see Fig. 3).

Scenario analyses

The ICER reduced significantly to US$86,337 per QALY gained in Scenario 1, where clinical outcomes for the SAPIEN 3 valve in the PARTNER S3i PSM study were used, due to the smaller incremental costs and greater incremental QALYs gained. In Scenario 2 when clinical outcomes were based on SURTAVI trial which used TAVI CoreValve® System, a much higher ICER of US$837,595 per QALY gained was estimated as the QALY gain was only 0.04. When the time horizon was increased from 5 to 20 years in Scenario 3, the TAVI intervention was dominated by SAVR as QALYs associated with SAVR was higher than TAVI at 0.31. When disutilities associated with AEs were incorporated, TAVI became slightly more favourable with a 5% reduction in ICER, at US$300,070 per QALY gained. Collectively, these scenario analyses suggested the ICER was highly sensitive to both the source of effectiveness data and the time horizon, but not to AE-related disutilities. More details are in Additional materials file 4.

Discussion

To the best of our knowledge, this is the first cost-effectiveness analysis that comprehensively considered the most up-to-date published trial data with time-to-event data including the PARTNER 2A, PARTNER S3i PSM, and SURTAVI trials. The use of scenario analyses also allowed an assessment of the impact on ICERs when the clinical trials, assumptions, and time horizons were varied.

With a base case ICER estimated to be US$315,760 per QALY gained, the use of TAVI in patients with intermediate surgical risk would unlikely be deemed cost-effective by conventional standards. Based on recognised reference international cost-effectiveness thresholds such as from the National Institute for Health and Clinical Excellence (NICE) and the World Health Organisation (WHO), the base case ICER obtained suggests that TAVI is unlikely to be cost-effective in this patient group [35,36,37,38,39]. The high ICER was attributed to the high implantation and procedure costs for TAVI relative to SAVR, and a marginal cumulative improvement of 0.10 QALYs as the simulated mortality in TAVI exceeded SAVR at around 45 months (3.75 years) post-implantation. When uncertainty was considered, deterministic one-way sensitivity analysis showed that ICERs continued to be high, ranging from US$248,605 to US$432,600 per QALY gained. Although there is no known ICER threshold in Singapore, ICERs for medical technologies previously recommended for subsidy in Singapore were generally below US$34,091 (S$45,000) per QALY gained [26, 40]. When US$34,091 was used as WTP in the model, an incremental net monetary benefit (INB) analysis culminated in a negative INB (-US$28,097; lower limit -US$11,133, higher limit -US$45,061), indicating that TAVI was unlikely a cost-effective option in Singapore when compared with SAVR [40]. This was also consistent with the probabilistic sensitivity analyses where TAVI became the more cost-effective option only when WTP exceeded approximately US$321,970 (S$425,000) per QALY gained.

The scenario analyses showed significant changes in ICER when the source of clinical trial data or time horizon were varied. The one-year outcomes data on PARTNER S3i PSM were used in Scenario 1 of this study instead of a conference abstract on its five-year outcomes data as no KM curves on all-cause mortality and stroke were available in the latter [14]. Nevertheless, the ICER estimated using observed five-year data from the PARTNER S3i PSM is likely to be higher than that estimated using the one-year data. This can be inferred from the higher reported all-cause mortality in TAVI at five years of 39.1% compared to 26.8% extrapolated from one-year data from the PARTNER S3i PSM, and relatively similar reported and extrapolated all-cause mortality at five years for SAVR (40.8% vs 41.3% respectively). PARTNER S3i PSM data showed that early all-cause mortality benefits from TAVI over SAVR at one year (TAVI 6.5% vs SAVR 12.2%) diminished at five years (TAVI 39.1% vs SAVR 40.8%), depicting how simulated data could differ from actual data as the expected trajectory of effects changes during the simulated period and early benefits did not persist in the reported trial results. It was difficult to determine if this discrepancy could also be influenced by the lack of randomisation in PARTNER S3i PSM. Similarly, for the CoreValve® System in SURTAVI trial, given the lack of utility data and clinical evidence beyond 24 months, the use of PARTNER 2A utility weights and the assumption that patients in both arms would follow linearly projected risks based on the second-year probability in the SAVR arm may have attributed to the higher ICER. We observed that the simulated mortality benefit conferred by CoreValve® System compared with SAVR was less than that in our base case, as shown in Additional materials file 1, where the probability of death in TAVI arm overtook that of SAVR arm from Month-11, until Month-24.

When compared with another published cost-effectiveness analysis (CEA) study on TAVI in intermediate risk study in Singapore, results from our study were much higher than their reported ICER of US$25,631 per QALY gained over a five-year time horizon despite also using five-year follow-up PARTNER 2A trial data [9]. These differences could be attributed to differences in model structure, health utilities, and cost differential between TAVI and SAVR. Unlike the published 2020 Singapore study, our study used only stroke health states and cost parameters for other AEs as published RCT data could not support mutually exclusive health states for different AEs [9]. While our model used a higher cost differential between TAVI and SAVR based on extractions from the national database than the 2020 study [9], the incremental QALY gain difference in our model was only about half of that from the 2020 model (0.10 QALYs vs 0.19 QALYs). For disutilities of AEs, the published 2020 study assumed a long-term disutility of 0.24 for stroke which was larger than the disutility value 0.161 used in the scenario analyses in our study, and ongoing AKI disutility was applied as a one-off occurrence in our study[9]. Furthermore, the scenario analyses in our study suggested that TAVI was worse off than SAVR based on the extrapolated trajectory for all-cause mortality beyond five years from the PARTNER 2A trial and became dominated when time horizon increased from five to 20 years. However, the published 2020 study reported lower ICER when time horizon increased from five to eight years [9]. Given the extrapolated trajectory, based on our model, it is unlikely that the ICER would be more favourable for TAVI when the time horizon increases. More details are in Additional materials file 5.

Beyond Singapore, CEA studies on TAVI in intermediate surgical risk patients in Australia [33], Canada [15, 17, 20], France [16], Japan [25], Ireland [41], Italy [42], Norway [43], Scotland [44], Spain [45], the United States (US) [8] and Wales [46] have been published. Published ICERs varied, with some reporting dominant ICERs favouring TAVI [8] while other studies reported ICERs as high as US$134,775 (£98,965) per QALY gained [44], showing inconsistency in cost-effectiveness of TAVI in the intermediate surgical risk group. Our study and the published 2020 study from Singapore used five year all-cause mortality data from PARTNER 2A and mortality rates between two to five years were estimated by assuming a linear increase over time [9]. All other published CEA studies had used one- to two-year data for TAVI from PARTNER 2A, PARTNER S3i registry, SURTAVI or OCEAN-TAVI registry, or a combination of them[8, 16, 17, 25, 33, 41,42,43,44, 46]. Extrapolation techniques for clinical inputs post two-year follow-up varied among the studies, with different studies making different assumptions on risks of complications and mortality. Studies that extrapolated all-cause mortality from the two-year PARTNER 2A trial yielded ICERs that were more favourable to TAVI as the extrapolated trajectory would extend TAVI’s early benefits on all-cause mortality. Similar to our study, costs relating to device, procedure, or both of TAVI and SAVR were consistently identified as key driver in the studies [15, 20, 43]. Data used to generate ICERs in these studies were limited to PARTNER 2A and did not reflect clinical outcomes of the latest generation TAVI device. A larger cost differential between SAVR and the more expensive TAVI would yield higher ICERs. It is plausible that the cost-effectiveness of TAVI in intermediate risk patients may change should the cost of TAVI be reduced

Globally, various health technology assessment agencies have evaluated TAVI in patient with intermediate surgical risk [7, 41, 43, 44, 46,47,48,49]. Health technology assessment agencies that performed de novo CEA – Healthcare Improvement Scotland (HIS), Health Technology Wales (HTW), Health Information and Quality Authority (HIQA) of Ireland, Norwegian Institute of Public Health (NIPH) – yielded ICERs that ranged from dominant ICERs favouring TAVI [41]to ICERs ranging from 1.04 million Krones to £98,965 (US$119,097 to US$134,775)[43, 44, 46]. All of them had used two-year follow-up data from PARTNER 2A which would have overestimated TAVI’s benefit as the early all-cause mortality benefits of TAVI were assumed to persist in the extrapolated trajectory for the simulated time horizon from two years to lifetime. When applied over longer time horizons of 15 years to lifetime, the ICERs would favour TAVI to an even greater degree, albeit with greater uncertainty given the lack of robust comparative evidence of the latest generation TAVI device for consideration in these models. Findings from these health technology assessment agencies had led to positive reimbursement decisions in three of the four jurisdictions. Had more mature five-year PARTNER 2A been available and incorporated in their assessments, higher ICER ranges would be expected. This exemplifies a key challenge faced by health technology assessment agencies when the evidence continues to evolve in a different trajectory or direction after reimbursement decisions are made and how often reassessment should take place when new or long-term evidence emerges.

From the payer perspective in Singapore, successful implementation of a positive reimbursement decision for TAVI for specific surgical risk groups is contingent on objective surgical risk assessment and susceptibility to leakage into lower risk groups. Locally, the likelihood of unintended leakage into lower surgical risk groups could yield poorer cost-effectiveness results due to the narrowing effectiveness between the intervention and comparator over time, and the need for valve replacement when implanted in patients in lower risk groups who tend to be younger. Although STS risk scores could be used to objectively determine the level of surgical risk, other clinical and patient factors such as frailty could also affect the multidisciplinary heart team’s determination of surgical risks assessment.

There were several limitations in this study. First, the published clinical evidence from PARTNER 2A and SURTAVI may not sufficiently reflect the current generation of TAVI models, current TAVI operator experience, and more rigorous patient selection for TAVI over time. Although PARTNER 2A and SURTAVI provided the most robust five- and two-year comparative evidence, they mainly used earlier generation TAVI models that were no longer used in current local clinical practice. Except for greater early quality of life improvement at one month which diminished by one year from current generation SAPIEN 3 model when compared to predicate SAPIEN XT from PARTNER S3i PSM [50], there was limited head-to-head comparative evidence in intermediate surgical risk patients to show different clinical outcomes between current and older generation TAVI. Second, although PARTNER S3i PSM used SAPIEN 3, a device that is currently offered in local clinical practice, it was difficult to determine if the risk of residual bias in the PSM was fully eliminated as there was insufficient published information on whether the final models included relevant interactions and higher order terms, and whether the distribution of baseline covariates was balanced within each propensity score quintile. Third, due to a lack of all-cause mortality data that differentiated patients who developed stroke from those who did not, hazard ratios from other PARTNER trials on high surgical risk or inoperable patients had to be applied to meet this data need [32]. However, these patients had inherently different level of surgical risk as our modelled population and the actual hazard ratios might differ. Although hazard ratio estimates based on actual survival curves from patients with intermediate surgical risk that did not develop stroke could improve the validity of these estimates, the availability of granular patient-level data would be needed to validate the assumption that the risk of adverse events would be the same for patients with and without stroke. Fourth, utility values for the “stroke” and “no stroke” health states were assumed to be the same due to the lack of specific utility estimates for patients with and without disabling or major stroke in the AS population. In the absence of longer-term utility data, the utility weights beyond follow-up were assigned based on the last observed value. However, we observed in the Markov traces that the largest difference in proportion with stroke between TAVI and SAVR was at most 2% at any point during the time horizon, while the proportion without stroke was almost the same. Hence, any potential difference in utility values between “no stroke” and “stroke” state would not lead to significant difference in incremental QALYs as shown in Scenario 4.

Conclusion

From the Singapore healthcare system perspective, the inclusion of longer-term trial data and the large cost difference between TAVI and SAVR ascertained that in patients with intermediate surgical risk, TAVI is unlikely to be cost-effective by conventional standards. Evolving and long-term evidence has the potential to significantly impact CEA findings and outcomes of reimbursement decisions.

Fig. 1
figure 1

Markov state transition diagram for the economic model. Abbreviations: SAVR, surgical aortic valve replacement; TAVI, transcatheter aortic valve implantation

Fig. 2
figure 2

Tornado diagram for deterministic one-way sensitivity analysis (top 10 drivers)

Fig. 3
figure 3

Cost effectiveness acceptability curve at varying willingness-to-pay levels for TAVI versus SAVR in intermediate surgical risk patients

Availability of data and materials

The dataset(s) supporting the conclusions of this article are included within the article and (and its additional file(s)).

Abbreviations

ACHI:

Australian classification of health interventions

AE:

Adverse Events

AF:

Atrial fibrillation

AKI:

Acute kidney injury

AS :

Aortic Stenosis

AVR:

Aortic valve replacement

EQ-5D:

EuroQol-5 Dimension

HR:

Hazard ratio

ICD:

International classification of diseases

ICER:

Incremental cost-effectiveness ratio

KM:

Kaplan-Meier

MI:

Myocardial infarction

MOH:

Ministry of Health Singapore

NICE:

The National Institute for Health and Care Excellence

NYHA:

New York Heart Association

PAR:

Paravalvular aortic regurgitation

PSM:

Propensity-scored matched studies

QALY:

Quality-adjusted life year

RCT:

Randomised controlled trial

SAVR:

Surgical aortic valve replacement

TAVI:

Transcatheter aortic valve implantation

TIA:

Transient ischaemic attack

US:

United States

WHO:

World Health Organisation

WTP:

Willingness-to-pay

References

  1. American Heart Association. Aortic Stenosis Overview. 2020; https://www.heart.org/en/health-topics/heart-valve-problems-and-disease/heart-valve-problems-and-causes/problem-aortic-valve-stenosis Accessed 2020 Mar 4

  2. National Heart Centre Singapore. Transcatheter aortic valve implantation (TAVI) - An alternative treatment for symptomatic severe aortic stenosis patients. 2017; https://www.singhealth.com.sg/news/medical-news/transcatheter-aortic-valve-implantation-tavi-alternative-treatment Accessed 2020 Mar 4

  3. Leon MB, Smith CR, Mack M, Miller DC, Moses JW, Svensson LG, et al. Transcatheter Aortic-Valve Implantation for Aortic Stenosis in Patients Who Cannot Undergo Surgery. N Engl J Med. 2010;363(17):1597–607. https://doi.org/10.1056/NEJMoa1008232.

    Article  CAS  PubMed  Google Scholar 

  4. Makkar RR, Fontana GP, Jilaihawi H, Kapadia S, Pichard AD, Douglas PS, et al. Transcatheter Aortic-Valve Replacement for Inoperable Severe Aortic Stenosis. N Engl J Med. 2012;366(18):1696–704. https://doi.org/10.1056/NEJMoa1202277.

    Article  CAS  PubMed  Google Scholar 

  5. Tay ELW, Lew PS, Poh KK, Saclolo R, Chia B-L, Yeo TC, et al. Demographics of severe valvular aortic stenosis in Singapore. Singapore Med J. 2013 Jan;54(1):36–9. https://doi.org/10.11622/smedj.2013009

  6. Medtronic. A minimally invasive alternative for severe aortic stenosis. https://europe.medtronic.com/xd-en/healthcare-professionals/therapies-procedures/cardiovascular/transcatheter-aortic-valve-implantation/tavi/about-the-therapy.html Accessed 2021 Jun 18

  7. Health Quality Ontario. Transcatheter Aortic Valve Implantation in Patients With Severe, Symptomatic Aortic Valve Stenosis at Intermediate Surgical Risk: A Health Technology Assessment. Ont Health Technol Assess Ser. 2020;20(2):1–121.

    Google Scholar 

  8. Baron SJ, Wang K, House JA, Magnuson EA, Reynolds MR, Makkar R, et al. Cost-Effectiveness of Transcatheter Versus Surgical Aortic Valve Replacement in Patients With Severe Aortic Stenosis at Intermediate Risk. Circulation. 2019;139(7):877–88. https://doi.org/10.1161/CIRCULATIONAHA.118.035236.

    Article  PubMed  Google Scholar 

  9. Kuntjoro I, Tay E, Hon J, Yip J, Kong W, Poh KK, et al. Cost-Effectiveness of Transcatheter Aortic Valve Implantation in Intermediate and Low Risk Severe Aortic Stenosis Patients in Singapore. Ann Acad Med Singapore. 2020;49(7):423–33.

    Article  PubMed  Google Scholar 

  10. Leon MB, Smith CR, Mack MJ, Makkar RR, Svensson LG, Kodali SK, et al. Transcatheter or Surgical Aortic-Valve Replacement in Intermediate-Risk Patients. N Engl J Med. 2016;374(17):1609–20. https://doi.org/10.1056/NEJMoa1514616.

    Article  CAS  PubMed  Google Scholar 

  11. Makkar RR, Thourani VH, Mack MJ, Kodali SK, Kapadia S, Webb JG, et al. Five-Year Outcomes of Transcatheter or Surgical Aortic-Valve Replacement. N Engl J Med. 2020;382(9):799–809. https://doi.org/10.1056/NEJMoa1910555.

    Article  PubMed  Google Scholar 

  12. Reardon MJ, Van Mieghem NM, Popma JJ, Kleiman NS, Søndergaard L, Mumtaz M, et al. Surgical or Transcatheter Aortic-Valve Replacement in Intermediate-Risk Patients. N Engl J Med. 2017;376(14):1321–31. https://doi.org/10.1056/NEJMoa1700456.

    Article  PubMed  Google Scholar 

  13. Thourani VH, Kodali S, Makkar RR, Herrmann HC, Williams M, Babaliaros V, et al. Transcatheter aortic valve replacement versus surgical valve replacement in intermediate-risk patients: a propensity score analysis. Lancet (London, England). 2016;387(10034):2218–25. https://doi.org/10.1016/S0140-6736(16)30073-3.

    Article  Google Scholar 

  14. Kodali S. SAPIEN 3 TAVR versus surgery in intermediate risk patients: a propensity score matched analysis of 5-year outcomes [Conference Abstract]. Cardiovasc Revascularisation Med. 2020. https://doi.org/10.1016/j.carrev.2020.07.029.

    Article  Google Scholar 

  15. Tam DY, Hughes A, Wijeysundera HC, Fremes SE. Cost-Effectiveness of Self-Expandable Transcatheter Aortic Valves in Intermediate-Risk Patients. Ann Thorac Surg. 2018;106(3):676–83. https://doi.org/10.1016/j.athoracsur.2018.03.069.

    Article  PubMed  Google Scholar 

  16. Goodall G, Lamotte M, Ramos M, Maunoury F, Pejchalova B, de Pouvourville G. Cost-effectiveness analysis of the SAPIEN 3 TAVI valve compared with surgery in intermediate-risk patients. J Med Econ. 2019;22(4):289–96. https://doi.org/10.1080/13696998.2018.1559600.

    Article  PubMed  Google Scholar 

  17. Tarride J-E, Luong T, Goodall G, Burke N, Blackhouse G. A Canadian cost-effectiveness analysis of SAPIEN 3 transcatheter aortic valve implantation compared with surgery, in intermediate and high-risk severe aortic stenosis patients. Clinicoecon Outcomes Res. 2019;11:477–86. https://doi.org/10.2147/CEOR.S208107.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Geisler BP, Huygens SA, Reardon MJ, Van MN, Kappetein AP, Osnabrugge RLJ, et al. Cost-Effectiveness and Projected Survival of Self-Expanding Transcatheter Versus Surgical Aortic Valve Replacement for High Risk Patients in a European Setting: A Dutch Analysis Based on the CoreValve High Risk Trial. Struct Hear. 2017;1(5–6):267–74. https://doi.org/10.1080/24748706.2017.1381357.

    Article  Google Scholar 

  19. Health Quality Ontario. Transcatheter Aortic Valve Implantation (TAVI) for Treatment of Aortic Valve Stenosis. Ont Heal Technol Assess Ser. 2016;16(19):1–94.

    Google Scholar 

  20. Tam DY, Hughes A, Fremes SE, Youn S, Hancock-Howard RL, Coyte PC, et al. A cost-utility analysis of transcatheter versus surgical aortic valve replacement for the treatment of aortic stenosis in the population with intermediate surgical risk. J Thorac Cardiovasc Surg. 2018;155(5):1978–1988.e1. https://doi.org/10.1016/j.jtcvs.2017.11.112

  21. Nishimura RA, Otto CM, Bonow RO, Carabello BA, Erwin JP 3rd, Fleisher LA, et al. 2017 AHA/ACC Focused Update of the 2014 AHA/ACC Guideline for the Management of Patients With Valvular Heart Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2017;135(25):e1159–95. https://doi.org/10.1161/CIR.0000000000000503.

    Article  PubMed  Google Scholar 

  22. Nishimura RA, Otto CM, Bonow RO, Carabello BA, Erwin JP 3rd, Guyton RA, et al. 2014 AHA/ACC Guideline for the Management of Patients With Valvular Heart Disease: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(23):2440–92. https://doi.org/10.1161/CIR.0000000000000029.

    Article  PubMed  Google Scholar 

  23. Smith CR, Leon MB, Mack MJ, Miller DC, Moses JW, Svensson LG, et al. Transcatheter versus surgical aortic-valve replacement in high-risk patients. N Engl J Med. 2011;364(23 CC-Heart):2187‐2198. https://doi.org/10.1056/NEJMoa1103510

  24. Adams DH, Popma JJ, Reardon MJ, Yakubov SJ, Coselli JS, Deeb GM, et al. Transcatheter aortic-valve replacement with a self-expanding prosthesis. N Engl J Med. 2014;370(19):1790–8. https://doi.org/10.1056/NEJMoa1400590.

    Article  CAS  PubMed  Google Scholar 

  25. Kodera S, Kiyosue A, Ando J, Komuro I. Cost effectiveness of transcatheter aortic valve implantation in patients with aortic stenosis in Japan. J Cardiol. 2018;71(3):223–9. https://doi.org/10.1016/j.jjcc.2017.10.008.

    Article  PubMed  Google Scholar 

  26. Agency for Care Effectiveness (ACE). MEDICAL TECHNOLOGIES EVALUATION METHODS AND PROCESS GUIDE. 2022;

  27. Department of Statistics, Ministry of Trade & Industry. Population trends, 2020. 2020; https://www.singstat.gov.sg/-/media/files/publications/population/population2020.pdf Accessed

  28. Money Authority Singapore. Exchange Rates [Internet]. 2021. https://eservices.mas.gov.sg/Statistics/msb/ExchangeRates.aspx Accessed

  29. Kappetein AP, Head SJ, Généreux P, Piazza N, van Mieghem NM, Blackstone EH, et al. Updated standardized endpoint definitions for transcatheter aortic valve implantation: the Valve Academic Research Consortium-2 consensus document. Eur Heart J. 2012;33(19):2403–18. https://doi.org/10.1093/eurheartj/ehs255.

    Article  PubMed  Google Scholar 

  30. Piazza N. An Introduction to VARC-3 Focused Update, New Endpoints and Definitions for Aortic Valve Clinical Research. Presentation (SHDS). 2018.

  31. Rohatgi A. WebPlotDigitizer [Internet]. https://automeris.io/WebPlotDigitizer/ Accessed 2021 Nov 10

  32. Kapadia S, Agarwal S, Miller DC, Webb JG, Mack M, Ellis S, et al. Insights Into Timing, Risk Factors, and Outcomes of Stroke and Transient Ischemic Attack After Transcatheter Aortic Valve Replacement in the PARTNER Trial (Placement of Aortic Transcatheter Valves). Circ Cardiovasc Interv. 2016 Sep;9(9). https://doi.org/10.1161/CIRCINTERVENTIONS.115.002981

  33. Zhou J, Liew D, Duffy SJ, Walton A, Htun N, Stub D. Cost-effectiveness of transcatheter aortic valve implantation compared to surgical aortic valve replacement in the intermediate surgical risk population. Int J Cardiol. 2019;294:17–22. https://doi.org/10.1016/j.ijcard.2019.06.057.

    Article  PubMed  Google Scholar 

  34. Baron SJ, Arnold SV, Wang K, Magnuson EA, Chinnakondepali K, Makkar R, et al. Health Status Benefits of Transcatheter vs Surgical Aortic Valve Replacement in Patients With Severe Aortic Stenosis at Intermediate Surgical Risk: results From the PARTNER 2 Randomized Clinical Trial. JAMA Cardiol. 2017;2(8):837–45. https://doi.org/10.1001/jamacardio.2017.2039.

    Article  PubMed Central  PubMed  Google Scholar 

  35. Tan-Torres ET, Baltussen, R; Adam, T; Hutubessy, R; Acharya, A; Evans, DB; Murray, DB; Murray C. Making choices in health: WHO guide to cost-effectiveness analysis. Vol. 1. World Health Organization; 2003.

  36. Zhang K, Garau M. International Cost-Effectiveness Thresholds and Modifiers for HTA Decision Making. OHE Consult Rep. 2020; https://www.ohe.org/publications/international-cost-effectiveness-thresholds-and-modifiers-hta-decision-making Accessed

  37. The National Institute for Health and Care. Guide to the methods of technology appraisal. Nice. 2013;2018:1–93. https://doi.org/10.1183/13993003.01815-2018Accessed.

    Article  Google Scholar 

  38. McDougall JA, Furnback WE, Wang BCM, Mahlich J. Understanding the global measurement of willingness to pay in health. J Mark access Heal policy. 2020;8(1):1717030. https://doi.org/10.1080/20016689.2020.1717030.

    Article  Google Scholar 

  39. Culyer A, Chalkidou K, Teerawattananon Y, Santatiwongchai B. Rival perspectives in health technology assessment and other economic evaluations for investing in global and national health. Who decides? Who pays? [version 1; referees: 1 approved, 2 approved with reservations]. F1000Research. 2018;7(May). https://doi.org/10.12688/f1000research.13284.1

  40. Cheng L-J, Wong G, Chay W-Y, Ngeow J, Tan Y, Soon SS, et al. Cost-effectiveness of olaparib maintenance therapy when used with and without restriction by BRCA1/2 mutation status for platinum-sensitive relapsed ovarian cancer. Expert Rev Pharmacoecon Outcomes Res. 2021;21(3):441–8. https://doi.org/10.1080/14737167.2021.1890587.

    Article  PubMed  Google Scholar 

  41. Health Information and Quality Authority. Health Technology Assessment of transcatheter aortic valve implantation ( TAVI ) in patients with severe symptomatic aortic stenosis at low and intermediate risk of surgical complications. 2019;(November). https://www.hiqa.ie/sites/default/files/2019-12/TAVI_HTA.pdf Accessed

  42. Lorenzoni V, Barbieri G, Saia F, Meucci F, Martinelli GL, Cerillo AG, et al. The cost-effectiveness of transcatheter aortic valve implantation: exploring the Italian National Health System perspective and different patient risk groups. Eur J Heal Econ. 2021;22(9):1349–63. https://doi.org/10.1007/s10198-021-01314-z.

    Article  CAS  Google Scholar 

  43. Norwegian Institute of Public Health. Health technology assessment of Transcatether aortic valve implantation (TAVI) as treatment of patients with severe aortic stenosis and intermediate surgical risk – Part 2. Health economic evaluation [Internet]. 2019. https://www.fhi.no/en/publ/2019/Transcatether-aortic-valve-implantation-as-treatment-of-patients-with-severe-aortic-stenosis-and-intermediate-surgical-risk/ Accessed

  44. Healthcare Improvement Scotland. Transcatheter aortic valve implantation (TAVI) for the treatment of patients with severe symptomatic aortic stenosis who are at intermediate surgical risk. SHTG Publ. 2019;(91). https://www.healthcareimprovementscotland.org/our_work/technologies_and_medicines/topics_assessed/shtg_04-19.aspx Accessed

  45. Pinar E, García de Lara J, Hurtado J, Robles M, Leithold G, Martí-Sánchez B, et al. Cost-effectiveness analysis of the SAPIEN 3 transcatheter aortic valve implant in patients with symptomatic severe aortic stenosis. Rev Española Cardiol (English Ed. 2021;(x). https://doi.org/10.1016/j.rec.2021.02.013

  46. Health Technology Wales. Evidence Appraisal Report: Transcatheter Aortic Valve implantation to treat people with severe symptomatic aortic stenosis , who are at intermediate surgical risk. 2020;(September):1–75. https://www.healthtechnology.wales/wp-content/uploads/2020/07/EAR024-Transcatheter-Aortic-Valve-implantation-1.pdf Accessed

  47. EUnetHTA. Transcatheter aortic valve implantation (TAVI) for the treatment of patients at intermediate surgical risk. 2018; https://www.eunethta.eu/wp-content/uploads/2018/12/OTCA06-TAVI-FOR-THE-TREATMENT-OFPATIENTS-AT-INTERMEDIATE-SURGICAL-RISK-FINAL-1.pdf Accessed 2020 Mar 4

  48. Medical Services Advisory Committee. Application No. 1652 – Transcatheter aortic valve implantation via transfemoral delivery for patients at intermediate risk for surgery. 2021;(March). http://www.msac.gov.au/internet/msac/publishing.nsf/Content/8C10EAD0A322460BCA258632000DACB7/$File/1652 Final PSD - Mar-Apr 2021_redacted.pdf Accessed

  49. Medical Services Advisory Committee. Application No. 1603 – Transcatheter Aortic Valve Implantation (TAVI) using SAPIEN 3 balloon-expandable valve system. 2020;(November). http://www.msac.gov.au/internet/msac/publishing.nsf/Content/5C3844FD549800CBCA25849300087D9F/$File/1603 Final PSD_Nov2020_redacted.pdf Accessed

  50. Baron SJ, Thourani VH, Kodali S, Arnold SV, Wang K, E.A. M, et al. Effect of SAPIEN 3 Transcatheter Valve Implantation on Health Status in Patients With Severe Aortic Stenosis at Intermediate Surgical Risk: Results From the PARTNER S3i Trial. JACC Cardiovasc Interv. 2018;11(12):1188–98. https://doi.org/10.1016/j.jcin.2018.02.032

Download references

Acknowledgements

This study was part of a health technology assessment evaluation on TAVI conducted by the Agency for Care Effectiveness (ACE), Ministry of Health, Singapore in 2020. The authors would like to thank Dr Li-Jen Cheng, Dr Hong Ju, Mr Chris Foteff, Mr Mohamed Ismail Abdul Aziz and Dr Louise Goh for their contributions to the health technology assessment evaluation. The authors would also like to thank Dr Tay Lik Wui Edgar and Professor Tan Huay Cheem for their clinical inputs during the course of the health technology assessment evaluation. Additionally, the authors would like to thank members of ScHARR at the University of Sheffield for conducting an external validation of the model.

Funding

This study did not receive any sponsorship or funding.

Author information

Authors and Affiliations

Authors

Contributions

Conception and design: R.S., X.W., K.S.M., S.S., K.N. Analysis and interpretation of the data: R.S., X.W., K.S.M., S.S., K.N., B.K., K.C. Drafting of the paper or revising it critically for intellectual content: R.S., X.W., K.S.M., S.S., K.N., B.K., K.C., K.H., I.K. Final approval of the version to be published: R.S., X.W., K.S.M., S.S., K.N., B.K., K.C., K.H., I.K. Both R.S. and X.W. contributed equally to this manuscript. All authors agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Kwong Ng.

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Not applicable.

Competing interests

H.K. and I.K. received honoraria as speakers from Edwards Lifesciences and Medtronic. All other authors declared no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Figure S1-1.

Actual death proportion from PARTNER 2A trial and simulated Markov traces in the economic model. Figure S1-2. Simulated Markov traces for TAVI and SAVR arms in SURTAVI trial for intermediate surgical risk.

Additional file 2: Table S2-1.

Cost parameters and corresponding codes

Additional file 3: Table S3-1.

Parameters, distributions and upper and lower values used in probabilistic sensitivity analyses

Additional file 4:Table S4-1.

Summary of clinical outcomes used in Scenario 1 (based on PARTNER S3i PSM). Table S4-2. Utility values and decrements applied in scenario analyses. Table S4-3. Summary of clinical outcomes used in Scenario 2 (based on SURTAVI trial). Table S4-4. Results of scenario analyses.

Additional file 5.

Comparison of current model with local published 2020 study [9].

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

See-Toh, R.SE., Wong, X.Y., Mahboobani, K.S.K.H. et al. Cost-effectiveness of transcatheter aortic valve implantation in patients with severe symptomatic aortic stenosis of intermediate surgical risk in Singapore. BMC Health Serv Res 22, 994 (2022). https://doi.org/10.1186/s12913-022-08369-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-08369-5

Keywords