Skip to main content
  • Research article
  • Open access
  • Published:

Reduction of routine use of radiography in patients with ankle fractures leads to lower costs and has no impact on clinical outcome: an economic evaluation



To evaluate the cost-effectiveness of a reduction in the number of routine radiographs in the follow-up of patients with ankle fractures.


We performed an economic evaluation alongside the multicentre, randomised WARRIOR trial. Participants were randomised to a reduced imaging follow-up protocol (i.e. radiographs at week 6 and 12 follow-up obtained on clinical indication) or usual care (i.e. routine radiography at weeks 6 and 12). The Olerud & Molander Ankle Score (OMAS) was used to assess ankle function and the EQ-5D-3L was used to estimate Quality-Adjusted Life Years (QALYs). Costs and resource use were assessed using self-reported questionnaires and medical records, and analysed from a societal perspective. Multiple imputation was used for missing data, and data were analysed using seemingly unrelated regression analysis and bootstrapping.


In total, 246 patients had data available for analysis (reduced imaging = 118; usual care = 128). Fewer radiographs were obtained in the reduced imaging group (median = 4) compared with the usual-care group (median = 5). Functional outcome was comparable in both groups. The difference in QALYs was − 0.008 (95% CI:-0.06 to 0.04) and the difference in OMAS was 0.73 (95% CI:-5.29 to 6.76). Imaging costs were lower in the reduced imaging group (−€48; 95% CI:- €72 to -€25). All other cost categories did not statistically differ between the groups. The probability of the reduced imaging protocol being cost-effectiveness was 0.45 at a wiliness-to-pay of €20,000 per QALY.


Reducing the number of routine follow-up radiographs has a low probability of being cost-effective compared with usual care. Functional outcome, health-related quality of life and societal costs were comparable in both groups, whereas imaging costs were marginally lower in the reduced imaging group. Given this, adherence to a reduced imaging follow-up protocol for those with routine ankle fractures can be followed without sacrificing quality of care, and may result in reduced costs.

Trial registration

The trial was registered on 26-05-2014 in the Netherlands Trial Registry, with reference number NL4477 (

Peer Review reports


Ankle fractures are common and account for about 9% of all fractures in the UK [1]. The incidence of ankle fractures around the world is reported to lie between 71 and 187 per 100,000 persons per year and has risen over the last decade due to ageing of the population and increased participation in athletic activities [2,3,4,5]. Routine imaging during the follow-up of ankle fractures is associated with relatively high healthcare costs [6, 7]. Healthcare costs are expected to rise in coming decades [8]. This has led to an increased interest in the effectiveness of imaging in clinical decision-making [9,10,11,12]. Despite increased costs, both national and international trauma protocols dictate that routine radiographs should be obtained at regular intervals during the follow-up of patients with an ankle fracture, although there is little scientific evidence to support this position [4, 13, 14]. For both non-operatively and operatively treated patients, it is recommended that four outpatient clinic visits including radiographs, are to be conducted after a follow-up of one, two, six and twelve weeks [13]. The goal of these radiographs are to monitor the position of the fracture fragments, the position of fixation material, the alignment of the joint and the bone-healing process.

In the Netherlands, with a population of over 17 million, approximately €5 million is spent annually on radiography for patients with ankle fractures. This estimate is based on an incidence of 30,000 per annum [15], with three to four follow-up radiographs [16], at a cost of €50 per radiograph [17]. Various studies have questioned the value of routine radiographs obtained at the first outpatient clinic visit and at intermediate-to-late follow-up (i.e. after the initial 3 weeks) of operatively treated ankle fractures [18, 19]. A recent retrospective analysis, involving a cohort of 528 patients with an ankle fracture, demonstrated that as few as 1.2% (n = 11/928) of routine radiographs obtained after 3 weeks of follow-up led to a change in treatment strategy [16]. These results suggest that current imaging protocols for the follow-up of ankle fracture patients might not be cost-effective.



The aim of this study was to evaluate the cost-effectiveness of a protocol with reduced numbers of routine radiographs in the follow-up of ankle fractures, in comparison with the current usual care.


This economic evaluation was conducted alongside a multicentre, randomised controlled trial. The methods of this trial, including its sample size calculation, are described in detail elsewhere [20]. Both a cost-effectiveness and cost-utility analysis were performed from a societal perspective. The time horizon of the economic evaluation was 12 months. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) guidelines were followed in preparing this report [21, 22].


Seven hospitals in the Netherlands participated in the study, including three university hospitals and four large teaching hospitals. Patients were enrolled between July 2014 and October 2017.


Inclusion/exclusion criteria

Patients could participate in the study if they provided written informed consent, were over 18 years of age, had a fracture of the ankle (Lauge Hansen classification types Supination Adduction [SA]2, Supination External rotation [SE]2–4, Pronation External rotation [PE] 1–4 or Pronation Abduction [PA] 1–3) [23] and were able to independently complete the Dutch questionnaires. Distortions or isolated Danis-Weber type A fractures [24] were not included. Exclusion criteria were the presence of fractures to multiple extremities and pathologic or open fractures (Gustillo grades 2–3). If patients were deemed unable to comply with follow-up they were also excluded.


Patients were informed about the study both verbally and by means of an information letter. All participants had to provide written informed consent. Participants were randomised by means of computerised allocation, using an online registration and randomisation program (ProMISe; Project Manager Internet Server; Participants were assigned in a 1:1 ratio to either the intervention group or the control group. Randomisation was carried out using a stratified, randomly varying block design (each block size containing 2 to 6 allocations). The tables were internally pre-generated within the secure data management system and stratified by hospital and initial treatment strategy.

Control group – usual care

Patients randomised to the usual care group were monitored at the outpatient clinic and received routine follow-up radiographs at one, two, six and twelve weeks of follow-up. Additional follow-up moments with or without the use of radiographs could be scheduled at any time if deemed necessary by the treating physician.

Intervention group – reduced imaging

In the reduced imaging group, radiographs were routinely obtained after one and two weeks. Radiographs could be obtained later during follow-up if a specific clinical indication was present or obtained at the discretion of the treating physician. Reasons for doing so included an additional trauma to the affected ankle, a pain score of 6 or higher on a 11-point Numerical Rating Scale (NRS), a decrease in Range-Of-Motion (ROM), or neurovascular abnormalities. Motivations to obtain additional radiographs were required to be logged in the medical file. Aside from the modified imaging protocol at follow-up, all aspects of treatment and follow-up were similar for both groups.

Outcome measures

Measurements at baseline included potential confounders [25], such as age, sex, medical history, smoking habits, alcohol intake, functional status, health-related quality of life (HR-QOL) and socioeconomic status. Follow-up questionnaires assessing the patients’ clinical outcomes as well as their resource use were administered after 6, 12, 26, and 52 weeks and could be completed either online or by post. Recall periods of these questionnaires varied from 6 weeks at 6-week follow-up to 26 weeks at 52-week follow-up to cover the complete duration of follow-up.

Clinical outcomes

Functional status of the affected ankle was evaluated using the Olerud and Molander Ankle Score (OMAS). This is a nine-item questionnaire assessing both pain and disability related to the affected ankle. OMAS scores were calculated for all of the measurement points separately, ranging from 0 to 100 with a score of 100 equalling no pain or disability [26]. HR-QOL was assessed using the Dutch version of the EQ-5D-3L. At baseline, participants were asked to complete the EQ-5D-3L for their health state just prior to their trauma. At all other time points, they were asked to consider their current health status. Utility scores per time point were estimated using the Dutch EQ-5D-3L tariff [27, 28]. Quality-adjusted Life Years (QALYs) per patient were estimated using linear interpolation of the utility scores for the different time points. Since the patients’ utility score right after the trauma was not available (i.e. the patients’ “true” baseline utility score), we assumed their utility score at baseline to be equal to that of 6 weeks of follow-up.

Cost measures

Resource use questionnaires were used to measure the patients’ use of primary and secondary healthcare, medication, informal care, as well as their levels of unpaid productivity losses, absenteeism and presenteeism. Costs of the intervention (i.e. costs for the radiographs) were gathered from electronic patient records. Primary healthcare use included the patients’ number of general practitioner consultations, visits to a company medical officer, physiotherapy treatments and visits to other specialised therapists. All of these visits were required to be associated with the ankle fracture. Information on the use of secondary healthcare services was gathered from electronic patient records and included admissions to hospital, nursing home or rehabilitation centre, outpatient clinic visits, all imaging other than plain radiographs (e.g. CT- or MRI-scans of the ankle) and re-operations. These services also included the initial admission right after the trauma occurred, and the primary intervention, if applicable. All healthcare costs were valued according to Dutch standard costs [29] or, if unavailable, tariffs. Medication costs were calculated as costs-per-day for each medication, which was based upon the standard dosage per day and unit prices of the Royal Dutch Society of Pharmacy [30]. Total medication costs were calculated by multiplying this cost per day with the total days of use. If the duration was not specified, we assumed patients used a certain medication during the complete recall period. Unpaid productivity losses (i.e. volunteer work, caregiving or domestic activities patients could not perform due to their trauma) and informal care (i.e. care provided by relatives, friends or volunteers) were valued at €14.13 per hour, a shadow price that is recommended by the Dutch National Health Care Institute [29]. Absenteeism was defined as the number of days of absence due to the ankle fracture. The Friction Cost Approach was used to value absenteeism, which assumes that costs are limited to the time it takes to replace an absent worker (in the Netherlands: 12 weeks) [29]. The participants’ number of presenteeism days were estimated by multiplying the number of days worked (i.e. workable days – sickness absence days) by a self-reported score reflecting their productivity level when they were present at work ranging from 0 (equalling no productivity) to 10 (equalling full productivity). Days of absenteeism and presenteeism were valued using gender-specific price weights [29]. The trial’s follow-up was 12 months and discounting of costs and effects was, therefore, not necessary. All costs were converted to Euros 2016 with the help of consumer price indices [31].

Statistical analysis

Analyses were performed in accordance with the intention-to-treat principle. Missing data were multiply imputed using STATA (Version 12 SE, Stata Corp, College station, TX). The imputation model included utility scores, OMAS scores, and all available cost values at baseline, 6, 12, 26 and 52 weeks as well as the baseline variables: fracture classification, BMI, ASA classification, smoking habits, alcohol intake, hospital, age, sex, randomisation result and operative- or non-operative treatment. These baseline variables were added because they were regarded as possible confounders, because they differed between groups at baseline, and/or because they were predictive of the ‘missingness’ of data. Five complete datasets were generated in order for the loss-of-efficiency to be lower than 5% [32]. Each dataset was analysed separately and estimates were pooled using Rubin’s rules. This method takes into account both imputation variability within each dataset, as well as imputation variability between the separate datasets [32]. Seemingly unrelated regression analyses (SUR) were used to estimate total cost (ΔC) and effect differences (ΔE). The advantage of SUR is that ΔC and ΔE are modelled simultaneously so that their possible correlation can be accounted for [33]. For OMAS, the patients’ 52-week follow-up scores were used as dependent variable. For total costs and QALYs, the patients’ total costs and QALYs experienced during follow-up were used as dependent variable, respectively. OMAS analyses were adjusted for baseline scores and other possible confounders measured at baseline (Table 1). In contrast to the recommendation of Manca et al. [34], we decided not to adjust QALYs for baseline utility scores, as a “true” utility score was lacking in the current study. That is, the baseline utility value in this study described the patients’ utility value prior to their fracture, instead of right after their fracture. The incremental cost-effectiveness ratio (ICER) was estimated by dividing the cost difference by the effect difference (ΔC/ΔE). To estimate the uncertainty around the ICER and to estimate 95% confidence intervals (95% CI) surrounding the cost differences, bias-corrected and accelerated bootstrapping was performed with 5000 replications. For all 5000 replications, the cost and effect pairs were plotted on a cost-effectiveness plane to graphically illustrate the uncertainty surrounding the ICER [35]. A summary measure of the joint uncertainty surrounding costs and effects was provided by constructing cost-effectiveness acceptability curves (CEACs). These curves give an indication of the probability that the reduced imaging protocol for ankle fractures is cost-effective for a range of willingness-to-pay values. CEACs were pooled using a combination of Rubin’s rules and the incremental net monetary benefit approach. Statistical significance was assumed at p < 0.05.

Table 1 Patient characteristics by treatment allocation

Sensitivity analyses

A total of six sensitivity analyses were planned. In the first sensitivity analysis, only data of participants with complete data were used (SA1). The second sensitivity analysis (SA2) made use of the measured utility score at baseline (prior to the fracture), instead of the value derived from the utility score at 6 weeks. The third sensitivity analysis (SA3) used the Human Capital Approach to calculate productivity losses instead of the Friction Cost Approach. The Human Capital Approach assumes that productivity losses occur during the complete period of absence instead of being limited to the friction period. For the fourth sensitivity analysis (SA4), costs were assessed from a healthcare perspective. A healthcare perspective regards only costs accruing to the formal Dutch healthcare system, meaning that costs of informal care, absenteeism, presenteeism and unpaid productivity losses were disregarded. The fifth (SA5) and sixth sensitivity analysis (SA6) only included patients with either a non-operative or an operative treatment strategy, respectively.



We enrolled 312 participants in the study (Fig. 1). Six were excluded after randomisation, because an exclusion criterion was present that was not identified before randomization (Fig. 1). Of the remaining 306 participants, 156 were randomised to usual care and 150 to reduced imaging. In total, 60 patients, 28 in the usual care group (18%) and 32 in the reduced imaging group (21%) did not return any of the follow-up questionnaires and were lost to follow-up. Thus, 246 participants were included in the analysis (n = 128 usual care and n = 118 reduced imaging). Aside from a higher mean Body-Mass-Index (BMI) in the reduced imaging group, no meaningful differences were observed between groups at baseline (Table 1). Surgery was performed in 60% of participants in the usual care group (n = 77/128) and in 65% of participants in the reduced imaging group (n = 77/118). Out of a total of 1230 (246*5) baseline and follow-up questionnaires, 1096 were returned by the participants (89%).

Fig. 1
figure 1

Flowchart of participants


There was no statistically significant difference in OMAS (0.73; 95% CI:-5.3 to 6.8) and QALYs (− 0.008; 95% CI:-0.04 to 0.03) between groups. An overview of the patients’ OMAS and EQ-5D-3L scores per time point can be found in Additional file 1.

Costs and use of resources

As a result of the intervention, patients randomised to the reduced imaging group had fewer radiographs taken of their ankle fracture than patients randomised to usual care, equalling a median number of radiographs of 4 (Interquartile Range [IQR] 3–5) in the reduced imaging group versus a median of 5 (IQR 4–6) in the usual care group. This resulted in a significant reduction in radiograph costs in favour of the reduced imaging group (−€48 per patient, 95% CI:-72 to − 25). All other costs, including total societal costs, were not statistically significantly different between groups (Table 2).

Table 2 Mean cost (in euros) per participant in the intervention and control group and mean cost differences between groups during follow-up


For QALYs, the intervention was dominated by the control, based on a cost difference (ΔC) of €131 and an effect difference (ΔE) of − 0.008 QALY. The ICER for functional outcome was 178, based on the same ΔC of €131 and a ΔE of 0.73 points on the OMAS (Table 3). The CE-plane for QALYs shows that the cost-effect pairs were scattered across all four quadrants of the CE-plane (Fig. 2). The CEAC in Fig. 3 indicates that if decision-makers are willing to pay €20,000 per QALY gained, the probability of reduced imaging being cost-effectiveness compared with usual care was 0.45. This probability reduced with increasing values of willingness to pay to about 0.37 at a willingness to pay of €80,000 per QALY. The CE-plane for the OMAS also shows that the cost-effect pairs were scattered across all four quadrants of the CE-plane (Fig. 4). For OMAS, the CEAC indicates that if decision-makers are not willing to pay anything per point improvement, the probability of reduced imaging being cost-effectiveness compared with usual care was 0.47. This probability increased with increasing values of willingness-to-pay to about 0.59 at a willingness to pay of €5000 per point improvement (Fig. 5).

Table 3 Differences in pooled mean costs and effects (95% Confidence intervals), incremental cost-effectiveness ratios, and the distribution of incremental cost-effect pairs around the quadrants of the cost-effectiveness planes for reduced imaging compared to usual care
Fig. 2
figure 2

Cost-effectiveness plane for QALYs, representing the results from the 5000 bootstrapped replications, and the point estimate. Higher on the Y-axis corresponds to costlier than control, more right on the X axis corresponds to more effective than control

Fig. 3
figure 3

Cost-effectiveness acceptability curve for QALYs, showing the probability of the intervention being cost effective at a certain willingness-to-pay value per QALY

Fig. 4
figure 4

Cost-effectiveness plane for OMAS, representing the results from the 5000 bootstrapped replications, and the point estimate. Higher on the Y-axis corresponds to costlier than control, more right on the X axis corresponds to more effective than control

Fig. 5
figure 5

Cost-effectiveness acceptability curve for OMAS, showing the probability of the intervention being cost effective at a certain willingness-to-pay value per point increase of the OMAS

Sensitivity analyses

Six sensitivity analyses were performed. Outcomes of the sensitivity analyses demonstrated many similarities with those of the main analysis (Table 3). Except for SA6, differences in QALYs were in favour of the usual care group. Cost per category for the non-operatively and operatively treated subgroup are reported separately (Additional file 2 and Additional file 3). Except for SA5, OMAS scores were higher in the reduced imaging group than in the usual care group and except for SA4 and SA5, total costs were highest in the reduced imaging group. However, all of these differences in costs and effects were not statistically significant. It is perhaps important to note that the relatively large differences in ICERs between the main analysis and some of the sensitivity analyses were due to small between-group differences in QALYs and OMAS scores.


The reduced imaging follow-up protocol resulted in a significant decrease in the number of radiographs as well as the associated cost compared to usual care. Other cost categories, including total healthcare costs and total societal costs, did not statistically differ between groups. Furthermore, no statistically significant differences were found between groups for QALYs and OMAS. This indicates that functional outcome and HR-QOL were unaffected by reducing the number of radiographs performed at 6 and 12 weeks follow-up. The probability of the reduced imaging protocol being cost-effective compared with usual care was relatively low (0.45) at a willingness-to-pay threshold of €20,000 per QALY. In the Netherlands, this is deemed an acceptable cost-per-QALY for interventions for diseases/disorders with a relatively low disease burden [36]. For OMAS, it is currently unknown how much decision-makers are willing to pay per unit of effect gained, so it is not possible to draw any firm conclusions for this outcome. Sensitivity analyses confirm these findings. Literature on the (cost-)effectiveness of omitting routine extremity radiography is scarce. This is discussed in our retrospective review [37] and has been confirmed by researchers investigating the usefulness of an additional shoulder radiograph [38]. Results from this study however were consistent with results from our study which examined the cost-effectiveness of reduced imaging in distal radius fractures [39]. In that study we also saw no difference in functional outcome, but a significant reduction in cost for radiographic imaging in the reduced-imaging group.

Strengths and limitations

This economic evaluation was performed alongside a pragmatic randomised controlled trial. Therefore, our results are likely to have a high internal validity, while their external validity is improved by the pragmatic nature of the trial. Of course, this study has limitations as do all studies. First, the sample size calculation was based upon a margin of non-inferiority [40] for the OMAS, rather than a meaningful difference in societal costs or QALYs. Wide confidence intervals surrounding the aggregate and disaggregate cost differences suggest that the study was underpowered to detect a meaningful difference in cost between groups. This is common in economic evaluations as powering to detect a meaningful difference in societal costs would have required many more participants. This would have been neither feasible nor ethical. Second, the number of radiographs omitted was lower than anticipated. This was due to a high number of protocol violations in the reduced imaging group. The protocol was adhered (i.e. no routine radiograph obtained at both 6 as well as 12 weeks of follow-up) to in just 59 of 118 participants (50%) in this group. We have reported on this in more detail in an earlier report [41]. Third, self-reported questionnaires were used to query the effect, and some costs. These questionnaires had a maximum recall period of 26 weeks, which might have introduced recall bias. However, as the recall period was similar in both groups we assume that if present, this bias was similar for both groups. Fourth, 79% (195/246) of the participants had at least one missing item on at least one of the questionnaires. The number of participants with complete cost and effect data was 242 at baseline (100%), 227 at week 6 (92%), 216 at week 12 (88%), 206 at week 26 (84%), and 201 at week 52 (82%). Multiple imputation was used to deal with missing data. In an economic evaluation, multiple imputation is considered the gold-standard for dealing with missing data [32]. Moreover, a sensitivity analysis using data of complete cases showed similar results as the main analysis, i.e. no significant differences between groups for costs, OMAS and QALYs. Finally, the patients’ EQ-5D-3L health status following the fracture was not assessed, but evaluated prior to the fracture and at the various follow-up measurement points. To deal with this issue, we assumed the patients’ 6-week EQ-5D-3L health state to be representative for the complete period between the occurrence of the fracture and 6-week follow-up and used this value for calculating QALYs. We opted for this strategy, instead of using their pre-injury EQ-5D-3L health state, since most patients would have had a cast, or non-weightbearing mobilisation during these 6 weeks. We do not expect this to have biased our outcomes, since a sensitivity analysis using the patients’ EQ-5D-3L health state before the occurrence of the fracture showed similar results as the main analysis.


Reducing the number of routine follow-up radiographs (on average one per patient) has a relatively low probability of being cost-effective compared with usual care. However, functional outcome, health-related quality of life and societal costs were comparable in both groups whereas imaging costs were lower in the reduced imaging group. In the light of these findings and the potential for further reduction of the number of routine follow-up radiographs in daily clinical ankle fracture care, we advise a reduced imaging follow-up protocol for patients with ankle fracture.

Availability of data and materials

The datasets analysed in the current study can be made available by the corresponding author upon request.



Body Mass Index


Cost-effectiveness acceptability curves


Consolidated Health Economic Evaluation Reporting Standards


Health-Related Quality Of Life


Incremental cost-effectiveness ratio


Inter-Quartile Range


Numerical Rating Scale


Olerud&Molander Ankle Score


Pronation Abduction


Pronation External rotation


Project Manager Internet Server


Quality-Adjusted Life Years


Range Of Motion


Supination Adduction


Supination External rotation


Seemingly unrelated regression analyses

95% CI:

95% confidence intervals


Cost difference


Effect difference


  1. Court-Brown CM, Caesar B. Epidemiology of adult fractures: a review. Injury. 2006;37(8):691–7.

    Article  Google Scholar 

  2. Daly PJ, Fitzgerald RH Jr, Melton LJ, Ilstrup DM. Epidemiology of ankle fractures in Rochester, Minnesota. Acta Orthop Scand. 1987;58(5):539–44.

    Article  CAS  Google Scholar 

  3. Thur CK, Edgren G, Jansson KA, Wretenberg P. Epidemiology of adult ankle fractures in Sweden between 1987 and 2004: a population-based study of 91,410 Swedish inpatients. Acta Orthop. 2012;83(3):276–81.

    Article  Google Scholar 

  4. Koehler S, Eiff P. Overview of ankle fractures in adults UpToDate; 2016. Available from:

    Google Scholar 

  5. Ranabhat CL, Atkinson J, Park MB, Kim CB, Jakovljevic M. The influence of universal health coverage on life expectancy at birth (LEAB) and healthy life expectancy (HALE): a multi-country cross-sectional study. Front Pharmacol. 2018;9:960.

    Article  Google Scholar 

  6. Ghattas TN, Dart BR, Pollock AG, Hinkin S, Pham A, Jones TL. Effect of initial postoperative visit radiographs on treatment plans. J Bone Joint Surg Am. 2013;95 e57(9):1–4.

    Google Scholar 

  7. Jakovljević M, Ranković A, Rančić N, Jovanović M, Ivanović M, Gajović O, et al. Radiology services costs and utilization patterns estimates in southeastern Europe-a retrospective analysis from Serbia. Value Health Reg Issues. 2013;2(2):218–25.

    Article  Google Scholar 

  8. Global burden of disease health financing collaborator network. Future and potential spending on health 2015–40: development assistance for health, and government, prepaid private, and out-of-pocket health spending in 184 countries. Lancet (London, England). 2017;389(10083):2005–30.

    Article  Google Scholar 

  9. Mushlin AI. Challenges and opportunities in economic evaluations of diagnostic tests and procedures. Acad Radiol. 1999;6(Suppl 1):S128–31.

    Article  Google Scholar 

  10. Cassel CK, Guest JA. Choosing wisely: helping physicians and patients make smart decisions about their care. Jama. 2012;307(17):1801–2.

    Article  CAS  Google Scholar 

  11. Otero HJ, Rybicki FJ, Greenberg D, Neumann PJ. Twenty years of cost-effectiveness analysis in medical imaging: are we improving? Radiology. 2008;249(3):917–25.

    Article  Google Scholar 

  12. Powe NR. Economic and cost-effectiveness investigations of radiologic practices. Radiology. 1994;192(1):11–8.

    Article  CAS  Google Scholar 

  13. Schipper IB, Termaat MF, Rhemrev S, Meylaerts SAG, Bartlema K, Stichter W, et al. Richtlijnen voor behandeling van letsels van het steun en bewegingsapparaat (clinical guidelines for the treatment of trauma to the musculoskeletal system). Rotterdam: Optima grafische communicatie; 2016.

    Google Scholar 

  14. Chaudhry S, DelSole EM, Egol KA. Post-splinting radiographs of minimally displaced fractures: good medicine or medicolegal protection? J Bone Joint Surg Am. 2012;94(17):e128(1)–5).

    Article  Google Scholar 

  15. (CBS) SN. Incidence of wrist fractures in the Netherlands: statistics Netherlands (CBS); 2014. Available from:,149&D6=4&VW=T.

    Google Scholar 

  16. van Gerven P, Weil NL, Termaat MF, Rubinstein SM, El Moumni M, Zuidema WP, et al. Routine follow-up radiographs for ankle fractures seldom add value to clinical decision-making: a retrospective, observational study. J Foot Ankle Surg. 2018;57(5):957–60.

    Article  Google Scholar 

  17. NZa. Tarievenlijst eerstelijnsdiagnostiek: Nederlandse Zorgautoriteit; 2015. Available from:

    Google Scholar 

  18. Ovaska MT, Nuutinen T, Madanat R, Makinen TJ, Soderlund T. The role of outpatient visit after operative treatment of ankle fractures. Injury. 2016;47(11):2575–8.

    Article  Google Scholar 

  19. McDonald MR, Bulka CM, Thakore RV, Obremskey WT, Ehrenfeld JM, Jahangir AA, et al. Ankle radiographs in the early postoperative period: do they matter? J Orthop Trauma. 2014;28(9):538–41.

    Article  Google Scholar 

  20. Weil NL, Termaat MF, Rubinstein SM, El Moumni M, Zuidema WP, Derksen RJ, et al. WARRIOR-trial - is routine radiography following the 2-week initial follow-up in trauma patients with wrist and ankle fractures necessary: study protocol for a randomized controlled trial. Trials. 2015;16:66.

    Article  Google Scholar 

  21. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS)--explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. 2013;16(2):231–50.

    Article  Google Scholar 

  22. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical research ed). 2010;340:c332.

    Article  Google Scholar 

  23. Lauge-Hansen N. Fractures of the ankle. II. Combined experimental-surgical and experimental-roentgenologic investigations. Arch Surg. 1950;60(5):957–85.

    Article  CAS  Google Scholar 

  24. Weber BG. Die Verletzungen des oberen Sprunggelenkes. Bern: Huber Verlag; 1966.

    Google Scholar 

  25. Calori GM, Albisetti W, Agus A, Iori S, Tagliabue L. Risk factors contributing to fracture non-unions. Injury. 2007;38(Suppl 2):S11–8.

    Article  Google Scholar 

  26. Olerud C, Molander H. A scoring scale for symptom evaluation after ankle fracture. Arch Orthop Trauma Surg. 1984;103(3):190–4.

    Article  CAS  Google Scholar 

  27. EuroQol Group. EuroQol--a new facility for the measurement of health-related quality of life. Amsterdam: Health Policy; 1990;16(3):199–208.

  28. Lamers LM, Stalmeier PF, McDonnell J, Krabbe PF, van Busschbach JJ. Measuring the quality of life in economic evaluations: the Dutch EQ-5D tariff. Ned Tijdschr Geneeskd. 2005;149(28):1574–8.

    CAS  PubMed  Google Scholar 

  29. Tan SS, Bouwmans CAM, Rutten FFH, Hakkaart-van RL. Update of the Dutch manual for costing in economic evaluations. Int J Technol Assess Health Care. 2012;28(2):152–8.

    Article  Google Scholar 

  30. Z-index. G-Standaard; 2016.

    Google Scholar 

  31. Statstics-Netherlands CBS. Consumer price indices; 2017.

    Google Scholar 

  32. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99.

    Article  Google Scholar 

  33. Willan AR, Briggs AH, Hoch JS. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data. Health Econ. 2004;13(5):461–75.

    Article  Google Scholar 

  34. Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ. 2005;14(5):487–96.

    Article  Google Scholar 

  35. Michael F, Drummond MJS. Karl Claxton, Greg L. Stoddart, and George W. Torrance. Methods for the economic evaluation of health care programmes. New York: Oxford University Press; 2015.

    Google Scholar 

  36. Smulders YM, Thijs A. The cost per year of life gained: trends and internal contradictions. Ned Tijdschr Geneeskd. 2006;150(45):2467–70.

    CAS  PubMed  Google Scholar 

  37. van Gerven P, Rubinstein SM, Nederpelt C, Termaat MF, Krijnen P, van Tulder MW, et al. The value of radiography in the follow-up of extremity fractures: a systematic review. Arch Orthop Trauma Surg. 2018;138(12):1659–69.

    Article  Google Scholar 

  38. Cruz SA, Castillo H, Chintapalli RTV, Adams OE, Morgan VK, Koh JL, et al. The clinical utility of additional axillary and Velpeau radiographs in the evaluation of suspected shoulder trauma. J Orthop Trauma. 2020;34(8):e261–e265.

  39. van Gerven P, van Dongen JM, S.M. R, Termaat MF, El Moumni M, Zuidema WP, et al. Is reduction of routine radiograph use in patients with distal radius fractures cost-effective? Analysis of data from the multicentre, randomised controlled WARRIOR trial. Submitted to: BMJ Open. 2020;10(7):e035370.

  40. Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. J Bone Joint Surg Am. 2020.

  41. van Gerven P, Krijnen P, Zuidema WP, El Moumni M, van Tulder MW, et al. Routine radiography following the initial 2-week follow-up of ankle fracture patients does not have added value. The Warrior-TRIAL: a multicenter randomized controlled trial. J Bone Joint Surg. 2020.

Download references


This study was performed by the Warrior Study Group. It includes the authors and (in alphabetical order):

L. van Bodegom-Vos PhD, (Leiden University Medical Center [LUMC] Leiden, the Netherlands);

R.S. Breederveld MD, PhD (LUMC);

R.J. Derksen MD, PhD (Zaans Medisch Centrum, Zaandam, the Netherlands);

B. van Dijkman MD (Flevoziekenhuis, Almere, the Netherlands);

J.C. Goslings MD, PhD (Amsterdam University Medical Center [Amsterdam UMC], Amsterdam the Netherlands);

J.H. Hegeman MD, PhD (Ziekenhuis Groep Twente, Almelo, the Netherlands);

J.M. Hoogendoorn MD, PhD (Haaglanden Medical Centre [HMC], Den Haag, the Netherlands);

C. van Kuijk MD, PhD (Amsterdam UMC);

S.A.G. Meylaerts MD, PhD (HMC);

F.R. Rosendaal MD, PhD (LUMC);

N.L. Weil MD (LUMC);

K.W. Wendt MD, PhD (University of Groningen, University Medical Centre Groningen, Groningen, the Netherlands).


This work was supported by ZonMw, The Netherlands Organization for Health Research and Development, grant number 837002403. The funding party required us to use a patient reported outcome measure (PROM) as the primary outcome. Apart from this requirement ZonMw had no involvement in trial design, data collection, interpretation and analysis or the writing of the manuscript.

Author information

Authors and Affiliations




PvG: Acquisition of data, analysis of data, interpretation of data, drafting of the primary manuscript and revising the manuscript based on co-author’s comments. JMvD: Analysis and interpretation of data, revising the manuscript. SMR: Conceptualization and design of the study, revising the manuscript. MFT: Conceptualization and design of the study, revising the manuscript. MEM: Conceptualization and design of the study, revising the manuscript. WPZ: Conceptualization and design of the study, revising the manuscript. PK: Conceptualization and design of the study, interpretation of data and revising the manuscript. IBS: Conceptualization and design of the study, revising the manuscript. MWvT Conceptualization and design of the study, interpretation of data and revising the manuscript. All authors read and approved the final manuscript.

Contributions of the study group:

LvB-V: Conceptualization and design of the study.

RSB: Conceptualization and design of the study.

RJD: Conceptualization and design of the study, Acquisition of data.

BvD: Conceptualization and design of the study, Acquisition of data.

JCG: Conceptualization and design of the study.

JHH: Conceptualization and design of the study, Acquisition of data.

JMH: Conceptualization and design of the study, Acquisition of data.

CvK: Conceptualization and design of the study.

SAGM: Conceptualization and design of the study, Acquisition of data.

FRR: Conceptualization and design of the study.

NLW: Conceptualization and design of the study, Acquisition of data.

KWW: Conceptualization and design of the study.

Corresponding author

Correspondence to P. van Gerven.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Medical Ethics Committee of the Leiden University Medical Centre (Protocol number P14.086). All participants provided written informed consent before randomisation.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Outcome scores per treatment allocation per timepoint.

Additional file 2.

Mean cost (in euros) per conservatively treated participant in the intervention and control group and mean cost differences between groups during follow-up.

Additional file 3.

Mean cost (in euros) per operatively treated participant in the intervention and control group and mean cost differences between groups during follow-up.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van Gerven, P., van Dongen, J.M., Rubinstein, S.M. et al. Reduction of routine use of radiography in patients with ankle fractures leads to lower costs and has no impact on clinical outcome: an economic evaluation. BMC Health Serv Res 20, 893 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: