Skip to main content
Download PDF

Interventions to reduce low-value imaging – a systematic review of interventions and outcomes

BMC Health Services Research volume 21, Article number: 983 (2021) Cite this article

Abstract

Background

It is estimated that 20–50% of all radiological examinations are of low value. Many attempts have been made to reduce the use of low-value imaging. However, the comparative effectiveness of interventions to reduce low-value imaging is unclear. Thus, the objective of this systematic review was to provide an overview and evaluate the outcomes of interventions aimed at reducing low-value imaging.

Methods

An electronic database search was completed in Medline – Ovid, Embase-Ovid, Scopus, and Cochrane Library for citations between 2010 and 2020. The search was built from medical subject headings for Diagnostic imaging/Radiology, Health service misuse or medical overuse, and Health planning. Keywords were used for the concept of reduction and avoidance. Reference lists of included articles were also hand-searched for relevant citations. Only articles written in English, German, Danish, Norwegian, Dutch, and Swedish were included. The Mixed Methods Appraisal Tool was used to appraise the quality of the included articles. A narrative synthesis of the final included articles was completed.

Results

The search identified 15,659 records. After abstract and full-text screening, 95 studies of varying quality were included in the final analysis, containing 45 studies found through hand-searching techniques. Both controlled and uncontrolled before-and-after studies, time series, chart reviews, and cohort studies were included. Most interventions were aimed at referring physicians. Clinical practice guidelines (n = 28) and education (n = 28) were most commonly evaluated interventions, either alone or in combination with other components. Multi-component interventions were often more effective than single-component interventions showing a reduction in the use of low-value imaging in 94 and 74% of the studies, respectively. The most addressed types of imaging were musculoskeletal (n = 26), neurological (n = 23) and vascular (n = 16) imaging. Seventy-seven studies reported reduced low-value imaging, while 3 studies reported an increase.

Conclusions

Multi-component interventions that include education were often more effective than single-component interventions. The contextual and cultural factors in the health care systems seem to be vital for successful reduction of low-value imaging. Further research should focus on assessing the impact of the context in interventions reducing low-value imaging and how interventions can be adapted to different contexts.

Peer Review reports

Background

The rapidly expanding use of health services is challenging and health care expenditures are mounting [1]. This has underscored the need for more efficient use of finite healthcare resources. However, according to the Organization for Economic Co-operation and Development (OECD), approximately 10–34% of health service spending is potentially inappropriate, representing ineffective and wasteful use of health care resources [2]. Such services are referred to as low-value care, which is defined “an intervention in which evidence suggest it confers not or very little benefit for patients, or risk of harm exceeds probable benefit or, more broadly, the added costs of the intervention do not provide proportional added benefits” [3].

While diagnostic imaging provides crucial information for the diagnostics of patients [4], inappropriate or low-value imaging are estimated to constitute 20–50% of radiological examinations internationally [2, 5,6,7,8]. Several interventions to reduce low-value imaging have been evaluated in the literature, including guidelines such as iRefer, iGuide, as well as national and international initiatives such as the National Institute for Health and Care Excellence (NICE) “Do-not-do list,” and the Choosing Wisely campaign [9,10,11,12]. However, the effect of such efforts on low-value diagnostic imaging has been limited due to barriers such as financial incentives, practice behavior, self-referral, lack of feedback, patient expectations, and duplicate imaging examinations [5, 11, 13,14,15,16,17]. Some interventions even seem to increase the use of inappropriate imaging [18, 19].

Several approaches to address the use of inappropriate health services, beyond low-value imaging, have been extensively evaluated. Education or training programs for health care personnel, clinical decision support, feedback, patient education, shared decision making, and economic incentives are but a few examples [5, 20,21,22,23,24]. However, the great quantity and variability of available approaches makes it unclear which measures are most suitable to target low-value imaging and overutilization. While research on interventions to reduce low-value care, in general, recommend implementation of multi-component interventions in complex health care systems [12, 25,26,27,28] there is still uncertainty as to why or when an intervention will be effective in diagnostic imaging specifically and/or in which clinical circumstances they are effective. Earlier systematic reviews on interventions in imaging have addressed specific interventions as image sharing or clinical decision support systems or specific imaging examinations or patient complaints [26, 29,30,31,32]. However, there is no encompassing systematic review assessing the outcome of various types of interventions to reduce low-value imaging. Thus, the objective of this systematic review was to provide an overview and evaluate the outcomes of interventions aimed at reducing low-value imaging.

Methods

This systematic review was conducted based on the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement (PROSPERO ID: CRD42020208072). The electronic database search was developed in Medline – Ovid (Table 1) and further adapted to Embase-Ovid, Scopus, and Cochrane Library. The terms used were built from medical subject headings (MESH) for Diagnostic imaging/Radiology, Health service misuse/Medical overuse, and Health planning. Keywords were used for the concept of reduction/avoiding. Also, the search was broadened with text word and keyword synonyms. The complete search strategy is available in Additional file 1. Searches were carried out in September and October 2020; last search made on 13th October 2020. Papers written in English, German, Danish, Norwegian, Dutch, and Swedish were eligible and language filters were used to exclude other languages. Keywords were used to exclude studies on animals, mass screening, and unnecessary care besides imaging services. No other limitations were applied.

Table 1 Search strategy developed in Medline (Ovid)
Full size table

Eligibility criteria

Primary empirical studies assessing interventions to reduce the use of low-value diagnostic imaging examinations were included. Studies designed as randomized controlled trials, non-randomized trials, descriptive studies, mixed-methods studies, and qualitative studies were included. While systematic reviews and meta-analyses were not included, the reference lists of relevant systematic reviews and meta-analyses were hand-searched for additional primary studies for inclusion. Studies published before 2010 were excluded due to the changes in perception on low-value imaging through the preparation and introduction of the Choosing Wisely campaign in 2012. The inclusion and exclusion criteria are provided in Table 2.

Table 2 Inclusion and exclusion criteria for assessing record eligibility
Full size table

Selection of records and methodological quality appraisal

The records were archived using Thomson Reuters EndNote X9.3.3 library and duplicates were removed. All remaining records were transferred to Rayyan QCRI [33] where titles and abstract review (EK and BMH) and full-text review and quality assessment (EK, ERA, LJJS, LvB-V and BMH) were completed by two teams of reviewers. Each study was quality assessed by one reviewer and double checked by EK for consistency. The Mixed Methods Appraisal Tool (MMAT) was used to assess the methodical quality of all included studies as it is considered to be an appropriate tool for appraisal of interventional studies of different methodologies [34]. Any disagreements during abstract or full-text screening were resolved through discussion and consensus. Reference lists of included articles were also hand-searched for relevant articles for inclusion. A grey literature search was also completed (ERA) according to the CADTH Grey Matters checklist [35]. Google Scholar was used for searching for eligible papers that cited the included studies.

Data extraction and synthesis

Data extraction was completed independently by EK, LJJS, LvB-V, BMH and ERA using a standardized summary table consisting of the following categories: author, title and year, country, design, population, clinical setting, outcome measures, low-value practice, intervention, targeted personnel or roles, control or comparator, use of low-value practice before or after intervention, and change in use of low-value practice. Data extraction was discussed in the research team for quality assurance purposes.

The findings from included studies were narratively synthesized. This synthesis was performed due to the variety of study designs among included studies and thus a meta-analysis was not feasible [36]. The narrative synthesis included familiarization, the development of a preliminary synthesis by organizing findings in tables. Then, relationships, patterns, and connections in the data were explored [36]. In addition, a subgroup analysis was done for interventions done in the USA separately.

Results

Search of the literature

As shown in Fig. 1, the electronic database searches resulted in 15,659 records. After the removal of 7468 duplicates, 8191 unique records were screened through title and abstract screening and 8108 records were excluded. An additional 103 records were identified through snowballing techniques and from the grey literature. A total of 186 articles were reviewed in full-text and 91 articles were excluded. Thus, 95 studies were included in the narrative synthesis. An overview of excluded studies with reason for exclusion is provided in Additional file 2.

Fig. 1
figure 1

Flow diagram for record selection

Full size image

Quality of included studies

The 95 included studies [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131] are summarized in Table 3. All included papers fulfill the screening questions in MMAT. Forty-six papers were given a full score in the MMAT appraisal. These are marked with a * in Table 3 (the full MMAT report is available in Additional file 3), while the others had one criterion unfulfilled, or lacked sufficient information in the report.

Table 3 Characteristics of the included studies, outcome of the intervention and quality assessment result
Full size table

Characteristics of included studies

A majority of the included studies applied quantitative study designs. Retrospective chart reviews (n = 26) and uncontrolled before-after studies (n = 14) were the most common. Seventy-eight of the studies were conducted in the USA (82%). The setting of the studies included hospital (n = 40), emergency department (n = 24), or outpatient medical center (n = 18). Musculoskeletal (n = 26), neurological (n = 23) and vascular (n = 16) imaging were most commonly evaluated. The most targeted imaging examinations were chest CTA (n = 15) and head CT (n = 12). Further, pulmonary embolism (n = 15), lower back pain (n = 14), and minor head injury (n = 12) were the most commonly explored medical conditions.

Interventions

Guidelines (n = 28) and education (n = 28), either alone or in combination with other measures, were the most common interventions evaluated to reduce low-value imaging. The outcome measures reported in the included studies varied, with the number or rate of imaging examinations (n = 75) most frequently reported primary outcomes. A majority of studies (n = 61) used a single component intervention and most studies (n = 90) targeted referring physicians. An overview of participants exposed to the intervention (referring physicians, imaging staff, patients and/or family members), types of interventions, and combinations of components in multi-component interventions are presented in Fig. 2.

Fig. 2
figure 2

Overview of participants, interventions, and combinations in multi-component interventions in the included studies. Figure legend: To the left an overview of single interventions used for each participant group. To the right an illustration for how multi-component interventions were combined. Overlapping circles illustrate different combinations of two of more components

Full size image

A variety of imaging modalities or patient diagnoses were targeted, and the primary outcomes varied among studies that reported improvements post-intervention. Among studies targeting several imaging modalities or diagnoses, 74–79% of the studies showed a reduction in use of low-value imaging. In contrast, studies targeting one specific modality only showed that targeting X-ray [37, 47, 48, 63, 64, 83, 86, 90, 92, 98, 104, 112, 119, 124, 130], CT [41, 46, 49, 53, 54, 58, 60, 62, 67, 69, 70, 77, 79, 82, 87, 88, 94,95,96, 101, 102, 106, 108, 121,122,123, 125,126,127, 129] or MRI [45, 68, 75, 105, 118] led to a 87, 86, and 83% reduction in low-value imaging, respectively. Few studies included other imaging modalities.

The most commonly targeted patient diagnosis was bronchiolitis [48, 90, 104, 112, 119], pulmonary embolism [49, 58, 60, 82, 100, 102, 121, 122, 125], and head injuries [67, 70, 77, 79, 87, 94, 101, 123, 126, 127]. In studies targeting these complaints, a reduction in use of low-value imaging were reported in 78–80% of the studies, while imaging in lower back pain [38, 45, 51, 53, 54, 65, 68, 71, 75, 89, 115, 118] were reduced in 58% of studies.

Among the 77 studies that reported improvements following the intervention [37,38,39,40,41,42,43,44,45, 47,48,49, 52, 53, 56, 57, 59, 60, 62,63,64, 67, 69, 71,72,73,74,75,76,77,78,79,80,81, 83, 85,86,87,88,89,90, 92,93,94,95,96,97,98,99,100,101,102,103,104,105,106, 108,109,110,111,112,113,114,115,116,117, 120,121,122,123, 125, 128,129,130,131], decreases in low-value imaging varied largely from < 1 to 62%. Of the remaining studies, three studies reported mixed results, where only some of the targeted low-value imaging examinations were reduced [46, 54, 68], and 16 studies showed a non-significant change or increase in the use of low-value imaging post-intervention [50, 51, 55, 58, 61, 65, 66, 70, 82, 84, 91, 107, 118, 119, 124, 127].

Implementation of multi-component interventions (2 or more components in combination) reportedly reduced the use of low-value imaging among 94% of the included studies [40,41,42,43,44, 47, 48, 51, 53, 56, 59, 63, 72, 78, 79, 81, 83, 86, 87, 94, 101, 104, 105, 111, 115, 116, 118, 120, 122, 123, 126, 128]. Multi-component interventions were found to be more effective when education was one of the components. Following implementation of a single component intervention, 74% of included studies reported decreases in low-value imaging [37,38,39, 45, 46, 49, 50, 52, 54, 55, 57, 58, 60,61,62, 64,65,66,67,68,69,70,71, 74,75,76,77, 80, 82, 84, 85, 88,89,90,91,92,93, 95,96,97,98,99,100, 102, 103, 106,107,108,109,110, 113, 114, 117, 119, 121, 124, 125, 127, 129,130,131]. Data analyses based on the USA studies demonstrated similar results as 96% of multi-component and 68% of single-component interventions showed reduction in the use of low-value imaging. Thus, county of intervention does not affect the result alone. Implementation of guidelines or clinical decision support systems were the most effective single-component interventions [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121, 126, 128, 130]. Furthermore, 23% of single-component interventions compared to 6% of multi-component interventions showed no statistically significant difference or an increase in the use of low-value imaging. In Fig. 3, the green bars represent studies with a reduction in the use of low-value imaging, red bars represent no significant change or increase, and orange bars represent mixed results.

Fig. 3
figure 3

The number of studies and outcome of different types of interventions to reduce low-value imaging

Full size image

Single-component interventions such as education, shared-decision-making, and financial measures alone often had no effect on use of low-value imaging [55, 61, 70, 82, 107]. Studies with more than 30% reduction in use of low-value imaging were both single-component (n = 11) and multi-component (n = 8) interventions [37,38,39, 41, 42, 44, 69, 73, 75, 80, 81, 83, 85, 86, 96, 108, 116, 117, 131]. All these studies targeted refereeing physicians while one also targeted imaging staff. Another targeted referrer, imaging staff, and patients. Of these studies, 16% were from countries other than the USA. Table 4 provides an overview of the type of interventions that resulted in more than a 30% reduction in low-value imaging.

Table 4 Overview of interventions with more than a 30% reduction in use of low-value imaging examinations
Full size table

Discussion

A large body of literature evaluating the outcome of interventions aimed at reducing low-value imaging was identified through this systematic review. Broadly, most interventions were found to be effective, with multi-component interventions more frequently reported to be effective compared to single-component interventions. All studies evaluating multi-component interventions with an education component reported reductions in low-value imaging. Multi-component interventions targeting the participants on several points providing education and then feedback and reminders over a longer period seems to be effective as change takes knowledge, motivation, and time [132]. Single-component interventions, particularly guideline implementation, clinical decision support systems, feedback, or actions required from the referrers, showed reduction in use of low-value imaging in several studies but not in all. This might be caused by organizational differences, differences in the clinical setting, or participants motivation [132]. Shared decision-making, new referring procedures, and financial measures demonstrated no effect; however, these interventions were only evaluated in a limited number of studies. Targeting specific examinations for specific conditions (e.g., bronchiolitis), targeting referrers, and only targeting one imaging modality seemed to be more effective than targeting several modalities or examinations referred from a variety of referrer groups (e.g., lower back pain). There was also a variety of outcome measure used among included studies. The number or rate of low-value imaging was the most common. Others included appropriateness and diagnostic yield. This warrants caution when comparing outcomes between different types of interventions.

The present results are in line with previous systematic reviews on interventions to reduce low-value services in general [12, 26, 29,30,31,32] and with a previous scoping review on unnecessary imaging, diagnostic tests, and procedures in hospitals [28]. The results indicate great variation in outcomes for many interventions. This is in accordance with research on innovation and interventions suggesting that the formal and informal network in the organization, motivation, flexibility, and fitness to the internal culture and core values in the organization where interventions are implemented, were key factors for a successful and long-lasting change of clinical practice [132,133,134,135]. Whether or not an intervention is successful in reducing the use of low-value imaging would thus depend on a variety of factors. Comparing studies conducted in the USA to those from other countries showed no difference in type of interventions used or in the rate of studies demonstrating > 30% reduction in the use of low-value imaging. Thus, the effect of intervention seems to be dependent on local culture and health care organizations rather than the national health system alone. In addition, only a few interventions were directed against patients, which is somewhat surprising as patients are also identified as drivers in the use of low-value imaging [136, 137]. Further research should include the patient perspective and the role of the radiology department in interventions to reduce low-value imaging in addition to a review on cost-effectiveness of interventions to reduce low-value imaging. Further investigation should focus on how interventions can be adapted to the culture and core values of the providers of health services in different contexts.

Our study has several limitations. Publication bias may have been introduced as articles with negative or nonsignificant findings are less likely to be published. Among the included studies, few reported null or low effect. Most studies had an uncontrolled before-after design not considering that there may be a secular downtrend in the use of the low-value imaging examinations due to the attention in campaigns, such as Choosing Wisely. Thus, the outcome may be overestimated. Further, the review may be subject to contextual bias and have limited generalizability, as most of the studies were conducted in the USA. Accordingly, caution is warranted when inferring from and applying the results in different settings. The proportion of single-center studies and observational studies may enhance the overall positive effect of the interventions [26]. Yet another limitation is related to indirect outcome measures, as many publications focus on interventions’ impact on volume and not on value. This is understandable as the change of low-value utilization is a warranted measure, but one should notice that the value of these services is not assessed. Moreover, it may be argued that the spectrum of imaging that is targeted by the interventions is biased by the methods to assess intervention outcomes. Nonetheless, we report a wide variety of interventions targeting many examinations. There are reasons to believe that the interventions are targeted strategically. For example, interventions that are believed to be effective may be targeted towards examinations documented to be of low value.

It may also be argued that retrospective chart reviews are not proper intervention studies, but as they are used systematically to assess change in practice, we have included them in this review. Additionally, the included studies afford providers’ perspective and as indicated, few studies elicited patient preferences or included patient-reported outcome measures. In addition, the snowballing uncovered a few studies published in 2021 not included in our analysis due to the inclusion criteria. The results of these studies are in line with the studies included in the analysis and thus not including these did not reduce the strengths of the analysis in this review [138,139,140,141,142].

Conclusions

This systematic review demonstrates that interventions to reduce low-value imaging can be very effective, but that there is a large variation in types of interventions and their outcomes. We found that multi-component interventions reported reduction in low-value imaging or increased diagnostic yield more frequently compared to single-component interventions. The context in which the intervention is introduced seems to be of vital importance for successful reduction of low-value imaging. Thus, in the future multi-component interventions that are adapted to the local context are more likely to be successful. Further research is needed to assess how interventions to reduce low-value imaging can best be adapted to specific contexts and how to reduce the use of low-value imaging cost-effectively.

Availability of data and materials

Not applicable.

Abbreviations

CT:

Computed tomography

CTA:

Computed tomography angiography

DEXA:

Dual-energy X-ray absorptiometry

MMAT:

Mixed Methods Appraisal Tool

MRI:

Magnetic resonance imaging

MUCG:

Micturating cystourethrogram

NM:

Nuclear medicine

PET:

Positron emission tomography

US:

Ultrasound

XR:

X-ray

References

  1. Expert Panel on effective ways of investing in Health (EXPH). Defining value in “value-based healthcare. 2019.

  2. Socha K, Couffinhal A, Forde I, Nader C, Cecchini M, Lee S, et al. Tackling Wasteful Spending on Health.OECD 20172017.

  3. Scott IA, Duckett SJ. In search of professional consensus in defining and reducing low-value care. Med J Aust. 2015;203(4):179–81. https://doi.org/10.5694/mja14.01664.

    Article  PubMed  Google Scholar 

  4. Brady A, Brink J, Slavotinek J. Radiology and value-based health care. Jama. 2020;324(13):1286–7.

    Article  Google Scholar 

  5. Hendee WR, Becker GJ, Borgstede JP, Bosma J, Casarella WJ, Erickson BA, et al. Addressing overutilization in medical imaging. Radiology. 2010;257(1):240–5.

    Article  Google Scholar 

  6. Sheng AY, Castro A, Lewiss RE. Awareness, utilization, and education of the ACR appropriateness criteria: a review and future directions. J Am Coll Radiol. 2016;13(2):131–6. https://doi.org/10.1016/j.jacr.2015.08.026.

    Article  PubMed  Google Scholar 

  7. Ingraham B, Miller K, Iaia A, Sneider MB, Naqvi S, Evans K, et al. Reductions in high-end imaging utilization with radiology review and consultation. J Am Coll Radiol. 2016;13(9):1079–82. https://doi.org/10.1016/j.jacr.2016.04.016.

    Article  PubMed  Google Scholar 

  8. Soltana K, Moore L, Bouderba S, Lauzier F, Clément J, Mercier É, et al. Adherence to clinical practice guideline recommendations on low-value injury care: a multicenter retrospective cohort study. Value Health. 2021. https://doi.org/10.1016/j.jval.2021.06.008.

  9. Choosing Wisely [Internet]. 2021. Available from: https://www.choosingwisely.org/getting-started/lists/.

  10. Improving health and social care through evidence-based guidance [Internet]. 2021. Available from: https://www.nice.org.uk/.

  11. Ryan JW, Hollywood A, Stirling A, Glynn M, MacMahon PJ, Bolster F. Evidenced-based radiology? A single-institution review of imaging referral appropriateness including monetary and dose estimates for inappropriate scans. Ir J Med Sci. 2019;188(4):1385–9.

    Article  Google Scholar 

  12. Cliff BQ, Avanceña ALV, Hirth RA, Lee SD. The impact of choosing wisely interventions on low-value medical services: a systematic review. Milbank Q. 2021.

  13. Barth JH, Misra S, Aakre KM, Langlois MR, Watine J, Twomey PJ, et al. Why are clinical practice guidelines not followed? Clin Chem Lab Med. 2016;54(7):1133–9. https://doi.org/10.1515/cclm-2015-0871.

    Article  PubMed  CAS  Google Scholar 

  14. DeAngelis J, Lou V, Li T, Tran H, Bremjit P, McCann M, et al. Head CT for minor head injury presenting to the emergency Department in the era of choosing wisely. West J Emerg Med. 2017;18(5):821–9. https://doi.org/10.5811/westjem.2017.6.33685.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Rosenberg A, Agiro A, Gottlieb M, Barron J, Brady P, Liu Y, et al. Early trends among seven recommendations from the choosing wisely campaign. JAMA Intern Med. 2015;175(12):1913–20.

    Article  Google Scholar 

  16. Anderson TS, Leonard S, Zhang AJ, Madden E, Mowery D, Chapman WW, et al. Trends in Low-Value Carotid Imaging in the Veterans Health Administration From 2007 to 2016. JAMA Network Open. 2020;3(9):e2015250-e.

  17. Berezin L, Thompson C, Rojas-Luengas V, Borgundvaag B, McLeod SL. Lumbosacral spinal imaging for patients presenting to the emergency department with nontraumatic low Back pain. The Journal of Emergency Medicine. 2020;58(2):269–74. https://doi.org/10.1016/j.jemermed.2019.12.017.

    Article  PubMed  Google Scholar 

  18. Pons E, Foks KA, Dippel DWJ, Hunink MGM. Impact of guidelines for the management of minor head injury on the utilization and diagnostic yield of CT over two decades, using natural language processing in a large dataset. Eur Radiol. 2019;29(5):2632–40. https://doi.org/10.1007/s00330-018-5954-5.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Carpenter CP, Johnston D, Tourville E, Sharadin C, Alzubaidi AN, Giel DW. Inappropriate imaging for management of cryptorchidism: Has the choosing Wisely® recommendation reduced occurrence? J Pediatr Urol. 2020;16(4):462.e1-.e6.

  20. Brink JA. Clinical decision-making tools for exam selection, reporting and dose tracking. Pediatr Radiol. 2014;44(Suppl 3):418–21. https://doi.org/10.1007/s00247-014-3015-z.

    Article  PubMed  Google Scholar 

  21. Armao D, Semelka RC, Elias J Jr. Radiology's ethical responsibility for healthcare reform: tempering the overutilization of medical imaging and trimming down a heavyweight. Journal of magnetic resonance imaging : JMRI. 2012;35(3):512–7. https://doi.org/10.1002/jmri.23530.

    Article  PubMed  Google Scholar 

  22. Allen J, King R, Goergen SK, Melder A, Neeman N, Hadley A, et al. Semistructured interviews regarding patients' perceptions of choosing wisely and shared decision-making: an Australian study. BMJ Open. 2019;9(8):e031831. https://doi.org/10.1136/bmjopen-2019-031831.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Iyengar R, Winkels JL, Smith CM, Meka AP, Porath JD, Meurer WJ. The effect of financial incentives on patient decisions to undergo low-value head computed tomography scans. Acad Emerg Med. 2019;26(10):1117–24. https://doi.org/10.1111/acem.13823.

    Article  PubMed  Google Scholar 

  24. Tejedor-Sojo J, Chan KN, Bailey M, Williams A, Killgore M, Gillard L, et al. Improving bronchiolitis Care in Outpatient Settings across a health care system. Pediatr Emerg Care. 2019;35(11):791–8. https://doi.org/10.1097/PEC.0000000000001966.

    Article  PubMed  Google Scholar 

  25. Colla CH. Swimming against the current--what might work to reduce low-value care? N Engl J Med. 2014;371(14):1280–3. https://doi.org/10.1056/NEJMp1404503.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  26. Colla CH, Mainor AJ, Hargreaves C, Sequist T, Morden N. Interventions aimed at reducing use of low-value health services: a systematic review. Med Care Res Rev. 2017;74(5):507–50. https://doi.org/10.1177/1077558716656970.

    Article  PubMed  Google Scholar 

  27. Ebdon-Jackson S, Frija G, European Society of R. Improving justification of medical exposures using ionising radiation: considerations and approaches from the European Society of Radiology. Insights into Imaging. 2021;12(1):2.

    Article  Google Scholar 

  28. Garrubba M, Melder A. Interventions to reduce unnecessary imaging, tests and procedures in hospitals. Centre for Clinical Effectiveness, Monash Health. 2016

  29. Vest JR, Jung HY, Ostrovsky A, Das LT, McGinty GB. Image Sharing Technologies and Reduction of Imaging Utilization: A Systematic Review and Meta-analysis. J Am Coll Radiol. 2015;12(12 Pt B):1371–9.e3.

  30. Muhiyaddin R, Abd-Alrazaq AA, Househ M, Alam T, Shah Z. The impact of clinical decision support systems (CDSS) on physicians: a scoping review. Stud Health Technol Inform. 2020;272:470–3. https://doi.org/10.3233/SHTI200597.

    Article  PubMed  Google Scholar 

  31. Suman A, Armijo-Olivo S, Deshpande S, Marietta-Vasquez J, Dennett L, Miciak M, et al. A systematic review of the effectiveness of mass media campaigns for the management of low back pain. Disabil Rehabil. 2020:1–29. https://doi.org/10.1080/09638288.2020.1743777.

  32. Desai S, Liu C, Kirkland SW, Krebs LD, Keto-Lambert D, Rowe BH. Effectiveness of implementing evidence-based interventions to reduce C-spine image ordering in the emergency department: a systematic review. Acad Emerg Med. 2018;25(6):672–83. https://doi.org/10.1111/acem.13364.

    Article  PubMed  Google Scholar 

  33. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Systematic Reviews. 2016;5(1):210. https://doi.org/10.1186/s13643-016-0384-4.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. Journal of Clinical Epidemiology. 2019;111:49–59.e1.

  35. CADTH. Grey Matters: a practical tool for searching health-related grey literature 2019 [Available from: https://www.cadth.ca/resources/finding-evidence/grey-matters.

  36. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, et al. Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version. 2006;1:b92.

    Google Scholar 

  37. Ashikyan O, Buller DC, Pezeshk P, McCrum C, Chhabra A. Reduction of unnecessary repeat knee radiographs during osteoarthrosis follow-up visits in a large teaching medical center. Skelet Radiol. 2019;48(12):1975–80. https://doi.org/10.1007/s00256-019-03247-4.

    Article  Google Scholar 

  38. Bailey JE, Pope RA, Elliott EC, Wan JY, Waters TM, Frisse ME. Health information exchange reduces repeated diagnostic imaging for back pain. Ann Emerg Med. 2013;62(1):16–24.

    Article  Google Scholar 

  39. Bailey JE, Wan JY, Mabry LM, Landy SH, Pope RA, Waters TM, et al. Does health information exchange reduce unnecessary neuroimaging and improve quality of headache care in the emergency department? J Gen Intern Med. 2013;28(2):176–83. https://doi.org/10.1007/s11606-012-2092-7.

    Article  PubMed  Google Scholar 

  40. Bairstow PJ, Persaud J, Mendelson R, Nguyen L. Reducing inappropriate diagnostic practice through education and decision support. Int J Qual Health Care. 2010;22(3):194–200. https://doi.org/10.1093/intqhc/mzq016.

    Article  PubMed  Google Scholar 

  41. Baker M, Jaeger C, Hafley C, Waymack J. Appropriate CT cervical spine utilisation in the emergency department. BMJ Open Qual. 2020;9(4).

  42. Bhatia RS, Dudzinski DM, Malhotra R, Milford CE, Yoerger Sanborn DM, Picard MH, et al. Educational intervention to reduce outpatient inappropriate echocardiograms: a randomized control trial. JACC Cardiovasc Imaging. 2014;7(9):857–66. https://doi.org/10.1016/j.jcmg.2014.04.014.

    Article  PubMed  Google Scholar 

  43. Bhatia RS, Ivers NM, Yin XC, Myers D, Nesbitt GC, Edwards J, et al. Improving the appropriate use of transthoracic echocardiography: the Echo WISELY trial. J Am Coll Cardiol. 2017;70(9):1135–44. https://doi.org/10.1016/j.jacc.2017.06.065.

    Article  PubMed  Google Scholar 

  44. Bhatia RS, Milford CE, Picard MH, Weiner RB. An educational intervention reduces the rate of inappropriate echocardiograms on an inpatient medical service. JACC Cardiovasc Imaging. 2013;6(5):545–55. https://doi.org/10.1016/j.jcmg.2013.01.010.

    Article  PubMed  Google Scholar 

  45. Blackmore CC, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision support in controlling inappropriate imaging. J Am Coll Radiol. 2011;8(1):19–25. https://doi.org/10.1016/j.jacr.2010.07.009.

    Article  PubMed  Google Scholar 

  46. Bookman K, West D, Ginde A, Wiler J, McIntyre R, Hammes A, et al. Embedded clinical decision support in electronic health record decreases use of high-cost imaging in the emergency department: EmbED study. Acad Emerg Med. 2017;24(7):839–45.

    Article  Google Scholar 

  47. Boutis K, Grootendorst P, Willan A, Plint AC, Babyn P, Brison RJ, et al. Effect of the low risk ankle rule on the frequency of radiography in children with ankle injuries. Cmaj. 2013;185(15):E731–8. https://doi.org/10.1503/cmaj.122050.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Breakell R, Thorndyke B, Clennett J, Harkensee C. Reducing unnecessary chest X-rays, antibiotics and bronchodilators through implementation of the NICE bronchiolitis guideline. Eur J Pediatr. 2018;177(1):47–51. https://doi.org/10.1007/s00431-017-3034-5.

    Article  PubMed  CAS  Google Scholar 

  49. Buntine P, Thien F, Stewart J, Woo YP, Koolstra M, Bridgford L, et al. Effect of a clinical flowchart incorporating Wells score, PERC rule and age-adjusted D-dimer on pulmonary embolism diagnosis, scan rates and diagnostic yield. Emerg Med Australas. 2019;31(2):216–24.

    Article  Google Scholar 

  50. Carpenter CP, Johnston D, Tourville E, Sharadin C, Alzubaidi AN, Giel DW. Inappropriate imaging for management of cryptorchidism: Has the choosing Wisely(R) recommendation reduced occurrence? J Pediatr Urol. 2020;16(4):462 e1- e6.

  51. Chandra K, Atkinson PR, Chatur H, Fraser J, Adams CL. To choose or not to choose: evaluating the effect of a choosing wisely knowledge translation initiative for imaging in low Back pain by emergency physicians. Cureus. 2019;11(2):e4002. https://doi.org/10.7759/cureus.4002.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Chang E, Buist DSM, Ley M, Johnson E, Fuller S, Pardee R, et al. Primary care physician resource use changes associated with feedback reports. Am J Manag Care. 2018;24(10):455–61.

    PubMed  Google Scholar 

  53. Char S, Yoon HC. Improving appropriate use of pulmonary computed tomography angiography by increasing the serum D-dimer threshold and assessing clinical probability. Perm J. 2014;18(4):10–5. https://doi.org/10.7812/TPP/14-007.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Chen D, Bhambhvani HP, Hom J, Mahoney M, Wintermark M, Sharp C, et al. Effect of electronic clinical decision support on imaging for the evaluation of acute low Back pain in the ambulatory care setting. World Neurosurg. 2020;134:e874–e7. https://doi.org/10.1016/j.wneu.2019.11.031.

    Article  PubMed  Google Scholar 

  55. Chien AT, Lehmann LS, Hatfield LA, Koplan KE, Petty CR, Sinaiko AD, et al. A randomized trial of displaying paid Price information on imaging study and procedure ordering rates. J Gen Intern Med. 2017;32(4):434–48. https://doi.org/10.1007/s11606-016-3917-6.

    Article  PubMed  Google Scholar 

  56. Depinet H, von Allmen D, Towbin A, Hornung R, Ho M, Alessandrini E. Risk Stratification to Decrease Unnecessary Diagnostic Imaging for Acute Appendicitis. Pediatrics. 2016;138(3).

  57. Doyle J, Abraham S, Feeney L, Reimer S, Finkelstein A. Clinical decision support for high-cost imaging: a randomized clinical trial. PLoS One. 2019;14(3):e0213373. https://doi.org/10.1371/journal.pone.0213373.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  58. Drescher FS, Chandrika S, Weir ID, Weintraub JT, Berman L, Lee R, et al. Effectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med. 2011;57(6):613–21. https://doi.org/10.1016/j.annemergmed.2010.09.018.

    Article  PubMed  Google Scholar 

  59. Dudzinski DM, Bhatia RS, Mi MY, Isselbacher EM, Picard MH, Weiner RB. Effect of educational intervention on the rate of rarely appropriate outpatient echocardiograms ordered by attending academic cardiologists: a randomized clinical trial. JAMA Cardiol. 2016;1(7):805–12. https://doi.org/10.1001/jamacardio.2016.2232.

    Article  PubMed  Google Scholar 

  60. Dunne RM, Ip IK, Abbett S, Gershanik EF, Raja AS, Hunsaker A, et al. Effect of evidence-based clinical decision support on the use and yield of CT pulmonary angiographic imaging in hospitalized patients. Radiology. 2015;276(1):167–74. https://doi.org/10.1148/radiol.15141208.

    Article  PubMed  Google Scholar 

  61. Durand DJ, Feldman LS, Lewin JS, Brotman DJ. Provider cost transparency alone has no impact on inpatient imaging utilization. J Am Coll Radiol. 2013;10(2):108–13. https://doi.org/10.1016/j.jacr.2012.06.020.

    Article  PubMed  Google Scholar 

  62. Fallon SC, Delemos D, Akinkuotu A, Christopher D, Naik-Mathuria BJ. The use of an institutional pediatric abdominal trauma protocol improves resource use. J Trauma Acute Care Surg. 2016;80(1):57–63.

    Article  Google Scholar 

  63. Ferguson CC, Gray MP, Diaz M, Boyd KP. Reducing Unnecessary Imaging for Patients With Constipation in the Pediatric Emergency Department. Pediatrics. 2017;140(1).

  64. Flamm M, Fritsch G, Hysek M, Klausner S, Entacher K, Panisch S, et al. Quality improvement in preoperative assessment by implementation of an electronic decision support tool. J Am Med Inform Assoc. 2013;20(e1):e91–6. https://doi.org/10.1136/amiajnl-2012-001178.

    Article  PubMed  PubMed Central  Google Scholar 

  65. French SD, McKenzie JE, O'Connor DA, Grimshaw JM, Mortimer D, Francis JJ, et al. Evaluation of a theory-informed implementation intervention for the management of acute low back pain in general medical practice: the IMPLEMENT cluster randomised trial. PLoS One. 2013;8(6):e65471. https://doi.org/10.1371/journal.pone.0065471.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  66. Gertz ZM, O'Donnell W, Raina A, Balderston JR, Litwack AJ, Goldberg LR. Implementation of a computerized order entry tool to reduce the inappropriate and unnecessary use of cardiac stress tests with imaging in hospitalized patients. Am J Cardiol. 2016;118(8):1123–7. https://doi.org/10.1016/j.amjcard.2016.07.021.

    Article  PubMed  Google Scholar 

  67. Goldberg J, McClaine RJ, Cook B, Garcia VF, Brown RL, Crone K, et al. Use of a mild traumatic brain injury guideline to reduce inpatient hospital imaging and charges. J Pediatr Surg. 2011;46(9):1777–83. https://doi.org/10.1016/j.jpedsurg.2011.02.052.

    Article  PubMed  Google Scholar 

  68. Graves JM, Fulton-Kehoe D, Jarvik JG, Franklin GM. Impact of an advanced imaging utilization review program on downstream health care utilization and costs for low Back pain. Med Care. 2018;56(6):520–8. https://doi.org/10.1097/MLR.0000000000000917.

    Article  PubMed  Google Scholar 

  69. Hardin L, Kilian A, Muller L, Callison K, Olgren M. Cross-continuum tool is associated with reduced utilization and cost for frequent high-need users. West J Emerg Med. 2017;18(2):189–200. https://doi.org/10.5811/westjem.2016.11.31916.

    Article  PubMed  Google Scholar 

  70. Hess EP, Homme JL, Kharbanda AB, Tzimenatos L, Louie JP, Cohen DM, et al. Effect of the Head Computed Tomography Choice Decision Aid in Parents of Children With Minor Head Trauma: A Cluster Randomized Trial. JAMA Netw Open. 2018;1(5):e182430-e.

  71. Hong AS, Ross-Degnan D, Zhang F, Wharam JF. Small decline in low-value Back imaging associated with the 'Choosing Wisely' campaign, 2012-14. Health Aff (Millwood). 2017;36(4):671–9. https://doi.org/10.1377/hlthaff.2016.1263.

    Article  Google Scholar 

  72. Hoo GW, Wu CC, Vazirani S, Li Z, Barack BM. Does a clinical decision rule using D-dimer level improve the yield of pulmonary CT angiography? AJR Am J Roentgenol. 2011;196(5):1059–64. https://doi.org/10.2214/AJR.10.4200.

    Article  PubMed  Google Scholar 

  73. Hui JS, Kramer DJ, Blackmore CC, Hashimoto BE, Coy DL. A quality improvement initiative to reduce unnecessary follow-up imaging for adnexal lesions. J Am Coll Radiol. 2014;11(4):373–7. https://doi.org/10.1016/j.jacr.2013.07.002.

    Article  PubMed  Google Scholar 

  74. Hurley P, Dhir A, Gao Y, Drabik B, Lim K, Curry J, et al. A statewide intervention improves appropriate imaging in localized prostate Cancer. J Urol. 2017;197(5):1222–8. https://doi.org/10.1016/j.juro.2016.11.098.

    Article  PubMed  Google Scholar 

  75. Ip IK, Gershanik EF, Schneider LI, Raja AS, Mar W, Seltzer S, et al. Impact of IT-enabled intervention on MRI use for back pain. Am J Med. 2014;127(6):512–8 e1. https://doi.org/10.1016/j.amjmed.2014.01.024.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Ip IK, Lacson R, Hentel K, Malhotra S, Darer J, Langlotz C, et al. JOURNAL CLUB: predictors of provider response to clinical decision support: lessons learned from the Medicare imaging demonstration. AJR Am J Roentgenol. 2017;208(2):351–7. https://doi.org/10.2214/AJR.16.16373.

    Article  PubMed  Google Scholar 

  77. Ip IK, Raja AS, Gupta A, Andruchow J, Sodickson A, Khorasani R. Impact of clinical decision support on head computed tomography use in patients with mild traumatic brain injury in the ED. Am J Emerg Med. 2015;33(3):320–5. https://doi.org/10.1016/j.ajem.2014.11.005.

    Article  PubMed  Google Scholar 

  78. Ip IK, Schneider L, Seltzer S, Smith A, Dudley J, Menard A, et al. Impact of provider-led, technology-enabled radiology management program on imaging. Am J Med. 2013;126(8):687–92. https://doi.org/10.1016/j.amjmed.2012.11.034.

    Article  PubMed  Google Scholar 

  79. Jennings RM, Burtner JJ, Pellicer JF, Nair DK, Bradford MC, Shaffer M, et al. Reducing Head CT Use for Children With Head Injuries in a Community Emergency Department. Pediatrics. 2017;139(4).

  80. Judkins A, Pascoe E, Payne D. Management of urinary tract infection in a tertiary children's hospital before and after publication of the NICE guidelines. Arch Dis Child. 2013;98(7):521–5. https://doi.org/10.1136/archdischild-2012-303032.

    Article  PubMed  Google Scholar 

  81. Kandiah JW, Chan VWY, Luo J, Dong F, Nugent JP, Forster BB. Reducing the volume of low-value outpatient MRI joint Examinations in Patients >/=55 years of age. Can Assoc Radiol J. 2020;71(1):83–91. https://doi.org/10.1177/0846537119885686.

    Article  PubMed  Google Scholar 

  82. Kanaan Y, Knoepp UD, Kelly AM. The influence of education on appropriateness rates for CT pulmonary angiography in emergency department patients. Acad Radiol. 2013;20(9):1107–14. https://doi.org/10.1016/j.acra.2013.05.005.

    Article  PubMed  Google Scholar 

  83. Keveson B, Clouser RD, Hamlin MP, Stevens P, Msn RN, Stinnett-Donnelly JM, et al. Adding value to daily chest X-rays in the ICU through education, restricted daily orders and indication-based prompting. BMJ Open Qual. 2017;6(2):e000072.

    Article  Google Scholar 

  84. Lacson R, Ip I, Hentel KD, Malhotra S, Balthazar P, Langlotz CP, et al. Medicare imaging demonstration: assessing attributes of appropriate use criteria and their influence on ordering behavior. AJR Am J Roentgenol. 2017;208(5):1051–7. https://doi.org/10.2214/AJR.16.17169.

    Article  PubMed  Google Scholar 

  85. Lu MT, Tellis WM, Fidelman N, Qayyum A, Avrin DE. Reducing the rate of repeat imaging: import of outside images to PACS. AJR Am J Roentgenol. 2012;198(3):628–34. https://doi.org/10.2214/AJR.11.6890.

    Article  PubMed  Google Scholar 

  86. Luther G, Miller PE, Mahan ST, Waters PM, Bae DS. Decreasing resource utilization using standardized clinical assessment and management plans (SCAMPs). J Pediatr Orthop. 2019;39(4):169–74. https://doi.org/10.1097/BPO.0000000000000873.

    Article  PubMed  Google Scholar 

  87. Masood S, Woolner V, Yoon JH, Chartier LB. Checklist for Head Injury Management Evaluation Study (CHIMES): a quality improvement initiative to reduce imaging utilisation for head injuries in the emergency department. BMJ Open Qual. 2020;9(1).

  88. McGrew PR, Chestovich PJ, Fisher JD, Kuhls DA, Fraser DR, Patel PP, et al. Implementation of a CT scan practice guideline for pediatric trauma patients reduces unnecessary scans without impacting outcomes. J Trauma Acute Care Surg. 2018;85(3):451–8. https://doi.org/10.1097/TA.0000000000001974.

    Article  PubMed  Google Scholar 

  89. Min A, Chan VWY, Aristizabal R, Peramaki ER, Agulnik DB, Strydom N, et al. Clinical decision support decreases volume of imaging for low Back pain in an urban emergency department. J Am Coll Radiol. 2017;14(7):889–99. https://doi.org/10.1016/j.jacr.2017.03.005.

    Article  PubMed  Google Scholar 

  90. Mittal V, Darnell C, Walsh B, Mehta A, Badawy M, Morse R, et al. Inpatient bronchiolitis guideline implementation and resource utilization. Pediatrics. 2014;133(3):e730–7. https://doi.org/10.1542/peds.2013-2881.

    Article  PubMed  Google Scholar 

  91. Moriarity AK, Klochko C, O'Brien M, Halabi S. The effect of clinical decision support for advanced inpatient imaging. J Am Coll Radiol. 2015;12(4):358–63. https://doi.org/10.1016/j.jacr.2014.11.013.

    Article  PubMed  Google Scholar 

  92. Mulders MAM, Walenkamp MMJ, Sosef NL, Ouwehand F, van Velde R, Goslings CJ, et al. The Amsterdam wrist rules to reduce the need for radiography after a suspected distal radius fracture: an implementation study. Eur J Trauma Emerg Surg. 2020;46(3):573–82. https://doi.org/10.1007/s00068-019-01194-2.

    Article  PubMed  Google Scholar 

  93. Mäenpää T, Asikainen P, Gissler M, Siponen K, Maass M, Saranto K, et al. Outcomes assessment of the regional health information exchange: a five-year follow-up study. Methods Inf Med. 2011;50(4):308–18.

    Article  Google Scholar 

  94. Nigrovic LE, Stack AM, Mannix RC, Lyons TW, Samnaliev M, Bachur RG, et al. Quality improvement effort to reduce cranial CTs for children with minor blunt head trauma. Pediatrics. 2015;136(1):e227–33. https://doi.org/10.1542/peds.2014-3588.

    Article  PubMed  Google Scholar 

  95. O'Connor SD, Sodickson AD, Ip IK, Raja AS, Healey MJ, Schneider LI, et al. JOURNAL CLUB: requiring clinical justification to override repeat imaging decision support: impact on CT use. Am J Roentgenol. 2014;203(5):W482–W90. https://doi.org/10.2214/AJR.14.13017.

    Article  Google Scholar 

  96. Ostby SA, Evans JG, Smith HJ, Boitano TKL, Toboni MD, Heimann MA, et al. Reducing emergency department (ED) computed tomography (CT) utilization in women treated for gynecologic cancers. Gynecol Oncol. 2020;156(2):288–92. https://doi.org/10.1016/j.ygyno.2019.11.024.

    Article  PubMed  Google Scholar 

  97. Palen TE, Sharpe RE Jr, Shetterly SM, Steiner JF. Randomized clinical trial of a clinical decision support tool for improving the appropriateness scores for ordering imaging studies in primary and specialty care ambulatory clinics. AJR Am J Roentgenol. 2019;213(5):1015–20. https://doi.org/10.2214/AJR.19.21511.

    Article  PubMed  Google Scholar 

  98. Parikh K, Hall M, Blaschke AJ, Grijalva CG, Brogan TV, Neuman MI, et al. Aggregate and hospital-level impact of national guidelines on diagnostic resource utilization for children with pneumonia at children's hospitals. J Hosp Med. 2016;11(5):317–23.

    Article  Google Scholar 

  99. Poeran J, Mao LJ, Zubizarreta N, Mazumdar M, Darrow B, Genes N, et al. Effect of clinical decision support on appropriateness of advanced imaging use among physicians-in-training. AJR Am J Roentgenol. 2019;212(4):859–66. https://doi.org/10.2214/AJR.18.19931.

    Article  PubMed  Google Scholar 

  100. Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med. 2013;126(11):975–81. https://doi.org/10.1016/j.amjmed.2013.04.018.

    Article  PubMed  PubMed Central  Google Scholar 

  101. Pugel S, Stallworth JL, Pugh LB, Terrell C, Bailey Z, Gramling T, et al. Choosing wisely in Georgia: a quality improvement initiative in 25 adult ambulatory medicine offices. Jt Comm J Qual Patient Saf. 2018;44(12):699–707. https://doi.org/10.1016/j.jcjq.2018.05.010.

    Article  PubMed  Google Scholar 

  102. Raja AS, Ip IK, Dunne RM, Schuur JD, Mills AM, Khorasani R. Effects of performance feedback reports on adherence to evidence-based guidelines in use of CT for evaluation of pulmonary embolism in the emergency department: a randomized trial. AJR Am J Roentgenol. 2015;205(5):936–40.

    Article  Google Scholar 

  103. Raja AS, Ip IK, Prevedello LM, Sodickson AD, Farkas C, Zane RD, et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012;262(2):468–74.

    Article  Google Scholar 

  104. Reiter J, Breuer A, Breuer O, Hashavya S, Rekhtman D, Kerem E, et al. A quality improvement intervention to reduce emergency department radiography for bronchiolitis. Respir Med. 2018;137:1–5. https://doi.org/10.1016/j.rmed.2018.02.014.

    Article  PubMed  Google Scholar 

  105. Rezaii PG, Fredericks N, Lincoln CM, Hom J, Willis M, Burleson J, et al. Assessment of the radiology support, communication and alignment network to reduce medical imaging overutilization: a multipractice cohort study. J Am Coll Radiol. 2020;17(5):597–605. https://doi.org/10.1016/j.jacr.2020.02.011.

    Article  PubMed  Google Scholar 

  106. Rosati SF, Maarouf R, Wolfe L, Parrish D, Poppe M, Manners R, et al. Implementation of pediatric cervical spine clearance guidelines at a combined trauma center: twelve-month impact. J Trauma Acute Care Surg. 2015;78(6):1117–21. https://doi.org/10.1097/TA.0000000000000643.

    Article  PubMed  Google Scholar 

  107. Sclafani JJ, My J, Zacher LL, Eckart RE. Intensive education on evidence-based evaluation of syncope increases sudden death risk stratification but fails to reduce use of neuroimaging. Arch Intern Med. 2010;170(13):1150–4. https://doi.org/10.1001/archinternmed.2010.205.

    Article  PubMed  Google Scholar 

  108. Shah SR, Sinclair KA, Theut SB, Johnson KM, Holcomb GW 3rd, St Peter SD. Computed tomography utilization for the diagnosis of acute appendicitis in children decreases with a diagnostic algorithm. Ann Surg. 2016;264(3):474–81. https://doi.org/10.1097/SLA.0000000000001867.

    Article  PubMed  Google Scholar 

  109. Singer AJ, Garra G, Thode HC Jr. Changes in practice patterns with the opening of a dedicated pediatric emergency department. Pediatr Emerg Care. 2014;30(10):705–9. https://doi.org/10.1097/PEC.0000000000000232.

    Article  PubMed  Google Scholar 

  110. Sodickson A, Opraseuth J, Ledbetter S. Outside imaging in emergency department transfer patients: CD import reduces rates of subsequent imaging utilization. Radiology. 2011;260(2):408–13. https://doi.org/10.1148/radiol.11101956.

    Article  PubMed  Google Scholar 

  111. Sy E, Luong M, Quon M, Kim Y, Sharifi S, Norena M, et al. Implementation of a quality improvement initiative to reduce daily chest radiographs in the intensive care unit. BMJ Qual Saf. 2016;25(5):379–85. https://doi.org/10.1136/bmjqs-2015-004151.

    Article  PubMed  Google Scholar 

  112. Tyler A, Krack P, Bakel LA, O'Hara K, Scudamore D, Topoz I, et al. Interventions to Reduce Over-Utilized Tests and Treatments in Bronchiolitis. Pediatrics. 2018;141(6).

  113. Vartanians VM, Sistrom CL, Weilburg JB, Rosenthal DI, Thrall JH. Increasing the appropriateness of outpatient imaging: effects of a barrier to ordering low-yield examinations. Radiology. 2010;255(3):842–9. https://doi.org/10.1148/radiol.10091228.

    Article  PubMed  Google Scholar 

  114. Walker D, Macdonald DB, Dennie C, Afkham A, Liddy C, Keely E. Electronic consultation between primary care providers and radiologists. AJR Am J Roentgenol. 2020;215(4):929–33. https://doi.org/10.2214/AJR.19.22270.

    Article  PubMed  Google Scholar 

  115. Wang KY, Yen CJ, Chen M, Variyam D, Acosta TU, Reed B, et al. Reducing Inappropriate Lumbar Spine MRI for Low Back Pain: Radiology Support, Communication and Alignment Network. J Am Coll Radiol. 2018;15(1 Pt A):116–22.

  116. Wu Y, Rose MQ, Freeman ML, Richard-Lany NP, Spaulding AC, Booth SC, et al. Reducing chest radiography utilization in the medical intensive care unit. J Am Assoc Nurse Pract. 2020;32(5):390–9. https://doi.org/10.1097/JXX.0000000000000256.

    Article  PubMed  Google Scholar 

  117. Xu SS, Berkovitz N, Li O, Garvin G. Reduction in inappropriate MRI knee studies after implementation of an appropriateness checklist: Experience at a tertiary care centre. Eur J Radiol. 2020;123:108781.

    Article  CAS  Google Scholar 

  118. Zafar HM, Ip IK, Mills AM, Raja AS, Langlotz CP, Khorasani R. Effect of clinical decision support-generated report cards versus real-time alerts on primary care provider guideline adherence for low Back pain outpatient lumbar spine MRI orders. AJR Am J Roentgenol. 2019;212(2):386–94. https://doi.org/10.2214/AJR.18.19780.

    Article  PubMed  Google Scholar 

  119. Zamora-Flores D, Busen NH, Smout R, Velasquez O. Implementing a clinical practice guideline for the treatment of bronchiolitis in a high-risk Hispanic pediatric population. J Pediatr Health Care. 2015;29(2):169–80. https://doi.org/10.1016/j.pedhc.2014.10.002.

    Article  PubMed  Google Scholar 

  120. Levitt K, Edwards J, Chow CM, Bhatia RS. Development of an educational strategy and decision support tool to enhance appropriate use of stress echocardiography at a large Academic Medical Center: a prospective, pre- and Postintervention analysis. J Am Soc Echocardiogr. 2015;28(12):1401–9. https://doi.org/10.1016/j.echo.2015.08.003.

    Article  PubMed  Google Scholar 

  121. Mills AM, Ip IK, Langlotz CP, Raja AS, Zafar HM, Khorasani R. Clinical decision support increases diagnostic yield of computed tomography for suspected pulmonary embolism. Am J Emerg Med. 2018;36(4):540–4. https://doi.org/10.1016/j.ajem.2017.09.004.

    Article  PubMed  Google Scholar 

  122. Ong CW, Malipatil V, Lavercombe M, Teo KG, Coughlin PB, Leach D, et al. Implementation of a clinical prediction tool for pulmonary embolism diagnosis in a tertiary teaching hospital reduces the number of computed tomography pulmonary angiograms performed. Intern Med J. 2013;43(2):169–74. https://doi.org/10.1111/j.1445-5994.2012.02926.x.

    Article  PubMed  CAS  Google Scholar 

  123. Puffenbarger MS, Ahmad FA, Argent M, Gu H, Samson C, Quayle KS, et al. Reduction of computed tomography use for pediatric closed head injury evaluation at a nonpediatric community emergency department. Acad Emerg Med. 2019;26(7):784–95. https://doi.org/10.1111/acem.13666.

    Article  PubMed  Google Scholar 

  124. Tajmir S, Raja AS, Ip IK, Andruchow J, Silveira P, Smith S, et al. Impact of clinical decision support on radiography for acute ankle injuries: a randomized trial. West J Emerg Med. 2017;18(3):487–95. https://doi.org/10.5811/westjem.2017.1.33053.

    Article  PubMed  PubMed Central  Google Scholar 

  125. Walen S, de Boer E, Edens MA, van der Worp CA, Boomsma MF, van den Berg JW. Mandatory adherence to diagnostic protocol increases the yield of CTPA for pulmonary embolism. Insights Imaging. 2016;7(5):727–34. https://doi.org/10.1007/s13244-016-0509-2.

    Article  PubMed  PubMed Central  Google Scholar 

  126. Arora R, White EN, Niedbala D, Ravichandran Y, Sethuraman U, Radovic N, et al. Reducing computed tomography scan utilization for pediatric minor head injury in the emergency department: a quality improvement initiative. Acad Emerg Med. 2020;28(6):655–65. https://doi.org/10.1111/acem.14177.

    Article  PubMed  Google Scholar 

  127. Ballard DW, Kuppermann N, Vinson DR, Tham E, Hoffman JM, Swietlik M, et al. Implementation of a clinical decision support system for children with minor blunt head trauma who are at nonnegligible risk for traumatic brain injuries. Ann Emerg Med. 2019;73(5):440–51. https://doi.org/10.1016/j.annemergmed.2018.11.011.

    Article  PubMed  Google Scholar 

  128. Carnevale TJ, Meng D, Wang JJ, Littlewood M. Impact of an emergency medicine decision support and risk education system on computed tomography and magnetic resonance imaging use. J Emerg Med. 2015;48(1):53–7. https://doi.org/10.1016/j.jemermed.2014.07.033.

    Article  PubMed  Google Scholar 

  129. Ehrlichman R, Dezman Z, Klein J, Jeudy J, Lemkin D. Quarterly reporting of computed tomography ordering history reduces the use of imaging in an emergency department. J Emerg Med. 2017;52(5):684–9. https://doi.org/10.1016/j.jemermed.2016.11.014.

    Article  PubMed  Google Scholar 

  130. Kobes KJ, Budau-Bymoen A, Thakur Y, Yong-Hing CJ. Multidisciplinary development of Mobile radiography guidelines reduced the number of inappropriate Mobile exams in patients receiving chest radiographs in British Columbia. Can Assoc Radiol J. 2020;71(1):110–6. https://doi.org/10.1177/0846537119888357.

    Article  PubMed  Google Scholar 

  131. Shelton JB, Ochotorena L, Bennett C, Shekelle P, Kwan L, Skolarus T, et al. Reducing PSA-based prostate Cancer screening in men aged 75 years and older with the use of highly specific computerized clinical decision support. J Gen Intern Med. 2015;30(8):1133–9. https://doi.org/10.1007/s11606-015-3249-y.

    Article  PubMed  PubMed Central  Google Scholar 

  132. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x.

    Article  PubMed  PubMed Central  Google Scholar 

  133. Keyworth C, Hart J, Armitage CJ, Tully MP. What maximizes the effectiveness and implementation of technology-based interventions to support healthcare professional practice? A systematic literature review. BMC Med Inform Decis Mak. 2018;18(1):93. https://doi.org/10.1186/s12911-018-0661-3.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  134. de Wit K, Curran J, Thoma B, Dowling S, Lang E, Kuljic N, et al. Review of implementation strategies to change healthcare provider behaviour in the emergency department. Cjem. 2018;20(3):453–60. https://doi.org/10.1017/cem.2017.432.

    Article  PubMed  Google Scholar 

  135. Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy: strategies for combining interventions at multiple levels. J Natl Cancer Inst Monogr. 2012;2012(44):34–41. https://doi.org/10.1093/jncimonographs/lgs001.

    Article  PubMed  PubMed Central  Google Scholar 

  136. Salerno S, Laghi A, Cantone MC, Sartori P, Pinto A, Frija G. Overdiagnosis and overimaging: an ethical issue for radiological protection. Radiol Med. 2019;124(8):714–20.

    Article  Google Scholar 

  137. You JJ, Levinson W, Laupacis A. Attitudes of family physicians, specialists and radiologists about the use of computed tomography and magnetic resonance imaging in Ontario. Healthc Policy. 2009;5(1):54–65. https://doi.org/10.12927/hcpol.2009.21002.

    Article  PubMed  PubMed Central  Google Scholar 

  138. Halpern DJ, Clark-Randall A, Woodall J, Anderson J, Shah K. Reducing imaging utilization in primary care through implementation of a peer comparison dashboard. J Gen Intern Med. 2021;36(1):108–13. https://doi.org/10.1007/s11606-020-06164-8.

    Article  PubMed  Google Scholar 

  139. Coombs DM, Machado GC, Richards B, Needs C, Buchbinder R, Harris IA, et al. Effectiveness of a multifaceted intervention to improve emergency department care of low back pain: a stepped-wedge, cluster-randomised trial. BMJ Qual Saf. 2021.

  140. Ehrman RR, Malik AN, Smith RK, Kalarikkal Z, Huang A, King RM, et al. Serial use of existing clinical decisions aids can reduce computed tomography pulmonary angiography for pulmonary embolism. Intern Emerg Med. 2021.

  141. Kharbanda AB, Vazquez-Benitez G, Ballard DW, Vinson DR, Chettipally UK, Dehmer SP, et al. Effect of Clinical Decision Support on Diagnostic Imaging for Pediatric Appendicitis: A Cluster Randomized Trial. JAMA Network Open. 2021;4(2):e2036344-e.

  142. Singh S, Hearps SJC, Borland ML, Dalziel SR, Neutze J, Donath S, et al. The effect of patient observation on cranial computed tomography rates in children with minor head trauma. Acad Emerg Med. 2021;27(9):832–43. https://doi.org/10.1111/acem.13942.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank Senior Research Librarian Karen Marie Øvern at NTNU and academic librarian Jana Myrvold at University of South-Eastern Norway for helping with the development of the search strategy. In addition, we would like to thank Dr. Fiona Clement for her useful suggestions for the search strategy.

Funding

This project received financial support from The Research council of Norway (Project number 302503).

Author information

Authors and Affiliations

  1. Institute for the Health Sciences at the Norwegian University of Science and Technology (NTNU) at Gjøvik, NTNU Gjøvik, Postbox 191, 2802, Gjøvik, Norway

    Elin Kjelle, Eivind Richter Andersen & Bjørn Morten Hofmann

  2. Department of Community Health Sciences and The Health Technology Assessment Unit, O’Brien Institute for Public Health, University of Calgary, 3280 Hospital Dr NW, Calgary, Alberta, T2N 4Z6, Canada

    Lesley J. J. Soril

  3. Medical Decision making, Department of Biomedical Data Sciences, Leiden University Medical Center, P.O. Box 9600, 2300, RC, Leiden, the Netherlands

    Leti van Bodegom-Vos

  4. Centre of Medical Ethics, University of Oslo, Postbox 1130, Blindern, 0318, Oslo, Norway

    Bjørn Morten Hofmann

Authors
  1. Elin Kjelle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eivind Richter Andersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lesley J. J. Soril
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leti van Bodegom-Vos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bjørn Morten Hofmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EK – planning and searching, screening, full-text and quality assessment, analysis, drafting, and revision of manuscript. ERA – planning, full-text and quality assessment, snowballing, analysis, and revision. LJJS – planning, full-text and quality assessment, analysis, and revision. LvB-V – planning, full-text and quality assessment, analysis, and revision. BMH – planning, screening, full-text and quality assessment, analysis, and revision. All authors read and approved the final manuscript

Corresponding author

Correspondence to Elin Kjelle.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Search strategy and hits from database searches.

Additional file 2.

Table of excluded studies.

Additional file 3.

MMAT registration forms.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kjelle, E., Andersen, E.R., Soril, L.J.J. et al. Interventions to reduce low-value imaging – a systematic review of interventions and outcomes. BMC Health Serv Res 21, 983 (2021). https://doi.org/10.1186/s12913-021-07004-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-07004-z

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us