- Research article
- Open Access
- Open Peer Review
Highlighting the need for de-implementation – Choosing Wisely recommendations based on clinical practice guidelines
BMC Health Services Researchvolume 19, Article number: 638 (2019)
The Choosing Wisely campaign has spread to many countries. Methods for developing recommendations are inconsistent. We describe our process of developing such recommendations from a comprehensive national set of clinical practice guidelines (Current Care, CC) and the results of a one-year Choosing Wisely Finland project.
Two of the authors drafted the quality and process criteria for all the Choosing Wisely Finland recommendations. The quality criteria were relevance, feasibility, evidence-based and strength. These were discussed in editors’ meetings and subsequently revised. Two different processes for developing recommendations within national clinical practice guidelines were designed and piloted (processes A and B). Process A was based on a published guideline. The recommendations are drafted by an editor and revised and approved by the guideline development group. In process B the development of the recommendations is integrated with guideline production or update. Choosing Wisely recommendations were then drafted for half of the published CC Guidelines. An additional process (process C) was designed for producing independent recommendations outside a guideline.
At least one Choosing Wisely recommendation could be identified from 39 out of 52 reviewed guidelines. Of the 106 recommendations drafted, 62 (58%) were accepted for publication. The main reasons for rejection were inability to give a strong recommendation (n = 18, 41%) and insufficient relevance (n = 14, 32%). Two thirds (n = 41, 66%) of the published recommendations were based on high to moderate level of evidence, and 18% (n = 11) on low or very low level of evidence, whereas for the rest, the quality of evidence was not critically appraised.
Choosing Wisely recommendations can be produced systematically from existing clinical practice guidelines. The rigorous methods of evidence-based medicine ensure high-quality recommendations. We welcome the use of our processes and methods describes in this article by other guideline-producing organizations.
De-implementation means reducing or stopping the use of a health service or practice provided to a patient. Reduction is appropriate in cases where interventions are used inappropriately or have low value, e.g. interventions that are shown to be ineffective or harmful . Low-value interventions can have an unfavorable balance between harms and benefits or be more costly than optional interventions. De-implementation should also be considered for interventions for which the evidence is uncertain or lacking.
Identifying and prioritizing interventions that should be avoided is a key challenge . The Choosing Wisely campaign was launched in 2012 to support discussions between physicians and patients  about avoiding low-value care. The campaign invited medical specialty societies in the US to produce top 5 lists of low-value services. Each society was free to develop lists with their own methods using documented, publicly available processes. The criteria for topic selection were that the procedure is used frequently, carries a significant cost, may expose patients to harm or burden, or may increase strain on the health care system. Each recommendation should be supported by strong evidence . The campaign has spread to over 20 countries . In Germany a manual and criteria have been published for developing Choosing Wisely recommendations .
In Germany, several methodological challenges in producing Choosing Wisely recommendations were identified . These include weak methodology, lack of transparency about prioritization of topics and unrealistic expectations for the recommendations as a solution to overuse. As one solution to these challenges, using high-quality clinical practice guidelines as a starting point was suggested. This would secure systematic literature searching and appraisal, multidisciplinarity, and consensus with independent moderation. In addition, the viewpoint would change from medical specialties to diseases.
In Finland, there is a library of 105 national evidence-based Current Care (CC) Guidelines. Here we describe methods for developing the Choosing Wisely Finland recommendations from these guidelines, and for integrating such recommendations back into the guidelines.
The work was inspired by international experiences, including Choosing Wisely campaigns as well as NICE’s Do-not-do -recommendations. We developed a methodology based on the scanty literature available, piloting methods on the way. We worked in close collaboration with guideline editors, a group of physicians with training in critical appraisal and facilitating guideline development groups.
The Finnish Medical Society Duodecim, in co-operation with medical specialty societies, has produced national clinical practice guidelines (Current Care) since 1994 . The guidelines cover prevention, diagnostics, medical treatment, and rehabilitation of diseases. CC Guidelines serve as a support for treatment decisions for healthcare professionals, with a particular emphasis on primary care. They are available for anyone through open access, and most of them include a patient version. CC Guidelines are accessed over 6 million times a year. Evidence based health care has a long history in Finland. In addition to guidelines, systematically produced health technology assessments (HTAs) have guided the implementation of new technologies in hospitals, including consensus recommendations to limit the spread of low-value technologies . The Council for Choices in Health Care in Finland produces recommendations and defines the service basket at a societal level .
The production of CC Guidelines follows international standards of development . The level of evidence is graded from A to D (high to very low)  based on the system launched by the Institute of Medicine, as endorsed by the Guidelines International Network . Guideline production is publicly funded and led by the medical profession. The editorial team includes 11 editors representing different medical specialties. They act as project managers and evidence-based medicine (EBM) methodologists. All guideline development groups are multiprofessional.
The Finnish health care system is decentralized, organized by municipalities and funded by taxation, apart from occupational health care which is funded by employers and produced largely by private companies. Specialized health care is provided in 20 hospital districts, five of which have a university hospital. CC Guidelines are well-known and widely accepted by clinicians and used also at organizational level , as all Finnish hospital districts expect their entire staff to apply CC Guidelines in care pathways.
We decided to base our de-implementation work on the existing comprehensive national guideline library. The Ministry of Social Affairs and Health funded the pilot project. The first step was to set up criteria for the recommendations. Although Choosing Wisely methods are described by the participating societies on the Choosing Wisely website, there were few scientific publications on the methodology. For Choosing Wisely Finland, we aimed to describe criteria that guaranteed transparent, consistent, and EBM-based methodology and that was suited for guideline producers. In line with the US Choosing Wisely criteria relevance for patients and health care providers was considered important. To limit the number of recommendations, feasibility was added as a criterion. Transparency was primary consideration when defining process criteria. The criteria were drafted by the CC Editor-in-Chief (JK) and Managing Editor (RS) and discussed in editors’ meetings. As a result, process criteria were separated from quality criteria for the recommendations (Table 1).
We developed a structured format for publishing [See Additional file 1]. According to the principles of EBM, each recommendation is accompanied by a justification, references and, when appropriate, an evidence summary. A short list of possible barriers for implementation is provided when relevant.
We identified three different processes for developing Choosing Wisely Finland recommendations (Table 2) and discussed these with the editors. Two of these are based on evidence gathered in a guideline production process. In process A, the Choosing Wisely recommendations were drawn from and incorporated into an existing guideline. In process B, the development of recommendations was integrated to the guideline development process. The third process (Table 2, process C) can be used when a national guideline on the subject is not available.
In the pilot phase, five CC Guidelines were reviewed according to process A (Table 2) and eight Choosing Wisely recommendations were drafted. The Ministry of Social Affairs and Health then funded a project for developing relevant Choosing Wisely recommendations from 50 existing CC Guidelines. In cooperation with the Ministry, a set of recently updated guidelines were selected from the national guideline library.
The editors were initially introduced to the Choosing Wisely ideology in a pilot phase workshop. Subsequently, the editors were trained to understand low-value care, and learn the processes and quality requirements of Choosing Wisely Finland recommendations. The training included educational sessions, presentations as well as feedback and problem solving.
Each editor received a set of guidelines to review. The editors were asked to draft up to 5 recommendations per each guideline. They identified topics and drafted recommendations using the existing guidelines and their evidence summaries as a basis (Table 2, process A) and proceeded with the most relevant ones. The recommendations could target treatment, diagnostic testing, screening, rehabilitation or follow-up. Topics were often identified and drafted in close co-operation with the chair of the guideline development group and members responsible for the topic. Recommendations were discussed at subsequent editors’ meetings. Before final approval, the guideline development group members were asked to comment on the recommendations and consider relevance and other criteria. Consensus was reached by discussions. The Managing Editor led the project, tutored other editors and provided detailed individual feedback.
Altogether 52 guidelines were reviewed during the project. In 13 (25%) guidelines, no recommendations fulfilling the Choosing Wisely criteria were identified. For the remaining 39 guidelines, 106 recommendations were drafted, and 62 (58%) of these were accepted for publication [see Additional file 2]. Because relevance was considered to be insufficient, recommendations were rejected in 14 cases. In another 8 cases the underlying evidence was judged to be too weak for a strong recommendation according to GRADE criteria. In 18 cases a strong recommendation against an intervention was not possible due to the complexity of the issue in the Finnish health care setting. In four cases the evidence was outdated, and a new literature search and review would have been required.
A third of the published recommendations were related to pharmacotherapy (n = 21), whereas a quarter dealt with diagnostics (n = 15). None were related to psychosocial treatments or rehabilitation. Twenty-two of the 62 recommendations were supported by high-level of evidence, 19 by moderate level, 10 by low level and 1 by very low level of evidence. In 10 cases an evidence summary was considered unnecessary, typically because they dealt with overdiagnosis, overtreatment or well-known, and potentially serious adverse events. A typical justification for a Choosing Wisely recommendation was an unfavorable balance between benefits and harms or ineffectiveness of the intervention (Table 3).
In contrast to most Choosing Wisely campaigns that start by identifying isolated" Things Providers and Patients Should Question", we utilized existing clinical practice guidelines. We also developed and applied systematic evidence-based methods to secure the trustworthiness of the recommendations. Methodological requirements in the US Choosing Wisely campaign are rather loose (e.g. being evidence-based) and the reported methodologies vary from Delphi method to adopting other medical societies’ recommendations .
A major strength of our work is that the recommendations are developed systematically and transparently, using EBM methodology. According to one report, only 32% of Choosing Wisely recommendations were judged sufficiently trustworthy . Admon et al. extracted all 320 recommendations published by the Society of Hospital Medicine and found that a majority (70%) were referring to guidelines of variable quality while only one in five was linked to primary research .
According to GRADE, a strong recommendation should usually be based on at least moderate confidence in effect estimates . Of our Choosing Wisely recommendations, 66% were based on a high to moderate level of evidence. For 16% the quality of the evidence was not formally assessed in the guideline. One study found that under 10% of the US Choosing Wisely recommendations were based on low-quality evidence , but in the primary care context, the majority of recommendations were self-evident, e.g. based on “consensus or disease-oriented evidence” . We have not formally compared our Choosing Wisely recommendations to the US recommendations but made some interesting observations. It seems that our criteria result in fewer recommendations per topic or disease. According to our results, the main reasons for rejecting a recommendation were a lack of relevance and complexity of the issue. Some recommendations may be acceptable in one setting but not in another one. For example, for type 2 diabetes, the US library has a recommendation to “Avoid routine multiple daily self-glucose monitoring in adults with stable type 2 diabetes on agents that do not cause hypoglycemia.”  We suggested a similar recommendation, but our guideline development group assessed the issue to be too complex as there are some patient groups that may benefit from such monitoring. The guideline development group pointed out that municipalities already have restricted the distribution of test strips.
Our method has several other strengths. The criteria are clearly described, making the process transparent. Each recommendation also includes a justification and, if appropriate, an evidence summary. The justification provides detailed reasoning for the reader. This is especially important if the evidence is of low quality or lacking. The reader thus gets an opportunity to make informed decisions to avoid low-value care and receives tools for shared decision-making. We also argue that the process changes the viewpoint from specialty-oriented to more disease-oriented when a guideline development group formulates recommendations. Furthermore, a multiprofessional group probably evaluates the relevance of a recommendation more widely than a group from one specialist society.
One limitation of our process is that the quality criteria are not formally assessed when producing recommendations from guidelines with processes A and B. The recommendations are, however, based on evidence gathered and assessed by a guideline development group during the guideline process. A more formal assessment would naturally increase the transparency of the process. A more extensive formal review is planned for recommendations that are not based on a guideline (Table 2, process C). Both quality and process criteria are checked by the Editor-in-Chief before final approval. At this point, in particular the level and quality of evidence as well as the possibility to give a strong recommendation are ascertained.
Another limitation is that our guideline development groups do not include patients. We plan to involve lay representatives in guideline production, including the steps for Choosing Wisely Finland recommendations. This may increase the relevance of the recommendations.
Forty-four of the draft recommendations were not published. The main reason for this was that we were unable to give a strong recommendation against an intervention due to complexity of the issue at hand. This underlines the need to discuss evidence and reach a consensus to avoid oversimplification and conflicts. We did not use a full GRADE process, where the importance of each outcome is weighed and tabulated. This would likely be useful when guidelines are updated and new Choosing Wisely recommendations are drafted.
Conflict of interest declarations are gathered from all authors during CC Guideline production or updates (Process B, Table 2). However, for the other two processes, formal updated declarations are not requested as of yet.
Awareness is the first step to change. To raise awareness of our recommendations we use our website, social media, press releases and sessions at continuing medical education events. Selected recommendations are also published in the scientific journal of the Finnish Medical Society Duodecim. Choosing Wisely campaigns aim to facilitate discussions between physicians and patients. The US campaign includes posters and videos for the public. Relevant Choosing Wisely recommendations are incorporated into the patient versions of CC Guidelines. To produce more extensive material to the public, however, we would need more resources and co-operation with patient organizations. So far, we have not studied changes in awareness or the actual impact of our recommendations. In the US, change in awareness was negligible during the campaign in 2014–2017 . Some small changes in intervention prevalence have been detected, but the clinical impact of the decrease has remained questionable .
Guideline implementation research shows that dissemination is not enough to change practices; active interventions are needed. There is little research on de-implementation and most behavioral theories do not differentiate between trying to increase and decrease the frequency of a behavior . Typical major barriers to avoiding low-value care include malpractice concerns, time pressures in the clinical encounter, patient demand, and physicians’ need for more information to reduce uncertainty [18, 21]. Choosing Wisely recommendations are one tool to facilitate change, but other interventions would be needed to overcome barriers.
We have shown that it is feasible to produce Choosing Wisely recommendations systematically from existing evidence-based clinical practice guidelines. A parallel production of guidelines and Choosing Wisely recommendations has obvious benefits: duplicate work is avoided and rigorous EBM methods and processes ensure high quality recommendations. Other guideline producers are welcome to make use of our processes and methods.
Availability of data and materials
All data generated or analyzed during this study are included in this published article and its supplementary information files.
Health technology assessment
Choosing Wisely. http://www.choosingwisely.org/. Accessed 27th May 2019.
Elshaug AG, Hiller JE, Tunis SR, Moss JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007; https://doi.org/10.1186/1743-8462-4-23.
Wolfson D, Santa J, Slass L. Engaging physicians and consumers in conversations about treatment overuse and waste: a short history of the choosing wisely campaign. Acad Med. 2014. https://doi.org/10.1097/ACM.0000000000000270.
Levinson W, Born K, Wolfson D. Choosing Wisely Campaigns: A Work in Progress. JAMA. 2018. https://doi.org/10.1001/jama.2018.2202.
The Association of the Scientific Medical Societies in Germany (AWMF). https://www.awmf.org/en/health-care/gemeinsam-klug-entscheiden/manual-und-ausfuellhilfe.html. Accessed 30th July 2019.
Strech D, Follmann M, Klemperer D, Lelgemann M, Ollenschlager G, Raspe H, et al. When Choosing Wisely meets clinical practice guidelines. Z Evid Fortbild Qual Gesundhwes. 2014. https://doi.org/10.1016/j.zefq.2014.10.014.
Current Care Guidelines. http://www.kaypahoito.fi/web/english/home. Accessed 10th June 2019.
Sihvo S, Ikonen T, Makela M. Implementing Health Technology Assessment-Based Recommendations in Finland: Managed Uptake of Medical Methods. Int J Technol Assess Health Care. 2017. https://doi.org/10.1017/S0266462317000587.
Choices in Health Care. https://palveluvalikoima.fi/en/frontpage. Accessed 22nd November 2018.
AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12:18–23.
Ketola E, Kaila M, Honkanen M. Guidelines in context of evidence. Qual Saf Health Care. 2007. https://doi.org/10.1136/qshc.2006.019752.
Qaseem A, Forland F, Macbeth F, Ollenschlager G, Phillips S, van der Wees P, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012. https://doi.org/10.7326/0003-4819-156-7-201204030-00009.
Kuronen R, Jallinoja P, Patja K. Use of and attitudes toward current care guidelines among primary and secondary care nurses in Finland. Clin Nurs Res. 2011. https://doi.org/10.1177/1054773811407765.
Horvath K, Semlitsch T, Jeitler K, Abuzahra ME, Posch N, Domke A, et al. Choosing Wisely: assessment of current US top five list recommendations’ trustworthiness using a pragmatic approach. BMJ Open. 2016;6:e012366. https://doi.org/10.1136/bmjopen-2016-012366.
Admon AJ, Gupta A, Williams M, Valley TS, Sjoding MW, Wiener RS et al. Appraising the Evidence Supporting Choosing Wisely(R) Recommendations. J Hosp Med. 2018. https://doi.org/10.12788/jhm.2964.
Andrews JC, Schunemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA et al. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength. J Clin Epidemiol. 2013. https://doi.org/10.1016/j.jclinepi.2013.02.003.
Lin KW, Yancey JR. Evaluating the Evidence for Choosing WiselyTM in Primary Care Using the Strength of Recommendation Taxonomy (SORT). J Am Board Fam Med. 2016. https://doi.org/10.3122/jabfm.2016.04.160003.
Colla CH, Mainor AJ. Choosing Wisely Campaign: Valuable For Providers Who Knew About It, But Awareness Remained Constant, 2014-17. Health Aff (Millwood). 2017; https://doi.org/10.1377/hlthaff.2017.0945.
Rosenberg A, Agiro A, Gottlieb M, Barron J, Brady P, Liu Y, et al. Early Trends Among Seven Recommendations From the Choosing Wisely Campaign. JAMA Intern Med. 2015. https://doi.org/10.1001/jamainternmed.2015.5441.
Patey AM, Hurt CS, Grimshaw JM, Francis JJ. Changing behaviour 'more or less'-do theories of behaviour inform strategies for implementation and de-implementation? A critical interpretive synthesis. Implement Sci. 2018; https://doi.org/10.1186/s13012-018-0826-6.
Embrett M, Randall GE. Physician perspectives on Choosing Wisely Canada as an approach to reduce unnecessary medical care: a qualitative study. Health Res Policy Syst. 2018. https://doi.org/10.1186/s12961-018-0370-5.
Perk J, De Backer G, Gohlke H, Graham I, Reiner Z, Verschuren M, et al. European Guidelines on cardiovascular disease prevention in clinical practice (version 2012). The Fifth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of nine societies and by invited experts). Eur Heart J. 2012;33:1635–701.
Gibbons RJ, Abrams J, Chatterjee K, Daley J, Deedwania PC, Douglas JS, et al. ACC/AHA 2002 guideline update for the management of patients with chronic stable angina--summary article: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on the Management of Patients With Chronic Stable Angina). Circulation. 2003;107:149–58.
We wish to thank the CC editorial team and the CC guideline development group members for producing the Choosing Wisely recommendations and discussing the process critically.
The project was funded by The Ministry of Social Affairs and Health. Duodecim and the funding body jointly selected the guidelines for the project. The funding body had no role in the design of the study, collection, analysis, or interpretation of data, or in writing the manuscript.
Ethics approval and consent to participate
According to the Finnish legislation, an ethical approval was considered unnecessary because the manuscript describes a development project of working and guideline methods, and no personal data was gathered. (Laki lääketieteellisestä tutkimuksesta 9.4.1999/488 [Medical research act]).
No consent to participate was obtained because reviewing the guidelines was part of the salaried editors’ work duties. In the analysis the editors’ work was not personally identified.
Consent to publish
RS, JK and MM were employees of the Finnish Medical Society Duodecim during the project. The authors declare that they have no other competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Template and an example for a Choosing Wisely recommendation, Current Care Guidelines. A structured template for publishing Choosing wisely recommendations used in the project and one example of a Choosing Wisely Finland recommendation. (DOCX 19 kb)
A list of reviewed Current Care guidelines and accepted and rejected Choosing Wisely recommendations. A list of guidelines reviewed during the project, all drafted Choosing Wisely recommendations and reasons for rejection. (DOCX 38 kb)