A survey to assess awareness and opinion of initiatives and recommendations on low-value diagnostic practices
BMC Health Services Research volume 20, Article number: 505 (2020)
The need to reduce healthcare practices that provide no value has led to the development of initiatives that generate and publish recommendations to improve the appropriateness of clinical practice by identifying potentially inappropriate services, making recommendations, and proposing improvements. DianaHealth (www.dianahealth.com) identifies, classifies, and publishes recommendations from numerous scientific societies. The purpose of this study was to determine the awareness and perceived usefulness and applicability of published recommendations on low-value diagnostic measures, as judged by physicians who are recognised clinical leaders in their respective centres.
We designed a questionnaire on the diagnostic recommendations considered relevant for each medical specialty and made it available, until September 2016, on DianaHealth. The survey was administered online to clinical leaders from 25 Spanish healthcare centres (hospitals and primary care centres).
A total of 413 (40.0%) physicians from 34 different specialties participated. The participation rate varied between centres (range 21.1%-100.0%) and specialties (range 12.5%-78.9%). Do Not Do (57.1%) was the most widely-known initiative. Most participants (82.6%; IQR 77.9%-94.9%) stated that they knew at least one of the 12 initiatives that identify non-recommended practices, and on average they were aware of four initiatives (range 1-12). The initiatives were perceived useful by 82.4% (IQR 73.3%-90.4%), and perceived applicable by 75.6% (IQR 67.4%-86.8%). A total of 531 recommendations were assessed. Sixty-three percent (IQR 53.6%-77.5%) of participants reported they were aware of the recommendations for their corresponding specialty. A total of 84.5% (IQR 75.0%-90.0%) stated they agreed with the recommendations and 84.5% (IQR 75.0%-90.0%) considered them useful.
Among those who agreed with their respective recommendations, a median of 51.5% (IQR 41.4%-60.9%) perceived the guidelines as being fully implemented, 40.1% (IQR 31.9%-46.8%) considered them partially implemented, and 7.1% (IQR 3.7%-12.9%), not implemented.
Clinical leaders’ awareness of initiatives that generate and publish recommendations to improve clinical appropriateness remains low, although they did consider them useful. In general, participants were familiar with their speciality-specific diagnostic recommendations, agreed with them, and perceived them as useful and implemented in their centres. More needs to be done to raise awareness among professionals who do not know of or apply these recommendations.
Currently, there is a need to identify and reduce clinical practices of questionable value. Around 20% to 25% of clinical practices have been reported to provide no benefit to the patient and, therefore, present unnecessary risks, with an economic impact that could even exceed 20% of total healthcare costs .
Nowadays, optimising health care services is a worldwide major interest, and several recently-published studies have provided a reference framework for appropriate health care [2,3,4,5]. Likewise, multiple national and international initiatives that generate and publish recommendations have been developed to improve the appropriateness of care and clinical practice, by identifying potentially inappropriate services, making recommendations, and proposing actions and clinical alternatives, when available, to reduce inappropriate use [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20].
Most initiatives are conducted by groups of scientific societies who gather clinical recommendations . Choosing Wisely  is one such example that is active in several countries [12,13,14,15,16,17,18,19,20]. Other initiatives include a) collections of articles identifying low-value health care practices (e.g. the Less is More series in the Journal of the American Medical Association, JAMA ); b) ‘do not do’ recommendations (e.g. the Do Not Do recommendations from the National Institute for Health and Care Excellence, NICE ); c) appropriateness criteria such as those by the American College of Radiology ; and d) local projects to improve clinical practice (e.g. ESSENCIAL  by the Agency for Health Quality and Assessment of Catalonia (AQuAS), and the MAPAC initiative (in English, Improvement of Appropriateness in Clinical Practice and Healthcare, from the Spanish Mejora de la Adecuación de la Práctica Asistencial y Clínica) . In order to overcome the scattered fragmentation of recommendations generated by such initiatives, DianaHealth (www.dianahealth.com) identifies, classifies and publishes them to help clinicians identify the low-value clinical interventions in their particular field .
Few studies have been conducted to determine health professionals’ awareness and opinion of the various published recommendations on low-value clinical practices. A study among primary care physicians was conducted in the United States regarding the Choosing Wisely initiative . It found that 66% of participants were aware of the initiative and 97% of those who were acquainted with it considered it an appropriate source of information on low-value tests and interventions. A similar study was developed in Spain at a tertiary hospital regarding the “Do Not Do” recommendations and it found that 90% of the directors of clinical units agreed with the recommendations .
The objective of the current study was to determine the awareness of the above-mentioned initiatives and perceived usefulness and applicability in clinical practice of its recommendations on diagnostic practices, in physicians from different healthcare centres in Spain.
We conducted a multi-centre, cross-sectional study co-ordinated by the Service of Clinical Epidemiology and Public Health at the Hospital de Sant Pau in Barcelona, and the Comisión de Mejora de la Práctica Clínica (Committee for Improving Clinical Practice) of the Consorci Sanitari de Terrassa (Barcelona).
From April to September 2017, we conducted a survey aimed at clinical leaders (directors, heads of departments, and heads of clinical units or areas) from hospitals and primary care centres in Spain that had previously communicated an explicit interest to participate in the study. We invited only those professionals whose specialties included on the survey. We sent an email to participate on the study which included a link to the anonymous online questionnaire.
We selected recommendations based on the following criteria:
Inclusion criteria: recommendations on low-value diagnostic tests, published on DianaHealth from inception (2014) to September 10th, 2016.
Exclusion criteria: recommendations on non-diagnostic interventions, high-value recommendations, assessments with no explicit recommendation, recommendations with no specialty assigned, or nursing recommendations.
DianaHealth allows users to select recommendations by specialty, so we generated a list for each one. When a recommendation could be classified to more than one specialty, we included it in both lists. We excluded specialties with six or fewer low-value diagnostic recommendations. As a result, 34 specialties were selected.
Between 7 and 25 recommendations were selected for each speciality based on their clinical relevance, general interest, potential clinical impact, and applicability.
The questionnaire consisted of two parts (Additional file 1): the first contained general questions on different initiatives that generate and publish recommendations and the second contained specific questions related to each of the selected recommendations. The responses to the questions about awareness, agreement, usefulness, and applicability were classified as disagreement (“totally disagree” and “disagree”), neither agree nor disagree, and agreement (“agree" and "totally agree”).
The survey was designed and carried out in accordance with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) . Before starting, we tested the survey for usability and technical functionality. Each physician was invited to answer both sections of the questionnaire through the electronic platform Clinapsis®. The survey was open for three months, from May to July 2017. The time needed to complete the survey was around 10 to 20 minutes, depending on the number of recommendations for each speciality.
We conducted a descriptive data analysis for all the collected variables, expressed as percentage of respondents, using the software Stata version 14.0. Participation rate was calculated considering those clinical leaders that answered the survey divided by the target population (clinical leaders who were invited to answer the survey).
The study was approved by the Clinical Research Ethics Committee of both co-ordinating centres. No informed consent was required since the questionnaire was completed anonymously. Our online survey did not require a signed consent, because it was completely anonymous since it did not have any information that could be used to glean the identity of the participants. Nevertheless, the protocol of the study was presented to two ethics committees and they considered not necessary to be review by an ethical research committee since it was not a biomedical research.
Initially, 25 centres agreed to participate in the study, although 7 were excluded from the analysis: 2 because they did not reach the minimal participation rate (< 5%) and 5 because finally they decided not to take part of the survey. The majority of participating centres were hospitals (15), but three primary centres also participated (3). The target population consisted of 1,030 physicians, 413 of whom completed the survey (participation rate: 40.1%). Six centres had a participation rate exceeding 50% (Table 1). If we would have considered the data of the two centres that were excluded the participation rate would have been similar (36.9%).
The median participation rate per specialty was 41.3% (range 12.5%-78.9%) (Table 2). Nine specialties had a participation rate over 50.0% (rheumatology, dermatology, nuclear medicine, medical oncology, internal medicine, occupational medicine, geriatrics, radiology and haematology); in two specialties less than 20.0% responded (anaesthesiology and urology).
A total of 82.6% of participants (IQR 77.9%-94.9%) reported that they knew of at least one of the 12 initiatives identifying non-recommended practices included in DianaHealth (Table 3 and Fig. 1). The mean number of known initiatives per specialty was 4 (range 1-12), the Do Not Do and Too Much Medicine initiatives being the most widely-known, with 57.1% and 54.5% or respondents, respectively, being aware of them (Fig. 1). Most initiatives were familiar to fewer than 30% of participants. Regarding national initiatives, Essencial, No Hacer (by the Spanish Society of Family and Community Medicine), and MAPAC were the most widely-known initiatives with awareness rates of 32.0%, 31.5%, and 24.5%, respectively.
Overall, perceived usefulness and applicability to clinical practice of the different initiatives was high: 82.4% (IQR 73.3%-90.4%) and 75.6% (IQR 67.4%-86.8%), respectively (Table 3). However, in four specialties (cardiology, haematology, immunology and allergy, and medical oncology) less than half of the participants considered the initiatives useful or applicable in their clinical practice.
Overall, a median of 63.1% of respondents were aware of the recommendations for their specialty (IQR 53.6%-77.5%) (Table 4). The three specialties with the highest awareness were endocrinology, anaesthesiology and radiology.
The overall median agreement with recommendations was 84.5% (IQR 75.0%-90.0%). The five specialties with the highest reported agreement were nuclear medicine otorhinolaryngology, radiology, family medicine and neurology (Table 4).
The recommendations were perceived as useful by a median of 84.5% overall (IQR 75.0%-90.0%). The specialists that considered the recommendations more useful were those of otorhinolaryngology, radiology and microbiology (Table 4).
Among those specialists who agreed with the recommendations, for all specialties, a median of 51.5% (IQR 41.4%-60.9%) perceived their department as being fully-compliant, and 40.1% (IQR 31.9%-46.8%) perceived partial compliance (Fig. 2). Dermatology, paediatrics, and immunology and allergy had the highest perception of full compliance. A median 7.1% perceived their department as non-compliant with the recommendations (IQR 3.7%-12.9%).
This study aimed to determine clinical leaders’ awareness, agreement and interest regarding existing initiatives to reduce low-value clinical practices as well as the published recommendations specific to each specialty.
In this first approach we decided to focus our survey on recommendations on diagnostic practices and leave the development of a similar survey on therapeutic aspects for future studies.
The DianaHealth portal made obtaining the existing recommendations very easy. This open-access portal, available in English and Spanish, periodically classifies the recommendations generated from national and international initiatives aimed at reducing low-value clinical practices and improving clinical appropriateness .
Survey participation was not particularly high (40.1%, or 47.4% with the exclusion of the centre that invited all medical staff instead of only clinical leaders), although this is in line with results from other similar surveys. The wide variability of participation across the different centres could reflect the presence or absence of active institutional policies to reduce low-value practices. The participation rate for the different specialties was probably influenced to some extent by the specific characteristics of each centre.
Around 60% of the participants reported that they knew the diagnostic recommendations for their specialty, with substantial variability between specialties. Three-quarters of the participants agreed with the recommendations and considered them useful. It would be interesting for future studies to analyse in depth the differences between specialties regarding perceived usefulness. Around 50% of the participants estimated that professionals at their centre always complied with the recommendations, 40% estimated partial compliance, and less than 10% estimated non-compliance, with some variability between specialties.
Despite the many initiatives and projects generating recommendations to reduce low-value clinical practices, very few studies have been conducted to assess target physicians’ perception of these recommendations. Our survey is the first to provide an overall assessment of existing local, national and global initiatives, by evaluating awareness and perceived usefulness. DianaHealth has proven to be a very valuable resource for identifying potential questionable practices.
One of the potential limitations of our study is that participation was lower than desired. Nevertheless, our response rate was in relation to has been described for other web surveys [25, 26]. Most of the participating centres were in Catalonia, because the survey coordinators were based there, and also because most of such initiatives have been developed there. These local initiatives are generally promoted in hospitals, which is why fewer primary care professionals were invited to participate. Another limitation of our study is a potential self-selection bias because of participants who answered the survey could be those more aware and familiar with the importance of preventing low-value care.
The response rate was quite good (40%) in relation to what is described for web surveys [25, 26]. The research group followed the methodological steps of this type of survey to improve the response: make a pilot of the survey (to assess the design, order and duration), make the invitation to participate, send a reminder and ensure easy access to the questionnaire . However, no incentive was offered.
Overall, there is insufficient awareness among Spanish clinical leaders regarding worldwide initiatives to improve health care appropriateness, despite perceived usefulness being high. Furthermore, according to the participants, diagnostic recommendations specific to each specialty are widely known, agreed with and adhered to at each centre, which is very encouraging. It seems necessary for such initiatives, which periodically generate recommendations on reducing low-value practices, to develop additional knowledge-translation actions addressed at the target physicians to improve awareness and compliance.
Availability of data and materials
The datasets generated and analysed during the current study are included in the Clinapsis® electronic platform repository, but restrictions apply to the availability of these data, and they are not publicly available. Data are however available from the authors upon reasonable request.
Journal of the American medical association
National institute for health and care excellence
In Spanish, “Mejora de la Adecuación de la Práctica Asistencial y Clínica”; in English, Improvement of Appropriateness in Clinical Practice and Healthcare
Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513–6.
Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, Nagpal S, Saini V, Srivastava D, Chalmers K, Korenstein D. Evidence for overuse of medical services around the world. Lancet. 2017;390(10090):156–68.
Glasziou P, Straus S, Brownlee S, Trevena L, Dans L, Guyatt G, Elshaug AG, Janett R, Saini V. Evidence for underuse of effective medical services around the world. Lancet. 2017;390(10090):169–77.
Saini V, Garcia-Armesto S, Klemperer D, Paris V, Elshaug AG, Brownlee S, Ioannidis JPA, Fisher ES. Drivers of poor medical care. Lancet. 2017;390(10090):178–90.
Elshaug AG, Rosenthal MB, Lavis JN, Brownlee S, Schmidt H, Nagpal S, Littlejohns P, Srivastava D, Tunis S, Saini V. Levers for addressing medical underuse and overuse: achieving high-value health care. Lancet. 2017;390(10090):191–202.
National Institute for Health and Care Excellence [Internet]. Savings and productivity collection: Do not do recommendations. 2016. Available: https://www.nice.org.uk/about/what-we-do/our-programmes/cost-savings-resource-planning.
Choosing Wisely.org. [Internet]. ABIM Foundation. [cited 2016 Sept 1]. Available: http://www.choosingwisely.org/.
Ministerio de Sanidad, Servicios Sociales e Igualdad [Internet]. Compromiso por la Calidad de las Sociedades Científicas. [cited 2016 Sept 1]. Available: http://www.msc.es/organizacion/sns/planCalidadSNS/cal_sscc.htm.
Agència d’Avaluació i Qualitat Sanitàries de Catalunya (AquAS). [Internet]. Essencial, Afegint valor a la pràctica clínica [cited 2016 Sept 1]. Available: http://essencialsalut.gencat.cat/ca/.
The JAMA Network. [Internet]. Less is more collection [cited 2016 Sept 1]. Available: https://jamanetwork.com/collections/44045/less-is-more.
U.S. Preventive Services Task Force. [Internet]. U.S. Preventive Services Task Force A and B recommendations. [cited 2016 Sept 1]. Available: http://www.uspreventiveservicestaskforce.org/Page/Name/uspstf-a-and-b-recommendations/.
Choosing Wisely Canada.org. [Internet]. Canadian Medical Association, University of Toronto. [cited 2016 Sept 1]. Available: http://www.choosingwiselycanada.org/.
Elshaug AG, Watt AM, Mundy L, et al. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2012;197(10):556–60.
Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, Chacko SJ, Borkar D, Gall V, Selvaraj S, Ho N, Cifu A. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc. 2013;88(8):790–8.
American College of Radiology. [Internet]. The ACR Appropriateness Criteria. [cited 2016 Sept 1]. Available: http://www.acr.org/Quality-Safety/Appropriateness-Criteria.
The Canadian Task Force for Preventive Health Care. [Internet]. The CTFPHC Guidelines. [cited 2016 Sept 1]. Available: https://canadiantaskforce.ca/.
The British Medical Journal. [Internet]. Too Much Medicine. [cited 2016 Sept 1]. Available: http://www.bmj.com/specialties/too-much-medicine.
Sociedad Española de Medicina de Familia y Comunitaria semFYC. [Internet]. Proyecto Recomendaciones «No hacer». [cited 2016 Sept 1]. Available: http://e-documentossemfyc.es/recomendacion-para-no-hacer-de-la-sociedad-espanola-de-medicina-de-familia-y-comunitaria/.
Sociedad Española de Radiología Médica (SERAM). Recomendaciones de No Hacer. [cited 2016 Sept 1]. Available: http://seram.es/modules.php?name=documentos&op=getDocument&iddocument=397.
ChoosingWiselyAustralia.org.au. [Internet]. NPS MedicineWise: Choosing Wisely Australia. [cited 2016 Sept 1]. Available: http://www.choosingwisely.org.au/about-choosing-wisely-australia.
Bonfill X, Osorio D, Solà I, Pijoan JI, Balasso V, Quintana MJ, Puig T, Bolibar I, Urrútia G, Zamora J, Emparanza JI, de la Gómez C. Ferreira-González I. DianaHealth.com, an on-line database containing appraisals of the clinical value and appropriateness of healthcare interventions: database development and retrospective analysis. PLoS One. 2016;11(2):e0147943.
Buist DS, Chang E, Handley M, Pardee R, Gundersen G, Cheadle A, Reid RJ. Primary Care Clinicians' Perspectives on Reducing Low-Value Care in an Integrated Delivery System. Perm J. 2016;20(1):41–6.
Martin S, Miñarro R, Cano P, Aranda JM. Applicability of "do not do recommendations" from the National Institute for Health and Care Excellence in a quaternary care hospital. Rev Calid Asist. 2015;30(3):117–28.
Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3):e34.
Cantuaria ML, Blanes-Vidal V. Self-reported data in environmental health studies: mail vs. web-based surveys. BMC Med Res Methodol. 2019;19(1):238.
Fan W, Yan Z. Factors affecting response rates of the web survey: A systematic review. Comput Hum Behav. 2010;26(2):132–9.
Group authorship: collaborating author names of DianaHealth Study Group Members:
Xavier Bonfill, M Jesús Quintana, Carolina Requeijo, Karla Salas, Ángela M Merchán. Hospital de Sant Pau (Barcelona), CIBERESP.
Antonio Sánchez, Elena Medarde. Consorci Sanitari de Terrassa (Terrassa).
Dimelza Osorio, Soledad Romea. Hospital Universitari Vall d’Hebron (Barcelona).
Francisco Baigorri. Corporació Sanitària Parc Taulí (Sabadell).
Agustín Urrútia. Hospital Universitari Germans Trías i Pujol (Badalona).
Josep Lluis Vega. Hospital Sant Joan de Déu (L’Hospitalet de Llobregat).
Pedro Armario. Hospital de Sant Joan Despí Moisès Broggi (Sant Joan Despí).
Matteo Fabbi. Hospital Universitari General de Catalunya (Sant Cugat del Vallès).
Anna Carol Pérez Segarra. Hospital Sant Rafael (Barcelona).
Xavier Martret. Parc Sanitari Sant Joan de Déu (Sant Boi de Llobregat).
Miquel Vila; Hospital General de Granollers (Granollers).
Marta Banqué. Yolima Cossio; Consorci Sanitari l’Anoia (Igualada).
Javier Zamora, Jesús López Alcalde, Alfonso Muriel. Hospital Universitario Ramón y Cajal (Madrid).
José Ignacio Emparanza, Iratxe Urreta. Hospital Universitario Donostia (Donostia).
José Ignacio Pijoan, Amaia Martínez Galarza. Hospital Universitario Cruces (Barakaldo), CIBERESP.
Agustín Gómez de la Cámara. Hospital Universitario 12 de Octubre (Madrid), CIBERESP.
Ana Royuela. Hospital Puerta de Hierro (Majadahonda).
Blanca Lumbreras. Hospital Universitari Sant Joan d’Alacant (Alicante).
Pere Plaja. Hospital Fundació Salut Empordà (Figueres).
Antoni Peris. Consorci Castelldefels Agents de Salut d'Atenció Primària (Castelldefels).
Carlos Brotons. Equip d’Atenció Primària Sardenya (Barcelona).
Montserrat Ureña. Centres d’Atenció Primaria, Consorci Sanitari de Terrassa (Terrassa).
Joan Fernández Náger. Equip d’Atención Primaria Penitenciari de Lledoners (Sant Joan de Vilatorrada).
We would also like to thank Mrs. Andrea Cervera Alepuz for translating and Dr. M Jane Marshall for editing the manuscript.
The development of the online survey was partially funded by the MAPAC program of the Clinical Epidemiology Program of the Centre for Biomedical Research in Epidemiology and Public Health Network (CIBERESP), Institute of Health Carlos III. Spain.
Ethics approval and consent to participate
No informed consent was required since the questionnaire was completed anonymously. Nevertheless, the protocol of the study was presented to two Clinical Research Ethics Committees (Hospital de Sant Pau and Consorci Sanitari de Terrassa) at it was not considered biomedical research since it was not carried out on patients, did not use biological samples, nor did it contemplate the recording of clinical data, and therefore it was not necessary to review it by an ethical research committee.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bonfill, X., Salas-Gama, K., Requeijo, C. et al. A survey to assess awareness and opinion of initiatives and recommendations on low-value diagnostic practices. BMC Health Serv Res 20, 505 (2020). https://doi.org/10.1186/s12913-020-05286-3