Characterization and analysis of guideline implementation tools (GItools) reveals opportunities for improving health service planning, delivery and quality improvement
BMC Health Services Research volume 14, Article number: O8 (2014)
Guidelines inform health care planning, delivery and quality improvement but are not consistently implemented [1–3]. Research shows that guidelines were more likely to be used when accompanied by guideline implementation tools (GItools), but few guidelines offered GItools [4–6]. Interviews with guideline developers and analysis of guideline instructional manuals revealed a need for information to support GItool development [7–9]. First it is necessary to characterize GItools. The purpose of this research was to generate a framework of desirable GItool features, and use the framework to describe a sample of GItools.
Materials and methods
Items representing desirable GItool features were first generated by a cross-sectional survey of the international guideline community [10, 11]. Then items were confirmed and refined by a panel of guideline developers, implementers and researchers in a two-round Delphi survey [12–14]. The resulting GItool framework was applied to describe a sample of GItools of various types, accompanying guidelines identified in the National Guideline Clearinghouse on various clinical topics produced within five years by organizations having developed at least ten guidelines.
The cross-sectional survey was completed by 96 respondents from Australia, Canada, United Kingdom, United States, The Netherlands, and several other countries. Nine items were rated by most as desirable but difficult to achieve given limited resources and a perceived imperative to make GItools accessible even if not rigorously developed or evaluated. Thirty-one panelists from ten countries including Australia, Canada, Germany, New Zealand, Peru, Saudi Arabia, Spain, United Kingdom, and the United States took part in a two-round Delphi survey. Twelve items achieved consensus as desirable GItool features. A total of 13 GItools were identified among a sample of 149 guidelines (8.7%). Most GItools named target users (92.3%) and described development methods (84.6%). Fewer possessed other features considered desirable such as instructions for use (61.5%), sources of content (61.5%), target users were involved in development (53.8%), underlying evidence identified (23.1%), evaluation described (7.7%), or pilot-tested with target users (0.0%).
Further work is needed to validate the framework with guideline users, and share the GItool framework with guideline developers. It can serve as the basis for evaluating and adapting existing GItools, or developing new GItools. Inclusion of higher quality GItools with more guidelines may support implementation and use of guidelines by target users, ultimately leading to improved care delivery and associated outcomes.
Shekelle P, Woolf S, Grimshaw JM, Schunemann H, Eccles MP: Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012, 7: 62-
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. NEJM. 2003, 348: 2635-2645.
Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, West P, Wright D, Wright J: What’s the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients’ notes, and interviews. BMJ. 2004, 329: 999-
Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are non-specific guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000, 34: 1429-1448.
Grilli R, Lomas J: Evaluating the message: The relationship between compliance rate and the subject of a practice guideline. Med Care. 1994, 32: 202-213.
Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011, 6: 26-
Gagliardi AR: “More bang for the buck”: exploring optimal approaches for guideline implementation through interviews with international developers. BMC Health Serv Res. 2012, 12: 404-
Gagliardi AR, Brouwers MC: Integrating guideline development and implementation: Analysis of guideline development manual instructions for generating implementation advice. Implement Sci. 2012, 7: 67-
Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T, Zhang Y, Raid U, Neumann I, Norris SL, Thornton J, Harbour R, Treweek S, Guyatt G, Alonso-Coello P, Reinap M, Brozek J, Oxman A, Akl EA: Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014, 186: E123-142.
Streiner DL, Norman GR: Health measurement scales. A practical guide to their development and use. 2003, Oxford (UK): Oxford University Press
Kelley K, Clark B, Brown V, Sitzia J: Good practice in the conduct and reporting of survey research. Int J Qual Health Care. 2003, 15: 261-266.
Jones J, Hunter D: Consensus methods for medical and health services research. BMJ. 1995, 311: 376-380.
Fink A, Kosecoff J, Chassin M, Brook RH: Consensus methods: Characteristics and guidelines for use. Am J Public Health. 1984, 74: 979-983.
Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting Delphi method for selecting healthcare quality indicators: a systematic review. PLOS One. 2011, 6: e20476-
About this article
Cite this article
Gagliardi, A.R. Characterization and analysis of guideline implementation tools (GItools) reveals opportunities for improving health service planning, delivery and quality improvement. BMC Health Serv Res 14 (Suppl 2), O8 (2014). https://doi.org/10.1186/1472-6963-14-S2-O8