Skip to main content
  • Oral presentation
  • Open access
  • Published:

Characterization and analysis of guideline implementation tools (GItools) reveals opportunities for improving health service planning, delivery and quality improvement


Guidelines inform health care planning, delivery and quality improvement but are not consistently implemented [13]. Research shows that guidelines were more likely to be used when accompanied by guideline implementation tools (GItools), but few guidelines offered GItools [46]. Interviews with guideline developers and analysis of guideline instructional manuals revealed a need for information to support GItool development [79]. First it is necessary to characterize GItools. The purpose of this research was to generate a framework of desirable GItool features, and use the framework to describe a sample of GItools.

Materials and methods

Items representing desirable GItool features were first generated by a cross-sectional survey of the international guideline community [10, 11]. Then items were confirmed and refined by a panel of guideline developers, implementers and researchers in a two-round Delphi survey [1214]. The resulting GItool framework was applied to describe a sample of GItools of various types, accompanying guidelines identified in the National Guideline Clearinghouse on various clinical topics produced within five years by organizations having developed at least ten guidelines.


The cross-sectional survey was completed by 96 respondents from Australia, Canada, United Kingdom, United States, The Netherlands, and several other countries. Nine items were rated by most as desirable but difficult to achieve given limited resources and a perceived imperative to make GItools accessible even if not rigorously developed or evaluated. Thirty-one panelists from ten countries including Australia, Canada, Germany, New Zealand, Peru, Saudi Arabia, Spain, United Kingdom, and the United States took part in a two-round Delphi survey. Twelve items achieved consensus as desirable GItool features. A total of 13 GItools were identified among a sample of 149 guidelines (8.7%). Most GItools named target users (92.3%) and described development methods (84.6%). Fewer possessed other features considered desirable such as instructions for use (61.5%), sources of content (61.5%), target users were involved in development (53.8%), underlying evidence identified (23.1%), evaluation described (7.7%), or pilot-tested with target users (0.0%).


Further work is needed to validate the framework with guideline users, and share the GItool framework with guideline developers. It can serve as the basis for evaluating and adapting existing GItools, or developing new GItools. Inclusion of higher quality GItools with more guidelines may support implementation and use of guidelines by target users, ultimately leading to improved care delivery and associated outcomes.


  1. Shekelle P, Woolf S, Grimshaw JM, Schunemann H, Eccles MP: Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012, 7: 62-

    Article  PubMed Central  PubMed  Google Scholar 

  2. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. NEJM. 2003, 348: 2635-2645.

    Article  PubMed  Google Scholar 

  3. Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, West P, Wright D, Wright J: What’s the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients’ notes, and interviews. BMJ. 2004, 329: 999-

    Article  PubMed Central  PubMed  Google Scholar 

  4. Shekelle PG, Kravitz RL, Beart J, Marger M, Wang M, Lee M: Are non-specific guidelines potentially harmful? A randomized comparison of the effect of nonspecific versus specific guidelines on physician decision making. Health Serv Res. 2000, 34: 1429-1448.

    PubMed Central  CAS  PubMed  Google Scholar 

  5. Grilli R, Lomas J: Evaluating the message: The relationship between compliance rate and the subject of a practice guideline. Med Care. 1994, 32: 202-213.

    Article  CAS  PubMed  Google Scholar 

  6. Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM: How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011, 6: 26-

    Article  PubMed Central  PubMed  Google Scholar 

  7. Gagliardi AR: “More bang for the buck”: exploring optimal approaches for guideline implementation through interviews with international developers. BMC Health Serv Res. 2012, 12: 404-

    Article  PubMed Central  PubMed  Google Scholar 

  8. Gagliardi AR, Brouwers MC: Integrating guideline development and implementation: Analysis of guideline development manual instructions for generating implementation advice. Implement Sci. 2012, 7: 67-

    Article  PubMed Central  PubMed  Google Scholar 

  9. Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T, Zhang Y, Raid U, Neumann I, Norris SL, Thornton J, Harbour R, Treweek S, Guyatt G, Alonso-Coello P, Reinap M, Brozek J, Oxman A, Akl EA: Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014, 186: E123-142.

    Article  PubMed Central  PubMed  Google Scholar 

  10. Streiner DL, Norman GR: Health measurement scales. A practical guide to their development and use. 2003, Oxford (UK): Oxford University Press

    Google Scholar 

  11. Kelley K, Clark B, Brown V, Sitzia J: Good practice in the conduct and reporting of survey research. Int J Qual Health Care. 2003, 15: 261-266.

    Article  PubMed  Google Scholar 

  12. Jones J, Hunter D: Consensus methods for medical and health services research. BMJ. 1995, 311: 376-380.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  13. Fink A, Kosecoff J, Chassin M, Brook RH: Consensus methods: Characteristics and guidelines for use. Am J Public Health. 1984, 74: 979-983.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  14. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C: Using and reporting Delphi method for selecting healthcare quality indicators: a systematic review. PLOS One. 2011, 6: e20476-

    Article  PubMed Central  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations


Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gagliardi, A.R. Characterization and analysis of guideline implementation tools (GItools) reveals opportunities for improving health service planning, delivery and quality improvement. BMC Health Serv Res 14 (Suppl 2), O8 (2014).

Download citation

  • Published:

  • DOI: