Insights about the process and impact of implementing nursing guidelines on delivery of care in hospitals and community settings
© Davies et al; licensee BioMed Central Ltd. 2008
Received: 19 April 2007
Accepted: 02 February 2008
Published: 02 February 2008
Little is known about the impact of implementing nursing-oriented best practice guidelines on the delivery of patient care in either hospital or community settings.
A naturalistic study with a prospective, before and after design documented the implementation of six newly developed nursing best practice guidelines (asthma, breastfeeding, delirium-dementia-depression (DDD), foot complications in diabetes, smoking cessation and venous leg ulcers). Eleven health care organisations were selected for a one-year project. At each site, clinical resource nurses (CRNs) worked with managers and a multidisciplinary steering committee to conduct an environmental scan and develop an action plan of activities (i.e. education sessions, policy review). Process and patient outcomes were assessed by chart audit (n = 681 pre-implementation, 592 post-implementation). Outcomes were also assessed for four of six topics by in-hospital/home interviews (n = 261 pre-implementation, 232 post-implementation) and follow-up telephone interviews (n = 152 pre, 121 post). Interviews were conducted with 83/95 (87%) CRN's, nurses and administrators to describe recommendations selected, strategies used and participants' perceived facilitators and barriers to guideline implementation.
While statistically significant improvements in 5% to 83% of indicators were observed in each organization, more than 80% of indicators for breastfeeding, DDD and smoking cessation did not change. Statistically significant improvements were found in > 50% of indicators for asthma (52%), diabetes foot care (83%) and venous leg ulcers (60%). Organizations with > 50% improvements reported two unique implementation strategies which included hands-on skill practice sessions for nurses and the development of new patient education materials. Key facilitators for all organizations included education sessions as well as support from champions and managers while key barriers were lack of time, workload pressure and staff resistance.
Implementation of nursing best practice guidelines can result in improved practice and patient outcomes across diverse settings yet many indicators remained unchanged. Mobilization of the nursing workforce to actively implement guidelines and to monitor the delivery of their care is important so that patients may learn about and receive recommended healthcare.
The gap between research evidence and clinical practice is one of the most persistent problems in the provision of quality health care. Approximately 30 to 40% of patients do not receive health care according to current scientific evidence and some patients receive unnecessary or harmful care [1, 2]. In primary care, only 3% to 49% of patients receive recommended care according to a systematic review of studies of quality of care . As one strategy to help health professionals and decision-makers address this gap, professional and government organizations are producing clinical guidelines with credibly appraised summaries of research results and recommendations for health care. The Guidelines International Network (GIN) was founded in 2002 and by 2004 there were 52 member organizations from 27 countries yet only two members were nursing organizations: The Joanna Briggs Institute for Evidence Based Nursing & Midwifery in Australia and the Royal College of Nursing Institute in the United Kingdom . In the United States, The Agency for Healthcare Research and Quality (AHRQ) sponsors the National Guideline Clearinghouse which contains 105 nursing guidelines developed by 14 organizations in Canada, China, the United Kingdom and the United Sates .
One might question whether or not discipline-specific guidelines are required. We take the position that while health care is delivered to patients from a multidisciplinary perspective, the responsibilities of each team member need to be informed by the best available research evidence and some of this evidence is discipline-specific. The nursing profession represents a large proportion of health care providers and a potential workforce to address gaps in the provision of recommended health care.
The effectiveness of guideline implementation in nursing and allied health professions has received relatively little research attention. A Cochrane review summarized the results of 18 intervention studies of which 13 included physicians and nurses, and only 4 specifically targeted nurses . The authors concluded that there is limited evidence supporting the effectiveness of guidelines for improved care in "professions allied to medicine" . A systematic review of research evidence in nursing was conducted but only four studies met the inclusion criteria and the authors concluded that little is known about how to increase research use in nursing .
In contrast, there is a considerable body of physician-oriented and multi-disciplinary research with 235 studies reviewed by Grimshaw and colleagues (2004) of the effectiveness and costs of guideline development, dissemination and implementation strategies . Physicians were the focus in 74% of these studies while the other studies targeted a multidisciplinary team. Results were promising in that eighty-six percent of studies reported improvements in care, however there were substantial variations in the results. Grimshaw and team concluded that further research is needed to support decisions about which guideline dissemination and implementation strategies are likely to be effective. No differences were found in effects according to the number of elements such as reminders, education, audit and feedback although Grimshaw et al. and others have suggested that using multifaceted strategies may be more effective than single interventions since they address multiple barriers to implementation [8–10]. While these systematic reviews help to advance our understanding of the potentially potent ingredients of guideline implementation programs, they summarize evidence from highly controlled research studies with targeted guidelines and pre-planned implementation settings. However, provincial government decision-makers who are determining whether or not to invest in nursing best practice guideline implementation programs, also need evaluation results from naturalistic implementation programs in diverse settings where decisions regarding which guidelines should be implemented and which recommendations from each guideline is a priority for implementation are determined by practitioners rather than by researchers.
In nursing, there are few empirical studies about the quality of care and it is crucial to examine the interrelationships between evidence-based nursing care processes and outcomes . A state of the science review of nursing-sensitive outcomes describes a rapidly developing field yet many unresolved issues including what, when and how to measure indicators of nursing care . Data about nursing practice is needed to answer questions about health human resource requirements and what nurses need to do to maximize their impact on the delivery of health care and patient outcomes.
In summary, there is a very limited evidence base for either the design of nursing guideline implementation interventions or for measuring the process and outcome of nursing care. An opportunity arose with the production of new nursing guidelines for a naturalistic study that examined implementation of various guidelines across settings to provide insights which are often absent from other studies where the protocol is tightly prescribed, only one guideline is implemented and it is implemented across homogeneous settings. The context for the study is described below.
Context: The Registered Nurses' Association of Ontario Initiative
For the past seven years, the Registered Nurses' Association of Ontario (RNAO) in Canada has been leading the development and implementation of nursing best practice guidelines funded by the provincial government. Priority topics are selected using focus groups, surveys and an open web-based invitation to nurses. An expert panel is convened for each topic with representatives from different health care sectors (hospital, public health, long-term care), and responsibilities (administration, education, practice, research). Guidelines undergo extensive review by multidisciplinary stakeholders. Further details about the methodology for guideline development are described elsewhere [13, 14]. To date, 30 guidelines have been published in English with twelve guidelines also translated into French . Each guideline includes recommendations for practice such as patient assessment, patient education, medication monitoring, supportive care and referral to community services
We report on the real-time implementation of six guidelines during 2002–2004: asthma, breastfeeding, delirium-dementia-depression (DDD), foot complications in diabetes, smoking cessation and venous leg ulcers . Specific objectives of this evaluation study were to document the process of best practice guideline implementation by topic, to describe facilitators and barriers to implementation and to determine the impact of indicators related to process and patient outcomes. The evaluation of these six guidelines with respect to patient referrals is reported elsewhere .
A one-year prospective before and after implementation project was conducted with three months allotted for the baseline data collection, six months for implementation and three months for the follow-up data collection. The one-year time-frame was determined by the requirements of the government contract funding. The approach used was to evaluate guideline implementation as it naturally occurred in clinical settings. Ethics approval was received from the University of Ottawa Health and Social Sciences Ethics Review Board and from participating organizations. Following a call for proposals and a peer review process, eleven health care organisations were selected to implement one of the six guidelines. Organizations selected included four acute care teaching hospitals, two community hospitals, one chronic care hospital, one mental health teaching hospital, two home visiting nursing agencies and one regional public health unit. Sites began implementation of four topics in 2002 (asthma, breastfeeding, delirium-dementia-depression, and smoking cessation) and two topics in 2003 (foot complications in diabetes, and venous leg ulcers).
Characteristics of organisations participating in the implementation of nursing guidelines
Best Practice Guidelines
Delirium, Dementia, Depression
Diabetes Foot Care
Venous Leg Ulcers
Type and Number of Hospitals
Teaching Mental Health (1)
Community Chronic Care (1)
Emergency, Urgent care, Medicine
Labour & Delivery, Post-partum
Surgery, Medicine, Rehabilitation
Regional Public Health Unit (1)
Home visiting (1)
Home visiting (1) Wound clinics (3)
Number of Clinical Resource Nurses (CRN's)
1 full-time manager as CRN, 7 APNs†
1 full-time manager as CRN, 7 APN's†
Number of nurses educated
Initial workshops coordinated by the RNAO were held with three to five key stakeholders from each site (e.g. CRN, manager, administrator). Leaders from each guideline development panel gave structured presentations about the guideline recommendations with the supporting research evidence. A Toolkit [17, 18] with a two-hour training session describing strategies for implementation of clinical practice guidelines was provided. The Toolkit was developed by an expert panel including members who have published on evidence-based nursing practice and theoretically-informed guideline implementation research [19–22]. The Toolkit was based on a model with six essential elements: guideline identification; stakeholder identification, assessment and engagement; environmental readiness; use of effective implementation strategies; evaluation of guideline implementation; and identification of resource requirements .
Bolded columns indicate guideline implementation that resulted in no change in > 80% of indicators
Delirium, Dementia, Depression (DDD)
Diabetes Foot Care
Venous Leg Ulcers (VLU)
Educational strategies for nurses
2 core sessions of 2 hours each for separate groups of nurses-inpatient and emergency
Articles about the project in internal newsletters
Placebos to practice skills
Core 4-hour training session off site
Written material team teaching, non-didactic, focus on attitudes, beliefs and values, use of stories by mother/baby dyads
Additional short in-services held
Powerpoint slides, facilitator guide, handouts, case studies, a game to review materials, standardized assessment tools
Hospital: 1 session 30 to 60 minutes, as a lunch and learn, handouts, self-learning package
Visiting nurses: 6 sessions for 1.5 hours each, practice sessions, word game, refresher training
2 hours, Powerpoint slides, informal and interactive, stages of change theory
Use of case scenarios depending on group e. g. in- patient, outpatient, long term care
Manual (basic wound and VLU care) and CD for self-directed learning, individual 2-hour session with quiz, demonstration and bandaging practice by nurses Discussion of newsletters (not mailed)
hospital nurses had demonstrations on bandaging and products
Champions (Local opinion leaders)
Encouraged stronger nurses to sign up early to be advocates and mentors
Champions on each unit, part of the steering committee with role to raise issues in day to day rounds and to encourage the nurses to use the recommended assessment tools
A senior leader physician champion
Several clinical resource nurses with one on each unit
Resource people trained in both community and hospital settings
Mentoring by consultants at client's home
Logo, mugs, posters, name tags for nurses who completed the training
Posters, binders, pocket cards listing symptoms of DDD
Project logo, posters, articles in newsletters, voicemail messages, special flyer
Buttons, posters, laminated pocket cards summarizing the steps of ask, advise, and assist strategy
Smoking room policy changes
Creation of new documentation
Newborn critical pathways chart
New charting tools and discharge sheets
Trigger questions added to initial pt assessment forms to help nurses maintain an index of suspicion
New assessment tool
Respirologists very supportive but emergency physicians reluctant due to concern for nurses' workload
"Little involvement of other disciplines"
Conflict with dieticians in public health unit about 6 months excusive breast feeding
Steering group formed with both hospital and community reps but did not have an active ongoing role
Strong senior physician champion 'interdisciplinary work was amazing...the camaraderie between the disciplines and meeting everyone else from the different sites was one of the major benefits"
Community physician support
Patient education toolkits with placebos, teaching booklets and laminated cards on all units
Patient education and referral resources
Patient education brochure initiated but not completed
Process and Patient Outcomes
In order to select key outcomes, a rapid three-way consultation occurred with the leader of the specific guideline development panel, decision-makers from each implementation site and a member of the research team. Criteria guiding final selection of outcomes included: outcomes with the strongest research evidence, outcomes feasible to measure in a short time frame, and outcomes relevant to priorities at participating organizations. Chart abstraction forms were adapted from tools used in previous research when available [23–27] and assessed for face validity and relevance by the decision-makers and research team.
Consecutive patient charts that met the inclusion/exclusion criteria were audited pre- and post-implementation. In addition, for one topic (asthma), patient techniques for use of medication inhaler device were observed . In four out of six topics (asthma, breast-feeding, diabetes foot care, and smoking cessation), patients were interviewed in hospital or at home about recent nursing care and again, either four or eight weeks later by telephone, about their access to specialist and community resources . For the topic of delirium-dementia-depression, participating units were concerned about ethical issues and the feasibility of obtaining informed proxy consent from family members within the three-month time frame for baseline data collection. The CRNs described a vulnerable population with many clients unable to give consent, substitute decision-makers difficult to access and some clients with no substitute decision makers. For the venous leg ulcers guideline, the home visiting nurse agency was willing to assist with the chart audit process but declined to participate in obtaining client interviews due to limited time and multiple obligations.
Best Practice Guideline Topics: Eligibility Criteria, Sources of Data, Response Rates
Sources of Data
Chart Audit N
Patient Interview In Hospital – N, (% of chart audits)
Patient Telephone Interview at home N, % of 1st Interview
Adults with a previous diagnosis of asthma or are carrying a puffer.
Decrease likely due to seasonal variations
Pre 31 38.8%
Post 14, 29.2%
Most seen in Emergency, few admitted to hospital
Pre 21, 67.7%
Post 10, 71.4%
4 weeks after discharge
Singleton infant, term pregnancy, no major abnormalities, not up for adoption, discharged home with mother
Pre 75, 72.8%
Post 80, 89.9%
Pre 51, 68.0%
Post 61, 76.3%
8–9 weeks after birth
Delirium, Dementia Depression
Include if ≥ 65 years
Diabetes Foot Care
Adults with Type 1 or Type 2 Diabetes excluding women with gestational diabetes
Pre 66, 67.3%
Post 58, 47.2%
Pre 38, 57.6%
Post 26, 44.8%
4 to 6 weeks after discharge
18 years or older, daily or occasional smoker in mental health facility Excluded if major depression or psychiatric illness
Pre 89, 76.7%
Post 84, 90.3%
Pre 42, 47.2%
Post 27, 32.1%
4 weeks after admission
Venous Leg ulcers
New or recurrent, Exclude if arterial ulcer, vasculitis, under diabetic management
The data collection process was supervised centrally by an experienced Project Manager who frequently communicated with the CRNs to maintain the quality of data collection. The CRNs extracted pilot data from five to 10 charts at each organization to develop an effective system for obtaining charts and to assess the feasibility of coding categories. Part-time data collectors were then trained and supervised by the CRN locally to conduct the chart audits. The Project Manager reviewed data from all sites on an ongoing basis and contacted local CRNs about missing data or inconsistencies.
Descriptive statistics and bivariate statistics (independent t-tests, Pearson chi-square) were used to describe characteristics of the sample, and to compare before and after measures of clinical outcomes within study sites. Fisher's exact test was used when expected frequencies were small.
Semi-structured interview schedules were developed by the co-investigators and administered to all CRNs, administrators and selected staff nurses immediately after the implementation. The staff nurses (2–3 per unit) were identified by the CRN to include those who had a favourable as well as those who had a less favourable experience with the guideline implementation.
Questions were designed to elicit information about the strategies used as well as the perceived factors that facilitated or provided barriers to guideline implementation. In addition, CRNs were interviewed at the mid-point of implementation to describe priority recommendations selected for implementation and reasons why some recommendations were not selected. Examples of these interview schedules as well as our approach used to conduct and analyze interview data are provided in an available monograph .
The Project Manager and trained interviewers conducted the interviews which were audio-taped with the consent of each respondent. Analysis of the interviews was conducted using QSR-N5. Interviews were initially coded by one individual using a coding guide. Content analysis was conducted looking for prominent themes and patterns across respondents for each guideline .
Chart audits and patient interviews
Completion rates for chart audits were greater than 95% for all eligible cases. Response rates varied for initial and follow-up patient interviews, as displayed in Table 3, with an average of 61.1%. Language barriers and refusals lowered the response rates at the initial interviews. For the follow-up interviews, the response rates decreased due to drop-outs, difficulty contacting patients after discharge, discharge of patients to other facilities and death. In addition, for the asthma guideline, lower response rates were due to an extremely busy emergency department discharging potential patients before they could be interviewed for the study.
Description of recommendations and proportion of indicators which improved or not from pre to post implementation
Delirium, Dementia, Depression
Foot Complications Diabetes
Venous Leg Ulcers
Content of recommendations implemented†
-Assess asthma control
-Educate re Asthma
-Develop action plans
-Assess inhaler device technique
-Endorse WHO Baby Friendly Hospital Initiative
Advocate friendly environments
-Promote community action
-Assess Postnatal factors
-Educate and support
-Maintain suspicion for DDD in the older adult
-Screen for cognition, function, behaviour and mood
-Use structured assessment & standardized tools
-Physical exam of feet to assess risk factors
-Educate about foot care Educate tailored to knowledge, needs and risk factors
Minimal smoking cessation intervention using ask, advice, assist, arrange protocol Recognize relapse is common and need to re-engage.
-Assess risk factors
-Measure surface areas
-Ankle Brachial Pressure Index
-Graduated compression bandaging
Number of chart audit items
Number of patient interview items
Improvements* (Inhaler use)
Number of items in follow-up telephone call
Total # indicators N%
Selected clinical outcomes at pre and post guideline implementation ‡
Selected process of care indicators and patient outcomes
Changes† Source of Data
N = 80 Pre
N = 48 Post
Assessment of patient use of β2 agonist
Individual action plan for client discharge
Teaching information provided
Breast-feeding in hospital N = 75 Pre; N = 76 Post
- Interview in hospital
Infant supplementation given N = 103 Pre; N = 89 Post
Delirium, Dementia and Depression
N = 196 Pre
N = 186 Post
Assessment on admission for memory problems
Assessment on admission for mood (i.e. depression)
Assessment during stay for memory problems
Diabetes Foot Care
N = 97 Pre
N = 119 Post
Assessment for risk factors for foot ulceration/amputation
Monofilament used to assess sensation in the feet
Received advice on stopping smoking or staying quit N = 116 Pre; N = 93 Post
Self-help information given N = 116 Pre; N = 93 Post
Have you tried to quit smoking in the past 2 months? N = 36 Pre; N = 21 Post
- Telephone call at home
Venous Leg Ulcers
N = 87 Pre
N = 52 Post
Assessed for clinical history and features associated with venous disease
Doppler measurement of Ankle Brachial Pressure Index (ABPI)
Overall, with respect to chart audit data, there were statistically significant improvements in nearly half of the indicators (46.7%, 35/75). It is interesting to note that improvements were found in items related to: patient assessment/risk classification 37%; patient education 29%; medication and treatments 14%; symptom monitoring 9%; supportive care 6%; and other 6%. Statistically significant improvements were also found in pre- and post-observation scores of asthma patients' correct use of an inhaler device (p =.014). With respect to patient interview data, it is notable that statistically significant improvements were found in all seven indicators for diabetes foot care as reported in Table 4.
No statistically significant differences were found > 80% of the indicators for three guidelines (breastfeeding, DDD, smoking cessation). In addition, two statistically significant negative results were found for two of these guidelines. Specifically, fewer mothers were exclusively breastfeeding their infants in hospital (80.0% pre-implementation, 67.1% post-implementation; p = 0.012) and fewer mental health patients reported that they had tried to quit smoking post-guideline implementation (66.7% pre-implementation, 33.3% post-implementation; p = .026).
The concordance of nursing practices with indicators measured varied considerably post-implementation. The lowest level was 17.4% for the provision of smoking cessation advice to smokers in a mental health organization and the highest level was 96.2% for the patients in the chronic care hospital and the clients receiving community-based nursing services for the care of their venous leg ulcers (Table 5).
Even in the three guidelines with statistically significant improvements in > 50% of indicators, there were still substantial gaps in the provision of evidence-based recommendations at post-intervention as follows: Only 23.9% of hospital patients with asthma had an action plan at discharge; only 60.5% of hospital and community-based patients with diabetes had a documented assessment of risk factors for foot ulcers; and only 45.7% of patients with venous leg ulcers from long-term care and a community-based nursing service received a Doppler Assessment of Ankle BrachialPressure Index (Table 5).
Participation in the qualitative interviews by the CRNs, nurses and administrators was strong from all units with an overall participation rate of 87.4% (83/95) with a range of 75%–100% per guideline. The proportion of recommendations implemented varied by guideline topic from 43% (e.g. 3/7) to 100% (e.g. 7/7). A very brief synopsis of the content of recommendations is described in Table 4 with a full description of recommendations available . For three guidelines, the CRNs reported at mid-point that their implementation action plan addressed all clinical recommendations (asthma, DDD and diabetes foot care). However, for the other three topics the CRNs explained reasons why certain recommendations were not a priority as described below.
The prenatal components which included a comprehensive assessment to facilitate a breastfeeding plan and small informal group education classes were not implemented because "these are very expensive and would require too much personnel time and it is felt not possible at this time". Also, the recommendation to conduct an in-person follow-up and assessment if a mother and baby were discharged within 48 hours of birth was not implemented due to perceived lack of available public health nurses. Controversy arose about the recommendation for exclusive breastfeeding for the first six months of life from the hospital nutrition/dietetic staff. According to the CRN, these nutritionists followed other national guidelines recommending exclusive breastfeeding for only four months.
The CRN reported that three recommendations were not implemented: provide intensive smoking cessation interventions, implement interventions with pregnant women and encourage smokers and non-smokers to have smoke-free homes. Reasons for non-implementation included lack of time for nurses to do intensive smoking cessation counselling and the availability of experts at a nicotine dependence clinic within the organization. The pregnant women population was not applicable to the participating mental health organization. With respect to the quest for smoke-free homes of patients/clients, the CRN stated "We felt that it was something that we did not have a lot of control over, some clients live in rooming houses and boarding houses and collective group settings."
Venous Leg Ulcers
Of the 56 recommendations, 52 were implemented. Recommendations not implemented related to secondary treatment options including venous surgery, electrical stimulation, hyperbaric oxygen, and therapeutic ultrasound. The CRN explained that "these are second wave treatment option, so the first part of our process has really been trying to educate the nurses around the guidelines, care and so that sort of comes after the initial care."
Participants' perceptions of most important facilitators for implementation
Delirium, Dementia, Depression
Diabetes Foot Care
Venous Leg Ulcers
Administrative support and buy-in
Nurses' paid time for attending education sessions
Support of CRNs‡, APNs† on units
Ease of implementation, easy to use tools
Organizational support and readiness
Buy-in of nursing staff
Support of CRN‡
Support of CRN‡
Steering committee with representation from all units
Support of CRN‡
Champions including physician
Support of CRNs‡ and other educators
Having education as mandatory program outside of work hours
Partnership between hospital and public health settings
Organizational and management support
Organizational commitment and management support
Organizational support and resources
Having credible person in department to facilitate implementation
Having all stakeholders on board, managers, APNs†, nurses, champions
Involving staff at start of process
Ongoing access to information
Increased number of VLU patients
Participants' perceptions of most important barriers to implementation
Delirium, Dementia, Depression
Diabetes Foot Care
Venous Leg Ulcers
Lack of time to work with patients in emergency department
Workload and competing demands
Time and workload pressure for nurses
Client resistance to smoking cessation
Time and workload pressures
Too few asthma patients on in-patient units (not peak asthma season)
Public Health Nurses' limited access to CRN and lactation consultant
Limited time spent with patient, patient stay too short
Difficulty getting support and buy- in from all levels of organization (managers, nurses, physicians)
Time and workload pressures, and competing demands
SARS outbreak created delay in education and implementation
Timing of project, timing of launch, lost momentum
Workload and limited availability of CRN‡in hospital
Complexity of skills required for RPNs§
Patient issues: cost of taking action, patient motivation, communication, follow-up
Challenges of administration and coordination across four sites
Lack of physicians willingness to order high compression bandaging
Change in management in two key units
Lack of communication between hospital and public health
Lack of buy-in from nurse managers at unit level and some nurses
Lack of CRN‡ for a period of time, delay in appointing new CRN
Attitudes re clients, past experience led to belief clients can't quit smoking
Format of education manual
Lack of physician and administrative support in emergency; physicians in a hurry to send patients home
Lack of management support
Changes in senior personnel and lack of consistent champion
Organizational change, reorganization
Documentation not changed for staff to record assessment of smoking cessation
Lack of educational material for clients
After the one-year implementation of newly developed nursing-oriented guidelines, a substantial proportion of patients were not receiving recommended nursing care. While our results reveal that some statistically significant positive improvements were found for all guideline topics, more than 80% of the indicators for breastfeeding, DDD and smoking cessation did not change from pre-implementation to post-implementation. However, there were statistically significant improvements in more than half of the indicators for the asthma, diabetes foot care and venous leg ulcer guidelines. These improvements are notable given the relatively short time frame of one-year for project completion.
Overall, this situation is consistent with findings of other studies of guideline implementation and assessment of the quality of care [1–3, 6, 8]. Perhaps changes in the processes of care and outcomes would have been enhanced if nurses, managers and other disciplines were aware of the small proportion of patients actually receiving guideline-based recommended care. In this study, feedback on the pre-intervention study results was not provided to the agency or staff. Comparative data of nursing practice and patient outcomes is important for health service delivery to determine the quality and quantity of care provided. Systematic reviews of audit and feedback conclude that telling people what they have been doing does impact on change and improve professional practice .
Recent studies of nursing practice including subsequent work of our own suggest that it is possible to obtain data in a reliable and valid way [31–33]. Electronic gathering and dissemination systems that document patients' responses to treatment, obtain real-time outcome feedback and support access to electronic resources including RNAO guidelines using hand-held computers at point of care are promising tools to support guideline implementation .
The implementation strategies used in this study reflect those generally classified as multi-faceted interventions by Grimshaw and colleagues . We observed two unique strategies applied to each of the three guideline implementations resulting in improvements in more than 50% of the indicators. Firstly, these nurses' received an opportunity for hands-on skill practice sessions (trial use of placebo devices for asthma medication, practice sessions for foot assessment of people with diabetes, and practice sessions for bandaging for people with venous leg ulcers). Skill practice has been identified as an element of interactive workshops which may also include role-play or case discussion. Interactive workshops can result in moderately large changes in professional practice compared to didactic lecture-only sessions which are unlikely to change behaviour . However, it is noteworthy that case study discussions were used in education sessions for the three other guidelines without subsequent changes in professional practice (breastfeeding, DDD and smoking cessation). The skill practice component appears to be a vital element and warrants attention.
The second unique implementation element used with the three guidelines with substantial practice changes was the development of patient education toolkits and brochures. The CRN for the asthma guideline suggested that the new patient information tool provided an opportunity to reinforce previous teaching provided to nurses. The CRN for diabetes foot care reported that patients appreciated the education suggesting a positive feedback process from clients may have encouraged implementation of the guideline. It is noteworthy that all seven process and outcome indicators from the patient interviews increased significantly from pre to post-implementation for the topic of diabetes foot care.
For other guideline topics, positive feedback from patient education was less likely in part due to the nature of the guideline topic and in part due to the application in the selected clinical context. While the diabetes guideline was directly applicable to the day-to-day care in the medical hospital in-patient units and the ongoing care of home visiting nurses, the smoking cessation guideline required a considerable shift in the norms at the participating mental health facility where previously, cigarettes had been provided as a positive reward for good patient behaviour. With respect to the breastfeeding guideline, the post-partum nurses did not receive feedback about the impact of their hospital-based teaching because public health nurses assumed care once the mother and infant were discharged from the hospital. The public health nurses did not receive feedback about long-term breast-feeding outcomes because their teaching was concentrated in the early post-partum period. The uptake of guidelines such as breastfeeding or smoking cessation with a preventive focus appears to be slower, a finding that is consistent with Rogers contention that preventive innovations have a particularly slow rate of adoption because individuals have difficulties perceiving the relative advantage and the consequences are distant in the future .
Management support was reported as an important facilitator for all guideline implementations except for breastfeeding. We report in this study and in previous evaluations of RNAO guidelines , that one of the most important facilitators for implementation of a guideline is management support and commitment. In addition, key barriers documented in previous work and in this report include lack of administrative support and changes in management. We note that "leadership support for evidence-based practice culture" is an explicit organizational element of Stetler's conceptual framework  and that leadership is a sub-element of "context" of the Promoting Action on Research Implementation in Health Services (PARIHS) framework . However, management support and leadership are not listed as an implementation strategy in either our implementation Toolkit [17, 18] or with the current EPOC Cochrane systematic reviews of consumer, professional or organizational change. We conceptualise "managerial leadership for research use" as a multidimensional process of influence to enable nurses to use research evidence in clinical practice [40, 41]. An integrative review of 12 published studies found that managerial support, policy revisions, auditing, role modelling and valuing research facilitated nurses' use of research evidence . We suggest that managerial leadership is an important element for guideline implementation in nursing
With respect to process and patient outcomes, we found that none of the organizations had existing patient care databases with sufficient detail or data about nursing process of care to be used in the evaluation of these RNAO guidelines. Thus, we endeavoured to rapidly adapt existing published tools and develop new tools (chart audit, observation and interview questionnaires) to measure the processes of nursing care that are meaningful and actionable by nursing managers and senior administrators. These data collection tools were assessed for content validity and applicability and are available on the web with the RNAO Cycle 3 guidelines  and may provide a launching point for other teams evaluating gaps in nursing care. Subsequent psychometric testing of the asthma observation tool found mean inter-rater reliability indicies of .82 . Other related articles, monographs and user-guides describing measures for evaluating the implementation of nursing best practice guidelines are available [27, 28, 33, 42, 43].
An environmental scan of facilitators and barriers as recommended in the implementation Toolkit [17, 18] was conducted by the CRNs for each guideline implementation in order to plan the intervention strategies in consultation with a multidisciplinary steering committee. However, knowing that a barrier(s) exists is only the first step. There are no quick fixes for many of the barriers identified by CRNs. For example, overcoming nurses' resistance to the breastfeeding guideline recommendations, addressing competing demands for time as reported for the DDD guideline and tackling long-standing issues of client resistance to smoking cessation are all complex issues. Tailored interventions to overcome identified barriers to change have been evaluated in a review of 15 studies with mixed results because it was not clear whether all barriers or important barriers were identified and addressed by pre-selected strategies . Pragmatic evaluations of guideline implementation in nursing are necessary to document important barriers for the design and evaluation of tailored strategies that fit real-world implementation.
A note of caution is required about the evaluation of recommendations that are sensitive to patient preferences and nurses' values. As one report advises , performance measures require attention to avoid defining high testing rates as good quality of care, since they may not take into consideration patient preferences for care options. Patient preferences and nursing values and beliefs are reported to influence evidence-based nursing  and may have contributed to the two statistically significant negative outcomes found in this study. With respect to breastfeeding, the CRN reported that tension existed about the extent to which staff should encourage individual mothers to breastfeed versus elicit mothers' choices to bottle-feed their infant. With respect to smoking cessation, client resistance to smoking cessation was reported as an important barrier by nurses. Evidence-based practice is defined to "integrate the best research evidence with clinical expertise and patient values ." There is a need to identify patient and provider values implicit in guideline recommendations and these values should be reported along with the research evidence for the recommendations .
Lack of time and workload considerations were reported as barriers across all guideline implementation initiatives. Cost considerations such as staffing for increased workload due to new procedures recommended in guidelines are critical for decision-makers , yet few authors have attempted to estimate these costs. In the subsequent Cycle 4 evaluation projects, cost estimates were derived for implementation of the pressure ulcer guideline and this report is available on request from the authors . The cost of implementation during the initial six month start-up phase and in the subsequent 18 months were determined using a balanced score card approach. Education and capital costs were high in the initial period, while patient operating costs accounted for the largest set of expenditures in the latter period. Future studies are needed to compare the cost implications of guideline implementation across settings and among different types of guidelines
Organizations in this evaluation study volunteered to participate and are therefore a self-selected group. Their volunteer status may have stimulated guideline recommendation uptake and we note that these organizations may reflect the early adopters and innovation champions . Indeed the CRNs in our study appeared to possess characteristics of early adopters with well developed interpersonal and negotiating skills, often holding key linking positions in their organization. Other limitations include the lack of concurrent control groups and the lack of inter-rater reliability testing of chart auditors. Thus, we cannot be certain that the observed changes would have happened without the intervention. Future studies with concurrent control groups and thorough process evaluations are recommended. Periodic feedback to CRNs, managers and staff nurses regarding the status of implementation, assessment of ongoing barriers and the design of repeated opportunities to infuse this information in a tailored implementation plan are suggested for future research.
A unique opportunity to study the process and impact of six newly developed nursing guidelines in hospital and community settings existed. We report a real-world implementation project that provides data about promising implementation strategies.
Specifically, these strategies include the opportunity to practice skills in interactive educational sessions for nurses, the development and use of patient educations tools, and the importance of managerial leadership for nurses' use of research-based guidelines. The lessons learned in this evaluation project may be helpful in planning future guideline implementations and in planning future research.
With funding from the Canadian Health Services Research Foundation, the Canadian Institutes of Health Research and the Canadian Nurses Foundation, we are currently conducting follow-up studies to examine the long-term impact of nursing guideline implementation on clinical and process outcomes. This follows from the comments of CRNs and managers who observed that it takes longer than the six months allocated to the intervention to make clinical changes. In addition, our longitudinal study examines the predictors of sustained implementation of guideline recommendations .
In Canada, there is a strong public and political agenda to continuously strive towards high quality health care. The 10-year action plan to strengthen health care includes a priority to share best practices and provide information to make progress transparent to citizens . The nursing profession, a major group of healthcare providers, needs to share intervention strategies and evaluation results to improve health care delivery. Developing, implementing and evaluating best practice guidelines in nursing are essential elements in high quality health care.
Clinical practice guidelines have the potential to improve the quality of care received by patients and thereby improve patient outcomes. Yet there is a lack of evidence about the impact of nursing best practice guidelines and the most effective strategies to implement such guidelines. We found that a multi-faceted intervention for the implementation of nursing guidelines about asthma, breastfeeding, delirium-dementia-depression, reducing foot complications in diabetes, smoking cessation and venous leg ulcers can result in some improvement in practice and patient outcomes across diverse healthcare settings yet many outcomes remained unchanged. Promising elements for future initiatives are identified including interactive education with skill practice sessions, attention to patient education, involvement of managerial leadership and electronic gathering and dissemination systems offering both real-time feedback and access to guidelines.
We acknowledge project funding from the Government of Ontario. Dr. Edwards holds a Nursing Chair funded by the Canadian Health Services Research Foundation, the Canadian Institutes of Health Research and the Government of Ontario. Dr. Davies held a Career Scientist award funded by the Ontario Ministry of Health and Long-Term Care at the time of this study. Dr. Ploeg held an Investigator Award from the Canadian Institutes of Health Research and St. Joseph's Health Care, Hamilton, Ontario. Tazim Virani holds a doctoral fellowship funded by the Ontario Ministry of Health and Long-term Care and awarded by the RNAO. The views expressed in this publication are those of the authors and not those of the RNAO or the Government of Ontario.
We acknowledge the leadership of Doris Grinspun at RNAO for the nursing best practice guideline initiative. We thank evaluation team collaborators Maureen Dobbins and Jennifer Skelly from McMaster University and Heather McConnell from RNAO. We are grateful for assistance from research staff including Diana Ehlers, Barbara Helliwell, Marilyn Kuhn, Elana Ptack, and Isabelle St. Pierre.
We especially thank the participating health care organizations and the following clinical resource nurses: Genevieve Courant, Sandy Haist, and Karen Ray.
- Grol R, Wensign M, Eccles M: Improving patient care. The implementation of change in clinical practice. 2005, Edinburgh: ElsevierGoogle Scholar
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348: 2635-2645. 10.1056/NEJMsa022615.View ArticlePubMedGoogle Scholar
- Seddon ME, Marshall MN, Campbell SM, Roland MO: Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand. Qual Health Care. 2001, 10: 152-158. 10.1136/qhc.0100152...View ArticlePubMedPubMed CentralGoogle Scholar
- Ollenschlager G, Marshall C, Qureshi S, Rosenbrand K, Burgers J, Makela M, Slutsky J: Improving the quality of healthcare using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). Qual Saf Health Care. 2004, 13: 455-460. 10.1136/qshc.2003.009761.View ArticlePubMedPubMed CentralGoogle Scholar
- Coopey M, Nix MP, Clancy CM: Translating research into evidence-based nursing practice and evaluating effectiveness. J Nurs Care Qual. 2006, 21: 195-202.View ArticlePubMedGoogle Scholar
- Thomas L, Cullum N, McColl E, Rousseau N, Soutter J, Steen N: Guidelines in professions allied to medicine. Cochrane Database of Systematic Reviews. 2000, 10.1002/14651858.. 2
- Thompson DS, Estabrooks CA, Scott-Findlay SS, Moore K, Wallin L: Interventions aimed at increasing research use in nursing: a systematic review. Implement Sci. 2007, 2: 15-10.1186/1748-5908-2-15.View ArticlePubMedPubMed CentralGoogle Scholar
- Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8 (6): 1-72.View ArticleGoogle Scholar
- Davis DA, Thomson MA, Oxman AD, Haynes RB: Changing physician performance a systematic review of the effect of continuing medical education strategies. JAMA. 1995, 274: 700-705. 10.1001/jama.274.9.700.View ArticlePubMedGoogle Scholar
- Wensing M, Van der Weijden T, Grol R: Implementing guidelines and innovations in general practice which interventions are effective?. J Gen Pract. 1998, 48 (427): 991-997.Google Scholar
- Maas ML, Delaney C: Nursing process outcome linkage research: issues, current status, and health policy implications. Med Care. 2004, 42 (Suppl 2): 40-48.Google Scholar
- Doran D: Nursing-Sensitive Outcomes State of the Science. 2003, Boston: Jones and BartlettGoogle Scholar
- Grinspun D, Virani T, Bajnok I: Nursing best practice guidelines: The RNAO project. Hosp Q. 2001, 5 (2): 56-60.PubMedGoogle Scholar
- Registered Nurses' Association of Ontario: Guideline Methodology Report. Toronto. 2006, [http://www.rnao.org/researchunit/]Google Scholar
- Registered Nurses Association of Ontario: Nursing Best Practice Guidelines. [http://www.rnao.org/Page.asp?PageID=861&SiteNodeID=133]
- Edwards N, Davies B, Ploeg J, Virani T, Skelly J: Implementing nursing best practice guidelines impact on patient referrals. BMC Nurs. 2007, 6: 4-10.1186/1472-6955-6-4.View ArticlePubMedPubMed CentralGoogle Scholar
- Registered Nurses' Association of Ontario: Toolkit: Implementation of clinical practice guidelines. Toronto. 2002, [http://www.rnao.org/Page.asp?PageID=861&SiteNodeID=133]Google Scholar
- DiCenso A, Virani T, Bajnok I, Borycki E, Davies B, Graham I, Harrison M, Logan J, McCleary L, Power M, Scott J: A toolkit to facilitate the implementation of clinical practice guidelines in healthcare settings. Hosp Q. 2002, 5: 55-60.PubMedGoogle Scholar
- DiCenso A, Guyatt G, Ciliska D: Evidence-Based Nursing: A guide to clinical practice. 2005, St. Louis, MO: ElsevierGoogle Scholar
- Graham ID, Logan J: Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004, 36: 89-103.PubMedGoogle Scholar
- Davies B: Sources and models for moving research evidence into clinical practice. JOGN Nurs. 2002, 31: 558-562.View ArticleGoogle Scholar
- ICEBeRG: Designing theoretically-informed implementation interventions. Implement Sci. 2006, 1: 4-10.1186/1748-5908-1-4.View ArticleGoogle Scholar
- Lorimer K, Harrison M, Graham I, Friedberg E, Davies B: Venous leg ulcer care How evidence-based is nursing practice?. J Wound Ostomy Continence Nurs. 2003, 30 (3): 132-142.PubMedGoogle Scholar
- Lorimer K, Harrison M, Graham I, Friedberg E, Davies B: Assessing venous leg ulcers population characteristics and practices in a home care community. Ostomy Wound Manage. 2003, 49 (5): 32-43.PubMedGoogle Scholar
- World Health Organization & Wellstart International: The Baby-Friendly Hospital Initiative, monitoring and reassessment: Tools to sustain progress. 2001, Geneva: World Health OrganizationGoogle Scholar
- Edwards N, Sims-Jones N, Hotz S, Cushman R: Development and testing components of a multifaceted intervention program to reduce the incidence of smoking relapse during pregnancy and postpartum of both women and their partners. Final Report to Health Canada. 1997, 1-112. (Prepared in both English and French)Google Scholar
- Davies B, Danseco E, Cicutto L, Higuchi KS, McConnell H, Edwards N, MacPherson A, Clarke D: Nursing Best Practice Guidelines Evaluation User Guide: Inhaler Device Assessment Tool for Promoting Asthma Control in Children. 2006, University of Ottawa: Nursing Best Practice Research Unit, [http://www.nbpru.ca]Google Scholar
- Edwards N, Davies B, Griffin P, Ploeg J, Skelly J, Danseco E, Helliwell B: Evaluation of nursing best practice guidelines Interviewing nurses and administrators. 2004, (CHRU Publication #M04-1), University of Ottawa: Community Health Research Unit, [http://aix1.uottawa.ca/~nedwards/chru/english/pdf/m04.01.pdf]Google Scholar
- Miles MB, Huberman AM: Qualitative Data Analysis: An Expanded Sourcebook. 1994, Thousand Oaks, CA: Sage Publications, 2Google Scholar
- Jamtvedt G, Young JM, Kristofferson O'Brien M, Oxman A: Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care. 2006, 15: 433-436. 10.1136/qshc.2006.018549.View ArticlePubMedPubMed CentralGoogle Scholar
- Doran D, Harrison MB, Laschinger H, Hirdes J, Rukholm E, Sidani S, McGillis Hall L, Tourangeau A, Cranley L: Relationship between nursing interventions and outcome achievements in acute care settings. Res Nurs Health. 2006, 29: 61-70. 10.1002/nur.20110.View ArticlePubMedGoogle Scholar
- Doran D, Harrison MB, Laschinger HS, Hirdes JP, Rukholm E, Sidani S, McGillis Hall L, Tourangeau AE: Nursing-sensitive outcomes data collection in acute care and long-term care settings. Nurs Res. 2006, 55: S75-S81. 10.1097/00006199-200603001-00012.View ArticlePubMedGoogle Scholar
- Edwards N, Danseco E, Heslin K, Ploeg J, Santos J, Stansfield M, Davies B: Development and testing of instruments to assess physical restraint use. Worldviews Evid Based Nurs. 2006, 3: 73-85. 10.1111/j.1741-6787.2006.00056.x.View ArticlePubMedGoogle Scholar
- Doran D, Sidani S: Outcomes-focused knowledge translation A framework for knowledge translation and patient outcomes improvement. Worldviews Evid Based Nurs. 2007, 4: 3-13. 10.1111/j.1741-6787.2007.00073.x.View ArticlePubMedGoogle Scholar
- O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews. 2001, 10.1002/14651858.. Art. No.: CD003030. , 2Google Scholar
- Roger EM: Diffusion of innovations. 2003, New York: Free Press, 5Google Scholar
- Ploeg J, Davies B, Edwards N, Gifford W, Elliot-Miller P: Factors influencing best-practice guideline implementation: Lessons learned from administrators, nursing staff and project leaders. Worldviews Evid Based Nurs. 2007, 4 (4): 210-219.View ArticlePubMedGoogle Scholar
- Stetler C: Role of organization in translating research into evidence-based practice. Outcomes Manag. 2003, 7: 97-103.PubMedGoogle Scholar
- Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Tichen A, Estabrooks C: Ingredients for change revisiting a conceptual framework. Qual Saf Health Care. 2002, 11: 174-180. 10.1136/qhc.11.2.174.View ArticlePubMedPubMed CentralGoogle Scholar
- Gifford W, Davies B, Edwards N, Graham I: Leadership strategies to influence the use of clinical practice guidelines. Can J Nurs Leadersh. 2006, 19: 72-88.View ArticleGoogle Scholar
- Gifford W, Davies B, Edwards N, Griffin P: Managerial leadership for nurses' use of research evidence An integrative review of the literature. Worldviews Evid Based Nurs. 2007, 4 (3): 126-145. 10.1111/j.1741-6787.2007.00095.x.View ArticlePubMedGoogle Scholar
- Edwards N, Davies B, Ploeg J, Dobbins M, Skelly J, Griffin P, Ralphs-Thibodeau : Evaluating best practice guidelines. Can Nurse. 2005, 101: 19-23.Google Scholar
- Dobbins M, Davies B, Danseco E, Edwards N, Virani T: Changing nursing practice: Evaluating the usefulness of a best-practice guideline implementation toolkit. Can J Nurs Leadersh. 2005, 18: 34-48.View ArticleGoogle Scholar
- Shaw B, Cheater F, Baker R, Gillies C, Hearnshaw H, Flottorp S, Robertson N: Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane database of Systematic Reviews. 2005, 20 (3): 10.1002/14651858. Art No.: CD005470.Google Scholar
- Walter LC, Davidowitz NP, Heineken PA, Covinsky KE: Pitfalls of converting practice guidelines into quality measures. Lessons learned from a VA performance measurement. JAMA. 2004, 291: 2466-2470. 10.1001/jama.291.20.2466.View ArticlePubMedGoogle Scholar
- Sackett DL, Straus S, Richardson W, Rosenberg R: Evidence-based medicine: How to practice and teach EBM. 2002, London: HarcourtGoogle Scholar
- Schunemann HJ, Fretheim A, Oxman AD: Improving the use of research evidence in guideline development 10. Integrating values and consumer involvement. Health Res Policy Syst. 2006, 4: 22-10.1186/1478-4505-4-22.View ArticlePubMedPubMed CentralGoogle Scholar
- Edwards N, Downey A, Griffin P, Lang A, Eldred S, Danseco E, Perrier A: Costing Models for the Implementation of RNAO Best Practice Guidelines. Cycle 4 Technical Report. 2005, University of OttawaGoogle Scholar
- Davies B, Edwards N, Ploeg J, Virani T, Skelly J, Dobbins M: Determinants of the sustained use of research evidence in nursing. 2006, Final Report: Canadian Health Services Research Foundation & Canadian Institutes for Health Research. OttawaGoogle Scholar
- Health Canada Health Care Renewal. A 10-year plan to strengthen health care. [http://www.hc-sc.gc.ca/hcs-sss/delivery-prestation/fptcollab/2004-fmm-rpm/index_e.html]
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/29/prepub