After the one-year implementation of newly developed nursing-oriented guidelines, a substantial proportion of patients were not receiving recommended nursing care. While our results reveal that some statistically significant positive improvements were found for all guideline topics, more than 80% of the indicators for breastfeeding, DDD and smoking cessation did not change from pre-implementation to post-implementation. However, there were statistically significant improvements in more than half of the indicators for the asthma, diabetes foot care and venous leg ulcer guidelines. These improvements are notable given the relatively short time frame of one-year for project completion.
Overall, this situation is consistent with findings of other studies of guideline implementation and assessment of the quality of care [1–3, 6, 8]. Perhaps changes in the processes of care and outcomes would have been enhanced if nurses, managers and other disciplines were aware of the small proportion of patients actually receiving guideline-based recommended care. In this study, feedback on the pre-intervention study results was not provided to the agency or staff. Comparative data of nursing practice and patient outcomes is important for health service delivery to determine the quality and quantity of care provided. Systematic reviews of audit and feedback conclude that telling people what they have been doing does impact on change and improve professional practice .
Recent studies of nursing practice including subsequent work of our own suggest that it is possible to obtain data in a reliable and valid way [31–33]. Electronic gathering and dissemination systems that document patients' responses to treatment, obtain real-time outcome feedback and support access to electronic resources including RNAO guidelines using hand-held computers at point of care are promising tools to support guideline implementation .
The implementation strategies used in this study reflect those generally classified as multi-faceted interventions by Grimshaw and colleagues . We observed two unique strategies applied to each of the three guideline implementations resulting in improvements in more than 50% of the indicators. Firstly, these nurses' received an opportunity for hands-on skill practice sessions (trial use of placebo devices for asthma medication, practice sessions for foot assessment of people with diabetes, and practice sessions for bandaging for people with venous leg ulcers). Skill practice has been identified as an element of interactive workshops which may also include role-play or case discussion. Interactive workshops can result in moderately large changes in professional practice compared to didactic lecture-only sessions which are unlikely to change behaviour . However, it is noteworthy that case study discussions were used in education sessions for the three other guidelines without subsequent changes in professional practice (breastfeeding, DDD and smoking cessation). The skill practice component appears to be a vital element and warrants attention.
The second unique implementation element used with the three guidelines with substantial practice changes was the development of patient education toolkits and brochures. The CRN for the asthma guideline suggested that the new patient information tool provided an opportunity to reinforce previous teaching provided to nurses. The CRN for diabetes foot care reported that patients appreciated the education suggesting a positive feedback process from clients may have encouraged implementation of the guideline. It is noteworthy that all seven process and outcome indicators from the patient interviews increased significantly from pre to post-implementation for the topic of diabetes foot care.
For other guideline topics, positive feedback from patient education was less likely in part due to the nature of the guideline topic and in part due to the application in the selected clinical context. While the diabetes guideline was directly applicable to the day-to-day care in the medical hospital in-patient units and the ongoing care of home visiting nurses, the smoking cessation guideline required a considerable shift in the norms at the participating mental health facility where previously, cigarettes had been provided as a positive reward for good patient behaviour. With respect to the breastfeeding guideline, the post-partum nurses did not receive feedback about the impact of their hospital-based teaching because public health nurses assumed care once the mother and infant were discharged from the hospital. The public health nurses did not receive feedback about long-term breast-feeding outcomes because their teaching was concentrated in the early post-partum period. The uptake of guidelines such as breastfeeding or smoking cessation with a preventive focus appears to be slower, a finding that is consistent with Rogers contention that preventive innovations have a particularly slow rate of adoption because individuals have difficulties perceiving the relative advantage and the consequences are distant in the future .
Management support was reported as an important facilitator for all guideline implementations except for breastfeeding. We report in this study and in previous evaluations of RNAO guidelines , that one of the most important facilitators for implementation of a guideline is management support and commitment. In addition, key barriers documented in previous work and in this report include lack of administrative support and changes in management. We note that "leadership support for evidence-based practice culture" is an explicit organizational element of Stetler's conceptual framework  and that leadership is a sub-element of "context" of the Promoting Action on Research Implementation in Health Services (PARIHS) framework . However, management support and leadership are not listed as an implementation strategy in either our implementation Toolkit [17, 18] or with the current EPOC Cochrane systematic reviews of consumer, professional or organizational change. We conceptualise "managerial leadership for research use" as a multidimensional process of influence to enable nurses to use research evidence in clinical practice [40, 41]. An integrative review of 12 published studies found that managerial support, policy revisions, auditing, role modelling and valuing research facilitated nurses' use of research evidence . We suggest that managerial leadership is an important element for guideline implementation in nursing
With respect to process and patient outcomes, we found that none of the organizations had existing patient care databases with sufficient detail or data about nursing process of care to be used in the evaluation of these RNAO guidelines. Thus, we endeavoured to rapidly adapt existing published tools and develop new tools (chart audit, observation and interview questionnaires) to measure the processes of nursing care that are meaningful and actionable by nursing managers and senior administrators. These data collection tools were assessed for content validity and applicability and are available on the web with the RNAO Cycle 3 guidelines  and may provide a launching point for other teams evaluating gaps in nursing care. Subsequent psychometric testing of the asthma observation tool found mean inter-rater reliability indicies of .82 . Other related articles, monographs and user-guides describing measures for evaluating the implementation of nursing best practice guidelines are available [27, 28, 33, 42, 43].
An environmental scan of facilitators and barriers as recommended in the implementation Toolkit [17, 18] was conducted by the CRNs for each guideline implementation in order to plan the intervention strategies in consultation with a multidisciplinary steering committee. However, knowing that a barrier(s) exists is only the first step. There are no quick fixes for many of the barriers identified by CRNs. For example, overcoming nurses' resistance to the breastfeeding guideline recommendations, addressing competing demands for time as reported for the DDD guideline and tackling long-standing issues of client resistance to smoking cessation are all complex issues. Tailored interventions to overcome identified barriers to change have been evaluated in a review of 15 studies with mixed results because it was not clear whether all barriers or important barriers were identified and addressed by pre-selected strategies . Pragmatic evaluations of guideline implementation in nursing are necessary to document important barriers for the design and evaluation of tailored strategies that fit real-world implementation.
A note of caution is required about the evaluation of recommendations that are sensitive to patient preferences and nurses' values. As one report advises , performance measures require attention to avoid defining high testing rates as good quality of care, since they may not take into consideration patient preferences for care options. Patient preferences and nursing values and beliefs are reported to influence evidence-based nursing  and may have contributed to the two statistically significant negative outcomes found in this study. With respect to breastfeeding, the CRN reported that tension existed about the extent to which staff should encourage individual mothers to breastfeed versus elicit mothers' choices to bottle-feed their infant. With respect to smoking cessation, client resistance to smoking cessation was reported as an important barrier by nurses. Evidence-based practice is defined to "integrate the best research evidence with clinical expertise and patient values ." There is a need to identify patient and provider values implicit in guideline recommendations and these values should be reported along with the research evidence for the recommendations .
Lack of time and workload considerations were reported as barriers across all guideline implementation initiatives. Cost considerations such as staffing for increased workload due to new procedures recommended in guidelines are critical for decision-makers , yet few authors have attempted to estimate these costs. In the subsequent Cycle 4 evaluation projects, cost estimates were derived for implementation of the pressure ulcer guideline and this report is available on request from the authors . The cost of implementation during the initial six month start-up phase and in the subsequent 18 months were determined using a balanced score card approach. Education and capital costs were high in the initial period, while patient operating costs accounted for the largest set of expenditures in the latter period. Future studies are needed to compare the cost implications of guideline implementation across settings and among different types of guidelines
Organizations in this evaluation study volunteered to participate and are therefore a self-selected group. Their volunteer status may have stimulated guideline recommendation uptake and we note that these organizations may reflect the early adopters and innovation champions . Indeed the CRNs in our study appeared to possess characteristics of early adopters with well developed interpersonal and negotiating skills, often holding key linking positions in their organization. Other limitations include the lack of concurrent control groups and the lack of inter-rater reliability testing of chart auditors. Thus, we cannot be certain that the observed changes would have happened without the intervention. Future studies with concurrent control groups and thorough process evaluations are recommended. Periodic feedback to CRNs, managers and staff nurses regarding the status of implementation, assessment of ongoing barriers and the design of repeated opportunities to infuse this information in a tailored implementation plan are suggested for future research.
A unique opportunity to study the process and impact of six newly developed nursing guidelines in hospital and community settings existed. We report a real-world implementation project that provides data about promising implementation strategies.
Specifically, these strategies include the opportunity to practice skills in interactive educational sessions for nurses, the development and use of patient educations tools, and the importance of managerial leadership for nurses' use of research-based guidelines. The lessons learned in this evaluation project may be helpful in planning future guideline implementations and in planning future research.
With funding from the Canadian Health Services Research Foundation, the Canadian Institutes of Health Research and the Canadian Nurses Foundation, we are currently conducting follow-up studies to examine the long-term impact of nursing guideline implementation on clinical and process outcomes. This follows from the comments of CRNs and managers who observed that it takes longer than the six months allocated to the intervention to make clinical changes. In addition, our longitudinal study examines the predictors of sustained implementation of guideline recommendations .
In Canada, there is a strong public and political agenda to continuously strive towards high quality health care. The 10-year action plan to strengthen health care includes a priority to share best practices and provide information to make progress transparent to citizens . The nursing profession, a major group of healthcare providers, needs to share intervention strategies and evaluation results to improve health care delivery. Developing, implementing and evaluating best practice guidelines in nursing are essential elements in high quality health care.