The relationship between organisational characteristics and the effects of clinical guidelines on medical performance in hospitals, a meta-analysis

Systematic reviews in various health care settings have demonstrated that different implementation interventions have varying effects. [1, 2]. Most interventions to implement clinical guidelines focused on changing professional behaviour, but there is increasing awareness that factors related to the social, organisational and economical context can also be important determinants of guideline implementation[3]. For instance, a recent study on the implementation of screening guidelines in ambulatory settings has confirmed the influence of a number of organisational factors, such as mission, capacity and professionalism[4]. Despite increasing attention to organisational determinants of guideline implementation, research evidence on the relevance of specific factors is still limited. Insight into these factors is important as it can improve the effectiveness of implementation interventions by tailoring interventions to local circumstances. For example, different interventions may be more effective at academic hospitals than at community hospitals.

Most reviews on guideline implementation were conducted on implementation across settings, or implementation in primary care settings[5]. The literature on guideline implementation in hospital settings has not yet been reviewed separately. Therefore, we reviewed the effect of different intervention strategies to implement clinical guidelines at hospitals, and explored the impact of specific organisational factors on the effectiveness of these interventions. Hospitals are complex organisational systems whose primary aim is to deliver clinical care to individual patients. Management theories on change and innovation were analysed to derive specific factors for this explorative study. We identified the following factors that moght modify the effects of interventions: sufficient management support, appropriate learning environment, functional differentiation and local consensus on the intended changes (figure 1).

Theories on leadership and on quality management have suggested that support for an innovation from hospital management has a positive impact on its adoption [7–9]. The impact of management support may be based on power, incentives or facilitation. Hospital managers may also act as role models by implementing the innovation. Thus we hypothesized that implementation interventions are more effective if the effort is clearly supported by the local leaders.

The learning environment comprises a second set of factors. The underlying mechanism is that the availability of knowledge in the organisation enhances the adoption of innovations. This is consistent with existing theory on organisational learning, which suggests that an organisation's capacity to learn as an organisation is a crucial feature[9]. Teaching hospitals create a specific learning environment for trainers and trainees. Therefore we expected that implementation interventions are more effective in teaching hospitals than in non-teaching hospitals.

Functional differentiation is another factor that is expected to influence the uptake of new information or procedures in practice[10]. A higher level of specialisation and a higher level of technical expertise in the organisation may enhance implementation. The level and diversity of knowledge may be larger in settings with a range of medical disciplines, in which there is involvement of consultants, other physicians and non-physician practitioners. We therefore hypothesized that higher functional differentiation is positively associated with the effectiveness of implementation interventions.

Finally, we expected that promoting ownership through local consensus about clinical guideline recommendations and implementation strategies may also be associated with better uptake[11]. Organisational learning theory suggests that information gathering, shared perceptions of performance gaps and an experimental mind-set are important factors for learning in organisations[9]. Specific group cultures at hospitals appear to be associated with patient outcomes[12]. Theory on complex adaptive systems suggests that innovations should not be specified in detail in order to promote ownership and that 'muddling-through' should steer the guideline implementation process[13], while theory on adult learning adds that implementation should be tailored to each individual's learning needs[14]. We hypothesized that guideline implementation interventions would be most effective when developed within a hospital rather than derived from sources outside a hospital.

This systematic literature review aimed to assess the effectiveness of implementation and quality improvement interventions in hospital settings and to test our hypotheses on the impact of organisational factors.

The 53 trials yielded 81 comparisons. The appendix gives an overview of these studies and the intervention components of each comparison compared to the intervention components, if present, in the control group [see Additional file 1]. The trials consisted of 39 randomised controlled trials (of which 32 were clustered randomised controlled trials), 7 controlled clinical trials and 7 controlled before-and-after studies; 19 were inpatients studies [17–36], 28 were outpatient studies [37–64] and 6 had mixed settings [65–70]. In the 81 comparisons, 22 involved a single intervention. Mean number of interventions per comparison was 2.5 (SD 1.3).

Table 1 shows the results of the meta-analysis on the effects of various intervention components. When taken together, the Odds ratio in all intervention strategies appeared to be 2.13 (SD 1.72–2.65). These total results are visualised in a Forest plot in figure 2. The Odds ratio for the forest plot is slightly different because we could not correct for different interventions within studies. Single interventions consisted of reminders or feedback; no other intervention strategies were applied as a single intervention strategy. Overall odds ratios were 2.18 in single intervention studies, versus 1.77 in studies that had more than one intervention component, the so-called multifaceted intervention studies. Intervention components applied most frequently were reminders, feedback, educational meetings and educational material. Only a few studies included outreach visits, consensus meetings, financial interventions or the role of an opinion leader. With regard to the specific components of individual intervention strategies, we found that all components showed positive effects, except for consensus meetings, outreach and financial interventions, possibly due to small numbers.

However, in outpatient studies, community hospitals showed significantly larger effects. Furthermore, interventions that had not been developed internally, but had originated from outside the hospital, led to better outcomes, especially in outpatient studies.

This is the first systematic review to make an in depth exploration of guideline implementation in relation to the organisational characteristics of hospitals. Not only multifaceted interventions seemed to be effective, but also single interventions, contrary to our expectation that multifaceted interventions would prevail[69]. Single intervention strategies, particularly reminders, known to be effective in other settings, also appeared to be effective strategies in hospitals. Although a multi-faceted intervention including reminders may be effective, a single reminder strategy might provide a clearer or more consistent message and thus have more impact. Furthermore, educational material, reminders, feedback and revision of professional roles had more effect than other intervention strategies. We did not confirm our hypotheses on the influence of organisational factors, except for a learning environment in inpatient settings. Contrary to our expectations, effects were greater at community hospitals in outpatient studies. In some multi-centre studies, no difference was found between academic and community hospitals [27, 28]. or only a moderate positive effect was found for academic hospitals[70]. External factors, such as the origin of the intervention, seemed to have more impact than internal factors.

This review of the research literature has the limitation that it was an explorative retrospective study that may have suffered from publication bias. There may also have been reporting bias, because different studies with different aims were brought together and compared, while their research question was not to measure the influence of contextual factors. Due to limited organisational data within the studies, the validity of the measurements of organisational effect modifiers can be challenged. Another limitation of the study concerned the validity of the determinants, e.g. with regard to teaching hospitals as a proxy for learning environment, when no other valid measurements for learning environment could be found in the manuscripts. Other potential effect modifiers had to be disregarded, because there were no data present, for example organisational slack[9]. Also the number of studies was limited, partly because studies were left out if they did not produce dichotomous measures or did not have a concurrent group (interrupted time series designs). Adding these studies might have given our analysis greater power and perhaps other conclusions. The analysis was not corrected for clustering effects, which may have led to an overestimation of the statistical significance of the effects. Finally the study is limited by the fact that the exhaustive search strategy and data extraction inhibited the inclusion of studies published after 1998. An update of this review in a new study is recommended, including the possibility of exploring the inclusion of interupted time series and studies with continous measures.

Theories on organisational effect modifiers are mostly based on study results from a wide range of fields inside and outside health care and when they are based on health care, this mostly concerns primary care. Despite the pertinence of regarding organisational factors as being crucial for quality improvement, there is only limited research evidence for the claims made. The influence of possible organisational effect modifiers on the quality of care at hospitals needs more research attention and more evidence is needed from inside not outside hospitals for the development of theories in this field. Ongoing research should develop validated measures of potentially important organisational constructs and explore their influence on quality of care.

There is no 'magic bullet' in terms of the most effective strategy or organisational effect modifiers for the implementation of change within hospitals. On an organisational level, barriers against and facilitators for effective interventions are unclear. Depending on the management policy and other local factors, such as funds available and motivation of the health care personnel, hospitals might wish to focus on building a learning organisation [71] or on adopting proven, effective, strategies from outside.