Evaluation of a tailored, multi-component intervention for implementation of evidence-based clinical practice guidelines in primary care physical therapy: a non-randomized controlled trial
BMC Health Services Research volume 14, Article number: 105 (2014)
Clinical practice guidelines are important for transmitting research findings into practice and facilitating the application of evidence-based practice (EBP). There is a paucity of knowledge about the impact of guideline implementation strategies in primary care physical therapy. The aim of this study was to evaluate the effect of a guideline implementation intervention in primary care physical therapy in western Sweden.
An implementation strategy based on theory and current evidence was developed. A tailored, multi-component implementation intervention, addressing earlier identified determinants, was carried out in three areas comprising 28 physical therapy practices including 277 physical therapists (PTs) (intervention group). In two adjacent areas, 171 PTs at 32 practices received no intervention (control group). The core component of the intervention was an implementation seminar with group discussions. Among other components were a website and email reminders. Data were collected at baseline and follow-up with a web-based questionnaire. Primary outcomes were the self-reported awareness of, knowledge of, access to, and use of guidelines. Secondary outcomes were self-reported attitudes toward EBP and guidelines. Analyses were performed using Pearson’s χ2 test and approximative z-test.
168 PTs (60.6%) in the intervention group and 88 PTs (51.5%) in the control group responded to the follow-up questionnaire. 186/277 PTs (67.1%) participated in the implementation seminars, of which 97 (52.2%) responded. The proportions of PTs reporting awareness of (absolute difference in change 20.6%, p = 0.023), knowledge where to find (20.4%, p = 0.007), access to (21.7%, p < 0.001), and frequent use of (9.5%, NS) guidelines increased more in the intervention group than in the control group. The proportion of PTs reporting frequent guideline use after participation in the implementation seminar was 15.2% (p = 0.043) higher than the proportion in the control group. A higher proportion considered EBP helpful in decision making (p = 0.018). There were no other significant differences in secondary outcomes.
A tailored, theory- and evidence-informed, multi-component intervention for the implementation of clinical practice guidelines had a modest, positive effect on awareness of, knowledge of, access to, and use of guidelines, among PTs in primary care in western Sweden. In general, attitudes to EBP and guidelines were not affected.
The concept of evidence-based practice (EBP)  is increasingly permeating physical therapy practice, although there is considerable variation in the extent to which EBP is actually applied [2–5]. Physical therapists (PTs) use treatment methods with strong or moderate evidence of effect, but also methods with limited or no effect [6–9]. The body of evidence for physical therapy treatment is growing at a fast pace , making it challenging for clinicians to keep up with the latest findings. To bridge the gap between research and practice and facilitate the uptake of research findings in physical therapy practice, evidence-based clinical practice guidelines are increasingly being produced.
The use of physical therapy guidelines has been shown to contribute to EBP, improve the quality of care, and decrease costs [2, 8, 11]. However, the availability and use of guidelines in different countries and settings also tend to vary [3, 5, 12–14]. In Sweden, few guidelines for physical therapy treatments are available and less than half of the PTs in a recent survey stated that they use guidelines on a regular basis .
Evidence-based guidelines need to be accompanied by evidence-based implementation . There is growing evidence that active, multi-component strategies are effective in implementing change in professional behavior [16–21], and that components such as printed educational material (e.g., guidelines) , interactive education  and reminders  may be beneficial. Tailored implementation targeting specific barriers and facilitators has received increased attention and evidence for this approach is also growing [21, 25–27]. A recent Cochrane review concluded that a tailored implementation intervention is more likely to improve professional practice than no intervention or dissemination of guidelines . Different approaches have been suggested to link determinants with strategies, from theory-based  to more pragmatic “common sense” approaches [29, 30].
Implementation of guidelines or other measures to change practice behavior needs to be supported by theories or models [17, 31–33]. Theory can facilitate the implementation and increase the possibility to draw general conclusions on the effectiveness of the implementation strategy . Using theory provides a process and structure to support the development of a strategy or intervention and to guide its evaluation, thereby facilitating a better understanding of the generalizability and replicability of implementation interventions . There is some evidence that behavior change interventions that are informed by theory are more effective than those that are not . However, few guideline implementation interventions have so far been based on theory .
In physical therapy, relatively little is known about the effectiveness of various implementation strategies. A tailored, active, multi-component implementation strategy has been recommended . No study on a tailored strategy has been found, and there is only tentative evidence that active, multifaceted strategies are more effective than passive, single intervention strategies [37–39]. A systematic review on implementation of guidelines in physical therapy concluded that an active, multifaceted strategy was effective in improving knowledge and behavior, but not attitudes, patient outcomes, or cost of care and that the effects were mostly small . Hence, there is inconclusive evidence as to which strategies are most likely to be effective in different contexts. Studies investigating the implementation of guidelines in primary care physical therapy are scarce [40, 41] and none have been performed in Sweden.
A project to develop and implement evidence-based clinical practice guidelines for primary care physical therapy was initiated by a regional health authority in western Sweden. To introduce the new guidelines, a tailored, multi-component implementation intervention was developed, combining a theory-informed, evidence-based strategy with a pragmatic approach. The aim of this study was to evaluate the effectiveness on process outcomes of this guideline implementation intervention compared with no intervention.
Design and setting
The design of this study was two-fold and comprised a systematic implementation strategy development component and a non-randomized controlled trial in which the effectiveness of the implementation intervention was evaluated in comparison with no intervention. Data were collected at baseline and follow-up. The study took place within the county council Region Västra Götaland in western Sweden, providing first-line primary care of both acute and long-term nature to 1.6 million inhabitants. At the physical therapy practices, PTs primarily treat patients with musculoskeletal disorders; the most common complaints are low back pain, neck pain, and subacromial pain. There are 60 physical therapy practices in five areas within the county council, which were allocated to intervention (three areas, 28 practices) or control group (two adjacent areas, 32 practices) based on geographic location. The study was conducted between November 2010 and November 2011.
All PTs employed at the 60 physical therapy practices were eligible to participate in the study. At the time of the baseline data collection, 425 PTs were employed in primary care by the authority and at follow-up, 454 PTs were employed. The guideline project team members, who were also employed by the county council, were excluded (n = 6). Figure 1 illustrates participant flow through the study.
The initiative to the guideline development and implementation was taken by PT managers within the county council, who perceived a variation in physical therapy practice and therefore a need for evidence-based practice guidelines. An integrated organizational development/research project was planned. Support from senior management of the county council, as well as funding of the project, was secured. Guideline development, needs analysis, and implementation strategy development were carried out in parallel processes within the integrated project (Figure 2). The guideline development was performed by a project team of six primary care PTs, in a systematic process guided by a 7-step guideline development model by Grol et al. . The development process included systematic database searches, critical appraisal of the evidence base using GRADE , and formulating evidence-based practice recommendations. The guideline format followed recommendations from AGREE II . The guidelines consisted of a brief summary on the first page; a brief introduction to the topic with up-to-date data on definition of the condition, prevalence and prognosis; recommendations on patient management according to strength of evidence; and a detailed reference list. Detailed search strategy and search results, summaries of reference articles, recommended outcome measures, and patient information leaflets were provided in appendices to the guidelines.
Implementation strategy development
An implementation strategy was developed based on prevailing implementation theory , barriers and facilitators identified in the target population , and evidence from previous research. The latter included strategies that are theory-based [31, 44], active , multi-faceted , and tailored . The 5-step model for implementation by Grol et al.  was selected as a framework for the implementation, because it is based on a comprehensive overview of theories for behavioral change, integrates several theories that we considered relevant, and provides a structured approach for the implementation process. The model, and how it was adapted for the specific clinical and organizational context and applied to the strategy development, is presented in Figure 3. Steps one and two of the model were carried out in parallel, as the needs analysis in step two guided the choice of topics for the development of the guidelines in step one. As part of step two, a previously reported cross-sectional survey  was conducted. The implementation strategy developed in step three focused on addressing the determinants of guideline use identified in the survey. Determinants and how they were addressed in our implementation intervention are shown in Figure 4.
Further, an educational element was included in the implementation strategy, to address misconceptions of the EBP concept that emerged from the survey. To guide topic selection, input was also solicited in the survey regarding which guidelines were needed the most. Among the most commonly demanded topics were guidelines for physical therapy treatment of low back pain, neck pain, and subacromial pain. These diagnoses are also referenced in the literature as three of the most common reasons to consult a PT in primary care . In step four, details of the implementation activities were planned and the plan was executed (see below). In step five, data for the evaluation of the implementation were collected with the same questionnaire .
The implementation plan carried out in step four of the model comprised a multi-component intervention with the following components:
Guidelines in printed and electronic formats, with treatment recommendations linked to evidence levels and summarized in a front-page box.
A 3-hour implementation seminar (core component): The guideline development process, results and recommendations were presented (1.5 hours) together with a learning component on EBP and guidelines (0.5 hour) and integrated interactive group discussions (1 hour).
A specially developed guideline website, providing easy access to the guidelines, recommended outcome measures, links to medical databases and other EBP resources, and other related materials.
Links from local intranets to the guideline website, further facilitating access.
Bi-monthly e-mail reminders with brief “newsletter” style information.
Patient information leaflets with information and advice on self-care in line with the guideline recommendations, in printed and electronic format.
E-mail and telephone support by the project manager.
The seminar and group discussion sessions were held on nine different occasions over a 3-month period, with 9–28 PTs attending each session. In the group discussion, the guideline recommendations and current practice were discussed. The sessions were conducted by the first author and a project team co-member, both primary care PTs with postgraduate training in EBP skills. The seminars were held during working hours. Participation was supported and encouraged by senior managers but it was up to each unit manager to decide whether to send all or some of the unit’s PTs. Both the website and the seminar content were tailored to address areas in need of improvement that had been identified earlier .
Physical therapists in the control group received no intervention and continued to practice as usual, i.e., according to individual knowledge and experience. They did not receive any of the newly developed physical therapy guidelines, nor any information about the guideline website or the ongoing guideline project. After the follow-up data collection, the guidelines were introduced also to the control group. Publically available physical therapy and multidisciplinary guidelines, searchable on the Internet, were available to both groups during the study.
Data were collected at baseline and follow-up via a web-based, self-report questionnaire covering various aspects of EBP and guidelines . Follow-up data collection took place three to six months after the implementation seminars were carried out in the intervention group, corresponding to one year after baseline data collection.
The primary outcomes were the self-reported awareness of the existence of guidelines, knowledge of how to find guidelines, access to guidelines, and use of guidelines. The first three outcomes were selected because of their role as facilitators of guideline use, and the fourth outcome because of its importance in measuring the extent to which EBP was applied in practice. Awareness, knowledge, and access to guidelines were assessed with 3-point scales “yes”, “partially”, or “no”. Use of guidelines was assessed on a 5-point scale with response alternatives “never or very infrequently”, “infrequently”, “sometimes”, “frequently”, or “very frequently or always”. Secondary outcomes were self-reported attitudes to EBP and guidelines, assessed with 5-point scales ranging from “strongly disagree” to “strongly agree”.
The questionnaire had, in an earlier study, been found to be valid and of acceptable reliability in a primary care physical therapy context . In that study, it was translated, cross-culturally adapted, and further developed from a previously used instrument to assess EBP . Invitation to respond to the questionnaire was distributed via e-mail, containing a link to the questionnaire. Participants responded on-line and the survey software (EPiServer CMS 5, EPiServer AB, Stockholm, Sweden) logged the responses and added them to a results database. At both baseline and follow-up, three reminder notices were e-mailed at 1-week intervals. Staff turnover was analyzed by comparing the e-mail addresses between the two data collection points.
Ethical approval for the study was obtained from the Regional Ethical Review Board of Gothenburg (Reference 780–11). All questionnaires were filled out anonymously and responses could not be traced back to the respondents.
Sample size was calculated based on the assumption that a relevant difference between groups in the primary outcomes would be 15%. We estimated that we needed 137 PTs in each group to be able to detect a relevant change, with 80% power and a significance level of 5%. Assuming a response rate of approximately 65%, we needed to send the questionnaire to approximately 400 respondents, which corresponded well with the number of PTs employed in primary care in the county council.
Participating practices were analyzed using individual PTs as measurement units. Baseline data were compared between groups for age, gender, years of primary care experience, education level, workplace size, and for primary and secondary outcomes. Differences in proportions between groups were analyzed using Pearson’s χ2 test. Before the analysis, the response categories were dichotomized into the two highest and lowest categories. For the primary outcomes, primary analyses of data for all PTs in the intervention group were performed, regardless of whether they had participated in the implementation seminar, the main component of the intervention (“intention-to-treat” analysis). Secondary analysis was restricted to those participants who actually participated in the seminar and responded to the follow-up questionnaire (“per-protocol” analysis).
For the primary outcomes, absolute changes between baseline and follow-up in proportions answering “frequently/almost always” or “yes” were computed. Differences in change between the groups were computed and analyzed using an approximative z-test. A significance level was set at p < 0.05. Analyses were performed using IBM SPSS Statistics, version 20.
Of the 448 eligible PTs at follow-up, 277 belonged to the intervention group and 171 to the control group. Responses were received from 168 (60.6%) and 88 (51.5%) PTs, respectively. One hundred and eighty-six PTs participated in the implementation seminars, of which 97 (52.2%) responded (Figure 1).
Respondent and workplace characteristics in the two groups were comparable, both at baseline and follow-up, with the exception of workplace size; fewer PTs in the intervention group worked at clinics with 6−10 PTs and more worked at clinics with 11−15 PTs (Table 1). There were no differences in respondent and workplace characteristics within the groups between baseline and follow-up. Staff turnover averaged 10%, meaning that 90% of the study populations in the two surveys were the same.
At follow-up, a significantly higher proportion of PTs in the intervention group (59%) than in the control group (44%) reported being aware of guidelines (p = 0.030), knowing where to find guidelines (40% vs. 16%; p < 0.001), and having easy access to guidelines (26% vs. 7%; p < 0.001) (Table 2). The difference in guideline use was not statistically significant in intention-to-treat analysis (55% vs. 48%; p = 0.081), but was significant in per-protocol analysis, i.e., PTs who participated in the implementation seminar reported frequent use of guidelines to significantly greater extent than those in the control group (63% vs. 48%; p = 0.043).
Within-group changes in proportions from baseline to follow-up are shown in Table 3. The differences between the groups in this change were also statistically significant for three of the four primary outcomes (Table 3). The proportions of PTs reporting awareness of (p = 0.023), knowledge of (p = 0.007), and easy access to (p < 0.001) guidelines increased more in the intervention group than in the control group. The change in self-reported use of guidelines did not differ significantly (p = 0.297). Differences in proportions were between 9% and 22%.
Table 4 shows that there were no statistically significant differences in secondary outcomes. The only exception was “EBP helps decision making”, but this difference was small. Per-protocol analyses of secondary outcomes produced similar results (not reported).
This study showed that a tailored, multi-component guideline implementation intervention, informed by theories as well as current evidence and structured around an implementation model, can affect the frequency of guideline use, as well as several determinants of guideline use, among PTs in primary care. Attitudes, in general, were not affected. Significant effects were seen in the intervention group in three of the four primary outcomes. The effects were of a moderately large magnitude (20% to 22%) and can be considered relevant. Awareness of the existence of the guideline is a critical prerequisite for the use of guidelines and is logically the first step toward adherence . Awareness and familiarity with the guidelines’ content are likely to affect implementation . Knowing where to find guidelines and having easy access to them are also important prerequisites for the use of guidelines, the latter often considered an organizational or contextual barrier .
However, the effect on these facilitators of guideline use did not carry through to the fourth primary outcome. Use of guidelines only increased by an insignificant 9% in the whole intervention group and a significant 17% among those who participated in the seminar, respectively, while remaining unchanged in the control group (Table 2). It is well established that changes in cognitive factors such as knowledge, beliefs, and attitudes do not necessarily translate into behavior change [49, 50]. Knowledge is but one of 12 different domains that have been identified to influence implementation of guidelines and EBP .
Comparison to previous studies
Our findings support the conclusions of two systematic reviews [38, 39], concluding that PTs’ knowledge and, to some extent, behavior but not attitudes were improved after active multi-component implementation interventions. However, a third systematic review  found that multifaceted guideline implementation interventions were no more effective than single intervention strategies across several allied health professions. Two randomized controlled studies (included in the reviews) compared an active implementation intervention versus passive dissemination of physical therapy guidelines for low back pain and whiplash disorders, respectively [40, 41]. The active interventions were similar to ours and included interactive training, group discussions, and reminders. The studies found a 12% and a 44%, respectively, higher self-reported guideline adherence rate after the active strategy; a better result than in our study. However, in the study on whiplash guidelines, the effects on adherence were mixed; two of the guideline’s five recommendations were adhered to more by the intervention PTs than by those in the control group, while this was not the case for the other three recommendations.
A recent study of PTs and other musculoskeletal practitioners in the UK, reported a 60% guideline adherence rate after an implementation of guidelines for low back pain via posted information, versus 55% in the no-intervention control group; only a 5% absolute difference .
Hence, effect sizes in the few guideline implementation studies that have been published in physical therapy varied between 5% and 44%. In other healthcare areas, effects of implementation interventions are known to be modest. Systematic reviews [53, 54] have reported average effects of guideline implementation on performance or process of care between 5% and 10%, consistent with the findings of this study.
The increase in the proportion using guidelines frequently, non-significant in the ITT analysis but significant in per protocol analysis, implies an encouraging trend and the impact is likely to be visible over time as these PT’s patients can be expected to receive guideline-informed, evidence-based treatment. The corresponding effect on infrequent guideline use may be even more important. The proportion of PTs who reported infrequent guideline use was reduced from 12% to 4% in the intervention group. The implication of this is that very few PTs practice without consideration of the guidelines after the implementation intervention, which should have a positive impact on the quality of care.
The post-intervention rate of frequent guideline use in our study, 55%, can still be considered quite low, leaving room for future improvement. Changing behavior to start using guidelines more frequently is a process that takes time, and we measured only a few months after the implementation activities. A Dutch survey examining guideline adherence three years after a postal dissemination of physical therapy guidelines for low back pain, reported a corresponding self-reported adherence rate of 61% , only slightly higher. In other cross-sectional surveys, varying rates of guideline adherence have been reported among PTs in different countries and settings, ranging from 40% in the United States  and 45% in Australia , to 75%-86% in Sweden [13, 14]. Comparisons are however treacherous, as data were collected and analyzed differently in the studies.
On the other hand, 55% may not be that low. If we add the proportion of PTs who reported using guidelines “sometimes”, the rate increases to a high 96%. “Sometimes” might be a perfectly adequate frequency, e.g., if you don’t see patients with diagnoses for which guidelines exist more often than “sometimes” or if you have integrated guideline recommendations in your clinical decision making to the point that you are not even aware of using them.
Possible explanations of the results
There are several possible explanations for the modest effect on guideline use. Rogers  has suggested that the adoption of new innovations is highly influenced by the innovation characteristics and has described five perceived attributes of the implementation object: relative advantage, compatibility, complexity, trialability, and observability. Grilli and Lomas  found that of those, trialability (the degree to which an innovation may be tried on a limited basis) was the most important attribute of guidelines, enhancing adherence, and that complexity was associated with low adherence rates. Compatibility with existing clinical practice has also been shown to enhance adherence . In our project, trialability and complexity were addressed already in the development of the guidelines. Because the guideline recommendations did not differ much from existing practice, compatibility was also achieved. However, this very fact could maybe explain the modest effect on the use of guidelines; since they did not introduce any new revolutionary methods but rather confirmed existing practice, using them frequently might not be perceived as necessary.
Generally, it has been suggested that the modest effect of most implementation interventions could be due to the complexity of implementation research . The primary health care context involves many stakeholders at multiple levels, creating significant conceptual and methodological challenges such as developing effective study designs and selecting appropriate outcomes. Furthermore, to achieve a practice change, managerial and organizational support for an implementation project is crucial.
The fact that the per-protocol analysis yielded significant effect on guideline use where the intention-to-treat analysis did not, could either suggest that participating PTs did not share the content of the implementation seminar sufficiently with non-attending colleagues, or that there was a diffusion effect between the intervention and control groups. This finding indicates the importance of implementation seminar attendance, which therefore should be more strongly encouraged or mandated by managers.
Naturally, the positive effect on awareness, knowledge, and access were likely related to the components of the intervention that addressed these outcomes, and the specifically created website was a main contributor. Verbal information at the implementation seminars, supplemented by well-structured guideline information and supporting documents on the website as well as regular e-mail reminders, is likely to have contributed to increased awareness, knowledge and perceived easy access.
Attitudes to EBP and guidelines were for the most part not affected. It would have been difficult to demonstrate statistical significance within this study because participants in both groups were already very positive at baseline; hence there was little room for improvement.
The rather high baseline levels of awareness and use of guidelines (in both groups), also likely contributors to the modest effects on guideline use, could be explained by how the survey questions were phrased and the availability of other guidelines. PTs with good search skills and an interest in searching for guidelines could have searched and found guidelines on their own initiative. The questions were phrased in a rather general way: “are you aware of evidence-based clinical practice guidelines relevant for your practice?” and “how often do you use evidence-based clinical practice guidelines?”. We didn’t ask about the study specific guidelines since they were to be developed. At the time of the follow-up, PTs in the intervention group most likely interpreted the questions as referring to the recently developed and introduced guidelines, whereas the control group most likely interpreted the questions in a general way. Thus, the high baseline rates of guideline use imply that other guidelines than those introduced in the intervention, were used in both groups.
The use of a structured model to guide the implementation project was perceived as useful by the project team. Because the content and recommendations of the guidelines did not differ much from existing practice, complex theories were not considered necessary. However, because the model by Grol et al.  is rather pragmatic and integrates several theories in a non-explicit way, it may be less suitable for gaining a deeper understanding of how change was achieved. Linking intervention components, or behavior change techniques, to specific theories of behavior change, could be a more effective strategy to reach this goal .
Tailoring implementation components to identified determinants was also perceived as a useful and logical strategy. The core component of the intervention, the implementation seminar, together with the website, addressed most of the identified determinants. The inclusion of interactive group discussions has been suggested as one of the possible tailoring mechanisms that could enhance the impact of educational meetings in implementation . Few studies have been performed that use a tailored strategy to implement change in physical therapy practice. Stevens and Beurskens developed and evaluated a tailored strategy to implement measurement instruments in a Dutch physical therapy context . They concluded, based on interviews with a small group of PTs, that the strategy was applicable to physical therapy practice and effective in improving awareness, knowledge, attitude and use of measurement instruments.
As illustrated in Figure 3, we linked the components of our intervention to the identified determinants, as recommended in the literature . In addition to identifying barriers and facilitators in our population via the baseline data collection, we supplemented these findings with a literature search. Using more than one method to identify determinants is recommended . The largest barrier identified, lack of time, was addressed indirectly by attempting to reduce several of the other barriers, such as poor awareness, availability of and access to guidelines, i.e., decreasing the time required to search for evidence. It was also addressed in the guideline development process, as the guidelines were developed with this barrier in mind. They were designed to be brief and concise. The characteristics of the guidelines were carefully considered as they are likely to affect actual use. Guidelines that are easy to understand, can easily be tried out, and do not require specific resources have a greater chance of implementation . Lack of time is however mainly an organizational or contextual barrier that was not possible to address directly in this intervention.
While social cognitive determinants such as the ones addressed in our study have been shown also in earlier research to influence clinical behavior , in particular the use of research evidence , a less explored factor is the role of habits in clinical behavior. In a fairly stable context such as the one in this study, clinical practice can be assumed to be habitual and habits, i.e., repeated behaviors, can in stable contexts be difficult to change . Especially in a situation where the guideline recommendations did not differ much from existing practice, many PTs may not have felt a necessity to change their habits and start using guidelines more frequently.
Other possible strategies would have been to use audit and feedback or local opinion leaders. However, according to Grimshaw et al’s recent review on various implementation strategies , audit and feedback seem to be less effective than educational strategies. The one strategy that stands out, with a median effect size of 12% versus 4-6% for other strategies, is the use of local opinion leaders. This was not known at the time of our strategy development, nor were there any obvious candidates in our study setting. This strategy would require the availability of credible and influential persons, as well as an intact social professional network . With those conditions satisfied, it would be a recommended strategy in future implementation endeavours.
The study population is likely to be broadly similar to the national population of primary care PTs in Sweden. Although the study was conducted in only one county council/geographic region, this particular county council is the second largest in Sweden. Thus, we believe that our results can be generalized to PTs working in primary care in other county councils in Sweden and possibly also to other countries, particularly those in which, like Sweden, PTs have a large degree of autonomy.
The study has several limitations. Only process outcomes at practitioner level were measured and we do not know whether the intervention had, or will have, an impact on patient outcomes. We only measured self-reported behavior, which entails a risk of overestimation and social desirability bias, and limits our ability to draw strong conclusions concerning actual behavior .
To minimize social desirability bias, the questionnaires were anonymous and it was therefore not possible to track individual responses and changes between the two test occasions. Statistically, only comparisons between independent groups were possible. However, staff turnover was rather low during the study period, with an average of 90% of PTs remaining the same in the respective groups from baseline to follow-up measurements. The study population can therefore be considered rather stable.
As with all non-randomized studies, it cannot be fully ascertained that the observed changes can be attributed to the intervention. Randomization was in this project not feasible, as a likely diffusion effect between participants would have made data interpretation difficult. Instead, the second strongest design, a controlled before–after design with a comparable control group, was used . Data were collected in both populations contemporaneously using the same method at baseline and follow-up. A ‘between group’ analysis comparing performance in the study and control groups was performed, and the observed differences could be assumed to be due to the intervention . Both groups were similar at baseline and could be expected to experience secular trends or changes in the same extent, increasing the confidence with which the observed changes can be attributed to the intervention .
Communication and interaction between PTs in the intervention and control group on individual or group levels, e.g., meetings, conferences, were not unlikely, causing a risk for contamination between the two groups and reducing the apparent effect of the intervention.
The short duration between implementation and follow-up evaluation is likely to have contributed to the modest effect on guideline use. The introduction of complex workplace change, especially as concerns EBP, takes considerable time . The actual use of, or adherence to, a guideline has been described as the last in a sequence of cognitive and behavioral steps that begin with awareness , and to expect a high impact on guideline use so soon after the intervention is probably not realistic. A longer follow-up period would have been useful but was not possible due to the guidelines and website being made available also to the control group after the follow-up survey was conducted.
Another limitation, possibly contributing to the modest effect on guideline use, could be that the education component of the implementation intervention comprised only one session. Two systematic reviews have shown that educational meetings can improve both professional practice and patient outcomes and that sessions with interactive components may have greater impact [23, 69]. However, the effectiveness of various educational strategies is not conclusive and factors such as time and intensity have been proposed to influence outcomes . One single session may not be enough, although we took effort to make it as interactive as possible. Interactive educational strategies have consistently been reported as more effective than traditional, didactic education and engaging the clinician is recommended . Additional educational sessions might improve the effect of the implementation, but this would need to be investigated.
The intervention focused on the individual level. Mid-level managers have a potentially pivotal role as implementation agents and their proactivity and commitment have been suggested to influence implementation effectiveness . A multi-level approach and particularly targeting also unit managers would likely have increased the impact of our implementation intervention.
Although the questionnaire used was previously validated in a similar population of primary care PTs and found to have acceptable reliability, there are some interpretation issues. The response categories (e.g., what is “frequently”?) may have been interpreted differently by the respondents. In the data analysis, the categorization/dichotomization of the responses is not obvious, and may have been performed differently in different studies, rendering comparison difficult.
Implications for research
Measuring the effect of implementation interventions as well as guideline use is very complex, but very important. To build upon the results of this study, further research is recommended that evaluates an implementation strategy that comprises more than one educational session, and that is designed to enable a longer follow-up period, e.g., two years. Using a more rigorous design, i.e., a randomized trial, is paramount to reduce risk of bias and be able to draw stronger conclusions. In view of the modest effect on guideline use and the considerable resources required for an active, multi-component intervention such as the one in this study, a comparison between different implementation strategies with various components could provide valuable knowledge. A strategy that needs to be evaluated in the physical therapy context would be the use of local opinion leaders. Linking intervention components, or behavior change techniques, to specific theories of behavior change could facilitate a greater understanding of what works and what doesn’t. Data should be collected so that individual changes from baseline to follow-up can be analyzed. It is also very important to measure the effect on patient outcomes as well as the cost-effectiveness of guideline implementation strategies. Furthermore, quantitative evaluation studies need to be accompanied by qualitative studies to achieve increased understanding of effect mechanisms of implementation interventions and practice behavior change. Lastly, the patient’s voice is seldom heard in implementation research and patient views and preferences should be included in an evaluation of the barriers and facilitators for implementation, development of the implementation intervention, and in choosing relevant outcomes.
Implications for practice
The implementation of the guidelines can be expected to have an impact on physical therapy practice in the target population in several phases. In a first phase, the short term effects of the implementation – improved awareness of guidelines, knowledge about where to find them, and ability to access them easily through the dedicated website – will set the stage for a second phase, increased use of the guidelines thereby facilitating the application of EBP. In a third, long-term phase, patients could be expected to benefit from this, as they will be treated with methods known to be the most effective according to current evidence. This could, however, be expected to take quite some time. The effect of a guideline implementation, such as the one performed in this study, might be enhanced by conducting additional educational sessions. On the other hand, it might also be relevant, and more cost-effective, to consider a scaled-down educational strategy, such as training “ambassadors” that can train PTs at their workplace instead of having them all come to larger, out-of-office seminars. Organizational barriers could also be addressed to a larger extent, e.g., by securing active support from senior management and arranging for further discussions of the guidelines at the workplace. Decision makers need to carefully consider whether the benefits of implementation efforts are important enough to outweigh the costs involved.
In conclusion, this study demonstrates that a tailored, multi-component intervention for the implementation of clinical practice guidelines, informed by theory as well as current evidence, had a positive effect on self-reported awareness of the existence of guidelines, knowledge of how to find guidelines, and access to guidelines in PTs in primary care in western Sweden. While these factors are important prerequisites to and determinants of guideline use, the self-reported use of guidelines was not affected to the same extent. The results support previous findings on physical therapy guideline implementation strategies that active, multi-component strategies are effective but that the effects are modest.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn't. BMJ. 1996, 312: 71-72. 10.1136/bmj.312.7023.71.
Fritz J, Cleland JA, Brennan GP: Does adherence to the guideline recommendation for active treatments improve the quality of care for patients with acute low back pain delivered by physical therapists?. Med Care. 2007, 45: 973-980. 10.1097/MLR.0b013e318070c6cd.
Jette DU, Bacon K, Batty C, Carlson M, Ferland A, Hemingway RD, Hill JC, Ogilvie L, Volk D: Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther. 2003, 83: 786-805.
Turner P, Whitfield TW: Physiotherapists' use of evidence based practice: a cross-national study. Physiother Res Int. 1997, 2: 17-29. 10.1002/pri.76.
Bernhardsson S, Johansson K, Nilsen P, Oberg B, Larsson ME: Determinants of guideline use in primary care physical therapy: a cross-sectional survey of attitudes, knowledge, and behavior. Phys Ther. 2013 Oct 31, [Epub ahead of print]
Holden MA, Nicholls EE, Hay EM, Foster NE: Physical therapists' use of therapeutic exercise for patients with clinical knee osteoarthritis in the United Kingdom: in line with current recommendations?. Phys Ther. 2008, 88: 1109-1121. 10.2522/ptj.20080077.
Walsh NE, Hurley MV: Evidence based guidelines and current practice for physiotherapy management of knee osteoarthritis. Musculoskeletal Care. 2009, 7: 45-56. 10.1002/msc.144.
Liddle SD, Baxter GD, Gracey JH: Physiotherapists' use of advice and exercise for the management of chronic low back pain: a national survey. Man Ther. 2009, 14: 189-196. 10.1016/j.math.2008.01.012.
Mikhail C, Korner-Bitensky N, Rossignol M, Dumas JP: Physical therapists' use of interventions with high evidence of effectiveness in the management of a hypothetical typical patient with acute low back pain. Phys Ther. 2005, 85: 1151-1167.
Maher CG, Moseley AM, Sherrington C, Elkins MR, Herbert RD: A description of the trials, reviews, and practice guidelines indexed in the PEDro database. Phys Ther. 2008, 88: 1068-1077. 10.2522/ptj.20080002.
Overmeer T, Linton SJ, Holmquist L, Eriksson M, Engfeldt P: Do evidence-based guidelines have an impact in primary care? A cross-sectional study of Swedish physicians and physiotherapists. Spine (Phila Pa 1976). 2005, 30: 146-151.
Iles R, Davidson M: Evidence based practice: a survey of physiotherapists' current practice. Physiother Res Int. 2006, 11: 93-103. 10.1002/pri.328.
Nilsagård Y, Lohse G: Evidence-based physiotherapy: a survey of knowledge, behaviour, attitudes and prerequisites. Adv Physiother. 2010, 12: 179-186. 10.3109/14038196.2010.503812.
Heiwe S, Kajermo KN, Tyni-Lenne R, Guidetti S, Samuelsson M, Andersson IL, Wengstrom Y: Evidence-based practice: attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care. 2011, 23: 198-209. 10.1093/intqhc/mzq083.
Grol R: Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001, 39: II46-54.
Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003, 362: 1225-1230. 10.1016/S0140-6736(03)14546-1.
Grol R, Wensing M, Eccles M: Improving Patient Care; The implementation of change in clinical practice. 2005, Edinburgh, Scotland: Elsevier Butterworth Heineman
Prior M, Guerin M, Grimmer-Somers K: The effectiveness of clinical guideline implementation strategies–a synthesis of systematic review findings. J Eval Clin Pract. 2008, 14: 888-897. 10.1111/j.1365-2753.2008.01014.x.
Francke AL, Smit MC, de Veer AJ, Mistiaen P: Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Dec Mak. 2008, 8: 38-10.1186/1472-6947-8-38.
Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative G: Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes. 2011, 4: 212-10.1186/1756-0500-4-212.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE: Knowledge translation of research findings. Implement Sci. 2012, 7: 50-10.1186/1748-5908-7-50.
Giguere A, Legare F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A, Makosso-Kallyth S, Wolf FM, Farmer AP, Gagnon MP: Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012, 10: CD004398
Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009, 2: CD003030.
Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J: Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Syst Rev. 2012, 1: 36-10.1186/2046-4053-1-36.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N: Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010, 3: CD005470.
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O'Brien MA: Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001, 39: II2-II45.
Bosch M, van der Weijden T, Wensing M, Grol R: Tailoring quality improvement interventions to identified barriers: a multiple case analysis. J Eval Clin Pract. 2007, 13: 161-168. 10.1111/j.1365-2753.2006.00660.x.
Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychol Int Rev. 2008, 57: 660-680. 10.1111/j.1464-0597.2008.00341.x.
Oxman AD, Fretheim A, Flottorp S: The OFF theory of research utilization. J Clin Epidemiol. 2005, 58: 113-116. 10.1016/j.jclinepi.2004.10.002. discussion 117–120
Bhattacharyya O, Reeves S, Garfinkel S, Zwarenstein M: Designing theoretically-informed implementation interventions: fine in theory, but evidence of effectiveness in practice is needed. Implement Sci. 2006, 1: 5-10.1186/1748-5908-1-5.
Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behaviour of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58: 107-112. 10.1016/j.jclinepi.2004.09.002.
Rycroft-Malone J, Bucknall T: Using theory and frameworks to facilitate the implementation of evidence into practice. Worldwide Evid Based Nurs. 2010, 7: 57-58.
Davies P, Walker AE, Grimshaw JM: A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Implement Sci. 2010, 5: 14-10.1186/1748-5908-5-14.
Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG): Designing theoretically-informed implementation interventions. Implement Sci. 2006, 1: 4.
Abraham C, Kelly MP, West R, Michie S: The UK National Institute for Health and Clinical Excellence public health guidance on behaviour change: a brief introduction. Psychol, Health Med. 2009, 14: 1-8. 10.1080/13548500802537903.
Zidarov D, Thomas A, Poissant L: Knowledge translation in physical therapy: from theory to practice. Disabil Rehabil. 2013, 35: 1571-1577. 10.3109/09638288.2012.748841.
Hakkennes S, Dodd K: Guideline implementation in allied health professions: a systematic review of the literature. Qual Saf Health Care. 2008, 17: 296-300. 10.1136/qshc.2007.023804.
Menon A, Korner-Bitensky N, Kastner M, McKibbon KA, Straus S: Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. J Rehabil Med. 2009, 41: 1024-1032. 10.2340/16501977-0451.
van der Wees PJ, Jamtvedt G, Rebbeck T, de Bie RA, Dekker J, Hendriks EJM: Multifaceted strategies may increase implementation of physiotherapy clinical guidelines. a systematic review. Aust J Physiother. 2008, 54: 233-241. 10.1016/S0004-9514(08)70002-3.
Bekkering GE, Hendriks HJM, van Tulder MW, Knol DL, Hoijenboes M, Oostendorp RA, Bouter LM: Effect on the process of care of an active strategy to impement clinical guidelines on physiotherapy for low back pain: a cluster randomised controlled trial. Qual Saf Health Care. 2005, 14: 107-112. 10.1136/qshc.2003.009357.
Rebbeck T, Maher CG, Refshauge KM: Evaluating two implementation strategies for whiplash guidelines in physiotherapy: a cluster-randomised trial. Aust J Physiother. 2006, 52: 165-174. 10.1016/S0004-9514(06)70025-3.
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, Group GW: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008, 336: 924-926. 10.1136/bmj.39489.470347.AD.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P, Makarski J, Zitselberger L, AGREE Next Steps Consortium: AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010, 182: E839-E842. 10.1503/cmaj.090449.
Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M: Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007, 85: 93-138. 10.1111/j.1468-0009.2007.00478.x.
Green S, Buchbinder R, Hetrick S: Physiotherapy interventions for shoulder pain. Cochrane Database Syst Rev. 2003, 2: CD004258.
Bernhardsson S, Larsson ME: Measuring evidence-based practice in physical therapy: Translation, adaptation, further development, validation, and reliability test of a questionnaire. Phys Ther. 2013, 93: 819-832. 10.2522/ptj.20120270.
Pathman DE, Konrad TR, Freed GL, Freeman VA, Koch GG: The awareness-to-adherence model of the steps to clinical guideline compliance. The case of pediatric vaccine recommendations. Med Care. 1996, 34: 873-889. 10.1097/00005650-199609000-00002.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR: Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999, 282: 1458-1465. 10.1001/jama.282.15.1458.
Scott SD, Albrecht L, O'Leary K, Ball GD, Hartling L, Hofmeyer A, Jones CA, Klassen TP, Kovacs Burns K, Newton AS, Thompson D, Dryden DM: Systematic review of knowledge translation strategies in the allied health professions. Implement Sci. 2012, 7: 70-10.1186/1748-5908-7-70.
Nilsen P, Roback K, Brostrom A, Ellstrom PE: Creatures of habit: accounting for the role of habit in implementation research on clinical behaviour change. Implement Sci. 2012, 7: 53-10.1186/1748-5908-7-53.
Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, Psychological Theory G: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14: 26-33. 10.1136/qshc.2004.011155.
Evans DW, Breen AC, Pincus T, Sim J, Underwood M, Vogel S, Foster NE: The effectiveness of a posted information package on the beliefs and behavior of musculoskeletal practitioners: the UK Chiropractors, Osteopaths, and Musculoskeletal Physiotherapists Low Back Pain ManagemENT (COMPLeMENT) randomized trial. Spine (Phila Pa 1976). 2010, 35: 858-866. 10.1097/BRS.0b013e3181d4e04b.
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004, 8: iii-iv. 1–72
Grol R: Improving the quality of medical care. Building bridges among professional pride, payer profit, and patient satisfaction. JAMA. 2001, 286: 2578-2585. 10.1001/jama.286.20.2578.
Rutten G, Kremers S, Rutten S, Harting J: A theory-based cross-sectional survey demonstrated the important role of awareness in guideline implementation. J Clin Epidemiol. 2009, 62: 167-176. 10.1016/j.jclinepi.2008.04.004. e161
Rogers EM: Diffusion of innovations. 2003, New York: Free Press, 5
Grilli R, Lomas J: Evaluating the message: the relationship between compliance rate and the subject of a practice guideline. Med Care. 1994, 32: 202-213. 10.1097/00005650-199403000-00002.
Foy R, MacLennan G, Grimshaw J, Penney G, Campbell M, Grol R: Attributes of clinical recommendations that influence change in practice following audit and feedback. J Clin Epidemiol. 2002, 55: 717-722. 10.1016/S0895-4356(02)00403-1.
Bhattacharyya OK, Estey EA, Zwarenstein M: Methodologies to evaluate the effectiveness of knowledge translation interventions: a primer for researchers and health care managers. J Clin Epidemiol. 2011, 64: 32-40. 10.1016/j.jclinepi.2010.02.022.
Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE: The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013, 46: 81-95. 10.1007/s12160-013-9486-6.
Gagliardi AR: Tailoring interventions: examining the evidence and identifying gaps. J Cont Educ Health Prof. 2011, 31: 276-282. 10.1002/chp.20141.
Stevens A, Beurskens AJMH: Implementation of measurement instruments in physical therapist pratice: development of a tailored strategy. Phys Ther. 2010, 90: 953-961. 10.2522/ptj.20090105.
Flottorp S, Oxman AD: Identifying barriers and tailoring interventions to improve the management of urinary tract infections and sore throat: a pragmatic study using qualitative methods. BMC Health Serv Res. 2003, 3: 3-10.1186/1472-6963-3-3.
Godin G, Bélanger-Gravel A, Eccles M, Grimshaw J: Healthcare professionals' intentions and behaviours: a systematic review of studies based on social cognitive theories. Implement Sci. 2008, 3: 36-10.1186/1748-5908-3-36.
Lizarondo L, Grimmer-Somers K, Kumar S: A systematic review of the individual determinants of research evidence use in allied health. J Multidiscip Healthc. 2011, 4: 261-272.
Streiner DL, Norman GR: Health measurement scales: a practical guide to their development and use. 2008, Oxford, New York: Oxford University Press, 4
Grimshaw J, Campbell M, Eccles M, Steen N: Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract. 2000, 17: S11-S16. 10.1093/fampra/17.suppl_1.S11.
McCluskey A, Cusick A: Strategies for introducing evidence-based practice and changing clinician behaviour: a manager’s toolbox. Aust Occup Ther J. 2002, 49: 63-70. 10.1046/j.1440-1630.2002.00272.x.
Marinopoulos SS, Dorman T, Ratanawongsa N, Wilson LM, Ashar BH, Magaziner JL, Miller RG, Thomas PA, Prokopowicz GP, Qayyum R, Bass EB: Effectiveness of continuing medical education. Evid Rep Technol Assess. 2007, 149: 1-69.
Birken SA, Lee SY, Weiner BJ, Chin MH, Schaefer CT: Improving the effectiveness of health care innovation implementation: middle managers as change agents. Med Care Res Rev. 70: 29-45.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/14/105/prepub
The authors wish to thank participating PTs for data provision and for engaged participation and many valuable contributions in the implementation seminars. Thanks also to Gunilla Jonsson and Marc Karlsson, who were instrumental in the initiation and execution of the project, respectively, and to Magnus Pettersson for statistical advice. This study was financially supported by the Health and Medical Care Committee of the Regional Executive Board, Region Västra Götaland. SB is partially supported by a doctoral grant from Linköping University. The funding bodies had no role in the design; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.
The authors declare that they have no competing interests. The first author was project manager for the guideline development and implementation projects. All authors are responsible for the content and writing of the paper.
SB contributed to the design, data acquisition and analysis, and drafted the manuscript. ML participated in design, data analysis and interpretation, and drafting of the manuscript. KJ, PN and BÖ contributed to data analysis and interpretation, and drafting of the manuscript. RE, MFO, LN and MvT contributed to the design and critically revised the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Bernhardsson, S., Larsson, M.E., Eggertsen, R. et al. Evaluation of a tailored, multi-component intervention for implementation of evidence-based clinical practice guidelines in primary care physical therapy: a non-randomized controlled trial. BMC Health Serv Res 14, 105 (2014). https://doi.org/10.1186/1472-6963-14-105