Skip to main content

Effectiveness of a tailored implementation strategy to improve adherence to a guideline on mental health problems in occupational health care



As compliance to guidelines is generally low among health care providers, little is known about the impact of guidelines on the quality of delivery of care. To improve adherence to guideline recommendations on mental health problems, an implementation strategy was developed for Dutch occupational physicians (OPs). The aims were 1) to assess adherence to a mental health guideline in occupational health care and 2) to evaluate the effect of a tailored implementation strategy on guideline adherence compared to traditional guideline dissemination.


An audit of medical records was conducted as part of a larger RCT study. Participants were 66 OPs (32 intervention and 34 control) employed at one of six sites of an Occupational Health Service in southern Netherlands. OPs in the intervention group received multiple-session peer group training which focused on identifying and addressing barriers to using the guideline, using a Plan-Do-Check-Act approach. The control group did not receive training.

Medical records of 114 workers sick-listed with mental health problems were assessed (56 intervention and 58 control). Guideline adherence was determined by auditing the records using 12 guideline-based performance indicators (PI), grouped into 5 PIs: process diagnosis, problem orientation, interventions/treatment, relapse prevention, and continuity of care. Differences in performance rates of the PIs between the intervention and control groups were analyzed, taking into account the cluster study design.


OPs who received the training showed significantly greater adherence compared to the controls (p < .028) in 4 out of 5 grouped PIs, i.e. process diagnosis, problem orientation, interventions/treatment and relapse prevention. In one out of 12 PIs adherence was found adequate (53% of the medical records), in 6 PIs adherence was found minimal, and in 5 PIs the majority of the records showed no adherence.


An implementation strategy which addressed key barriers for change and tailor-made interventions improves adherence to an occupational health guideline for mental health problems compared to traditional guideline dissemination. However, adherence to the guideline recommendations is still far from optimal. To optimize adherence, it is recommended that implementation strategies focus on the workers level, organizational level, and the professional level.

Trial registration

ISRCTN86605310. Registered 30 June 2010.

Peer Review reports


Adherence to guidelines is generally low among health care professionals, even though many evidence-based practice guidelines exist and are recommended to be used in health care [1, 2]. Lack of guideline adherence can lead to refraining from essential care, suboptimal patient outcomes and wasted resources [3]. To improve the quality of patient treatment and decrease variability in care, it is important to improve implementation of and adherence to practice guidelines [4].

Unfortunately, many studies have demonstrated a lack of compliance to guidelines [5, 6]. As a result, it remains unclear whether practice guidelines have any impact on the performance of the providers and on outcomes on patient level [7]. Multiple factors can be of influence on guideline implementation, such as patient and provider characteristics, environmental factors, and the socio-political context [6, 8]. Cabana and colleagues [9] have shown that barriers to guideline adherence can be prevalent on different levels; barriers might be knowledge-related like a lack of awareness with the guideline, or attitude-related such as a lack of agreement with specific recommendations, lack of self-efficacy and skills to apply recommendations, or lack of motivation to change physician routine. Also external barriers can obstruct guideline use, such as patient factors (i.e. preferences of patients), guideline factors (i.e. not clearly written), and environmental factors like lack of time or resource, or organizational constraints hindering guideline use. To improve guideline adherence, passive implementation strategies, such as guideline dissemination, are found to be ineffective. Also, education meetings that solely focus on improving knowledge (such as lectures) will not lead to the desirable change in behavior of professionals [10]. To be effective in improving adherence, active strategies are needed that aim to eliminate barriers that hinder professionals from adhering to a specific guideline [11]. Therefore, firstly it is important to identify the barriers that are perceived by the target group [12]. Furthermore, it is recommended to use implementation strategies tailored to the needs of the target group to overcome perceived barriers of specific guideline recommendations [11, 13].

In occupational health care, scientific evidence is growing and practice guidelines are increasingly developed for their use in occupational health care settings [14,15,16]. Specifically for the treatment of mental health problems by occupational health professionals, various practice guidelines have been developed worldwide [17,18,19]. The Dutch guideline entitled ‘The management of mental health problems of workers by occupational physicians (OPs)’ (further referred to as ‘MHP guideline’) is one of these [20, 21]. Currently, work disability is primarily caused by mental health problems, such as mood and anxiety disorders and provokes major challenges for societies all over the world [22,23,24]. The MHP guideline aims to establish improved and sustainable work functioning and relapse prevention among workers with mental health problems. One of the central aspects of this guideline for OPs is to follow an activating approach aiming to enhance the problem solving capacity of workers, particularly in relation to their work context. Despite various efforts to implement the MHP guideline, research shows that OPs only minimally adhere to the guideline’s recommendations in practice [25, 26]. However, OPs do have a positive attitude towards the guideline in general and they have the intention to use it [27].

This study is part of a larger project which aims to explore how the management of sick-listed workers with mental health problems by OPs can be improved [28]. In the larger project, an implementation strategy (i.e. interactive peer group training) is developed, implemented and tested to improve guideline adherence among OPs, and to evaluate its effect on workers’ return to work using a cluster randomized controlled trial [28]. In a qualitative study as part of the larger project, it was found that OPs perceived multiple barriers to use the guideline in their practice, including a lack of knowledge of the content of all guideline parts and attitude related barriers such as lack of self-efficacy to perform certain recommendations, and difficulties with changing their routines and habits [29]. Additionally, external barriers, most commonly work contextual barriers, obstructed guideline use. These were a lack of time and perceived work pressure, and tight contracts between employers and occupational health services in which for example agreements are made about the frequency of consultations. Also a lack of collaboration with other stakeholders such as the employer and other health care providers, and conflicting policy was perceived as a barrier to adequately use the guideline as recommended [29]. In a process evaluation, it was found that the peer group training was a feasible, and a highly valued method among participating OPs. The training added to OPs’ knowledge, attitudes and skills, but OPs still perceived various external barriers using the guideline [30]. When assessing workers’ outcomes, this revealed that the guideline implementation approach did not lead to shortened sickness duration in workers with mental health problems [n = 3228] [31]. Additional analyses on the association between the use of the guideline by OPs and workers’ outcome, found that better guideline use was not associated with earlier return to work [32]. From these previous studies we know that the implementation strategy was not effective on the workers’ level, i.e. did not lead to shorter duration of return to work. On the OPs’ level, we found that OPs were highly satisfied with the guideline training and self-perceived adherence was high. However it is unknown to what extend OPs are actually using the guideline in practice. The current study aims to assess adherence to the MHP guideline by OPs and the effect of a tailored implementation strategy on guideline adherence in addition to implementation as usual (that is, dissemination of the guideline among Dutch OPs and providing short continuing medical education courses) by means of an audit of medical records.

Research questions:

  1. 1.

    To what extent do occupational physicians (both intervention group and control group) adhere to the MHP guideline?

  2. 2.

    What is the effect of a tailored implementation strategy on guideline adherence among occupational physicians compared to implementation as usual?


Study design

The current study was part of a larger project, examining the effect of an intervention to enhance guideline adherence in OPs on return to work of workers with mental health problems [28]. In the larger project workers with mental health problems were included and received guidance by an OP who also participated. Randomization to the control and intervention group was performed at the level of participating OPs. The OPs in the intervention group received an innovative peer group guideline training and received educational credits after completing the training. OPs in the control group received no extra training in the guideline and performed care as usual. In the years previous to this study, the guideline was distributed among Dutch OPs and became part of their continuing medical education. Therefore, it is assumed that most of the OPs had at least minimal knowledge of the guideline.

After the training, data on guideline adherence were collected by means of an audit of medical records of sick-listed workers with mental health problems who were participating in the larger project, and who were guided by an OP in the in the intervention or control condition. These data were collected between November 2012 and January 2014. The current study reports on the data analysis of the audit of medical records.

Approval was obtained from the Medical Research Ethics Committee of St. Elisabeth Hospital in Tilburg. This study was registered in the ISTCTN trial register, ISRCTN86605310. The “CONSORT 2010 statement: extension to cluster randomized controlled trials” was used for reporting [33].


In the Netherlands, according to the Dutch Gatekeeper Improvement Act [34, 35], the employer is responsible for the return to work of sick-listed workers during the first 2 years of sickness absence. During this period the employer is obliged by law to continue paying wages (at least 70%). This is irrespective of cause and work-relatedness. During these 2 years the sick-listed worker cannot be fired because of his sickness.

The OP has a central role in the Dutch social security system, and is the link between workers’ health and the work situation. An OP is a qualified medical doctor specialized in occupational health who assists employers and workers in occupational health issues, safety and sickness absence management [34]. In the case of sickness absence of a worker, the employer is obligated by law to provide access to an OP within 6 weeks of the sickness absence. The OP provides return to work support to the sick-listed worker and provides advice to the employer regarding return to work activities and work adaptations if necessary. Most Dutch employers have contracted independently operating OPs or Occupational Health Services (OHSs) for these services. Within these contracts employers and the OHS or OP agree on the tasks for and conditions under which the OPs can operate, including the frequency and, available time for consultations with the worker, as well as providing access to work sites. OPs often work for several organizations (ranging from large companies to small businesses) at multiple locations.

Since 1998, The Netherlands Society of Occupational Physicians (NVAB), has developed and implemented evidence-based guidelines for OPs for a variety of health conditions [36]. In this study we used the MHP guideline for OPs, which was developed in 2000 [20] and revised in 2007 [21]. The guideline is of ‘moderate to high developing and reporting quality’ according to appraisal using the internationally validated AGREEII instrument [19].

The MHP guideline [21] recommends OPs to use a process-based approach in both case and care management, and to document their findings in the workers’ medical record. The content of the guideline is based on cognitive behavioral principles aiming to enhance the problem solving capacity of both workers and employers, particularly in relation to the work context (See Table 1). Depending on the needs of the worker, the OP can choose to provide care management in addition to case management. Most workers will be receiving care from their general physician as well. Additionally, the OP can refer the worker to specialized (mental) health care. It is recommended that the OP communicate with other health care providers to try to align the treatments and promote healthy functioning and the value of work in addition to medical care. With regard to the employer, the OP advises and supports the employer/supervisor on the return to work process and any necessary work adaptation to achieve sustainable participation. This can be done through a wide variety of means (e.g. email contact, face-to-face meetings).

Table 1 Summary of the content of the MHP guideline [21, 29, 30]

Implementation strategy – intervention group

The peer group training for OPs was developed as a tailored implementation strategy aimed at improving OPs’ adherence to the MHP guideline. The training consisted of eight two-hour-meetings which were scheduled over the course of 1 year; January 2011 – January 2012. The training sessions were held at six regional offices of the OHS across the southern part of the Netherlands; each group attended the training at one location.

Small interactive groups of four to six OPs were formed to stimulate involvement and encourage in-depth discussion among OPs and to learn from their peers. The trainer (MJ) guided the groups by structuring the meetings, facilitating the discussions and monitoring the progress. The training focused on identifying and addressing barriers OPs perceived in using the guideline recommendations in practice. The framework of Cabana was used to identify barriers related to knowledge, attitude and external factors [9]. The training incorporated the different parts of the guideline, with each session focusing on a different topic and guideline recommendations. A Plan-Do-Check-Act (PDCA) approach was used by the trainer to structure and moderate the discussions; explore barriers perceived by OPs, find suitable solutions to overcome these barriers, test these solution in practice and evaluate the results. The PDCA cycle follows a learning approach aiming to change behavior and is flexible in adapting the changes according to feedback, which helps to ensure that fit-to-purpose solutions are developed [37]. The focus on perceived barriers (i.e. the Cabana model) in combination with a PDCA approach formed the basis of the training on the MHP guideline (see Table 2).

Table 2 Structure of the guideline training ‘Mental Health Problems’ [30]

In each session the trainer (MJ) asked the OPs to discuss which barriers they perceived in adhering to that specific topic. Next, OPs suggested solutions to address these barriers, taking into account the context of their daily practice. Subsequently, OPs tested the solutions in their daily practice. Finally, results were evaluated and, if necessary, solutions were adjusted. This PDCA cycle was repeated in each meeting and for all topics stated in the guideline. An additional file illustrates how OPs were engaged in the implementation of the guideline recommendations and how a PDCA cycle was applied (See Additional file 1).

In a previous reported study, Joosen and colleagues [30] reported on the compliance and feasibility of the peer group training, showing that the protocol was carried out as planned, all participating OPs attended all 8 training sessions and 90% of the OPs agreed that the peer-learning groups and the meetings spread over 1 year were highly effective training components. A detailed description of the implementation strategy and its feasibility can be found elsewhere [30].


Occupational physicians

OPs were recruited from a large OHS in the Netherlands between October 2010 and January 2011. All OPs who were employed at one of six sites of the OHS in the southern part of the Netherlands (n = approximately 155) were invited to participate. First, presentations by the researchers (MJ and EB) and information about the larger project were provided at several meetings for OPs at the OHS, after which OPs could register for participation. Also, an email invitation was sent to all OPs; a reminder email was sent after 2 weeks, and all OPs who had not yet responded were contacted by telephone.

Medical records

Medical records of sick-listed workers who were guided by participating OPs were used to assess adherence. Workers were selected from the registration system of the OHS based on the following inclusion criteria: 1) CMD was the primary reason for sick leave diagnosed by an OP according to the Dutch Classification of Diseases, based on the ICD-10 [38], 2) being on current sick leave when selected from the registration system of the OHS after the first meeting with the OP (between January 1st 2012 and January 15th 2013), and 3) having adequate command of the Dutch language. Exclusion criteria were: being suicidal, and a physical problem being the primary reason for sick leave at the time of study inclusion. The OHS invited the eligible workers to participate in this study. Participating workers gave their written informed consent and signed a separate consent form when they gave permission to audit their medical records. The audited period was 12 months, starting the first day of sickness absence of each worker. After inclusion, questionnaires were filled out regarding socio-demographic characteristics (age, gender, educational level), personal and work factors (contract hours and workability measured with a single question of the workability index (WAI) [39]) and clinical characteristics (measured with the Four-Dimensional Symptom Questionnaire (4-DSQ) [40]). More details about these questionnaires are described elsewhere [28].

Randomization, stratification and blinding

In each of the six regional offices of the participating OHS, a peer training group of OPs (intervention group) and a control group was formed. Randomization took place on the level of OP by computerized allocation. To establish equal intervention and control groups at all six sites, pre-stratification was used and OPs working at each site were randomly allocated to one of the two groups. To limit the risk of contamination across OPs working at the same site we specifically asked the OPs in the intervention group not to discuss the content of the guideline training.

OPs were not informed about the inclusion of specific workers into the study, but they were told which workers were invited to participate (about 500 workers in total). Workers and employers were blinded for randomization since they were not aware of the group allocation of their OP. During the recruitment, intervention period and data collection researchers were not blinded. During data analysis, the researchers were blinded for allocation of the groups since information related to the OP were stripped from the data.

Data collection on guideline adherence

Performance indicators (PIs)

Adherence to the guideline was determined by assessing medical records in which OPs record the provided care. In previous studies a set of performance indicators (PIs) was used to evaluate adherence to the MHP guideline developed in 2000 [25, 26, 41]. As the guideline was revised in 2007 the initial PIs did not fully cover the content of the revised guideline. Therefore, a new set of PIs was developed that covered the essence of the revised guideline.

The set of PIs was systematically developed using an iterative consensus rating procedure in three steps: (i) preselection of recommendations; (ii) expert consensus procedure; and (iii) transcription and classification of final set of indicators [42,43,44].

  1. 1.

    Preselection of recommendations. Three authors (JvdK, DR, ML) independently preselected all recommendations from the MHP guideline. They focused on the most important guideline recommendations that would have impact on the quality of occupational health care. This resulted in a list of 24 recommendations.

  2. 2.

    Expert consensus procedure. An expert meeting was organized with mental health specialists, work and health specialists, quality of care researchers, OPs, an occupational therapist, a general practitioner and a patient representative. In a two-round consensus procedure, the panel of 9 experts discussed the relevance of the recommendations to physicians’ performance and patients’ health benefit. Moreover, the experts discussed which recommendations reflected the essence of the guideline and how the PIs should be best formulated. This resulted in a selection of 20 recommendations.

  3. 3.

    Transcription, pilot testing and classification of final set of indicators. The selected 20 recommendations were transcribed into indicators and a subsequent scoring set. Ten medical records were pilot tested by two researchers independently. Based on the comments of the reviewers about feasibility (e.g. usability), measurability (e.g. is all information needed available in medical records) and relevance (e.g. are indicators relevant to quality of care), the list of PIs was adjusted. The final set of PI’s consisted of 12 indicators grouped into 5 categories of indicators (further referred to as grouped PIs). The PIs measured different aspects of the management of mental health problems, including diagnosis, management of mental health problems, relapse prevention and continuity of care. The PIs are presented in Table 3.

Table 3 Description of 12 PIs for OP’s guideline adherence in workers’ medical records and criteria for their scoring [31]

Audit of medical records

From each record, we used the recordings of all consultations from the first day of sick leave until the involvement of the OP ended, or after 1 year sickness absence. Each record was assessed by two assessors independently. The assessors used an audit form which included detailed description of the PIs and instructions for rating each PI. If the rating was not congruent, the two assessors would discuss the case. If no consensus was reached a third assessor audited the medical record and decided about the final score. To guarantee blinding of the outcome assessors, medical records were stripped of information relating to the OP (name, allocation to intervention or control group).

Each of the 12 PIs was rated as 0 (no adherence), 1 (minimal adherence), or 2 (adequate adherence). For each of the five grouped PIs a sum score was calculated by summing the scores of the corresponding PIs divided by the number of PIs. Post hoc, the performance scores were dichotomized because there were too few medical records showing adequate adherence (score 2), see Table 4. Scores were dichotomized into ‘minimal-to-adequate guideline adherence’ (scores ≥1) and ‘no guideline adherence’ (scores < 1). Finally, performance rates were calculated as the percentage of medical records in which guideline-based care was provided.

Table 4 Worker’s characteristics in the intervention group and control group

Statistical analysis

To describe guideline adherence among the total group of OPs, descriptive statistics were used among all 12 PIs describing the frequencies of scores on no adherence, minimal adherence and adequate adherence of the guideline.

To evaluate the differences in guideline adherence between intervention group and control group, for each PI the performance rates were compared. We first checked whether it was necessary to control for the cluster level effect of the OPs (i.e. clustering of workers within OPs), using Logistic multilevel analyses within Generalized Linear Mixed Models (GLMM). For all variables the best-fitting model, with or without correction for the cluster level, was chosen based on the Akaike information criterion (smallest AIC represents best-fitting model). If the model with correction for the cluster level was the best fitting model, GLMM analyses were performed. Chi-square analyses were performed if the model without correction for the levels was best fitting. In addition, effect sizes (risk differences) were calculated. The intention-to-treat principle was used in the analyses.

All analyses were performed with SPSS version 19.0.


A total of 66 OPs agreed to participate and were randomized to the intervention group (N = 32) or the control group (N = 34). All 32 OPs from the intervention group attended the eight training meetings, in six groups of 4–6 OPs. In six cases an OP was not able to attend a meeting and joined another group for that specific meeting. During the 1 year training period, 10 OPs left their job at the OHS (due to reorganization within the OHS or other reasons) thereby leaving the study. Of the remaining 56 OPs, 26 were in the intervention group and 30 in the control group.

In the larger project, 116 out of 128 workers gave their written consent for auditing their medical record. Two workers were not included in this study; one record was not available at the OHS and in another case mental health problems were not the primary cause for the sickness absence. Therefore, data from 114 workers were used for this study. The included workers were guided by 34 different OPs, 16 in the intervention group and 18 in the control group. From the remaining 22 OPs in this study, no medical records were assessed because the workers in their caseload were not participating in the larger project. Workers’ characteristics are shown in Table 4 and there were no statistically significant differences between workers in the intervention group and control group.

In 109 records the two assessors agreed on the ratings of the PIs. In five records a third assessor was consulted to reach consensus. For the analysis, Chi-square analyses were performed since the model without correction for the cluster levels was best fitting.

Guideline adherence among all OPs

As can be seen in Table 5, guideline adherence was found to be minimal in 6 out of 12 PIs. In another 5 PIs the majority of medical records showed no adherence. Guideline adherence was especially low in PI 4.1 ‘Relapse prevention by OP’ and PI 5.3 ‘Regular contact employer’ (in respectively 79.8 and 78.9% of the records guideline-based care was not provided). Adequate guideline adherence was found in PI 5.1 ‘Rapid first consultation worker’ (in 52.6% of the records guideline-based care was optimally provided).

Table 5 Guideline adherence in medical records (n = 114) of OPs in both intervention and control group. Number of medical records in which guideline-based care was not provided (no adherence), minimally provided (minimal adherence) or optimally provided (adequate adherence) and their percentage score (performance rate)

Effect of a guideline training on guideline adherence

Table 6 shows the guideline adherence per PI in percentage (performance rate) for both the intervention and the control group. A significantly higher performance rate was found in the intervention group in 4 out of 5 grouped PIs: Process diagnosis (p = .011), Problem orientation (p = .015), Intervention/treatment (p = .015) and Relapse prevention (p = .028). No significant differences were found between the groups in grouped PI5 ‘continuity of care’.

Table 6 Differences in minimal-to-adequate guideline adherence between intervention and control group. Number of medical records in which guideline-based care was minimal-to-adequate (score 1 and 2) consistent with the guideline, their percentage scores (performance rate) and differences (p-value, risk differences and 95% confidence interval) between intervention group and control group (chi-square test)

In 6 out of 12 of the individual PIs the performance rates of the intervention group were significantly higher than in the control group (p < .05). Low effect sizes (risk differences) were found between the differences of all PIs (< 27.8%).


In this study we found that OPs who received a tailored guideline training showed significantly greater adherence rates to the guideline for mental health problems in occupational health care compared to OPs who were exposed to traditional guideline dissemination. However, in both groups documented guideline adherence was low. Especially, OPs did not record that relapse prevention was addressed and they did not have regular contact with the employer. Also, symptoms of mental health problems were not documented well and in almost half of the records treatment was not in accordance with the recovery process of the worker. OPs did identify the problems that workers face at work and at home and in most records a rapid first consultation was recorded (within 2 weeks after the 1st day of sick leave).

Overall we found that guideline adherence was poor; in only one PI adherence was found adequate in the majority of the medical records assessed. In previous studies [25, 26], adherence to the MHP guideline was also found to be suboptimal. Although the results cannot be compared on the level of PIs, because the revised version of the guideline with a different content and different set of PIs was used, it is evident that the uptake of the guideline has been problematic for several years.

Several explanations can be given for why we found low guideline adherence. First, an audit of medical records does not reveal what actually happens during the encounters between OP and worker. OPs might not register all their findings and activities in the record. Negative findings and routine activities may not have been documented systematically, with the exception of the frequency of contacts between OP and worker and employer (i.e. continuity of care). Here, the OHS routinely lists the date of each consultation which rules out the possibility of inaccurate registration. Also, the PI criteria were developed to reflect the content of the guideline, but they might not adequately reflect what OPs perceive to be important and relevant to report. Secondly, many employers contract a minimum of services from their OHS (including service by OP). As is shown in the previous reported qualitative study [29], these minimal contracts can be in conflict with guideline recommendations and obstruct OPs from adhering to some recommendations [29]. For example, PI 4.1 ‘relapse prevention’ had one of the lowest performance rates. Possibly in some cases, OPs were restricted in scheduling a relapse prevention consultation because the contract did not cover consultations after full return to work. Besides these organizational constraints, OPs themselves also might not have made optimal use of their position to provide high quality occupational health care. From the analyses of the medical records low performance rates were found on treatment and guidance (PI 3.2). Here, OPs did not act in accordance with the recovery process of the worker (i.e. not intervening when recovery stagnates). Particularly in more complex situations or in case of stagnation the data in the medical records suggested that OPs failed to act as a proactive case manager, e.g. interact with the worker, work system, and other care providers. Especially in these cases acting according to the guideline might result in better worker outcomes.

In a previous outcome study [32], part of the larger project, the relationship between guideline use and workers’ outcomes was investigated using sickness absence registration data from the OHS. The analysis showed that low overall guideline adherence was not associated with earlier return to work. However, when evaluating specific items of the guideline, it was found that regular contact between the OP and the employer was associated with earlier full return to work of workers, even when OPs only minimally adhered to the guideline recommendation [32]. This finding stresses the importance of collaboration between work environment and occupational health professionals in facilitating the return to work process of workers with mental health problems. However, since overall guideline adherence was so poor, it was not possible to evaluate the effect of adequate guideline adherence on return to work. This still leaves the question unanswered whether good guideline use is positively associated with return to work and other workers’ outcomes (such as work functioning or sustainable employability). Meanwhile, it seems important to find strategies to improve adherence to guidelines and at the same time invest in developing/updating guidelines that include high quality evidence and take into account the daily practice and barriers of the target group.

Although overall guideline adherence was poor, we did find that the guideline training resulted in a statistically significant improvement in professional behavior. OPs who received the training reported stronger guideline adherence compared to their colleagues who had not received the training. In a previous study of Rebergen and colleagues [25], no effect of a three-day educational guideline course was found. The additional effect of the current intervention above traditional dissemination and education efforts might be explained by various elements: 1) a peer-group training was used which is known to activate the pre-knowledge of participants, leads to high-quality learning groups, and can impart sustainable knowledge and performance change [45, 46]. In addition, people adopt new information better through their trusted social networks [47]; 2) the training was a participant-focused programme, focusing on barriers OPs perceived in their daily practice (knowledge-related, attitude-related and external barriers), which ensured covering relevant clinical and practical topics. In addition, OPs themselves developed solutions that were tailored to the needs of the OPs and tested the solutions in practice using a Plan-Do-Check-Act approach [37]; 3) The 8 training sessions were spread over the course of 1 year, improving knowledge and allowing OPs to adopt a new working style and actually change their behavior. By this approach, all participating OP were actively involved and felt engaged because they could decide and act on the topics that were most relevant for their ability to use the guideline in their daily practice. Using tailored implementation strategies in small interactive sessions is found to be effective in changing professional behaviour in other studies [48,49,50].

As part of the larger project, qualitative analyses on the barriers OPs perceive using the guideline were conducted and reported by Lugtenberg and colleagues [29]. For the analyses, the Cabana framework was used to structure barriers into knowledge related barriers, attitude related barriers and external barriers that can influence guideline adherence [9]. It was found that the training had the most impact on knowledge related and attitude related barriers, such as lack of outcome expectancy and lack of self-efficacy, but external barriers remained [30]. The perceived external barriers were mostly work-contextual constraints, such as a lack of time, minimal contracts between OHSs and employers, and conflicting policies of and a lack of collaboration with other stakeholders involved (e.g. employer, healthcare providers) [29]. These kind of external barriers are be too extensive and complex to be changed by a professional-directed intervention [30]. For example, for an individual OP it is difficult to change policy or influence the conditions of contracts as these contracts are usually made between employers and the management of the OHS without interference of the OP. This might explain why we did not find an effect on Continuity of care (PI5), which involves the start of the first consultation, intervals between consultations and contact between OP and employer. Even though the OP knows what to do and wants to perform a certain behavior, remaining external barriers may prevent actual adherence to all guideline recommendations.

Lack of involvement of different stakeholders, such as the management OHS, employers and other health care professionals during the training might be another reason why OPs were not able to address external barriers. Therefore, to improve the implementation strategy, it would be advised to involve stakeholders at the organizational level (e.g. management of OHS), the work environment (e.g. employers, HR management), and other health care professionals to addressing the conditional external barriers for guideline use and organisational constraints.

In addition to work-contextual constraints OPs perceived other external barriers related to the guideline itself. OPs perceived the guideline as being unclear and inconsistent, with complex terminology [29]. These factors in particular need to be changed by guideline developers in order to help professionals use the guideline in their daily practice. At present the Netherlands Society of Occupational Medicine is revising the guideline which is expected to be released in 2019 [51].

Strengths and limitations

A strength of this study is its randomized controlled design, which is rare in this field of research [52]. By using cluster randomization the risk of contamination between the intervention and control condition was low. Another strengths is that we evaluated the use of the guideline in daily practice in two groups of OPs after completing 1 year of guideline training. The risk for recording desired performance by the OP is minimal, since the data collection started 3.5 year after the OPs gave their consent. In addition, by means of a conscientious and thorough development process a new set of PIs was developed for the revised version of the practice guideline. This may facilitate development and evaluation of international guidelines on this relevant and growing topic.

Assessing guideline adherence by an audit of medical files is, as with any observational study, susceptible to bias. A possible source of bias is that physicians did not document all their findings systematically in the medical records, resulting in an underestimation of the true performance. However, the method of auditing medical records is also a strength, since it hardly interferes with actual performance, in contrast to actual or video observation of consultations. Moreover, it is found that objective measures of adherence such as medical records are more accurate than self-perceived adherence which tend to result in an overestimation of adherence [53]. Another limitation is that the performance indicators might not influence guideline adherence in an equal way, that is, some performance indicators might have been conditional for others. For example, if an OP does not have regular contact with the employer (PI5.3), he/she will presumably report less information about the perspective of the employer regarding the recovery of the worker (PI1.3). To prevent interpretation bias, all medical records were blindly assessed by two researchers independently and a third researcher in case no consensus was reached. Since this is a pragmatic trial, in which we tested the effectiveness of guideline training in a real life situation, we have used intention to treat analysis [54]. Ideally, an OP guides a worker throughout the entire sickness and recovery process. However, in practice the worker might be allocated to another OP, because of (holiday) leave of the OP, or because the OP changes location. In all cases, the worker’s medical record was analyzed in the way the worker was randomized at the beginning of the trial, regardless of whether the worker completed their guidance with the same OP. By using this approach type II errors may occur and this should be taken into account when translating the results to another setting. Another limitation is the small sample size achieved; from 22 out of 56 participating OPs medical recordings were not assessed because none of the workers guided by these OPs were included in the study, which might have caused selection bias. Unfortunately, no information was available from workers who did not participate in the study, for which reason a non-response analysis could not be conducted. However, no significant differences were found between workers characteristics in the control and intervention group. In addition, using the GLMM analyses showed that adding OP as a random effect did not significantly improve the model.


The results from this study support the idea that a tailored implementation strategy in small interactive peer sessions during a long interval is effective in implementing guidelines but has limited impact if external barriers continue to hinder guideline adherence. We found that peer-group guideline training, focusing on perceived barriers, improved adherence to the guideline for mental health problems in occupational health care. As a generic approach to address key barriers for change, the implementation strategy might also be an effective method for implementing other guidelines, in other health care professionals and/or in other countries. To optimize the implementation process of guidelines, future research should focus on the implementation of interventions that target different levels (provider level, patient/worker level and organizational level), and should involve relevant stakeholders who are committed to implementing guideline recommendations, such as OPs, management of occupational health services, employers, workers and other health care professionals.



Four Dimensional Symptom Questionnaire (4DKL in Dutch)


Generalized linear mixed models


Mental health problems


Netherlands Society of Occupational Physicians (NVAB in Dutch)


Occupational health service(s)


Occupational physician(s)




Performance indicator(s)




Workability index


  1. Hepner KA, Rowe M, Rost K, Hickey SC, Sherbourne CD, Ford DE, Meredith LS, Rubenstein LV. The effect of adherence to practice guidelines on depression outcomes. Ann Intern Med. 2007;147(5):320–9.

    Article  PubMed  Google Scholar 

  2. Kennedy PJ, Leathley CM, Hughes CF. Clinical practice variation. Med J Aust. 2010;193(8 Suppl):S97–9.

    PubMed  Google Scholar 

  3. Mulley AG. Inconvenient truths about supplier induced demand and unwarranted variation in medical practice. BMJ. 2009;339:b4073.

    Article  PubMed  Google Scholar 

  4. Ilott I, Rick JO, Patterson M, Turgoose C, Lacey A. What is protocol-based care? A concept analysis. J Nurs Manag. 2006;14(7):544–52.

    Article  PubMed  Google Scholar 

  5. Grol R. Implementation of evidence and guidelines in clinical practice: a new field of research? Int J Qual Health Care. 2000;12(6):455–6.

    Article  CAS  PubMed  Google Scholar 

  6. Francke AL, Smit MC, de Veer AJ, Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Bighelli I, Ostuzzi G, Girlanda F, Cipriani A, Becker T, Koesters M, Barbui C. Implementation of treatment guidelines for specialist mental health care. Cochrane Database Syst Rev. 2016;12:CD009780.

  8. Fleuren M, Wiefferink K, Paulussen T. Determinants of innovation within health care organizations: literature review and Delphi study. Int J Qual Health Care. 2004;16(2):107–23.

    Article  PubMed  Google Scholar 

  9. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–65.

    Article  CAS  PubMed  Google Scholar 

  10. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane effective practice and organization of care review group. BMJ. 1998;317(7156):465–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Lugtenberg M, Zegers-van Schaick JM, Westert GP, Burgers JS. Why don’t physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009;4:54.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–30.

    Article  PubMed  Google Scholar 

  13. Davis D, O'Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867–74.

    Article  CAS  PubMed  Google Scholar 

  14. Kwak L, Wåhlin C, Stigmar K, Jensen I. Developing a practice guideline for the occupational health services by using a community of practice approach: a process evaluation of the development process. BMC Public Health. 2017;17(1):89.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Phillips JA, Holland MG, Baldwin DD, Gifford-Meuleveld L, Mueller KL, Perkison B, Upfal M, Dreger M. Marijuana in the workplace: guidance for occupational health professionals and employers: joint guidance statement of the American Association of Occupational Health Nurses and the American College of Occupational and Environmental Medicine. Workplace Health Saf. 2015;63(4):139–64.

    Article  PubMed  Google Scholar 

  16. Kuijer PPF, Verbeek JH, Visser B, Elders LA, Van Roden N, Van den Wittenboer ME, Lebbink M, Burdorf A, Hulshof CT. An evidence-based multidisciplinary practice guideline to reduce the workload due to lifting for preventing work-related low back pain. Ann Occup Environ Med. 2014;26(1):16.

    Article  PubMed  PubMed Central  Google Scholar 

  17. NIP/LVE. Werk en psychische klachten: richtlijn voor psychologen. Utrecht: Nederlands Instituut voor psychologen (NIP); 2005.

    Google Scholar 

  18. Oomens P, Huijs J, Blonk R. Effectiviteit van de richtlijn ‘Werk en psychische klachten’ voor psychologen. Hoofddorp: TNO; 2009.

    Google Scholar 

  19. Joosen MC, Brouwers EP, van Beurden KM, Terluin B, Ruotsalainen JH, Woo JM, Choi KS, Eguchi H, Moriguchi J, van der Klink JJ, et al. An international comparison of occupational health guidelines for the management of mental disorders and stress-related psychological symptoms. Occup Environ Med. 2015;72(5):313–22.

    Article  PubMed  Google Scholar 

  20. van der Klink JJ, van Dijk FJ. Dutch practice guidelines for managing adjustment disorders in occupational and primary health care. Scand J Work Environ Health. 2003;29(6):478–87.

    Article  PubMed  Google Scholar 

  21. Van der Klink JJL, Ausems CMM, Beijderwellen BD, Blonk R, Bruinvels DJ, Dogger J, Hoedeman R, Janssen J, Loo M, Nieuwenhuijsen K, et al. Richtlijn: Handelen van de bedrijfsarts bij werkenden met psychische problemen, herziene uitgave. [Guideline:The management of mental health problems of workers by occupational physicians; revised version]. Utrecht: NVAB [Netherlands Society of Occupational Medicine]; 2007.

    Google Scholar 

  22. World Health Organisation. Global burden of disease 2004 update. Geneva: World Health Organisation; 2008.

    Google Scholar 

  23. OECD. Mental health and work: United Kingdom, mental health and work. Paris: OECD Publishing; 2014.

  24. OECD. Mental health and work: Netherlands. Paris: OECD Publishing; 2014.

  25. Rebergen DS, Bruinvels DJ, Bos CM, van der Beek AJ, van Mechelen W. Return to work and occupational physicians’ management of common mental health problems--process evaluation of a randomized controlled trial. Scand J Work Environ Health. 2010;36(6):488–98.

    Article  PubMed  Google Scholar 

  26. Nieuwenhuijsen K, Verbeek JH, Siemerink JC, Tummers-Nijsen D. Quality of rehabilitation among workers with adjustment disorders according to practice guidelines; a retrospective cohort study. Occup Environ Med. 2003;60(Suppl 1):i21–5.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Rebergen D, Hoenen J, Heinemans A, Bruinvels D, Bakker A, van Mechelen W. Adherence to mental health guidelines by Dutch occupational physicians. Occup Med (Lond). 2006;56(7):461–8.

    Article  Google Scholar 

  28. van Beurden KM, Brouwers EP, Joosen MC, Terluin B, van der Klink JJ, van Weeghel J. Effectiveness of guideline-based care by occupational physicians on the return-to-work of workers with common mental disorders: design of a cluster-randomised controlled trial. BMC Public Health. 2013;13:193.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Lugtenberg M, van Beurden KM, Brouwers EPM, Terluin B, van Weeghel J, van der Klink JJL, Joosen MCW. Occupational physicians’ perceived barriers and suggested solutions to improve adherence to a guideline on mental health problems: analysis of a peer group training. BMC Health Serv Res. 2016;16:271.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Joosen MC, van Beurden KM, Terluin B, van Weeghel J, Brouwers EP, van der Klink JJ. Improving occupational physicians’ adherence to a practice guideline: feasibility and impact of a tailored implementation strategy. BMC Med Educ. 2015;15:82.

    Article  PubMed  PubMed Central  Google Scholar 

  31. van Beurden KM, Brouwers EPM, Joosen MCW, de Boer MR, van Weeghel J, Terluin B, van der Klink JJL. Effectiveness of an intervention to enhance occupational physicians’ guideline adherence on sickness absence duration in workers with common mental disorders: a cluster-randomized controlled trial. J Occup Rehabil. 2017;27(4):559–67.

    Article  PubMed  Google Scholar 

  32. van Beurden KM, Joosen MCW, Terluin B, van Weeghel J, van der Klink JJL, Brouwers EPM. Use of a mental health guideline by occupational physicians and associations with return to work in workers sick-listed due to common mental disorders: a retrospective cohort study. Disabil Rehabil. 2018;40(22):2623–31.

    Article  PubMed  Google Scholar 

  33. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661.

    Article  PubMed  Google Scholar 

  34. Weel AN, van der Beek AJ, Kroon PJ, Verbeek JH, van Dijk FJ. Recent changes in occupational medicine in the Netherlands. Int Arch Occup Environ Health. 1999;72(5):285–91.

    Article  CAS  PubMed  Google Scholar 

  35. Wet verbetering poortwachter. [Dutch Gatekeeper Improvement Act] [] Accessed 21 Mar 2019.

  36. Hulshof C, Hoenen J. Evidence-based practice guidelines in OHS: are they agree-able? Ind Health. 2007;45(1):26–31.

    Article  PubMed  Google Scholar 

  37. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290–8.

    Article  PubMed  Google Scholar 

  38. World Health Organization. International statistical classification of deseases and related health problems. ICD-10. 2010 edition. Geneva: World Health Organization; 2011.

  39. Ahlstrom L, Grimby-Ekman A, Hagberg M, Dellve L. The work ability index and single-item question: associations with sick leave, symptoms, and health-a prospective study of women on long-term sick leave. Scand J Work Environ Health. 2010;36(5):404–12.

    Article  PubMed  Google Scholar 

  40. Terluin B, van Marwijk HW, Ader HJ, de Vet HC, Penninx BW, Hermens ML, van Boeijen CA, van Balkom AJ, van der Klink JJ, Stalman WA. The four-dimensional symptom questionnaire (4DSQ): a validation study of a multidimensional self-report questionnaire to assess distress, depression, anxiety and somatization. BMC Psychiatry. 2006;6:34.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Smits PB, de Buisonje CD, Verbeek JH, van Dijk FJ, Metz JC, ten Cate OJ. Problem-based learning versus lecture-based learning in postgraduate medical education. Scand J Work Environ Health. 2003;29(4):280–7.

    Article  PubMed  Google Scholar 

  42. Grol R, Wensing M, Eccles M. Improving patient care: the implementation of change in clinical practice. Oxford: Elsevier; 2005.

    Google Scholar 

  43. Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care. 2003;15(6):523–30.

    Article  PubMed  Google Scholar 

  44. Rubin HR, Pronovost P, Diette GB. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care. 2001;13(6):489–96.

    Article  CAS  PubMed  Google Scholar 

  45. Kuhne-Eversmann L, Fischer MR. Improving knowledge and changing behavior towards guideline based decisions in diabetes care: a controlled intervention study of a team-based learning approach for continuous professional development of physicians. BMC Res Notes. 2013;6:14.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Kiessling A, Henriksson P. Efficacy of case method learning in general practice for secondary prevention in patients with coronary artery disease: randomised controlled study. BMJ. 2002;325(7369):877–80.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Rogers EM. Diffusion of innovations. New York: Simon and Schuster; 2010.

    Google Scholar 

  48. Baker R, Reddish S, Robertson N, Hearnshaw H, Jones B. Randomised controlled trial of tailored strategies to implement guidelines for the management of patients with depression in general practice. Br J Gen Pract. 2001;51(470):737–41.

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Sinnema H, Terluin B, Wensing M, Volker D, Franx G, van Balkom A, de Lange J. Systematic tailoring for the implementation of guideline recommendations for anxiety and depressive disorders in general practice: perceived usefulness of tailored interventions. BMC Fam Pract. 2013;14:94.

    Article  PubMed  PubMed Central  Google Scholar 

  50. de Beurs DP, de Groot MH, de Keijser J, Mokkenstorm J, van Duijn E, de Winter RF, Kerkhof AJ. The effect of an e-learning supported train-the-trainer programme on implementation of suicide guidelines in mental health care. J Affect Disord. 2015;175:446–53.

    Article  PubMed  Google Scholar 

  51. []. Accessed 21 Mar 2019.

  52. Weinmann S, Koesters M, Becker T. Effects of implementation of psychiatric guidelines on provider performance and patient outcome: systematic review. Acta Psychiatr Scand. 2007;115(6):420–33.

    Article  CAS  PubMed  Google Scholar 

  53. Adams AS, Soumerai SB, Lomas J, Ross-Degnan D. Evidence of self-report bias in assessing adherence to guidelines. Int J Qual Health Care. 1999;11(3):187–92.

    Article  CAS  PubMed  Google Scholar 

  54. Armijo-Olivo S, Warren S, Magee D. Intention to treat analysis, compliance, drop-outs and how to deal with missing data in clinical research: a review. Phys Ther Rev. 2009;14(1):36–49.

    Article  Google Scholar 

Download references


We would like to thank all participating OPs and workers for their collaboration and the Netherlands Organisation for Health Research and Development (ZonMw) for financially supporting this project.


This study was financially supported by the Netherlands Organisation for Health Research and Development (ZonMw) (grant number 208030001). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Trail registration: ISRCTN86605310.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations



MJ, EB, KvB, ML, BT, JvdK, JvW contributed to the original design of the study. The data collection was carried out by MJ and KvB. MJ, EB, KvB, DR, ML, BT, JvdK, JvW participated in the data analysis and interpretation of the data. The manuscript was written by MJ and was edited by all authors, who also approved of the final manuscript.

Corresponding author

Correspondence to Margot C. W. Joosen.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was provided by the Medical Research Ethics Committee of St. Elisabeth Hospital in Tilburg (MREC number 1162). All participants gave their written informed consent to participate in the study.

Consent for publication

Not applicable.

Competing interests

JvdK and ML were involved in the development of the guideline, commissioned by the Netherlands Society of Occupational Medicine; JvdK was chair of the guideline committee, ML was involved as one of the members of the large guideline committee. Both do not receive fees for the use of the guideline. JvdK, MJ, and EB developed the guideline training for occupational physicians in this study and MJ was the trainer for this training. No fees were received for conducting the training. KvB, BT, DR and JvW declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Example of the implementation of a guideline recommendation by OPs participating in the guideline training’ an example is presented of how OPs were engaged in the implementation of the guideline recommendations and how a PDCA cycle was applied. (DOCX 64 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Joosen, M.C.W., van Beurden, K.M., Rebergen, D.S. et al. Effectiveness of a tailored implementation strategy to improve adherence to a guideline on mental health problems in occupational health care. BMC Health Serv Res 19, 281 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: