Research article | Open | Open Peer Review | Published:
Implementing a guideline for the treatment of type 2 diabetics: results of a Cluster- Randomized Controlled Trial (C-RCT)
BMC Health Services Researchvolume 7, Article number: 79 (2007)
In Italy many diabetics still lack adequate care in general practice. We assessed the effectiveness of different strategies for the implementation of an evidence-based guideline for the management of non-complicated type 2 diabetes among General Practitioners (GPs) of Lazio region.
Three-arm cluster-randomised controlled trial with GPs as units of randomisation (clusters). 252 GPs were randomised either to an active strategy (training module with administration of the guideline), or to a passive dissemination (administration of the guideline only), or to usual care (control). Data on prescriptions of tests and drugs were collected by existing information systems, whereas patients' data came from GPs' databases. Process outcomes were measured at the cluster level one year after the intervention. Primary outcomes concerned the measurement of glycosilated haemoglobin and the commissioning of micro- and macrovascular complications assessment tests. In order to assess the physicians' drug prescribing behaviour secondary outcomes were also calculated.
GPs identified 6395 uncomplicated type 2 patients with a high prevalence of cardiovascular risk factors. Data on GPs baseline performance show low proportions of glycosilated haemoglobin assessments. Results of the C-RCT analysis indicate that the active implementation strategy was ineffective relating to all primary outcomes (respectively, OR 1.06 [95% IC: 0.76–1.46]; OR 1.07 [95% IC: 0.80–1.43]; OR 1.4 [95% IC:0.91–2.16]. Similarly, passive dissemination of the guideline showed no effect.
In our region compliance of GPs with guidelines was not enhanced by a structured learning programme. Implementation through organizational measures appears to be essential to induce behavioural changes.
In Italy diabetes mellitus (DM) is a major health problem with a prevalence of 3–4% [1–3] and considerable resources are committed to addressing treatment of micro- and macrovascular complications . In Italy, patients with type 2 non-complicated DM are mainly treated in general practice, whereas patients with type 1 DM and complicated patients usually receive specialist care. Data from an unpublished survey, conducted in 2003 in a local health district of Lazio, showed that the overall quality of DM care often fails to achieve established standards, thus needing urgent intervention to raise standards and protect those most vulnerable from the consequences of DM. Strategies for changing general practitioner's (GPs) behaviour by dissemination and implementation of guidelines and translating evidence into practice are notoriously difficult areas [5–7]. A large systematic review by Grimshaw et al concluded that there is an imperfect evidence base to support decisions on which strategy is likely to be effective . Further uncertainties concern knowledge on service organisation and delivery of diabetic care and the lack of comparative studies on the effects of guideline implementation strategies in Italy.
We report on a cluster-randomised trial (C-RCT) to assess the effects of two different strategies of introducing an evidence-based guideline for the treatment of type 2 DM in primary care. The C-RCT design (randomisation at the level of professional practice or health care organization) represents the optimal design when evaluating dissemination and implementation strategies . As systematic reviews can inform and contribute to the correct design of randomised controlled trials, our protocol was based on three Cochrane and one UK Health Technology Assessment Programme systematic reviews of guideline dissemination and implementation strategies [10–12].
Eligible study participants were GPs taking part in an electronically-linked disease surveillance network and receiving a specific fee for computer-made prescriptions; they were about 50% of the whole population of Lazio GPs, and on average younger than the non-using informative instruments in their clinical practice. In order to minimize the risk of contamination due to GPs working in associated forms, we used a software which assigned a zero probability to be selected to members of the same practice, once one of them has already been extracted. Participants were recruited during the month of December 2003 and follow-up took place from January 2004 to December 2004.
Recruitment was performed by an invitation letter written in a standardized format, giving information about the study project. Every GP represented a cluster. The study was carried out in the primary care setting of Italian National Health Service in the Lazio region of Central Italy, and was focused on the management of patients with non-complicated type-2 DM, defined as diabetes disease in absence of even only one of the following conditions: ocular complications (retinopathy, cataract, or glaucoma); renal complications (nephropathy, chronic renal failure); neurological complications (autonomic neuropathy, peripheral neuropathy); diabetic foot (deformity, Charcot foot, infection, ulcer, gangrene, amputation); TIA/ictus; obstructive disease of lower limbs.
In our design, GPs were "units of measure" (clusters) of the process of diabetic care delivery, as they managed laboratory and preventive care for most patients with type 2 DM. They were asked to identify their patients accordingly to the above case definition and in observance with the following exclusion criteria: age < 40 years; gestational diabetes; diabetes in pregnancy.
Two different data sources were used. Patient descriptive data (baseline information) were transmitted on-line by GP participants who were given a login and a password to access personal web pages. Data on prescriptions of drugs, tests and outpatient appointment visits were extracted by current information systems, which routinely assemble data for reimbursement purposes, in particular the Regional Drug Information System, which collects all prescriptions of drugs reimbursed by the Italian NHS, and the Ambulatory Care Information System, which gathers all prescriptions of diagnostic tests and outpatient visits.
Data on the process of care for DM were collected during the 12 months of follow-up after the intervention (year 2004) and main outcomes were constructed and assessed from GPs' prescribing and commissioning behaviour.
The intervention was structured learning and distribution of the guideline to participant GPs (clusters).
In order to choose an appropriate guideline we examined the methodological issues on guideline evaluation and implementation strategies and we conducted a systematic search and evaluation of existing guidelines on diabetes care [13–15]. We used the quality assessment criteria and the quality assurance scores of the National Guideline Program of the Italian Institute of Health  and we performed a qualitative assessment of guidelines applicability to our regional context. We chose a French guideline , which we translated, updated and adapted for Italian GPs.
The intervention program
We structured the intervention program through a process of identification of barriers to implementation of recommendations and factors that may facilitate changing professional behaviour, together with an estimation of the resources available within the governance budget. We developed the intervention to be preferably a single and not a multi-faceted intervention, to be easy to carry out and to be reproducible in the implementation of other guidelines. The intervention program included either a two-day training course with CME credits and dissemination of the guideline (arm 1, active implementation) or dissemination of the guideline without the training course but with a written request to implement the guideline (arm 2, passive dissemination). The control group represented the usual care. The training course was organized as parallel sessions of teaching modules together with interactive and group work sessions with discussion of the content of the guideline. An entry questionnaire was given to participants to obtain a picture of current care for diabetic patients.
The primary objective of the study was to assess the effectiveness of different strategies for the implementation of an evidence-based guideline for the management of non-complicated type 2 DM among GPs of the Lazio region. Our null hypothesis was that a structured intervention would be no more effective than a passive dissemination of the guideline or of a no-intervention strategy. The secondary objective was to estimate the efficiency of resource use associated with the intervention through a cost-effectiveness analysis. Both the objectives were to be assessed at cluster level.
Process of care variables were considered study outcomes, all aimed at assessing physician-changing behaviour for the 12 months following the intervention, by using data from existing information systems. As Lazio GPs are not trained to register clinical data for epidemiological purposes, patient's health outcomes were not collected. Outcomes were expressed as proportions of patients who were prescribed requests for tests, for outpatient appointment visits and drugs in the follow-up year. All outcomes were evaluated at the patient level and aggregated to represent clusters.
We evaluated GPs' adherence to guideline recommendations for diabetes management by the following assessments: assessment of glycaemic control: we chose as primary outcome the measurement of glycosilated haemoglobin, as the proportion of patients who were prescribed 3 measurements of glycosilated haemoglobin with at least two months' interval per year; assessment of macrovascular complications aimed at early detection of the risk of macrovascular complications of DM: we chose as primary outcome the proportion of patients who were prescribed all macrovascular complications assessment tests (ECG and complete lipid profile, i.e. total cholesterol, HDL cholesterol and tryglicerides simultaneously) per year; assessment of microvascular complications aimed at early detection of the risk of microvascular complications of DM: we chose as primary outcome the proportion of patients who were prescribed all microvascular complications assessment tests (eye examination or fundus and blood creatinine or creatinine clearance and microalbuminuria) per year.
To explore participant GPs' drug prescribing behaviour we also calculated secondary outcomes, as indicators on drugs prescribed for the management of the disease and of correlated cardiovascular risk factors: pharmacological management of DM, assessed by the proportion of overweight/obese diabetics who were prescribed methformin in monotherapy as first-choice therapy and by the proportion of elderly diabetics (>70 years) who were prescribed long-life sulphonylureas (contraindicated in old age for their long biological action which increases the risk of hypoglycaemia); pharmacological management of cardiovascular risk factors, assessed by the proportion of diabetics with at least 1 cardiovascular risk factor who were prescribed anti-platelet drugs and by the proportion of hypertensive diabetics who were prescribed first-choice antihypertensive drugs in monotherapy; the proportion of patients with dyslipidemia receiving lipid-lowering-drugs.
For pharmacological indicators, patients were considered as previously untreated if they had no medication prescriptions recorded in the 12 months prior to the beginning of the trial.
The sample size calculation  took into account the intra-cluster correlation coefficient, the number of events, the expected effect, and the power of the study. On the basis of the unpublished survey we assumed an intracluster correlation of ρ = 0.1 , an average of 15 diabetic patients for each GP, and a worst-case control rate of 50%. Under these assumptions we anticipated a power of 90% to detect a difference of 10% in rates between one arm and the other with α = 0.05 with 84 GP for each group. The total sample size was 252 GPs.
Our randomisation sequences was computer-generated. GPs who accepted to take part in the study, were assigned by simple random allocation by the REXSCO  software, which assigns to same-practice partners a nil probability of being randomised, thus minimising the chances of participant contamination. Randomisation was performed by a researcher not involved in the study and who was blind to the identity of the practices.
Given the cluster design participant concealment was not possible but data extraction and analysis was carried out in blind fashion by one of us (DM).
We carried out comparisons of primary outcomes between the intervention and control arms at cluster (GP) level as this was the unit of randomisation  and data were analysed using a cluster specific method, Generalised Estimating Equation (GEE) model [23, 24].
A flow-chart of the progress of clusters through the phases of the trial is at Figure 1. We wrote to 2148 of the 4744 Lazio GPs who were known to be computer-literate because they were in receipt of a computer allowance and were users of regional surveillance software. Twenty five percent (545) of eligible GPs agreed to take part (potential participants). Of these we randomly assigned 84 "first choices" and 36 "stand-ins" for each arm. As 43 GPs declined to take further part after they had been notified of the outcome of randomisation, we contacted a total of 295 potential participants (110 in arm 1, 91 in arm 2 and 94 in arm 3). Post-randomisation attrition was highest in arm 1: 26 (60%) vs 6 (14%) vs 11 (26%) probably because of the requirement to attend our course.
Tables 1 and 2 show the distribution of characteristics of (respectively) participant GPs (clusters) and diabetic patients cared by GPs (individuals). Baseline information was available for 6290/6395 patients.
We carried out two analyses, one by including all 252 participant GPs (84 in arm 1, 85 in arm 2 and 83 in arm 3) who had identified their DM type 2 patients and the second, per protocol analysis (PPA), on the 235 participants who had identified their patients, provided their baselines characteristics and taken part in the full course (arm 1 only) (71 in arm 1, 81 in arm 2 and 83 in arm 3).
Outcomes and estimation
Table 3 shows pre-intervention (year 2003) and post-intervention (year 2004) values of primary outcomes together with their individual components and change over time for each of the 3 arms.
C-RCT results obtained processing data from all 252 clusters are shown in Table 4.
Active implementation vsusual care
Analyses of data measuring the impact of the active strategy shows that there was no effect either on the metabolic control indicator (OR 1,06 [0,76–1,46]), or on macrovascular complications assessment tests (OR 1,07 [0,80–1,43]) or on microvascular complications assessment tests (OR 1,4 [0,91–2,16]).
Passive dissemination vsusual care
Analyses of data measuring the effect of passive dissemination of the guideline shows similar results for all the three primary outcomes, respectively OR 0,93 [0,67–1,30] for the metabolic assessment, OR 0,93 [0,70–1,24] for the macrovascular complications assessment test and OR 1,11 [0,73–1,69] for the microvascular complications assessment test.
Per protocol analyses gives similar results (data not shown). On the basis of analyses of per protocol data and of data relating to all 252 clusters we cannot reject our null hypothesis, as our findings show an absence of correlation between active or passive educational training of GPs and physicians attitude at following the indications of the guideline. As results showed the non-effectiveness of the intervention strategy, we did not perform any economic evaluation or carry out analysis on participant sub-clusters.
Discussion and Conclusion
Our trial did not show significative differences between arms in the effectiveness of two different strategies to implement a good quality guideline for caring for people with type 2 uncomplicated diabetes. The dissemination of our guideline and a specifically targeted educational program seemed to unable to persuade GPs to change their current clinical practice. Although disappointing to us, these findings represent the first experimental evidence of the problems of putting evidence into practice available in Italy. They suggest that at both regional and national level there is an urgent requirement to attempt complex and articulated paths in an effort to increase quality of care. There is a growing body of evidence about the importance of an existing definite organizational framework as the basis for any changing behaviour by health care providers [25, 26] and a recent systematic review highlights a substantial improvement in patient glycaemic control using case management . Disease management models have been recently introduced in some regions, but although some pilot studies have shown promise , substantial differences between regional contexts may make these results not generally applicable. In Lazio many community services are performed by hospital outpatient departments with scarce integration between hospital and community and between primary and secondary care. This lack of integration is reflected in patchy information exchange between operators. An important issue is the generalisability (or external validity) of the trial's findings at the cluster (GP) level. Available systematic reviews [7–9] reported possible non-specific effects on the control arms of C-RCTs which could hinder generalisability of results. In Lazio there are very scarce data on knowledge and perception of guidelines and commonly GPs have difficulty in being involved in institutional programs for improving the quality of care. For a number of reasons [29, 30]. Internal barriers include time constraints, possibly inadequate reimbursement and disagreement with innovative programs. External barriers include individual patient needs, limited systems to support chronic disease management and poor patient adherence to treatment. In addition a large part of GPs may not be accustomed to web-based data loading and may not be accustomed to taking part in trials. It is possible that the performance of the control group (current practice arm) could have been different from other current practice with unconscious improvement from the beginning of the trial when control participants were asked to define and transmit a set of data relating to their DM patients. This may have had an impact on the applicability of our results. Conversely some contamination with participants belonging to different arms cannot be discounted, although we have no proof of its occurrence. The conclusions of our study are not very different from those reported by a survey conducted in 2004 by the Italian Institute of Health on a population of diabetics sampled in all Italian regions, which reported scarce adherence to clinical guidelines either by GPs or by specialists and large opportunities for improvement of care . Evidence from systematic reviews is at present unable to point to the most effective way of changing doctors' behaviour because of poor methodological quality of the majority of the included studies (major recurring problems were unit of analysis errors, baseline imbalance and within-group rather than between groups comparisons). In addition most studies used process measures for their primary end-point (like we did) and real improvements in health remained an issue of debate. In conclusion the effectiveness of implementation strategies is strongly correlated to the local context and could largely vary under different circumstances. Efforts should be directed to increase the body of evidence with more and better quality comparative studies. At the same time, policy makers have the responsibility to administrate the limited resources for clinical governance on the basis of a thorough knowledge of settings and problems.
The following are apparent limits of trial: uncertain representativeness of enrolled GPs compared to the remaining GPs which could affect external validity of our results; better performance of enrolled GPs; non-attendance for teaching sessions rate (13 out of 84, 15%).
Garancini MP: L'epidemiologia del diabete non-insulino-dipendente e la ridotta tolleranza glucidica. Il Diabete in Italia. Edited by: Vaccaro O, Bonora E, Bruno O, Garancini MP, Muntoni S. 1996, Milano, Editrice Kurtis, 17-30.
Bruno G, Carta Q, Runzo C, Prina Cerai S, Pagano G: Incidenza e prevalenza del diabete mellito tipo 2. Il diabete. 2004, 16: 295-299.
Sistema Statistico Nazionale, Istituto Nazionale di Statistica. Annuario Statistico Italiano, ISTAT. 2005
Lucioni C, Garancini MP, Massi Benedetti M, Mazzi M, Serra G: Il costo sociale del diabete di tipo 2 in Italia: lo studio Code-2. Pharmacoeconomics, Italian Research Articles. 2000, 1: 1-21.
Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003, 362: 1225-1230. 10.1016/S0140-6736(03)14546-1.
NHS Centre for Reviews and Dissemination, University of York: Getting evidence into practice. Effective Health Care. 1999, 3: 1-16.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, Rubin HR: Why don't physicians follow clinical practice guidelines?. JAMA. 1999, 282: 1458-65. 10.1001/jama.282.15.1458.
Grimshaw JM, Thomas RE, Maclennan G, Fraser C, Ramsay CR, Vale R, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dikstra R, Donaldson C, Hutchinson A: Effectiveness and Efficiency of Guideline Dissemination and Implementation Strategies. Health Technol Assess. 2004, 8: 1-72.
Grimshaw J, Campbell MK, Eccles M, Steen I: Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract. 2000, 17: S11-S18. 10.1093/fampra/17.suppl_1.S11.
Thomson O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J: Continuing education meetings and workshops: effects on professional practice and health care outcomes (Cochrane Review). The Cochrane Library. 2004, Chichester, UK: John Wiley & Sons Ltd, 1
Thomson O'Brien MA, Oxman AD, Davis DA, Haynes RB, Freemantle N, Harvey EL: Educational outreach visits: effects on professional practice and health care outcomes (Cochrane Review). The Cochrane Library. 2004, Chichester, UK: John Wiley & Sons Ltd, 1
Thomas L, Cullum N, McColl E, Rousseau N, Soutter J, Steen N: Guidelines in professions allied to medicine (Cochrane Review). The Cochrane Library. 2004, Chichester, UK: John Wiley & Sons Ltd, 1
Duff L, Batstone G, Courtney D, Kelson M, Loftus-Hills A, Hudson R, Odegbami Y, Palmer C, Scrivener R: La scelta delle linee guida. Linee guida cliniche. Strategie di implementazione. Italian translation of Implementing Clinical Guidelines. A Resource for the Health Care Team. McGraw-Hill. 2002, 17-49.
Grilli R, Magrini N, Penna A, Mura G, Liberati A: Practice guidelines by specialty societies: the need for a critical appraisal. Lancet. 2000, 355: 103-106. 10.1016/S0140-6736(99)02171-6.
Mele A, Bianco E, Cirrincione R, Jefferson T, Magrini N, Ciardullo AV, Marata AM, Satolli R, Carra L, Sagliocca L, Pellegrini L, Rusticali B, Gensini G, Conti A, Trabucchi M: PNLG. Come produrre, diffondere e aggiornare raccomandazioni per la pratica clinica. Manuale metodologico. Arti Grafiche Passoni Srl, Milano. 2002
National Guideline Program of the Italian Instituteof Health. (accessed 2 April 2003), [http://www.pnlg.it]
Agence Nationale d'Accréditation et d'Evaluation en Santé. Stratégie de prise en charge du patient diabétique de type 2 à l'exclusion de la prise en charge des complications (Mars 2000). [http://www.anaes.fr]
Drummond MF, Jefferson TO: Guidelines for authors and peer-reviewers of economic submissions to the British Medical Journal. The BMJ Economic Evaluation Working Party. BMJ. 1996, 313: 275-83.
Cluster Randomisation Sample Size Calculator version 1.0.2. Health Services Research Unit, Aberdeen University
Empirical estimates of ICCs from changing professional practices studies. University of Aberdeen, [http://www.abdn.ac.uk/hsru/epp/cluster.shtml]
A Delphi INTELx86 WIN32 platform software developed for internal use only by Agency of Public Health.
Murray DM: Design and Analysis of Group-Randomized Trials. 1998, Oxford University Press
Horton NJ, Lipsitz SR: Review of Software to Fit Generalized Estimating Equation Regression Models. The American Statistician. 1999, 53: 160-69. 10.2307/2685737.
Liang KY, Zeger SL: Longitudinal data analysis using generalized linear models. Biometrika. 1986, 73: 13-22. 10.1093/biomet/73.1.13.
Renders CM, Valk GD, Griffin SJ, Wagner EH, Eijk Van JT, Assendelft WJ: Interventions to improve the management of diabetes in primary care, outpatient and community settings. A systematic review. Diabetes Care. 2001, 24: 1821-1833. 10.2337/diacare.24.10.1821.
Task Force on Community Preventive Services: Recommendations for Healthcare System and self management education interventions to reduce morbidity and mortality from diabetes. Am J Prev Med. 2002, 22 ([4S]): 10-14.
Shojania KG, Ranji SR, McDonald KM, Grinshaw JM, Sundaram V, Rushakoff RJ, Owens DK: Effects of quality improvement strategies for type 2 diabetes on glycaemic control. JAMA. 2006, 296: 427-440. 10.1001/jama.296.4.427.
Manicardi V, Giorda CB: Modelli di assistenza diabetologica. Il diabete. 2004, 16: 369-378.
Danish Institute for Health Services Research and Development (DSI): Changing Professional Practice. Theory and Practice ofClinical Guidelines Implementation. DSI rapport 99.05. Copenhagen. 1999, [http://www.dsi.dk]
Duff L, Batstone G, Courtney D, Kelson M, Loftus-Hills A, Hudson R, Odegbami Y, Palmer C, Scrivener R: Ideare una strategia di implementazione. Linee guida cliniche. Strategie di implementazione. Italian translation of Implementing Clinical Guidelines. A Resource for the Health Care Team. McGraw-Hill. 2002, 121-163.
Studio Quadri (Qualità dell'Assistenza alle Persone Diabetiche nelle Regioni Italiane). (accessed 5 June 2006), [http://www.epicentro.iss.it/quadri/default.htm]
Campbell MK, Elbourne DR, Altman DG, for the CONSORT Group: CONSORT statement: extension to cluster randomised trials. BMJ. 2004, 328: 702-708. 10.1136/bmj.328.7441.702.
Strategies for the introduction and implementation of a guideline for the treatment of type 2 diabetics by general practitioners (GPs) of the Lazio region of Italy (IMPLEMEG study): Protocol for a cluster randomised controlled trial [ISRCTN80116232]. BMC Health Serv Res. [http://www.biomedcentral.com/1472-6963/4/13]
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/7/79/prepub
NB – Design: A C-RCT in which participants are randomly assigned to three arms and allocation is based on clusters. The report of this C-RCT is written according to the recommendations of the CONSORT statement extension to cluster-randomised trials .
The study is funded by the Italian Ministry of Health ("Special Programs" art. 12 bis D.lgs 229/99) and the Lazio Region. The Agency of Public Health of Lazio region provided computers for data collection and the resources for planning and organizational support. The project was exempt from ethical clearance according to the Italian Ministry of Health law number ex art. 12bis, Dlgs 229/1999. The Italian Data Protection act was followed in the handling of patient data.
The study protocol has been published on BMC Health Services Research 2004, 4:13 (14 Jun 2004) and is available on line .
The scientific Committee thanks Professor Jeremy Grimshaw and Paolo Giorgi Rossi for the helpful advice and the methodological contribution given.
The author(s) declare that they have no competing interests.
AP, CP, PB, CG, TJ, SL, CS, AF WM and VC wrote the protocol, chose and adapted the guideline. DM wrote the statistical methods and carried out data analysis. All authors contributed to data interpretation and inputted to the final version of the manuscript which was written by CP, DM and TJ.