Skip to main content

Table 1 Summary of studies included in the review

From: The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review

Source Country Study Design Background of external change agent Study Arms Intervention strategies Primary Outcomes Results
Aspy et al., 2008 [25] USA Cluster RCT of 16 small-sized practices Quality improvement agent Intervention arm (8 practices): feedback with benchmarking, academic detailing, assistance of practice enhancement
Vs. usual care (8 practices)
1. Audit and feedback
2. Academic detailing
3. Practice facilitation
- proportion of mammograms recommended
-rate of performed mammograms
-Intervention arm offered significantly more mammograms than control arm (p = .043)
− 52% in intervention arm completed mammograms vs. 35% in control arm (P < .015)
Bertoni et al. 2009 [26] USA Cluster RCT: 29 practices in intervention and 32 in control (mixed sizes of small and middle) physician-investigator Both arms: treatment guidelines, intro lecture, 1 feedback report, 4 academic detailing visits
Intervention arm only: Personal digital assistant-based decision support
1. Academic detailing
2. System support (Personal digital assistant-based decision support)
- screening rate for lipid levels
- appropriate management of lipid levels
- appropriate drug prescription
- overtreatment
The screening rate for lipid levels increased in intervention but was not significant (p = .22). Appropriate management of lipid levels decreased in both arms but the difference favored intervention arm (p = .01).
Appropriate drug prescription decreased in both arms (p = .37). Overtreatment declined from 6.6% to 3.9% in intervention but rose in control from 4.2% to 6.4% (p = .01).
Clyne et al., 2015 [30] Ireland Cluster RCT: 21 mid-sized GP practices pharmacist Intervention (11 practices): academic detailing; web-based review of medicines, tailored patient info leaflets
Control (10 practices): Usual care + simple pt.-level feedback
1. Academic detailing
2. System support (Web-based treatment options)
3. Provision of educational materials (Patient information leaflets)
-proportion of patients with inappropriate prescribing
-mean no. of potentially inappropriate prescriptions
-Patients in intervention group had lower odds of inappropriate prescribing (p = .02)
-Mean no. of inappropriate prescriptions also significantly lower in intervention group (p = .02)
Dickinson et al., 2014 [18] USA Cluster RCT of 40 small to mid-sized PCPs Quality improvement agent 3 arms:
1. Practice facilitation for 6 mo. using reflective adaptive process (RAP)
2. Practice facilitation for up to 18 months using continuous quality improvement (CQI)
3. Self-direction (SD) practices with model info and resources—no facilitation
1. Practice facilitation
2. Audit and feedback
- diabetes quality measures (chart audits)
- Practice Culture Assessment surveys of clinicians & staff
- Quality of diabetes care improved in all 3 groups (all P < .05). Improvement was greater in CQI practices compared with both SD practices (P < .0001) and RAP practices (P < .0001), and in SD practices vs. RAP practices (P < .05).
- Change Culture scores in RAP practices showed trend of improvement at 9 mo. (P = .07) but decreased below baseline at 18 months (P < .05), and Work Culture scores decreased from 9 to 18 months (P < .05). In CQI and SD practices, culture scores were stable over time.
Dignan et al., 2014 [28] USA Cross-over cluster RCT of 66 mixed-size PC practices, 33 per arm Local people who knew primary care were trained in academic detailing Intervention (33 practices): “Early” clinics received academic detailing for 6 months
Control (33 practices): “Delayed” clinics received no intervention until after data were collected at 6 mo. Then delayed clinics received the academic detailing intervention the “early” clinics had received
1. Academic detailing - recommendations for screening
- completed screenings
- No increase in recommendations for screening
- Rates of completed screenings were higher for all practices for the most common screening methods (colonoscopy and fecal occult blood testing), though rates of completed colonoscopy were higher in early clinics vs. delayed (p = 0.01).
Engels et al., 2006 [19] Netherlands Cross-over cluster RCT of 49 large PC practices Trained outreach visitors Intervention arm (26 practices): Assessment of practice mgmt. Using VIP; detailed written and oral feedback; workbook with CQI tools; trained facilitator used in 5 monthly team meetings; use of QI cycles; transfer of task from facilitator to team.
Control (23 practices): Assessment using VIP with written feedback delivered in 1-h. meeting
1. Audit and feedback
2. Practice facilitation
-the number of improvement projects undertaken
-the number of improvement
steps taken for each QI project (to measure quality)
-the number of self-defined
objectives met
- Intervention group practices had significantly better results on all 3 outcomes vs. control
Feldstein et al., 2006 [29] USA Cluster RCT of 15 large sized clinics from one HMO physicians Intervention 1 (7 clinics): alerts in EHR + group academic detailing
Intervention 2 (8 clinics): Alerts only
1. Academic detailing
2. System support (EMR alerts)
- the rate of interacting prescriptions Reduction in the interacting medication prescription rate resulting in a 14.9% relative reduction at 12 months (p < .001).
Group academic detailing did not enhance alert effectiveness.
Hennesy et al., 2006 [31] USA Cluster RCT with 93 PC providers (clinic size NA) clinical pharmacist Intervention (39 providers): academic detailing visit, provider-specific data, provision of educational materials,
Control: (54 providers): no intervention
1. Academic detailing
2. Audit and feedback
3. Provision of educational materials
- the rate of blood pressure measurement below 140/90 mmHg No significant difference was detected between study arms
Hogg et al., 2008 [36] Canada Match-paired Cluster RCT of 54 small to mid-sized PC practices Masters level nurses trained in facilitation Intervention arm (27 practices): monthly visits + delivery of preventive interventions (goal setting, learning about tools, planning for reaching goals, adapting)
Control (27 practices): no services from facilitator
1. Audit and feedback
2. Practice facilitation
- Practices’ delivery of preventive maneuvers, measured by preventive performance indices from chart reviews and patient survey data. No difference was detected between the trial’s arms for the primary outcome.
Lowrie et al., 2014 [22] UK Cluster RCT of 31 small PC practices pharmacist Intervention (16 practices): org support (id patients, id barriers to prescribing change, plan for overcoming barriers, plans for indiv patients) + 3 face-to-face mtgs
Control (15 practices): usual care
1. Audit and feedback
2. Academic detailing
-the proportion of patients achieving cholesterol targets Intervention patients were significantly more likely to have cholesterol at target (69.5% vs 63.5%; OR 1.11, CI 1.00–1.23; p = 0.043) as a result of improved simvastatin prescribing.
Magrini et al., 2014 [23] Italy Two separate
Cluster RCTs: TEA vs SIDRO—115 PC groups in both studies (clinic size NA)
pharmacist Intervention: Facilitator has biannual 3–4 h. meetings with audit & feedback and problem-based learning
In both TEA and SIDRO trials, 57 primary care groups in arm 1 (TEA: benign prostatic hyperplasia; SIDRO: prulfloxacin) and 58 in arm 2 (TEA: osteoporosis; SIDRO: basmidapine)
1. Academic detailing (biannual 3–4 h meetings)
2. Audit and feedback
-changes in the six-months prescription of targeted drugs:
-TEA: Prescription of alfuzosin compared to tamsulosin and terazosin. (disease oriented)
-SIDRO: Single drug oriented, pharmacist as the main actor or facilitator of an easy-to-use
evidence synthesis, and has more clear-cut outcomes based on prescribing of a single drug
In the TEA trial, one of the four primary outcomes showed a reduction (prescription of alfuzosin compared to tamsulosin and terazosin in benign prostatic hyperplasia: prescribing ratio 28.5%, p = 0.03). Another primary outcome
(prescription of risedronate) showed a reduction at 24 and 48 months (27.6%, p = 0.02; and 29,8%, p = 0.03), but not at six months (25.1%, p = 0.36). In the SIDRO trial both primary outcomes showed a statistically significant reduction (prescription of barnidipine 29.8%, p = 0.02; prescription of prulifloxacin 211.1%, p = 0.04), which persisted or increased over time.
Conclusion: Intervention worked better for more straightforward, single-drug trial.
Note: Randomization was to arm that covers one disease or another (for TEA) or one drug or another (for SIDRO). Everybody got same intervention.
Mold et al., 2008 [17] USA Individual RCT
(small to mid-sized practices)
Practice facilitator and IT person Intervention (12 clinics): One clinician/nurse team per practice received performance feedback peer-to-peer education (academic detailing), a practice facilitator, and computer (IT) support)
Control (12 clinics): Performance feedback and benchmarking alone
1. Audit and feedback
2. Academic detailing
3. Practice facilitation
4. System support (automated reminders)
- Standing orders: protocols or policies that authorize staff to deliver services (measure is 50%use)
- Reminders: paper based or electronic (measure is 50%use)
-Wellness visits devoted to providing preventive services. (measure is 50%use)
- Standing orders: 9/14 vs 1/8 (p = .02)
- Reminders: 6/8 vs 1/4 (p = .10)
- Visits: 5/16 vs 2/10 (p = .53)
(P is calculated by binomial proportions test)
Authors’ conclusion: The multicomponent strategy increased implementation of evidence-based processes to a greater extent than performance feedback and benchmarking alone.
Mold et al., 2014 [32] USA Cluster RCT in 43 (mixed sizes) PC practices from 3 research networks Practice facilitator All practices rec’d performance feedback, academic detailing, summaries of guidelines, and a toolkit of asthma tests and action plan templates. 4 arms:
1. Practice facilitation (PF) alone (10 practices): visits ½ day/week or 1 day/every other week for 6 months.
2. Local learning collaboratives (LLCs) (10 practices): monthly meetings among practices to review data and plans
3. PF + LLC (12 practices)
4. Control (11 practices)
1. Audit and feedback
2. Academic detailing
3.Practice facilitation
4. Local Learning collaboratives (LLC)
Adherence to 6 recommendations:
- Documentation of severity assessment
- Assessment of exposure to environmental triggers,
-Assessment of level of
control,
-Prescription of controller medications
-Written asthma action plan,
-Planned asthma visits
- Statistically significant adoption rates at each arm:
Control group: 2 out of 6 recommendations
Practice Facilitation: 3 out of 6
LLC: 4 out of 6
PF + LLC: 5 out of 6
PF practices improved assessment of asthma severity and assessment of asthma level of control (p = .005)
- LLCs are not significantly effective.
Naughton et al., 2009 [21] Ireland Cluster RCT (98 GP clinics) (clinic size NA) Pharmacist 1. Audit and feedback via postal bulletin containing educational materials (50 GP clinics)
2. Audit and feedback via postal bulletin containing educational materials + Academic detailing (48 GP clinics)
1. Academic detailing
2. Provision of educational materials (postal bulletin)
Prescription data pulled from national prescribing database Antibiotic prescribing was significantly reduced in both groups, suggesting that receiving prescribing feedback was effective in reducing prescribing rates; however, there was no significant difference reported between the AF and AD groups.
Ornstein et al., 2010 [33] USA 2-arm cluster RCT (32 small-sized PC clinics) A physician and a nurse (PIs) 1. Quality improvement (QI) intervention combining EMR based audit and feedback, practice site visits for academic detailing and participatory planning (4 half day site visits over 2 years), and “best-practice” dissemination on CRC screening delivered via bi-annual in person meetings of participants vs.
2. Control (usual care)
1. Audit and feedback
2. Academic detailing
3. Practice facilitation
Proportion of active patients aged 50–75 up to date with CRC screening; proportion of active patients among those not up to date with CRC screening having screening recommended within past year. Patients 50–75 years in intervention practices exhibited significantly greater improvement in being up-to-date with CRC screening than patients in control practices (p < .001; adjusted difference 4.9%); recommendations for screening also improved in the intervention group
(p < .001; adjusted difference 7.9%).
Ornstein et al., 2013 [34] USA Delayed intervention, group-randomized trial of 19 small to mid-sized PC clinics. Physician (PI) Intervention consisted of quarterly feedback reports; 4 in-person site visits for academic detailing and participatory planning; and 2 in-person meetings of participants for networking and sharing of best practices. 1. Academic detailing
2. Audit and feedback
3. Practice facilitation
Improving screening rates for problem alcohol use, provision of brief interventions, and use of pharmacotherapy for patients with diabetes and/or hypertension Patients in early-intervention practices were significantly more likely than patients in delayed-intervention practices to have been screened ((odds ratio [OR] = 3.30, 95% CI [1.15,
9.50]) and more likely to have been provided a brief intervention (OR = 6.58, 95% CI [1.69, 25.7].) The intervention had little effect on use of pharmacotherapy for alcohol use disorders.
Parchman et al., 2013 [20] USA Stepped-wedge study design with block
randomization of practices in groups of 10 (40 small PC clinics total).
Quality Improvement experts Practice facilitation with integral audit and feedback. Facilitator held a minimum of six one-hour team meetings within each practice over a 12-month period 1. Audit and feedback
2. Practice facilitation (CCM model)
3. System support
Assessment of Chronic Illness Care (ACIC) survey score, a survey instrument designed to measure concordance with tenets of the chronic care model - Practices randomized to early intervention showed a significant improvement in ACIC scores (p < 0.05) compared to the delayed intervention (control) practices. This increase was sustained after one year.
Rognstad et al., 2013 [24] Norway 2-arm cluster RCT (449 GP providers) (mixed sizes of clinics) GP physicians associated with a university (including investigators) Intervention consisted of 2 academic detailing visits and review of a personalized audit and feedback report of providers’ potentially inappropriate prescriptions for older adults, plus an in-person full day workshop. vs.
2. GPs in the control group were assigned to another educational intervention targeting antibiotic prescribing practice for respiratory tract infections.
1. Academic detailing
2. Audit and feedback (mailed report)
Percentage of patients with potentially inappropriate prescriptions (PIPs) – based on thirteen explicit criteria. A reduction relative to baseline of 10.3% in PIPs per 100 patients aged ≥70 years was obtained in the intervention group compared to the control group.
Sheffer et al., 2012 [27] USA Two-arm cluster RCT (49 PC small sized clinics) Study physician and outreach specialist (health educator) Control condition: clinic is provided with a manual that describes the roles and responsibilities required of members of the healthcare delivery team to successfully implement a clinic-based fax referral program. In addition, clinics receive audit/feedback reports and access to educational materials.
Vs.
Intervention condition: in addition to above, clinics received On-site training at launch and 6 months (by outreach specialist to key clinic staff); Telephone check-in and performance feedback at 1 and 9 months (by outreach specialist to clinic manager); Telephone check-in and performance feedback at 3, 6, and 9 months (by study physician to clinic physician leader). .
1. Academic detailing
2. Audit and feedback
3. Practice facilitation
4. Provision of educational materials
-Number of referrals
-Number of quality referrals
- Mean number of post-intervention referrals/clinician to the Wisconsin Tobacco Quitline was 5.6 times greater in the intervention group (p = 0.001).
-Number of quality referrals was higher in intervention group (p = 0.001)
Smidth et al., 2013 [35] Denmark 2-group Cluster RCT with additional non-randomized control group (clinic size NA) Trained QI facilitators Intervention group: practices were invited to participate in four two-and-a-half-hour sessions. The Breakthrough Series was used as a framework for implementation. One facilitator visited each practice to address challenges encountered in pursuing their goals.
Control: standard governmental implementation protocols.
1. Audit and feedback
2. Practice facilitation (CCM model) (4 sessions)
3. System support
- Adherence to disease management programs for chronic obstructive pulmonary disease, measured using the Patient-Assessment-of-Chronic-Illness-Care (PACIC) instrument. There was a statistically significant change in the PACIC score in the intervention group than in the control group (intervention effect = 0.12 [95% CI: 0.00;0.25].
Varonen et al., 2007 [13] Finland 2 group Cluster RCT (30 large sized PC clinics) General practice physicians Intervention group: Academic detailing
Our modification of AD included
use of information sources, feedback of own practices and visits of external experts.
Control: Problem-based learning, a clinician education method based on group work facilitated by a local GP tutor that utilized case scenarios, information retrieval and reflection.
1. Academic detailing
2. Problem-based learning
The effect of guideline implementation on acute maxillary sinusitis management Implementation of guidelines produced only modest changes in the management of AMS. There were no significant differences between academic detailing and problem based learning education methods..
  1. PC Primary Care, GP General Practice, RCT Randomized Controlled Trial