Skip to main content

Table 4 Evidence for questions addressed by the Cochrane Review.

From: What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review

Questions

Most relevant analyses from Cochrane Review

Evidence from all trials reviewed (n = 85)

Evidence from chronic disease management trials (n = 15)

Evidence from trials of diabetes care (n = 4)

Does audit and feedback work?

Any intervention involving audit and feedback versus no intervention +/- educational materials

83 comparisons: for dichotomous outcomes, median adjusted relative risk (RR) of non-compliance was 0.85 [Interquartile range (IQR) 0.74 to 0.96]*

Small to moderate effects in 11 of 19 comparisons

Moderate to large effects in two comparisons [12;13]

 

Audit and feedback versus other interventions

Five comparisons: two show audit and feedback more effective than reminders; one that local opinion leaders more effective; one no effect over patient education; one no effect of audit and feedback with educational meetings over educational meetings alone

Small effect of audit and feedback over reminders from one comparison

None

Does it work equally across all dimensions of care?

No direct comparisons; exploration of heterogeneity

No heterogeneity explained by complexity of the targeted behaviour

None

None

How should it be prepared? Should data be comparative and if so, what should the comparator group be? Should data be anonymised?

Content. Patient information, such as blood pressure or test results, compliance with a standard or guideline, or peer comparison; versus information about costs or numbers of tests ordered or prescriptions

Two comparisons: no difference between peer comparison and individual feedback without peer comparison; nor between feedback on medication and feedback on performance

No difference between feedback on medication versus feedback on performance in one comparison

None

How intensive should feedback be?

Recipients. Individual or group

No difference between individual versus group feedback in one comparison

None

None

 

Frequency. Once only or more frequent feedback

None

None

None

 

Length. Once only feedback versus audit and feedback over a period of time

None

None

None

 

Short term effects compared to longer term effects after audit and feedback stops

Mixed results from 11 comparisons

No difference from one comparison [14]

No difference from one comparison [14]

 

Exploration of heterogeneity

No heterogeneity explained by intensity of audit and feedback

  

Questions

Most relevant analyses from Cochrane Review

Evidence from all trials reviewed (n = 85)

Evidence from chronic disease management trials (n = 15)

Evidence from trials of diabetes care (n = 4)

How should it be delivered – by post or by a messenger in person? And if by a messenger who should this be?

Format. Verbal, written or both

None

None

None

 

Source. Influential source [seen to be credible and trustworthy by the professional] or feedback from any other source

Two comparisons: peer feedback better than non-physician observer feedback; no difference between peer physician versus nurse feedback

No difference between peer physician versus nurse feedback in one comparison [11]

No difference between peer physician versus nurse feedback in one comparison [11]

What activities, if any, should accompany feedback?

Audit and feedback with complementary interventions versus audit and feedback alone

No clear effect of complementary interventions from 14 studies including various comparisons except for small effect of audit and feedback combined with educational outreach. Lower baseline compliance associated with larger effect sizes.

Small or mixed effects in two out of four comparisons

Outreach by peer or nurse more effective than feedback alone [11]

What should be done about the poorest performers detected by the audit?

None

None

None

None

  1. *Relative risk [RR] is given for non-compliance. Therefore a lower RR is equivalent to greater effect size.