Skip to main content

Table 2 Sources of variation and reliability of quality assessments by physician implicit review

From: Profiling quality of care: Is there a role for peer review?

  HTN COPD Diabetes Acute
Mean rating (1 – 6) * 3.41 3.09 3.39 2.90
Variance §     
   Reviewer 0.15 0.34 0.34 0.33
   Patient (between record) 0.75 0.30 0.88 0.26
   Noise (within record) 0.72 0.50 0.69 1.03
Total variance 1.62 1.14 1.90 1.62
Proportion of total variance     
   Reviewer 0.09 0.30 0.18 0.20
   Patient (between record) 0.46 0.26 0.46 0.16
   Noise (within record) 0.44 0.44 0.36 0.64
Correlation of quality ratings at patient level     
   COPD 0.382    
   Diabetes 0.767 0.106   
   Acute 0.359 0.314 0.262  
  1. *Ratings are on a 1–6 scale where 1 = very good care and 6=very poor care. This is the rating for the average patient accounting for differences in reviewer severity. Other than rounding error, the proportion of total variance sums to 100 across the 3 components. The proportion of variance at the patient level represents the reliability of structured implicit review for detecting differences between patients when assessed for samples of patients from a similar population and assessed by reviewers drawn from a population of similar reviewers. § Variation at the patient level represents the true score differences in physician ratings of quality of care across patients in regard to the specified condition. Variation at the reviewer level represents the idiosyncratic differences in rating severity the between physician reviewers. Variation at the review occasion level (repeated measurements of the same record) is termed "noise."