Skip to main content

Advertisement

Table 3 Inter-rater reliability of AGREE instrument domain scores

From: Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines

  Scope & Purpose Stakeholder Involvement Rigor of Development Clarity & Presentation Applicability Editorial Independence
  Single Rater ICC (95% CI) Average of Raters ICC- all 6 raters or 3 pairs of 2 raters? (95% CI) Single Rater ICC (95% CI) Average of Raters ICC (95% CI) Single Rater ICC (95% CI) Average of Raters ICC (95% CI) Single Rater ICC (95% CI) Average of Raters ICC (95% CI) Single Rater ICC (95% CI) Average of Raters ICC (95% CI) Single Rater ICC (95% CI) Average of Raters ICC (95% CI)
Generalists 0.37 (0.11–0.59) 0.54 (0.19–0.74) 0.71 (0.54–0.83) 0.83 (0.70–0.90) 0.81 (0.68–0.89) 0.89 (0.81–0.94) 0.41 (0.15–0.62) 0.58 (0.26–0.76) 0.65 (0.46–0.79) 0.79 (0.63–0.88) 0.60 (0.39–0.75) 0.75 (0.56–0.86)
Clinical Specialists 0.35 (0.10–0.56) 0.52 (0.18–0.72) 0.59 (0.39–0.74) 0.74 (0.56–0.85) 0.65 (0.47–0.78) 0.79 (0.64–0.88) 0.51 (0.29–0.69) 0.68 (0.45–0.81) 0.43 (0.19–0.63) 0.61 (0.32–0.77) 0.32 (0.06–0.54) 0.49 (0.11–0.70)
Researchers 0.17 (0.14–0.46) 0.30 (-0.32–0.63) 0.73 (0.54–0.85) 0.84 (0.70–0.92) 0.77 (0.61–0.87) 0.87 (0.76–0.93) 0.47 (0.18–0.69) 0.64 (0.31–0.81) 0.47 (0.19–0.68) 0.64 (0.32–0.81) 0.59 (0.34–0.77) 0.75 (0.51–0.87)
Overall 0.40 (0.25–0.58) 0.80 (0.66–0.89) 0.67 (0.54–0.80) 0.93 (0.87–0.96) 0.79 (0.68–0.88) 0.96 (0.93–0.98) 0.55 (0.40–0.72) 0.88 (0.80–0.94) 0.50 (0.35–0.68) 0.86 (0.76–0.93) 0.35 (0.19–0.54) 0.76 (0.59–0.88)
  1. CI – confidence interval; ICC – intraclass correlation coefficient
  2. Results statistically significant at the p < 0.5 level except where indicated by bold.