Skip to main content

Table 1 Items and reliability of the revised version of the peer review-manual for quality assurance in work capacity evaluation (n = 325)

From: An instrument for quality assurance in work capacity evaluation: development, evaluation, and inter-rater reliability

Item no.

Test question

Percentage agreement

rw (n = 325)

Quality domain: formal structure

 A.1

To what extent does the report structure complied with the requirements?

67.5%

0.19

 A.2

To what extent is the unified set form, consisting of cover and back banner page, used?

82.5%

0.89

Quality domain: clarity

 B.1

To what extent is the linguistic expression correct and unambiguous?

69.7%

0.27

 B.2

To what extent are technical terms and abbreviations that are essential for the understanding of the report explained?

50.5%

0.36

 B.3

To what extent is the specific social medicine terminology applied correctly?

68.1%

0.16

 B.4a

To what extent are socio-legal implementations / conclusions omitted?

80.6%

0.29

Quality domain: transparency

 C.1

To what extent is the origin of medical information described?

53.8%

0.27

 C.2

To what extent does the report illustrated by which processes, methods and tools the medical results are collected?

66.0%

0.37

 C.3

To what extent does the report illustrated which measured values, reference ranges and graduations are basis of the medical assessment?

57.9%

0.30

Quality domain: completeness

 D.1

To what extent is the medical anamnesis depicted completely?

48.9%

0.27

 D.2a

To what extent are medical findings documented to answer the social medicine report questions?

64.6%

0.31

 D.3

To what extent are ICD diagnosis illustrated with their functional limitations?

49.2%

0.45

 D.4a

To what extent are complaints, diseases and functional limitations expressed by the insured included in the discharged summary?

46.2%

0.15

 D.5a

To what extent are medical findings included in the discharged summary?

52.2%

0.22

 D.6a

To what extent are functional limitations in relation to performance in working life evaluated?

52.7%

0.09

 D.7

To what extent are statements on previous therapy and future therapeutic options given?

39.8%

0.36

 D.8

To what extent are substantial differences in the work capacity evaluation compared to earlier medical reports explained?

79.9%

0.25

 D.9

To what extent are all social medicine report questions fully answered?

43.9%

0.17

Quality domain: medical-scientific principles

 E.1

To what extent is the widely accepted state of medical knowledge applied?

88.6%

0.27

 E.2

To what extent is the existing literature for work capacity evaluation of the German Pension applied?

65.6%

0.24

Quality domain: efficiency

 F.1

To what extent is the diagnostic investigation appropriate and necessary?

72.5%

0.20

 F.2

To what extent is the diagnostic investigation sufficient?

57.3%

0.22

Superordinate criterion: experts’ report confirmability

 Evaluate the confirmability of the medical report on the basis of the argumentation used.

47.3%

0.39

  1. Quality domains: four-point rating scale (no deficiencies, mild deficiencies, clear deficiencies, serious deficiencies); superordinate criterion: three-point rating scale (no argument interruptions; argument interruptions that can be bridged by the assessing peer; argument interruptions that cannot be bridged by the assessing peer); rw = Kendall’s coefficient of concordance W
  2. a items has been removed from the manual in agreement with the German Statutory Pension Insurance after this present inter-rater reliability study was completed