Skip to main content

Table 1 Items in the questionnaire, distribution of scores, intra class correlation and percentages of agreement between external raters per item

From: Development and testing of the KERNset: an instrument to assess the quality of telephone triage in out-of-hours primary care services

 

Score (%)

Inter-rater agreement (%)

ICC

0

1

2

3

4

N/A

Complete

Reasonable

 

N = 342

N = 114

N = 114

N = 114

PHASE 1 INTAKE (item 1–8)

      

48.0

74.3

.16

Collect personal and residence information

         

1. Gathers, at an appropriate time, the personal and residence information (c)

1

3

10

18

64

3

48.2

78.9

.20

ABCDE-check (vital signs)

         

2. Asks to speak to the patient (c/m)

17

3

5

2

12

61

78.1

86.0

.70

3. Makes the right choice whether or not to perform the ABCDE-check (m)

13

12

13

14

49

–

36.8

65.8

-a

4. Checks the ABCDE-criteria adequately and draws the right conclusion (m)

11

14

7

11

9

49

40.4

57.9

.11

Open orientation

         

5. Gives caller sufficient time to describe the situation (c)

0

2

12

23

63

–

45.6

84.2

.16

Medical problem and help request

         

6. Asks for the medical problem and its development (m)

0

4

13

30

53

–

36.8

86.0

.24

7. Determines and explicitly states the help request (c)

26

8

10

16

40

–

47.4

71.9

.06

Complaint (in triage system)

         

8. Selects (in the triage system) the appropriate complaint which yields the highest urgency (m)

5

8

9

11

67

–

50.9

64.0

.15

PHASE 2 TRIAGE AN FOLLOW-UP (item 9–17)

      

50.9

73.5

.27

Questioning according to triage methodology

         

9. Asks at least the essential questions belonging to the specific complaint (m)

4

9

17

30

40

–

36.0

82.5

.27

10. Works according to the triage system of the GP cooperative (m)

1

4

3

12

21

61

61.4

62.3

.19

History and medication

         

11. Asks the relevant questions with regard to history and medication (m)

19

6

5

12

53

5

52.6

81.6

.57

Urgency estimation

         

12. Recognizes changes in the status of the patient and reacts adequately (m)

2

3

1

2

2

91

85.1

86.8

-a

13. Makes an appropriate urgency estimation (m)

3

13

13

16

56

–

37.7

65.8

.27

Follow-up action

         

14. Chooses the right follow-up action (m)

1

9

10

14

66

–

48.2

73.7

.16

15. Gives the right (selfcare-) advice (m)

5

5

5

9

28

47

53.5

66.7

-a

16. Gives the right safety net advice about how to act in case of a change in the situation (m)

21

8

6

13

33

20

43.0

64.9

.10

17. Provides concise information which can be clearly understood by the caller (c)

0

3

11

23

64

–

40.4

77.2

.30

PHASE 3 FINISHING-OFF (item 18–19)

      

25.4

60.5

.15

Check follow-up action

         

18. Asks if the follow-up action is understood and feasible (c)

13

8

17

26

35

–

27.2

60.5

.15

19. Checks whether the caller agrees with the follow-up action and shows an open attitude if the caller does not agree (c)

10

9

18

25

37

 

23.7

60.5

.14

GENERAL (item 20–24)

      

36.9

72.5

.19

Structure

         

20. Structures the conversation (c)

1

5

23

33

38

–

32.5

78.9

.10

21. Makes use of open and closed questions adequately (c)

1

8

18

32

42

–

31.6

79.8

.19

Summary

         

22. Gives a summary at an appropriate time, verifies and adjusts the summary if necessary (c)

31

12

11

20

26

–

33.3

71.1

.27

Sympathize

         

23. Pays attention to the experience of the caller (c)

12

10

15

21

30

12

24.6

52.6

.20

Consultation general practitioner

         

24. Consults the general practitioner only if necessary (m)

4

2

6

9

79

–

62.3

79.8

.05

MEDICAL ITEMS

      

51.6

73.1

.19

COMMUNICATION ITEMS

      

39.3

72.9

.19

TOTAL

      

44.9

72.5

.19

  1. aICC could not be estimated
  2. m = medical item; c = communication item
  3. N/A = not applicable