Skip to main content

Table 1 Items in the questionnaire, distribution of scores, intra class correlation and percentages of agreement between external raters per item

From: Development and testing of the KERNset: an instrument to assess the quality of telephone triage in out-of-hours primary care services

  Score (%) Inter-rater agreement (%) ICC
0 1 2 3 4 N/A Complete Reasonable  
N = 342 N = 114 N = 114 N = 114
PHASE 1 INTAKE (item 1–8)        48.0 74.3 .16
Collect personal and residence information          
1. Gathers, at an appropriate time, the personal and residence information (c) 1 3 10 18 64 3 48.2 78.9 .20
ABCDE-check (vital signs)          
2. Asks to speak to the patient (c/m) 17 3 5 2 12 61 78.1 86.0 .70
3. Makes the right choice whether or not to perform the ABCDE-check (m) 13 12 13 14 49 36.8 65.8 -a
4. Checks the ABCDE-criteria adequately and draws the right conclusion (m) 11 14 7 11 9 49 40.4 57.9 .11
Open orientation          
5. Gives caller sufficient time to describe the situation (c) 0 2 12 23 63 45.6 84.2 .16
Medical problem and help request          
6. Asks for the medical problem and its development (m) 0 4 13 30 53 36.8 86.0 .24
7. Determines and explicitly states the help request (c) 26 8 10 16 40 47.4 71.9 .06
Complaint (in triage system)          
8. Selects (in the triage system) the appropriate complaint which yields the highest urgency (m) 5 8 9 11 67 50.9 64.0 .15
PHASE 2 TRIAGE AN FOLLOW-UP (item 9–17)        50.9 73.5 .27
Questioning according to triage methodology          
9. Asks at least the essential questions belonging to the specific complaint (m) 4 9 17 30 40 36.0 82.5 .27
10. Works according to the triage system of the GP cooperative (m) 1 4 3 12 21 61 61.4 62.3 .19
History and medication          
11. Asks the relevant questions with regard to history and medication (m) 19 6 5 12 53 5 52.6 81.6 .57
Urgency estimation          
12. Recognizes changes in the status of the patient and reacts adequately (m) 2 3 1 2 2 91 85.1 86.8 -a
13. Makes an appropriate urgency estimation (m) 3 13 13 16 56 37.7 65.8 .27
Follow-up action          
14. Chooses the right follow-up action (m) 1 9 10 14 66 48.2 73.7 .16
15. Gives the right (selfcare-) advice (m) 5 5 5 9 28 47 53.5 66.7 -a
16. Gives the right safety net advice about how to act in case of a change in the situation (m) 21 8 6 13 33 20 43.0 64.9 .10
17. Provides concise information which can be clearly understood by the caller (c) 0 3 11 23 64 40.4 77.2 .30
PHASE 3 FINISHING-OFF (item 18–19)        25.4 60.5 .15
Check follow-up action          
18. Asks if the follow-up action is understood and feasible (c) 13 8 17 26 35 27.2 60.5 .15
19. Checks whether the caller agrees with the follow-up action and shows an open attitude if the caller does not agree (c) 10 9 18 25 37   23.7 60.5 .14
GENERAL (item 20–24)        36.9 72.5 .19
Structure          
20. Structures the conversation (c) 1 5 23 33 38 32.5 78.9 .10
21. Makes use of open and closed questions adequately (c) 1 8 18 32 42 31.6 79.8 .19
Summary          
22. Gives a summary at an appropriate time, verifies and adjusts the summary if necessary (c) 31 12 11 20 26 33.3 71.1 .27
Sympathize          
23. Pays attention to the experience of the caller (c) 12 10 15 21 30 12 24.6 52.6 .20
Consultation general practitioner          
24. Consults the general practitioner only if necessary (m) 4 2 6 9 79 62.3 79.8 .05
MEDICAL ITEMS        51.6 73.1 .19
COMMUNICATION ITEMS        39.3 72.9 .19
TOTAL        44.9 72.5 .19
  1. aICC could not be estimated
  2. m = medical item; c = communication item
  3. N/A = not applicable