| Rating Year |
---|
Coder | 2016 (n = 7a) | 2017 (n = 8) | 2018 (n = 9) | 2019 (n = 5) | 2020 (n = 6) |
---|
Coder 1 | .886 | .831 | .864 | .831 | .789 |
| (n = 5) | (n = 8) | (n = 9) | (n = 4) | (n = 6) |
Coder 2 | .866 | .869 | .854 | .927 | .768 |
| (n = 7) | (n = 8) | (n = 8) | (n = 5) | (n = 6) |
Coder 3 | .712 | .936 | .894 | .882 | .801 |
| (n = 6) | (n = 7) | (n = 8) | (n = 4) | (n = 6) |
Coder 4 | .732 | .879 | .831 | .700 | .727 |
| (n = 7) | (n = 7) | (n = 7) | (n = 4) | (n = 6) |
Coder 5 | .852 | .848 | .918 | .749 | .812 |
| (n = 7) | (n = 7) | (n = 7) | (n = 5) | (n = 6) |
Total Approved Spots | 8 | 9 | 10 | 6 | 7 |
- The coder designation represents the same coders in Table 1. ICC scores represent a coder’s ratings minus their lowest score. The number of approved spots varies due to coders’ attendance. For example, Coder 1 attended nine meetings of 11 meetings in 2016, but was only present at 6 of the 8 meetings where spots were rated and approved for testing
-
aThis value represents the number of approved spots minus one. The lowest score is dropped for each coder. For example, in 2016 there were 8 possible test spots; for a coder with perfect attendance, ICCs were calculated on their 7 best scores