Table 5 Results, summary of the judgements made by the 17 evaluators of the overall quality in the 12 examples in the pilot study
Example | High | Moderate | Low | Very low | Consensus | Comments |
|---|---|---|---|---|---|---|
1 | 2/15 | 4/15 | 5/15 | 4/15 | Moderate | |
2 | 12/17 | 5/17 | - | - | High | |
3 | 1/17 | 6/17 | 5/17 | 5/17 | Need more information before consensus | |
4 | 4/17 | 13/17 | - | - | High | Based on the new rule |
5 | 12/16 | 4/16 | - | - | High | |
6 | 7/17 | 10/17 | - | - | High | Based on new rule |
7 | - | 11/17 | 6/17 | - | Moderate | |
8 | - | 6/17 | 3/17 | 8/17 | High | Based on new rule |
9 | 2/16 | 3/16 | 5/16 | 6/16 | High/Moderate depending if there are fatal flaws | |
10 | - | 1/17 | 4/17 | 12/17 | Very low | |
11 | 1/17 | 3/17 | 8/17 | 5/17 | High | Changed question |
12 | 11/13 | 2/13 | - | - | High | |