Table 1 IDSR evaluation attributes and indicators, south and southeast of Madagascar, 2014–15
Attribute/ topic | Indicators | Numerator/ denominator |
|---|---|---|
Simplicity | ||
Ease of understanding | Presence of Terms of Reference (TOR) in the HS | Number of HS agents possessing a (SIMR) TOR document/ Number of interviewed HS agents |
Proportion of HS agents capable of describing the activities linked to the surveillance (according to TOR) | Number of HS agents who could describe the activities linked to the surveillance/ Number of interviewed HS agents | |
Ease of execution | Proportion of HS agents who | |
• Master selected case definitions (acute respiratory infection (ARI), diarrhoea, malaria, dengue-like syndrome (DLS), and measles) | • Number of HS agents who correctly cited all case definition aspects/ Number of interviewed HS agents | |
• Presence of case definitions guidelines in the HS | • Number of HS agents possessing hardcopy case definitions/ Number of interviewed HS agents | |
Distribution of data collection mode and kind of tools used | Number per mode or tool/ Number of modes or kind of tools used | |
Time of data collection | Median and range of minutes needed each week | |
Time of SMS editing | Median and range of minutes needed to write one SMS | |
Data quality | ||
Missing data | Number and proportion of SMS with ≥1 missing observation among the ten last SMS sent for frequent diseases (see under definitions below)/ syndrome | Number of SMS with ≥1 missing observation/ 10 last SMS sent |
Proportion of SMS with ≥1 missing observation among the ten last SMS sent for rare diseases (see under definitions below)/ syndrome | Number of SMS with ≥1 missing observation/ 10 last SMS sent | |
Erroneous data | Comparison of consultation register and sent SMS archived on the HS’s mobile phone, when this was not possible (no SMS archive), data from the consultation register was compared to the databases at district or central level | |
Proportion of erroneous observations among 10 last SMS sent | Number of erroneous observations/ Number of observations sent | |
Number of erroneous observations within the 10 last SMS sent | Median and range of erroneous observations | |
Number of supervision visits in 2014 | Median and range of supervision visits in 2014 | |
Completeness & Timeliness | ||
Routine completeness | Proportion of SMS reports received over last 4 weeks | Number of SMS received/ Number of SMS expected |
Distribution of reasons for not sending SMS reports | Number of HS agents invoking each reason/ Number of reasons (for not sending SMS) quoted | |
Routine timeliness | Proportion of routine SMS received in time (see under definitions below) for the last 4 weeks | Number of SMS in time per week/ Number of expected SMS |
Distribution of reasons for not sending the SMS in time over last 4 weeks | Number of HS agents invoking each reason/ Number of quoted reasons | |
Alert notifications | Number of HS that notified alerts | |
Number of alerts notified by HS in 2014 | ||
Type of notified alerts | ||
Proportion of alert notifications received in time (see under definitions below) for the last 4 alerts | Number of alerts received in time / Number of alerts received | |
Technological evaluation | ||
Geographical mobile phone network coverage and coverage at/around HS with the three available providers | Verification during HS visits or during telephone interview | |
Sources of mobile phones used for data transfer | Number of HS by phone source/ Number all HS mobile phones | |
Mobile phone changes/ replacements since arrival on job | Number of mobile phone changes/ replacements | |
Mobile phone handling capacity by HS agents (following demonstration by evaluation team) | Number of HS agents by capacity/ Number of all interviewed HS agents | |
Energy sources, availability and capacity | Evaluation by the evaluators in the field during HS visits | |
Last problem experienced with mobile phone charging | Interview with the HS director | |
Number of HS by time of last problem/ Number all HS | ||