Table 4 Program evaluation
Outcome | Timing | Data sources | Time required | Data collection process | Form of analytic variable |
|---|---|---|---|---|---|
Intervention assessment | |||||
Medication Reconciliation Intervention Score | Monthly throughout the intervention | Surveys to site leaders at each site, confirmed by mentor | 1 hour for baseline assessment, 15–20 minutes for subsequent assessments | Survey completed in QuesGen | 0-24 scale for each facet of medication reconciliation; total score |
Front-line staff Surveys to inform medication reconciliation intervention score | As needed throughout the study period as interventions implemented likely to affect results | Surveys completed by stakeholders (separate survey for outpatient clinicians) | 10 minutes per survey | Survey administered to all potential stakeholders using on-line survey software | Results used descriptively and to inform Medication Reconciliation intervention score |
Measures of context | |||||
Macro- and Micro-organizational structure | Prior to intervention | Modification of RAND ICICE organizational survey [29] completed by site leaders with help from administrative/financial personnel | 1 hour per site | Survey emailed to respondents | Varies by question type |
Safety culture, work climate, and teamwork | Prior to intervention | Modification of AHRQ patient safety culture survey [30] completed by stakeholders (e.g., pharmacists, nurses, physicians) | 10 minutes per survey | Survey administered to all potential stakeholders using on-line survey software | Composite frequency of positive responses in each of 10 dimensions of safety |
Satisfaction with medication reconciliation process and software, perceptions of errors related to medication reconciliation | Prior to intervention, again post-intervention | Survey completed by stakeholders | 5 minutes per survey | As with safety culture | Frequency of positive responses |
Job satisfaction and burnout | Prior to intervention, again post-intervention | 5 minutes per survey | As with safety culture | Frequency of positive responses | |
Qualitative information | |||||
Focus Groups | At first site visit | 5 focus groups of 6–8 representative stakeholders each, grouped by type, 35 total per site | 60-90 minutes | Administered by qualitative researcher | |
Individual Interviews | Follow-up phone calls one year after focus groups | One-on-one interviews with champions and 2–3 key opinion leaders per site | 30-45 minutes | Administered by phone by qualitative researcher | |
Intervention fidelity | |||||
Intervention Fidelity | At each site visit | Direct observation of medication reconciliation process | 6 hours at each visit (over two days) | Mentor | Mean percent completion of each process component; process fidelity scale (1–4) |