Table 1 Evaluation framework for NHS Scotland: Clinical Enquiry and Response Service (CLEAR)
From: Developing a framework to evaluate knowledge into action interventions
|  | Outcome level | Outcomes chain for CLEAR | Measures for CLEAR | Evidence gathered for CLEAR for 2015–16 |
|---|---|---|---|---|
Activity/organisation measures | Resource What resources will support the activities? | Existing team of information professionals and health service researchers. Website developers. | Adequate staffing to deliver service in line with demand. Website availability and accessibility. | Staffing data: adequate staffing to cover demand. No issues reported with website. |
Activities What K2A activities will take place? | Training of staff in writing summaries; development of efficient processes; website development | Adequate trained staff to deliver service in line with demand. Website usage. | No training required/delivered during this period. | |
Outputs What will the products/methods/services be? | Web based evidence summaries answering clinical questions. | Number of requests & responses. | For 2015: 92 enquiries received, 63 responses, (remainder referred or declined). | |
Immediate outcomes | Reach Which targeted groups are being engaged at micro, meso and macro levels? | Target audience general practice and staff in remote locations. All NHS Scotland practitioners. NHS Scotland Knowledge Services. | Number of enquiries per board/professional groups. | General Practice: 21 enqs Nursing/Midwid: 12 enqs Hospital Medicine: 11 enqs Allied Health Prof: 10 enqs |
Reactions How do the target groups react to the service? Satisfaction/timely/relevance/ efficiency… | Useful and relevant service, which saves practitioners time. | Number of visits to website. Number of repeat users. Users indicate they would recommend the service. Users indicate satisfaction with response received. Users indicate it saves time. Knowledge services staff promote CLEAR. | Google Analytics for 2015–16: 31,311 unique page views; 12,865 unique visits. 18% returning visitors.. 16 survey responses during 2015–16: 100% would recommend service and use again; 100% report query being fully or partially resolved; 100% report their practice being partially or fully informed by the response. 6 reported saving 4–6 h of their time, others gave ad hoc comments to this effect. | |
Intermediate outcomes | Knowledge, Attitudes, skills aspirations What knowledge, skills, attitudes change as a result of using the activity/outputs? | Improved practitioner understanding of diagnosis, aetiology, prognosis and appropriate therapies relating to patient care. Increased practitioner awareness of evidence resources available. Increase in skills base within knowledge services. | Users indicate they were presented with new information or are aware of new resources. Knowledge service staff indicate they have improved or acquired new skills. | 44% of survey respondents reported receiving new information. |
Practice, behaviour change What practices/behavious do you expect to change as a result of the activity/outputs? | Clinical decisions informed by best available evidence. Greater engagement of clinical staff with evidence resources. | Increase in usage of evidence based resources via Knowledge Network. Staff indicate their practice was informed by the information provided. | Athens access statistics. User survey. Annual evaluation report by project manager | |
Final outcomes | More effective practice and wider outcomes How will practice be more effective as a result of the activity/outputs? What outcomes will this contribute to? | Improved patient care leading to better patient outcomes. | Case studies of improved care. | Individual follow up with practitioners via survey responses if appropriate. |