Interview observations | Design solution - initial design | Rationale | Iterative improvement based on follow-up interviews |
---|---|---|---|
Metrics | |||
a) Summary measures | n patients, n patients by area, n patients by shift, n patients by discharge disposition | To give physicians an overview of the patients they saw in a month. Show the area of care and shifts as it might affect case mix | |
b) Length of stay metrics | Provider to decision time, overall LOSa (median) | Overall LOS is most important for ED, but provider-decision time is easier to influence by provider | ‘Provider to decision time’ changed to ‘Room to decision time’ as patients might see another provider in the waiting area (iteration 1) |
c) Utilization of tests | CT, MR, US, lab utilization (%) | Can be affected by physician and is known to affect LOS | |
d) Outcomes | 72-h return rates, deaths (%), LOS after admission (median) | To address concerns about negative outcomes, return-rates and deaths were included. LOS after admission was included as a proxy measure of appropriateness of admissions | Removed deaths as an outcome as it is not feasible to reliably obtain data from EMR (iteration 1) |
Comparisons | |||
a) Over time | Monthly intervals | Balance between too frequent reports with random variation and too infrequent where physicians don’t remember what happened | |
b) To peers | Blinded ranking (e.g. ‘your ranking 46/60’, with outcomes of peers with better or worse numbers shown). Overall ED medians are shown on separate page | Blinded since all interviewed physicians agreed un-blinded was not desired/needed. A ranking showing neighboring peers was included to give physicians an attainable goal | Changed the ranking to interquartile range of peers instead, since the optimum rate is likely someplace in the middle, outliers in either direction can be a problem (iteration 1) |
Functionality | |||
a) Ease of access | Monthly email summary with 3 measures that can be selected by ED leadership based on priorities | Easy to access | |
b) Drilldown functionality | Option to access full dashboard through a link in monthly email | Drilldown functionality | Added tabs to drill down based on the type of shift (e.g. night) and assigned area. (iteration 1) |
c) Customization | Physicians can select measures to show up on their own favorites page Leadership can select measures in monthly email | Customization options for individual physicians and leadership based on ED priorities | |
Barriers | |||
a) Adverse consequences on quality of care | Inclusion of outcomes on dashboard | To avoid focus only on throughput and utilization measures, which might result in adverse consequences | |
b) Conflicting teaching responsibilities | – | As this was not the goal of the dashboard, no measures related to teaching were included | |
c) Data accuracy | – | Extensive validation of data is required | Included definition of the measures on the dashboard. Made section headers very clear (iteration 2) |
d) Case-mix adjustment | Show total number of patients during different shifts and in different areas of care. | By showing these measures, physicians can put other measures in context. | Added tabs to drilldown by area of care and type of shift (iteration 1) |