Skip to main content

Table 3 Proposed core effectiveness metrics for electronic consultation programs, using the Quadruple Aim framework

From: Evaluating diverse electronic consultation programs with a common framework

Arm of the Quadruple Aim Measure Definition Rationale
Financial E-consult management Percentage of e-consults that are not scheduled for a face-to-face visit in the ensuring 12 months/Total number of e-consults per year Calculate the number of avoidable face-to-face visits
Financial Out of network specialty care requests Number of out-of-network specialty care requests/Total number of specialty care requests Examine changes in out-of-network specialty care visits
Population Health Time to third next available new in-person appointment for e-consult specialties Third next available new patient appointment if patient calls to make appointment Direct measure of impact on specialty care access
Population Health Demographics of patients who received an e-consult compared to in-person specialty care Insurance status of patients who received at least one e-consult/Insurance status of all patients who received specialty expertise (e-consult + in-person specialty visits) Program reach
Impact on equity
Population Health PCP capacity Percentage of PCPs who self-report educational value of the e-consult program on a survey Effectiveness of e-consult
Population Health Number of specialties offering e-consult and what they are Raw number of specialties offering e-consult Measure of adoption
Population Health Unclosed loop by PCP Number of specialist responses that are not read by PCP per year/Total number of specialist responses via e-consult per year Patient safety; unanticipated impact
Population Health Unclosed loop by Specialist Number of e-consults that did not receive a specialist response per year/total number of e-consults per year Patient safety; unanticipated impact
Population Health Average time to e-consult response Average lapsed number of days between time e-consult was generated and time specialist responded Access to specialty care
Care team experience PCP satisfaction/dissatisfaction Percentage of PCPs who report satisfaction with the program on a survey Program sustainability
Care team experience Specialist satisfaction/dissatisfaction Percentage of specialists who report satisfaction with the program Program sustainability
Care team experience Medical Assistant/Nurse/Referral Coordinator satisfaction/dissatisfaction Percentage of non-MD team primary care team members who report satisfaction with the program Program sustainability
Patient experience Satisfaction with access to specialty care in general Percentage of patients who report satisfaction with access to specialty care pre- and post- implementation. Program sustainability
Patient experience Concerns about limitations in care N/A
Patient experience Patient acceptability of having an e-consult N/A
Patient experience Travel/time saved by patients for avoided clinic visits Number of hours that patients must forgo for each in-person visit Business case for managed care plans