Skip to main content

Table 3 Proposed core effectiveness metrics for electronic consultation programs, using the Quadruple Aim framework

From: Evaluating diverse electronic consultation programs with a common framework

Arm of the Quadruple Aim

Measure

Definition

Rationale

Financial

E-consult management

Percentage of e-consults that are not scheduled for a face-to-face visit in the ensuring 12 months/Total number of e-consults per year

Calculate the number of avoidable face-to-face visits

Financial

Out of network specialty care requests

Number of out-of-network specialty care requests/Total number of specialty care requests

Examine changes in out-of-network specialty care visits

Population Health

Time to third next available new in-person appointment for e-consult specialties

Third next available new patient appointment if patient calls to make appointment

Direct measure of impact on specialty care access

Population Health

Demographics of patients who received an e-consult compared to in-person specialty care

Insurance status of patients who received at least one e-consult/Insurance status of all patients who received specialty expertise (e-consult + in-person specialty visits)

Program reach

Impact on equity

Population Health

PCP capacity

Percentage of PCPs who self-report educational value of the e-consult program on a survey

Effectiveness of e-consult

Population Health

Number of specialties offering e-consult and what they are

Raw number of specialties offering e-consult

Measure of adoption

Population Health

Unclosed loop by PCP

Number of specialist responses that are not read by PCP per year/Total number of specialist responses via e-consult per year

Patient safety; unanticipated impact

Population Health

Unclosed loop by Specialist

Number of e-consults that did not receive a specialist response per year/total number of e-consults per year

Patient safety; unanticipated impact

Population Health

Average time to e-consult response

Average lapsed number of days between time e-consult was generated and time specialist responded

Access to specialty care

Care team experience

PCP satisfaction/dissatisfaction

Percentage of PCPs who report satisfaction with the program on a survey

Program sustainability

Care team experience

Specialist satisfaction/dissatisfaction

Percentage of specialists who report satisfaction with the program

Program sustainability

Care team experience

Medical Assistant/Nurse/Referral Coordinator satisfaction/dissatisfaction

Percentage of non-MD team primary care team members who report satisfaction with the program

Program sustainability

Patient experience

Satisfaction with access to specialty care in general

Percentage of patients who report satisfaction with access to specialty care pre- and post- implementation.

Program sustainability

Patient experience

Concerns about limitations in care

N/A

Patient experience

Patient acceptability of having an e-consult

N/A

Patient experience

Travel/time saved by patients for avoided clinic visits

Number of hours that patients must forgo for each in-person visit

Business case for managed care plans