Table 2 Evaluation data pertinent to the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework for each delivery system
From: Evaluating diverse electronic consultation programs with a common framework
RE-AIM dimensions and Quadruple Aim domains and example measures | Champlain BASE | Mayo | San Francisco Health Network | Veterans Administration |
|---|---|---|---|---|
REACH | ||||
Approximate annual number of e-consults | 10,000 | 18,000 | 46,500 | 443,600 |
Percentage of requests for specialty care among participating services, initiated an as e-consult | unknown | unknown | 100% | 2% |
Approximate number of e-consults per 1000 patient-lives | 8 | 14 | 465 | 74 |
Demographic information of patients who received an e-consult | 84% adult, 16% pediatric | unknown | 93% adult; 54% female; 29% Hispanic, 25% Asian, 20% White, 17% Black | 32% were for patients older than 65 years; 55% female |
EFFECTIVENESS | ||||
Quadruple aim: Population Health | ||||
E-consult response time | Mean response 1 day | Mean response 2 days | 91% response within 3 days | 95% within 3 days |
Third next available in-person new patient appointment | unknown | unknown | Decreased after e-consult implementation | unknown |
E-consult management: % e-consults without a face-to-face visit in the same specialty | 71% | 82% | 23% | 37% |
Quality of care: specialty-specific patient-level outcomes | unknown | unknown | unknown | unknown |
Quality of care: provider perceptions | 92% PCPs believed that overall value of e-consults to patients was excellent/very good; 56% of specialists believed that e-consults improved access to care | 73% of PCPs agree e-consults provide “good medical care” | 72% PCPs agree/strongly agree that e-consults improve clinical care | 56% PCPs obtained specialty input for patients who would not travel to see a specialist; 61% specialists agree that e-consult provide “high quality medical care” |
Educational value for referring providers | 93% of PCPs report high educational value | unknown | 84% of PCPs report that e-consults have educational value | unknown |
Quality of care: potential harms/safety-implications | 3.4% of e-consult cases led to initiation of a face-to-face referral when one was not originally considered | 10% of specialty recommendations were not completed by PCPs | 1–2% of patients who received a gastroenterology or general surgery e-consult experienced unintended emergency department visits or hospital admissions | 6.3% e-consults lacked appropriate specialist follow-up after initial communication; 7.4% of PCPs did not appropriately follow-up |
Quadruple Aim: Patient experience | 46% considered e-consult a viable alternative to an Endocrine face-to-face visit | unknown | Patients identified benefits to e-consults and a desire for more information about the PCP-specialist communication | Median satisfaction score of 5 on a 5-point Likert scale |
Quadruple Aim: Care team experience | 95% of PCPs reported high satisfaction; Interview data suggest high PCP and specialist satisfaction | 80% of PCPs reported good or excellent satisfaction with e-consults | 80% of PCPs agree/strongly agree that they are satisfied; qualitative data from specialists suggest high satisfaction | 93% of PCPs and 53% of specialists are satisfied; qualitative data suggest high satisfaction |
Quadruple Aim: Financial implications | Cost savings from decreased specialty care visits | unknown | unknown | Decreased costs related to patient travel |
ADOPTION | ||||
Number of e-consult specialty services | 92 | 53 | 55 | Over 50, varies by region |
Types of e-consult specialty services | Medical, Surgical, Women’s health, Pediatric, Mental Health | Medical, Surgical, Women’s health, Pediatric | Medical, Surgical, Women’s health, Pediatric | Medical, Surgical, Women’s Health, Mental Health |
Number and percentage of PCPs using the service | 75% (n = 1240) | 96% (n = 350)a | 100% (n = 76)b | unknown |
Characteristics of PCPs using the service | Family physicians, Internal Medicine physicians (in the U.S.), Nurse Practitioners, Physician Assistants, General pediatricians (Mayo, SFHN) | |||
IMPLEMENTATION | ||||
Predisposing drivers for implementation | Supply-demand mismatch for specialty care with resulting poor access to specialty services | Desire to improve access for in-person specialty care visits and expand primary care scope to manage more complex patients | Supply-demand mismatch for specialty care with resulting poor access to specialty services; inefficient referral process | Variable access to specialty services |
Reinforcing organizational factors | Identification of specialty champions | Integration in to EHR; automated e-consults for certain clinical situations | Primary care workflow re-design; inclusion of trainees in the e-consult workflow; mandatory for all requests for specialty care | Primary and specialty care workflow re-design; identification of specialty champions; local autonomy to develop new templates and workflows |
Barriers to implementation | Legal implications; lack of clinical oversight | Increased specialist workload; changes in specialist workflow; variation in how specialties value the work involved | Increased primary care workload and changes in workflow; lack of clinical oversight; legal implications | Increased PCP and specialist workload; lack of widespread training |
MAINTENANCE | ||||
Inclusion into routine practice | yes | yes | yes | yes |
Reinforcing individual-level factors | Remuneration of PCPs and specialists per e-consult | Salaried specialists who receive work credit | Salaried specialists who receive work credit | Salaried specialists who receive work credit |
Reinforcing system-level factors | Dedicated project team for customer service; ongoing quality improvement; regional healthcare policy buy-in | Ongoing quality improvement | Dedicated project team for onboarding, dissemination, and analysis; executive leadership | Local autonomy to develop new workflows; executive leadership; strong direct communication and pre-existing relationships between PCPs and specialists |