Skip to main content

Table 2 Evaluation data pertinent to the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework for each delivery system

From: Evaluating diverse electronic consultation programs with a common framework

RE-AIM dimensions and Quadruple Aim domains and example measures Champlain BASE Mayo San Francisco Health Network Veterans Administration
REACH
 Approximate annual number of e-consults 10,000 18,000 46,500 443,600
 Percentage of requests for specialty care among participating services, initiated an as e-consult unknown unknown 100% 2%
 Approximate number of e-consults per 1000 patient-lives 8 14 465 74
 Demographic information of patients who received an e-consult 84% adult, 16% pediatric unknown 93% adult; 54% female; 29% Hispanic, 25% Asian, 20% White, 17% Black 32% were for patients older than 65 years; 55% female
EFFECTIVENESS
 Quadruple aim: Population Health
 E-consult response time Mean response 1 day Mean response 2 days 91% response within 3 days 95% within 3 days
 Third next available in-person new patient appointment unknown unknown Decreased after e-consult implementation unknown
 E-consult management: % e-consults without a face-to-face visit in the same specialty 71% 82% 23% 37%
 Quality of care: specialty-specific patient-level outcomes unknown unknown unknown unknown
 Quality of care: provider perceptions 92% PCPs believed that overall value of e-consults to patients was excellent/very good; 56% of specialists believed that e-consults improved access to care 73% of PCPs agree e-consults provide “good medical care” 72% PCPs agree/strongly agree that e-consults improve clinical care 56% PCPs obtained specialty input for patients who would not travel to see a specialist; 61% specialists agree that e-consult provide “high quality medical care”
 Educational value for referring providers 93% of PCPs report high educational value unknown 84% of PCPs report that e-consults have educational value unknown
 Quality of care: potential harms/safety-implications 3.4% of e-consult cases led to initiation of a face-to-face referral when one was not originally considered 10% of specialty recommendations were not completed by PCPs 1–2% of patients who received a gastroenterology or general surgery e-consult experienced unintended emergency department visits or hospital admissions 6.3% e-consults lacked appropriate specialist follow-up after initial communication; 7.4% of PCPs did not appropriately follow-up
 Quadruple Aim: Patient experience 46% considered e-consult a viable alternative to an Endocrine face-to-face visit unknown Patients identified benefits to e-consults and a desire for more information about the PCP-specialist communication Median satisfaction score of 5 on a 5-point Likert scale
 Quadruple Aim: Care team experience 95% of PCPs reported high satisfaction; Interview data suggest high PCP and specialist satisfaction 80% of PCPs reported good or excellent satisfaction with e-consults 80% of PCPs agree/strongly agree that they are satisfied; qualitative data from specialists suggest high satisfaction 93% of PCPs and 53% of specialists are satisfied; qualitative data suggest high satisfaction
 Quadruple Aim: Financial implications Cost savings from decreased specialty care visits unknown unknown Decreased costs related to patient travel
ADOPTION
 Number of e-consult specialty services 92 53 55 Over 50, varies by region
 Types of e-consult specialty services Medical, Surgical, Women’s health, Pediatric, Mental Health Medical, Surgical, Women’s health, Pediatric Medical, Surgical, Women’s health, Pediatric Medical, Surgical, Women’s Health, Mental Health
 Number and percentage of PCPs using the service 75% (n = 1240) 96% (n = 350)a 100% (n = 76)b unknown
 Characteristics of PCPs using the service Family physicians, Internal Medicine physicians (in the U.S.), Nurse Practitioners, Physician Assistants, General pediatricians (Mayo, SFHN)
IMPLEMENTATION
 Predisposing drivers for implementation Supply-demand mismatch for specialty care with resulting poor access to specialty services Desire to improve access for in-person specialty care visits and expand primary care scope to manage more complex patients Supply-demand mismatch for specialty care with resulting poor access to specialty services; inefficient referral process Variable access to specialty services
 Reinforcing organizational factors Identification of specialty champions Integration in to EHR; automated e-consults for certain clinical situations Primary care workflow re-design; inclusion of trainees in the e-consult workflow; mandatory for all requests for specialty care Primary and specialty care workflow re-design; identification of specialty champions; local autonomy to develop new templates and workflows
 Barriers to implementation Legal implications; lack of clinical oversight Increased specialist workload; changes in specialist workflow; variation in how specialties value the work involved Increased primary care workload and changes in workflow; lack of clinical oversight; legal implications Increased PCP and specialist workload; lack of widespread training
MAINTENANCE
 Inclusion into routine practice yes yes yes yes
 Reinforcing individual-level factors Remuneration of PCPs and specialists per e-consult Salaried specialists who receive work credit Salaried specialists who receive work credit Salaried specialists who receive work credit
 Reinforcing system-level factors Dedicated project team for customer service; ongoing quality improvement; regional healthcare policy buy-in Ongoing quality improvement Dedicated project team for onboarding, dissemination, and analysis; executive leadership Local autonomy to develop new workflows; executive leadership; strong direct communication and pre-existing relationships between PCPs and specialists
  1. aData are pertinent to the Rochester site only
  2. bData are pertinent to SFHN primary care clinics only