Concordance between 8-1-1 HealthLink BC Emergency iDoctor-in-assistance (HEiDi) virtual physician advice and subsequent health service utilization for callers to a nurse-managed provincial health information telephone service

Background British Columbia 8–1-1 callers who are advised by a nurse to seek urgent medical care can be referred to virtual physicians (VPs) for supplemental assessment and advice. Prior research indicates callers’ subsequent health service use may diverge from VP advice. We sought to 1) estimate concordance between VP advice and subsequent health service use, and 2) identify factors associated with concordance to understand potential drivers of discordant cases. Methods We linked relevant provincial administrative databases to obtain inpatient, outpatient, and emergency service use by callers. We developed operational definitions of concordance collaboratively with researcher, patient, VP, and management perspectives. We used Kaplan–Meier curves to describe health service use post-VP consultation and Cox regression to estimate the association of caller factors (rurality, demography, attachment to primary care) and call factors (reason, triage level, time of day) with concordance as hazard ratios. Results We analyzed 17,188 calls from November 16, 2020 to April 30, 2021. Callers advised to attend an emergency department (ED) immediately were the most concordant (73%) while concordance was lowest for those advised to seek Family Physician (FP) care either immediately (41%) or within 7 days (47%). Callers unattached to FPs were less likely to schedule an FP visit (hazard ratio = 0.76 [95%CI: 0.68–0.85]). Rural callers were less likely to attend an ED within 48 h when advised to go immediately (0.53 [95%CI:0.46–0.61]) compared to urban callers. Rural callers advised to see an FP, either immediately (1.28 [95%CI:1.01–1.62]) or within 7 days (1.23 [95%CI: 1.11–1.37]), were more likely to do so than urban callers. Interpretation Concordance between VP advice and subsequent caller health service use varies substantially by category of advice and caller rurality. Concordance with advice to “Go to ED” is high overall but to access primary care is below 50%, suggesting potential issues with timely access to FP care. Future research from a patient/caller centered perspective may reveal additional barriers and facilitators to concordance. Supplementary Information The online version contains supplementary material available at 10.1186/s12913-023-09821-w.


Background
Many Canadian provinces have publicly funded, nursemanaged telephone services to support citizens with urgent health concerns.These services provide information on management and on whether, when, and where to seek appropriate care [1][2][3][4][5].In British Columbia (BC), the nurse-managed telephone service involves an 8-1-1 phone line overseen by HealthLink BC (HLBC).The COVID-19 pandemic greatly increased both the volume and complexity of calls to 8-1-1, with many callers seeking to avoid the potential infection risk associated with in-person healthcare visits.This prompted HLBC to add a virtual physician (VP) consultation service, called HLBC Emergency iDoctors in-assistance (HEiDi) [6].Through the HEiDi service, callers identified by 8-1-1 nurses as requiring medical attention immediately or as soon as feasible can be referred for supplemental virtual assessment by telephone or video by off-site emergency physicians, with the potential to safely divert some callers away from emergency departments (EDs).

Nurse-led telephone triage and advice
Existing research suggests that concordance between advice given on nurse-managed telephone triage services and subsequent health service use ranges between 62 and 75%, [7,8] with concordance with advice to see a family physician (FP) being particularly low [8].A 2010 study of a nurse-led 8-1-1 line in Calgary, Alberta, found high "follow-through" for individuals advised to monitor symptoms at home, but relatively low "follow-through" on advice to seek care in an ED immediately (52.3%) or to seek care in the community within 24-h (43.2%) [9]."Follow-through" also varied significantly by health concern [9].A systematic review by Veteran's Affairs found little evidence that nurse-led remote triage reduced downstream service use [10].However, a more recent study found that patients who undergo initial remote triage using a validated algorithm are less likely to be given a low urgency triage when presenting to an ED [11], suggesting improvement in service provision.

The context of ED and FP care and potential drivers of concordance
When examining concordance, it is important to consider trends documented in the literature around the use of health services.Of particular interest are patterns around the "appropriate" use of emergency care among subpopulations we may see reflected in BC.Lower levels of education have been positively associated with likelihood of both high and low acuity visits, with income showing a similar pattern [12].Persons with a regular medical doctor have been found to be less likely to attend EDs [12].Those living in a rural area have been found to be more likely to do so, and but less likely to present with high acuity [12].Finally, a 2021 study found that patients living in areas with high scores for "deprivation" (Area Deprivation Index or ADI) or "stress" (neighborhood stress score or NSS) have a higher frequency of ED utilization [13].
A key concern for triage is that the ED often serves as a "safety net" for patients who lack access to appropriate primary care [14].Lavergne et al. 's 2022 analysis of primary care service use and neighbourhood income in BC also found evidence of inequity in access to care [15].From 1999/2000 through 2017/18, the number of primary care encounters declined in both urban and rural settings while ED visits increased [15].This raises concerns for equity generally, but particularly for Indigenous patients, as those living in urban areas were found to have less access to primary care and consistently higher "preventable hospital admissions", with an age-standardized mortality rate 2-5 times higher compared to other residents that authors felt was likely attributable to "gaps in access to primary care services." [14].

Physician assistance in telephone triage and advice
Research on the impact of the addition of VP consultation to nurse-managed telephone triage is extremely limited as few jurisdictions offer this service.Li et al. have published an evaluation of a similar program for veterans in the US which compares the clinical outcomes of telephone triage utilizing emergency physicians in addition to nurses versus nurses alone.However, the focus of this program was to better direct patients to in-network emergency departments rather than identify the most appropriate modality of care, precluding direct comparison to HEiDi [16].In a previously published description and process evaluation of the HEiDi service, VPs advised 72.1% of urgently triaged callers away from visiting EDs or urgent care clinics within 24 h, while advising 15.0% of callers whose concerns were deemed less urgent by the nurse to attend an ED immediately [6].Concordance between the VP's recommendations and subsequent health service use has not been explored.Examining concordance with advice is an important step in understanding both the potential value of including VPs in nurse-led telephone services, and what shapes patients' abilities and decisions to seek recommended follow-up care.

Objective
This exploratory, hypothesis-generating analysis uses population-based administrative data to examine concordance between virtual physician advice and subsequent health services use.
We sought to 1) estimate concordance between VP advice and subsequent health service use, and 2) identify factors associated with concordance to understand potential drivers of discordant cases.

Study design
We conducted a retrospective, time-to-event (survival) analysis of routinely collected administrative health data.This study was approved by the University of British Columbia's Clinical Research Ethics Board (H21-02209).
The study was conducted in British Columbia, Canada.As of April 2021, BC had approximately 5.2 million residents [17] distributed across five health administration regions (locally referred to as "health authorities" or HAs) [18] with a total of 120 hospitals [19] and 103 emergency departments [20].

Data sources, linkage, and variables
We used BC Ministry of Health's Health Data Platform BC (HDPBC) to access anonymized, linked administrative health data in a secure environment.Linkage between datasets was conducted using anonymized Personal Health Numbers (PHN).
HealthLink BC provided access to their 8-1-1 dataset to identify HEiDi callers during the study period.8-1-1 data also included the level of nurse triage ("red-MD" or "yellow" indicating high and moderate urgency, respectively), health concern associated with call (e.g., "cardiovascular", "pediatric", "pharmaceutical (medications)"), time of call, and VP category of advice given ("disposition" -e.g."Go To ED Now").To mitigate potential confounding of concordance from multiple calls for the same or new issues, only the first call from each caller in a two-week period was used for this analysis.We used "cardiovascular" as the reference health concern for VP advice as this is widely understood as an urgent concern by both clinicians and the lay public.The most urban HA in BC, Vancouver Coastal, was used as the baseline for regional comparison.Finally, the 21-40 year-old age category was the most frequent and so used as the reference for comparisons across age.
We used the Discharge Abstract Database (DAD) [21] and National Ambulatory Care Reporting System (NACRS) [22] datasets to identify inpatient hospitalizations and ED visits up to seven days after the HEiDi call, and Medical Services Plan (MSP) billing information [23] to track use of outpatient services, particularly FP visits in the community for that same period.Both in-patient and remote (virtual/telehealth) services were tracked in MSP billings.This dataset also included a measure of "FP attachment", based on a Ministry of Health algorithm that estimates whether a person has a consistent FP [24].For the purposes of this analysis FP refers to practitioners identified by a speciality code of "00″, "Family Medicine", for purposes of MSP billing [23].Finally, demographic information (age, sex, rurality as per Statistics Canada's definitions [25], and socioeconomics (SES) status via the "Canadian Index of Multiple Deprivation" [26] on callers was linked to the above using the "Client Roster" [27], and mortality was tracked by linking callers' to Vital Events Deaths database [28].
The NACRS dataset, only includes data for approximately 29 EDs, typically at larger, urban hospitals constituting 70% of BC ED visits [29].We therefore supplemented these data with billings from the MSP and DAD databases, following a previously described algorithm to improve ED visit ascertainment [30].
Table 1 Operational definitions of VP advice "dispositions" and their interpretation

HEiDi virtual physician advice and 8-1-1 "Disposition codes":
We used consensus-derived operational definitions of existing HEiDi VP advice categories assigned by VPs at the end of each call, and the interpretation in Table 1 to describe the advice VPs gave to callers regarding seeking follow-up care.

Study population
We have previously reported details of the 8-1-1 and HEiDi service and call-flow [6].All of the "red-MD" or "yellow" callers interviewed by 8-1-1 nurses and referred for VP consultation between November 16, 2020 and April 30, 2021 -representing approximately 26% and 38% of 8-1-1 calls respectively -were included this analysis.We excluded any individuals with missing demographic data or whose PHN corresponded to a deceased individual at the time of call.We also excluded callers who received the "Other" disposition from the HEiDi physician, as this was insufficiently specific to allow assessment of concordance.

Concordance
We used the following definitions of concordance, developed through consultation with HEiDi virtual physicians, health service researchers and a patient partner from the BC Emergency Medicine Network's Patient Council.Virtual physicians strongly indicated that concordance could only reliably be assessed over a one-week (seven day) time-horizon, as over extended periods the likelihood that service use stems from issues unrelated to the reason for the index call increases (Table 2).
Concordance is considered achieved if the first service used by the caller subsequent to the call aligns with physician advice.Use of other services after this initial encounter is still considered concordant, but use of other services prior to the advised service does not.For example, for "Go to ED Now": a caller who saw an FP or was hospitalized prior to their ED visit would not be counted as concordant.However, a caller who saw an FP and/or was hospitalized following their ED visit would be.

Data analysis
We conducted a time-to-event (TTE) analysis using a Cox proportional hazard model [31] for each HEiDi disposition.Each of the models included the same set of independent variables, described in "Data Sources, Linkage and Variables" above.This was a hypothesisgenerating analysis to determine the hazard ratios for each independent variable and for each disposition (as opposed to optimizing a predictive model with fewer predictors).A TTE analysis allowed us to estimate the effect of multiple predictors over the range of hours or days for each disposition.
We used SQL Server Management Studio (version 18.5, Microsoft) to query and link records across databases.Eligible records were extracted and imported to R (version 4.1, R Core Team) for statistical analysis.

Sensitivity analysis
We also conducted sensitivity analysis with expanded definitions of concordance, incorporating broader health services use, for the three dispositions (Table 3).
VPs expressed concern that strictly "binary" definitions of concordance may be misleading as advice given to callers often includes conditional statements and may be layered "You should do X unless Y changes, then look into doing Z".This means the patient-caller could be initially told to "Seek FP care now" unless a given symptom worsens, in which case "Go to ED now".We can only observe the initial layer of advice, but this sensitivity analysis allows us to view patterns of callers' behaviour with the Table 2 Operational definitions of VP advice and corresponding concordance criteria a In conversations with VPs, they indicated that "seek FP care now" was considered more urgent than "schedule FP appointment" but noted that time to access FPs varies.A consensus existed that the appointment should be as soon as possible in the former scenario, and ideally within a week for the latter.Given potential access issues that may affect a plurality of patients, we included both dispositions with a one-week timeframe to assess if concordance patterns demonstrate a similar perception of urgency based on subsequent service use b Note the final category advises that service use is unnecessary, and so "time-to-concordance" is inverted as "time-to-discordance. "

VP Advice
Concordance Criteria

Go to ED Now
The caller has a recorded ED visit within 48 h of the HEiDi call and did not have a recorded hospitalization or FP encounter before the ED visit Seek FP care now a The caller has a recorded FP encounter within seven days of the HEiDi call and did not have a recorded hospitalization or ED visit before the FP encounter Schedule FP appointment a The caller has a recorded FP encounter within seven days of the HEiDi call and did not have a recorded hospitalization or ED visit before the FP encounter

Self-Manage b
The caller has no recorded ED visits, hospitalizations, or FP encounters within seven days of the HEiDi call allowance for conditional advice unobservable in the data.
In this exploratory analysis, we examined sensitivity analysis results for any significant, systematic difference from primary analyses.

Caller and call characteristics
Most callers were women (64.3%) over age 21, with age 21-40 being the most frequent age group and median age of callers ranging from 35.6 to 42.1 years by disposition.Callers were disproportionately high SES as compared to the general population reported in BC census data, with 25.1% in the top quintile.Rurality varied slightly by disposition but was generally slightly higher than from provincial population estimates of ~ 13% [32].Approximately 84% of callers were "attached" to a FP (Table 4).
Seventeen percent of calls triaged to VPs were advised to seek immediate emergency care, 48% were advised to seek care in the community, and 36% were advised to self-manage (Table 4) Most calls were placed during daytime business hours (69.2%) on a weekday (~ 70%).These characteristics have been evaluated in previously published work [24].

Post-HEiDi health service utilizations
Callers assigned "Go to ED Now" were the most concordant at seven days (72.9%).Advice to "See FP now" was 49.6% and "Schedule FP Appointment" was 50.1%.Similarly, 48.9% of callers advised to "Self-Manage" did not use any additional services (Table 5).The "ED now" group was also 4 times more likely to be hospitalized than the others (8% vs 2%), suggesting high concern was justified.Specialist encounters and diagnostic testing appeared associated with the urgency of VP advice, with higher levels of concern seeing greater service use.FP visits were only slightly less frequent for "Go to ED Now" callers than FP-specific dispositions (45% vs 50%), and about 1/3 of those assigned to self manage saw an FP by week's end (Table 5).

Time-to-event results
As with absolute values, the rate of concordance varied across the four dispositions (Fig. 1, see Additional File 1: Table S2).A key finding evident in Fig. 1 is that concordance was highest "Go to ED Now", where 70.4% of callers were found to have an ED visit within 48 h.We note this advice appears to have been acted on quickly-over 90% (1840/2025) of the subset of concordant callers visit an ED within the first 4 h (see Additional File 1: Table S2).Concordance was lowest for those told to seek FP care now or within seven days at 41% and 47% respectively.Sixty percent of callers in the "Self-Manage" disposition did not have an identified ED visit, hospitalization, or FP encounter within seven days.
While several geographic signals emerged from this analysis, only the Northern Health Authority was significantly associated with concordance across all four dispositions (Table 6).For the "Go to ED Now" disposition, HEiDi callers from Northern were less likely (hazard ratio = 0.56 [95%CI: 0.35-0.61])while Fraser (1.16 [1.03-1.30])and Island (1.39 [1.21-1.59])were more likely to be concordant as compared to Vancouver Coastal.Rurally-located callers were significantly  Callers in the "Go to the ED Now" disposition were more likely to be concordant with advice if  ), as compared to the least deprived, were more likely to be concordant.We observed no difference in concordance by the nature of the callers' health concern for this disposition.
We observed no significant difference in concordance by sex or weekends vs weekdays across any of the four dispositions (Table 6).

Sensitivity analysis results
With expanded definitions of concordance that consider broader health service utilizations, we found higher rates of concordance over 48 h and seven days for the "Go to ED Now", "Seek FP Care Now", and "Schedule FP Appointment" dispositions.For the latter seven-day dispositions, we found a marked increase in events during the first two days, mainly due to the inclusion of ED visits.For the "Go to ED Now" disposition, there were increases at 12 and 24 h with the inclusion of physician encounters outside of the ED.We observed some additional small changes to the significance of independent variables in some models, but no changes in the direction of observed relationships (see Additional File 1: Table S3).

Discussion
Few studies have examined the impact of the inclusion of virtual physicians in a telephone triage services on subsequent health service use [24] and these did not examine the concordance between physician advice and subsequent care seeking.
We found variable rates of concordance across the four dispositions "Go to ED Now", "See FP now", "Schedule FP Appointment", and "Self-manage".Concordance was highest for individuals instructed to go to an ED (70% concordant) and lowest for those instructed to seek care in their community (41% and 47%).The rate of concordance with advice to seek ED care we found was consistent with or higher than those reported in previous studies of nurse-led telephone triage services that do not include virtual physicians -which suggests VPs may reinforce urgency.However, rates of concordance with advice to seek the care of a FP was low compared with previous studies [7][8][9].
We found a significant effect of geography in all four models, with individuals living in rural areas (or in Northern Health Authority, which is predominantly rural and remote) having a fundamentally different pattern of service use than urban areas, with ED use being lower and FP use higher.This may be because rural FPs in BC may be more likely to provide urgent after hours care and visits outside of office compared to their urban counterparts [33].Evidence from Alberta suggests that Family Medicine graduates who receive a rural as opposed to urban training program tend to have a broader scope of practice overall and urban-trained graduates who practice in rural areas tend to behave similarly, which supports the notion that rural care is typically more generalist and flexible [34].This variation may also be driven at least in part by a substitution effect due to lack of accessible EDs and rural providers serving as both family and emergency physicians.Further research examining both distance from a callers' home to the nearest ED and the difficulty of the route (e.g., mountainous regions, islands) may also reveal further trends, as we note that 90% of those concordant with advice to "Go to ED Now" did so within 4 h or lesssuggesting relative proximity to an ED.
Current literature on the general population level impact of rurality on ED use has mostly focussed on lower-acuity presentations and signals are not always clear, however they suggest that lower SES and rurality are associated with higher ED use [12,13], which is not evident in our results.Importantly, when we included visits to a FP in our concordance definition for our sensitivity analysis, we observed only marginal improvement in concordance by rurality.It is worth nothing that the impact of rurality on ED utilization appears to be driven at least in part by characteristics and health needs of the population under study [35][36][37].As this study examines the general population, signals regarding the impact of rurality are likely to be mixed and would require examination of specific cohorts to quantify and address.It is also worth noting that "RED/YELLOW" HEiDi callers, particularly those told to "Go to ED Now", are being assessed as likely high acuity, and so comparisons to the literature are not exact.
Concordance with advice to "Seek FP care now" was below 50%, and there was little difference in concordance between the "Seek FP care now" and "Schedule FP Appointment" dispositions.Primary care access challenges are pervasive and persistent in BC [15,33,38] and it is likely that this lack of access is driving at least some of the poor concordance.A 2022 qualitative study of patients seeking primary care attachment by Marshall et al. describes unattached patients as experiencing additional burdens due to lost opportunities for care and having to self-manage navigation of their health information and system resources [39].They further note that these burdens are often worse for at-risk groups.Browne et al. examined access to primary care from an Indigenous perspective at an urban ED and found that ED use should be understood as "a function of patient's prior and ongoing healthcare experiences in other settings" [14].
Lavergne et al. 's study on declining access to primary care in BC noted that within urban settings, encounters declined more rapidly in low-income areas while ED visits increased more rapidly.In this period, the percentage of physicians providing support for neighborhoods in the two lowest income quintiles declined from 30.6% to 26.3% [15].Hedden et al. found a similar erosion of primary care in BC from 2006 to 2012, with the proportions of physicians providing care in alternative locations or after hours declined significantly across all practices in the range of 5-22% [33].However, they also noted that rural physicians were significantly more likely to provide alternative locations or hours [33].
Concerningly, lack of attachment to a FP does not have a statistically significant effect on concordance for the most urgent FP advice category.This suggests that having a regular FP does not guarantee timely access to primary care, at least within the 7-day follow-up period we used.Additionally, if it is infeasible for patients to access an FP in the timely manner implied by the "Seek FP care now" disposition, we would suggest this advice category be modified, and possibly combined, with "Schedule FP Appointment" to reflect the real-world access to care of callers.
We did not observe differences in concordance by health concern, with the exception of callers with obstetric/postpartum and gynecological issues.Such patients appear to be less likely to attend an ED, more likely to schedule an FP appointment, and less likely to stay home when advised, suggesting that this group of patients may require additional communication efforts by VPs when provided advice and perhaps additional care pathways.
When conceptualizing potential strategies for HEiDi and Healthlink moving forward, we find some suggestions in the telehealth literature.In a 2016 systematic review of telehealth strategies in rural and remote Australia, Bradford et al. identified 6 key factors influencing success and sustainability: vision, ownership, adaptability, economics, efficiency, and equipment [40]."Adaptability", which the authors define as a strategy to "trial and model the service according to the needs of patients and health service" and "remain[ing] responsive to requirements of all stakeholders" [40], seems to be the most salient to the current BC context.Our results suggest that, while concordance rates on advice for ED use are high in aggregate, more work needs to be done to identify and address the needs of specific groups -particularly rural BC residents.Furthermore, advice to seek care from a family physician does not appear to be functioning as intended and may need to be recontextualized based on current access issues.
A final consideration for how callers' may consider their health service use in BC, but arguably throughout Canada, should be the 2020 "In Plain Sight" report on Indigenous-specific racism and discrimination in healthcare.While out of scope of for this population-level analysis, the report highlights concerns around the perception of EDs and/or hospitals as unsafe may deter Indigenous persons from seeking care [41].Similar perceptions may be shared by other racialized or historically marginalized groups and this should be examined by any future patientcentered work.Overall, it should be noted that ED utilization is driven at least in part by social determinants of health and should be considered using an equity lens.

Limitations
This is a hypothesis-generating analysis using routinely collected provincial administrative data and is subject to all the limitations thereof including missing or incorrect data.We conducted this analysis during earlier phases of the COVID-19 pandemic when there were significant concerns about seeking in-person care, and this may have reduced ED and in-person physician visits; patients with access to virtual primary care (and this might include urban or wealthier patients) may have been more concordant.
It merits consideration that health categories may be miscategorized due to overlapping symptomology or miscommunication.Furthermore, it is not possible to determine whether the reason for a HEiDi call is the same reason for which a patient ultimately seeks inperson care as the HEiDi/811 service is not able to clinically determine and capture a standardized diagnostic code.
Furthermore, we did not have access to unique identifiers for nurse and/or VP providers in our data.This precluded the identification of baselines and trends within and across providers, including analyses such as clustering.

Conclusion
Seventy percent of callers to a virtual physician triage service who are advised to go to an emergency department do so within 48-h, suggesting an overall high-degree of concordance with this advice.However, this is markedly lower in rural areas and the Northern health authority, which suggests the need for targeted assessment of access to emergency care, particularly with regard to distance and difficulty travelling from home to ED.
Less than half of those advised to seek follow-up care with a physician in the community do so within seven days, and there is little difference between those advised to seek FP care now and those advised to seek such care within seven days.In conjunction with recent literature, this may be driven by long-term trends restricting access to primary care in BC.Qualitative analysis of patientcaller's decision making from their perspective would be beneficial in further understanding these findings.The drivers for the low rates of concordance with advice, particularly with advice to seek care in the community, merit future research.The HEiDi service and the evaluation of this virtual care pathway is funded by the British Columbia Ministry of Health through HealthLink BC (HLBC).The Executive Director of HLBC is a member of the steering committee and involved in the design of this evaluation, interpretation, drafting the manuscript, and as a co-author for this publication.

Table 3
Operational Definitions of VP advice and corresponding sensitivity criteria for concordance

Table 4
Summary of HEiDi caller and call characteristics by dispositionNote: cell values represent count (percentage) or mean (standard deviation) as appropriate1The "Other" health concern collapses the remaining 18 concerns, with the five most common health concerns displayed in this table.For a full list of health concerns, see Supplemental TableS1 less likely to be concordant with advice to "Go to ED Now" (0.53 [0.46-0.61])andthoseadvised to see a family physician either immediately (1.28 [1.01-1.62])orsoon(1.23 [1.11-1.37])were significantly more likely to record an FP encounter within seven days than urban callers.

Table 5
Summary of health service utilizations within seven days by disposition following callers' HEiDi encounterConcordance numbers can be seen at the intersection of advice and service use (ie.Go to ED Now -ED Visits, FP Encounters -See FP Now/Schedule FP Appointment, Self-Manage -No Health Service Utilization

Table 6
Summary of caller and call characteristics impact on concordance by category of advice (adjusted hazard ratios of multivariate cox regression) 1 "Self-Manage" hazard ratios are inversely calculated, i.e., so that higher HRs for all dispositions indicate callers being more likely to be concordant * Significant at alpha < 0.05 ** Significant at alpha < 0.01 *** Significant at alpha < 0.001