Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Moderate sensitivity and high specificity of emergency department administrative data for transient ischemic attacks

BMC Health Services Research volume 17, Article number: 666 (2017) Cite this article

Abstract

Background

Validation of administrative data case definitions is key for accurate passive surveillance of disease. Transient ischemic attack (TIA) is a condition primarily managed in the emergency department. However, prior validation studies have focused on data after inpatient hospitalization. We aimed to determine the validity of the Canadian 10th International Classification of Diseases (ICD-10-CA) codes for TIA in the national ambulatory administrative database.

Methods

We performed a diagnostic accuracy study of four ICD-10-CA case definition algorithms for TIA in the emergency department setting. The study population was obtained from two ongoing studies on the diagnosis of TIA and minor stroke versus stroke mimic using serum biomarkers and neuroimaging. Two reference standards were used 1) the emergency department clinical diagnosis determined by chart abstractors and 2) the 90-day final diagnosis, both obtained by stroke neurologists, to calculate the sensitivity, specificity, positive and negative predictive values (PPV and NPV) of the ICD-10-CA algorithms for TIA.

Results

Among 417 patients, emergency department adjudication showed 163 (39.1%) TIA, 155 (37.2%) ischemic strokes, and 99 (23.7%) stroke mimics. The most restrictive algorithm, defined as a TIA code in the main position had the lowest sensitivity (36.8%), but highest specificity (92.5%) and PPV (76.0%). The most inclusive algorithm, defined as a TIA code in any position with and without query prefix had the highest sensitivity (63.8%), but lowest specificity (81.5%) and PPV (68.9%). Sensitivity, specificity, PPV, and NPV were overall lower when using the 90-day diagnosis as reference standard.

Conclusions

Emergency department administrative data reflect diagnosis of suspected TIA with high specificity, but underestimate the burden of disease. Future studies are necessary to understand the reasons for the low to moderate sensitivity.

Peer Review reports

Background

Over half of all ischemic cerebrovascular disease patients present with transient or mild symptoms [1]. Patients with transient ischemic attack (TIA) and minor stroke are at high risk of stroke recurrence or progression in the following days to weeks [2,3,4]. Further, despite mild or resolved symptoms, 15% of TIA and minor stroke patients develop permanent disability [5]. Accurate surveillance of TIA and minor stroke allows for monitoring disease burden and temporal trends in the population. It assists in planning health resource allocation and conducting future stroke prevention studies.

While active surveillance is defined as case ascertainment in real-time, passive surveillance is a time and cost-effective method to identify cases using existing data, such as administrative data [6]. Prior studies have shown that the coding of hospital charts using the International Classification of Diseases (ICD) has moderate to high predictive value for cerebrovascular diseases [7, 8]. However, the few validation studies that included TIA as a separate diagnostic category have found that codes for TIA in hospital discharge abstract data have lower predictive value compared to those for ischemic or hemorrhagic strokes [9,10,11,12].

Patients with TIA are increasingly being evaluated on an outpatient basis, from the emergency department (ED) to rapid-access clinics without requiring an inpatient admission [13,14,15]. Chart records are more sparse in the outpatient setting, which may decrease the sensitivity and specificity of passive surveillance [16, 17]. The diagnosis of TIA in the ED is challenging [18, 19] and subject to disagreement even among neurologists [20]. Although the ED is the most relevant setting to validate TIA coding, prior studies have focused on discharge diagnoses after admission to acute-care hospital and therefore are vulnerable to a specific selection bias [9, 10, 21,22,23].

In a cohort of patients presenting with acute and minor neurological symptoms, we aimed to determine the validity of the ICD 10th Canadian Revisions (ICD-10-CA) codes for TIA in the National Ambulatory Care Reporting System (NACRS). We hypothesized that coding for the diagnosis of TIA has lower accuracy in the ED compared to coding after inpatient hospitalization and that ED administrative data codes reflect the ED diagnosis more accurately than the 90-day diagnosis. A key secondary objective was to determine the accuracy of the ED working diagnosis compared to the final 90-day diagnosis.

Methods

In this administrative data validation study for identifying TIA in the ED, we compared the ICD-10-CA coding algorithms to two reference standards: 1) the ED diagnosis from chart re-abstraction and 2) the final 90-day neurologist-determined diagnosis.

National Ambulatory Care Reporting System (NACRS)

In Alberta, ED discharge diagnoses are reported in the NACRS, the Canadian national ambulatory administrative database, containing information on ED visits, day surgery admissions, and selected outpatient clinic visits [24]. The database contains one ICD-10-CA code for the “main problem” per ED visit, up to 10 codes for “other problems”, and has the option to identify queried diagnoses with a prefix “Q”. In Canada, ICD codes are assigned by non-health professional health records technicians who undergo a 2-year technical degree training. Coding is performed according to Canadian standards published and updated by the Canadian Institute for Health Information [25]. We tested four algorithms to identify TIA in NACRS: 1) TIA codes (G45.x except G45.4) in the main position without querying prefix Q (MP); 2) TIA codes in any position without Q (AP); 3) TIA codes in the main position with or without Q (MP + Q); and 4) TIA codes in any position with or without Q (AP + Q). We defined ischemic cerebrovascular disease as coding for either TIA or ischemic stroke (H34.1, H34.2, I63.x, I64.x) [10].

Study population

The study population was obtained from two ongoing Canadian studies: SPECTRA (Spectrometry for Transient Ischemic Attack Rapid Assessment) and DOUBT (Diagnosis Of Uncertain-origin Benign Transient neurological symptoms). SPECTRA aims to identify a blood biomarker to differentiate TIA and minor strokes from mimics and enrolls patients within 24 h after symptom onset. DOUBT is a neuroimaging study of patients with TIA, minor strokes, and stroke mimics and enrolls patients within 7 days after symptom onset. Both studies prospectively enroll consenting patients and require a clinical evaluation by a neurologist at least once between symptom onset and 90 days. Study inclusion and exclusion criteria are outlined in Additional file 1: Table S1. Minor stroke is defined as National Institute of Health Stroke Scale (NIHSS) score ≤ 3. A key feature of both studies is that patients with stroke mimics are actively recruited. These are patients with acute and focal neurological symptoms in whom the working diagnosis is non-ischemic in etiology, such as syncope, seizure, migraine, or other. Recruitment for both studies largely took place in a tertiary-care teaching hospital with an ED volume of about 78,000 visits per year, including 905 annual acute stroke consults. Residents of Alberta, Canada enrolled between December 1st 2013 and October 30th 2015 with at least one ED visit prior to the enrolment date of SPECTRA (within 24 h) and DOUBT (within 7 days) were included into the current study. The Alberta unique personal healthcare number was used for deterministic linkage with NACRS. When linkage to multiple ED visits occurred, including between-ED transfers, the last visit prior to enrolment was retained for analysis. Patients who did not have a 90-day SPECTRA or DOUBT diagnosis were excluded.

Sample size estimation

Prior studies have reported moderate to high accuracy of administrative data to identify TIA after hospital admission (77% to 97%) [9, 10]. Thus, we determined that a sensitivity or specificity below 0.85 would be a clinically meaningful cut-off. We expected a third of patients enrolled into SPECTRA and DOUBT to be true TIAs. With a sample size of 400 patients, we expected estimating sensitivity and specificity with a confidence interval width of 0.10 around the proportion estimates.

Reference standards: ED chart re-abstraction and 90-day neurologist diagnosis

A stroke neurologist (AY) performed chart abstractions to retrospectively adjudicate an ED diagnosis, the first reference standard. The abstractor was blinded to the NACRS code and only evaluated documentation dated and timed prior to the disposition from the ED to replicate the Canadian coding standards [25]. The ED charts of four patients were missing and the SPECTRA and DOUBT case report forms were abstracted instead. The diagnosis of TIA was adjudicated based on the World Health Organization time-based criteria [26]. For example, a patient with symptoms lasting less than 24 h but a magnetic resonance imaging showing evidence of tissue ischemia, such as a small diffusion weighted imaging lesion, was adjudicated as a TIA and specifically not a stroke. TIA was further classified into “possible” or “definite,” depending on whether the adjudicator judged if there was a high likelihood of a competing non-vascular diagnosis. A senior stroke neurologist (SBC) independently reviewed a random sample of thirty charts to determine inter-abstractor reliability.

The second reference standard was the final 90-day diagnosis from the DOUBT and SPECTRA studies. In both studies, a final diagnosis was determined by a stroke neurologist at 90 days, with knowledge of all investigations and clinical follow-up completed in clinical routine.

Given the purpose of this study was to determine the validity of coding for mild or transient ischemic events, we categorized the rare intracranial hemorrhage into the stroke mimic category. These cases had a specific neuroimaging finding, heterogeneous underlying pathologies (e.g. cortical subarachnoid hemorrhage due to reversible cerebral vasoconstriction syndrome, multifocal parenchymal hemorrhage due to cerebral hyperperfusion post carotid revascularization), and all were clinically mild, NIHSS ≤3 as per inclusion criteria of the SPECTRA and DOUBT studies.

Statistical methods

Descriptive statistics were used to determine the frequency of baseline characteristics and prevalence of disease. We first used the diagnosis from ED chart re-abstraction to determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the four TIA case definition algorithms. The analyses were repeated with the final 90-day diagnosis. The categories of definite and possible TIA were combined for the primary analysis. Sensitivity analyses were performed to determine the validity of the algorithms for definite TIA as well as for ischemic cerebrovascular diseases. Accuracy of the working ED diagnosis was compared to the final 90-day diagnosis. All analyses were performed using STATA 14.0 (STATA Corp, College Station, Tex).

Results

Among 417 patients included (n = 375 from SPECTRA and n = 42 from DOUBT), the median age was 67 years (IQR 22). Most patients, 390 (93.5%) were discharged from a tertiary comprehensive stroke centre and 27 patients were discharged from one of the four community hospitals. Re-abstracted diagnoses from ED charts showed 163 (39.1%) TIA, including 121 definite and 42 possible TIA, 155 (37.2%) ischemic strokes, and 99 (23.7%) stroke mimics. Inter-abstractor agreement was 76.7% (κ = 0.50) for the diagnosis of TIA. There were eight intracranial hemorrhages (two subarachnoid and six parenchymal hemorrhages).

Table 1 shows the baseline characteristics as well as the final 90-day diagnosis according to the adjudicated ED diagnosis. The sensitivity and specificity of the ED working diagnosis to accurately determine the 90-day final diagnosis were 61.4% [CI95 52.9, 69.3] and 72.8% [CI95 67.1, 78.0] for TIA; 77.3% [CI95 69.1, 84.3] and 80.6% [CI95 75.6, 85.0] for ischemic stroke; and 56.9% [CI95 48.4, 65.2] and 93.8% [CI95 90.2, 96.3] for stroke mimic.

Table 1 Baseline characteristics and 90-day diagnosis per ED chart adjudicated diagnosis
Full size table

Table 2 presents the sensitivity, specificity, PPV, and NPV with 95% confidence intervals of the four administrative data coding algorithms using the re-abstracted ED diagnosis. Algorithm MP identified 79 (18.9%) TIA in the cohort. It had the highest specificity (92.5%) and PPV (76.0%), but lowest sensitivity (36.8%). In comparison, algorithm AP + Q identified 151 (36.2%) TIA and had the highest sensitivity (63.8%) at the cost of lower specificity (81.5%) and PPV (68.9%). Sensitivity increased when considering all cerebral ischemia, including TIA and minor stroke, with similar specificity. The sensitivity, specificity, PPV, and NPV were overall lower when comparing administrative data with the final 90-day diagnosis as reference standard (Table 3). The results remained similar when evaluating definite TIA exclusively (Tables 4 and 5).

Table 2 Sensitivity, specificity, PPV, and NPV of ICD-10-CA with adjudicated ED diagnosis as reference standard
Full size table
Table 3 Sensitivity, specificity, PPV, and NPV of ICD-10-CA with 90-day diagnosis as reference standard
Full size table
Table 4 Sensitivity, specificity, PPV, and NPV of ICD-10-CA with ED definite TIA as reference standard
Full size table
Table 5 Sensitivity, specificity, PPV, and NPV of ICD-10-CA with 90-day definite TIA as reference standard
Full size table

The most specific algorithm MP found 19 (4.6%) false positive and 103 (24.7%) false negative cases when compared to the re-abstracted ED diagnosis. Table 6 outlines the diagnosis in the main position for the false negative cases and shows that 66% of these codes are still related to cerebrovascular diseases.

Table 6 First position diagnoses amongst the false negatives (n = 103)
Full size table

Discussions

Using the Canadian NACRS database, we showed that suspected cases of acute TIA can be identified with high specificity in the ED, but the sensitivity was low to moderate. Even the most inclusive algorithm AP + Q, evaluating codes in any position and including queried codes, only achieved a sensitivity of 64%, meaning that more than a third of suspected TIA cases discharged from the ED will be inappropriately identified as a stroke or non-stroke syndrome.

Administrative data coding can only be as good as the attending physician’s clinical diagnostic abilities. We found that administrative data reflect the ED diagnosis better than the final 90-day diagnosis, indicating that the coding reflects the episode of care. The process of translating diagnoses during the health encounter into ICD codes is a static snapshot in time, while a physician’s diagnostic impression is fluid and changes based on new information from the clinical course, collateral history, and investigation results. Empirically, transient neurological events are described and diagnosed by the clinical history and show high diagnostic variability. As a result, we show an ED diagnosis of TIA carries only a 61% sensitivity and 73% specificity compared to the final diagnosis, a finding that is consistent with prior reports [18,19,20].

Our results have implications for the use of administrative data for TIA surveillance and research. Suspected acute TIAs can be identified with high specificity and used to determine the temporal trends of disease, but the low sensitivity will underestimate the burden of disease. The most inclusive algorithm AP + Q has moderate sensitivity 64% and PPV 69% and may be useful to evaluate patterns of healthcare utilization in suspected acute TIA or to determine adherence rates to secondary stroke prevention guidelines. However, because the coding is less accurate when compared to the 90-day diagnosis, if administrative data were used to select a cohort of suspected TIAs for a prospective study to determine outcomes of interest, such as rates of stroke recurrence or mortality after TIA, misclassification errors must be addressed by chart review or other case ascertainment method. Finally, there are implications for the use of administrative data for other conditions primarily managed in the ED. Outpatient cases, for example angina or a single seizure, should be validated separately from their inpatient counterparts, acute myocardial infarction or complex seizure disorder [27, 28].

We show that all four algorithms yielded a similar PPV (68.9 to 76.0%), which is lower than the estimates reported by prior TIA validation studies on discharge diagnostic codes after hospital admission (88.9 to 97%) [9, 10], but higher than other studies on TIA patients discharged from the ED (48.8 to 56.1%) [12, 29]. Our study used different methods compared to prior publications. By including patients with TIA mimics, we were able to determine both true negative and false positive cases, allowing for the calculation of sensitivity and specificity. The majority of prior studies evaluating the validity of administrative data in cerebrovascular diseases identified a priori patients with high-yield stroke codes and then verified the coding accuracy or agreement [9, 10, 23, 30, 31]. By design, these studies had a high prevalence of disease, which may lead to an overestimation of the agreement [32] and PPV [33]. Others examined the validity of cerebrovascular disease coding as a comorbidity of another condition (e.g. diabetes, hypertension), such that the prevalence of cerebrovascular diseases is low, less than 5%, yielding estimates with wide confidence intervals and prevalent disease is not differentiated from incident cases [34, 35]. Finally, a third type of study validated administrative data against a registry or other active case finding methods [11, 12, 16, 36]. Stroke mimics are excluded by design, limiting the ability to evaluate true negative and false positive cases.

Our study has several strengths. By using two reference standards, we addressed the question of accuracy of administrative data in two different ways: how well do the codes reflect 1) the clinical encounter and 2) the patient’s actual condition? By including stroke mimics, we evaluated true negative and false positive rates in order to report specificity estimates. Some limitations are worth mentioning. SPECTRA and DOUBT prospectively enrolled patients who consented to participation, resulting in a sample of non-consecutive patients presenting to the ED with acute neurological symptoms. The TIA prevalence in our study is higher than that of a randomly selected sample from the ED; because prevalence impacts the predictive value, our results may not be directly applicable to a specific ED population. Nevertheless, the prevalence of TIA and stroke mimics in this study is similar to prior publications on patients with acute neurological presentations referred to a stroke prevention clinic [19, 37]. The majority of the patients were discharged from a single center. This may limit the generalizability of our results because administrative data coding is variable between jurisdictions. Finally, although chart abstractors were blinded to NACRS codes and the 90-day diagnosis, the retrospective derivation of the ED diagnosis may be subject to residual bias.

Conclusions

We show that administrative data identify suspected TIA in the ED with high specificity while underestimating the incidence of acute TIA cases. We also highlight the diagnostic challenges that are presented to the physician for patients with TIA in the ED. Understanding the detailed reasons for the moderate sensitivity of administrative data for this diagnosis and whether the miscoded patients are systematically different from the accurately coded ones will require prospective diagnostic studies.

Abbreviations

DOUBT:

Diagnosis Of Uncertain-origin Benign Transient neurological symptoms

ED:

Emergency Department

ICD:

International Classification of Diseases

ICD-10-CA:

International Classification of Diseases, 10th Canadian iteration

NACRS:

National Ambulatory Care Reporting System

NPV:

Negative Predictive Value

PPV:

Positive Predictive Value

SPECTRA:

Spectrometry for Transient Ischemic Attack Rapid Assessment)

TIA:

Transient Ischemic Attack

References

  1. Reeves M, Khoury J, Alwell K, Moomaw C, Flaherty M, Woo D, Khatri P, Adeoye O, Ferioli S, Kissela B, et al. Distribution of National Institutes of Health stroke scale in the Cincinnati/northern Kentucky stroke study. Stroke. 2013;44(11):3211–3.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Johnston SC, Gress DR, Browner WS, Sidney S. Short-term prognosis after emergency department diagnosis of TIA. JAMA. 2000;284(22):2901–6.

    Article  CAS  PubMed  Google Scholar 

  3. Hill MD, Yiannakoulias N, Jeerakathil T, Tu JV, Svenson LW, Schopflocher DP. The high risk of stroke immediately after transient ischemic attack: a population-based study. Neurology. 2004;62(11):2015–20.

    Article  CAS  PubMed  Google Scholar 

  4. Amarenco P, Lavallee PC, Labreuche J, Albers GW, Bornstein NM, Canhao P, Caplan LR, Donnan GA, Ferro JM, Hennerici MG, et al. One-year risk of stroke after transient ischemic attack or minor stroke. N Engl J Med. 2016;374(16):1533–42.

    Article  CAS  PubMed  Google Scholar 

  5. Coutts SB, Modi J, Patel SK, Aram H, Demchuk AM, Goyal M, Hill MD. What causes disability after transient ischemic attack and minor stroke?: results from the CT and MRI in the triage of TIA and minor cerebrovascular events to identify high risk patients (CATCH) study. Stroke. 2012;43(11):3018–22.

    Article  PubMed  Google Scholar 

  6. Yu AY, Holodinsky JK, Zerna C, Svenson LW, Jette N, Quan H, Hill MD. Use and utility of administrative health data for stroke research and surveillance. Stroke. 2016;47(7):1946–52.

    Article  PubMed  Google Scholar 

  7. Andrade SE, Harrold LR, Tjia J, Cutrona SL, Saczynski JS, Dodd KS, Goldberg RJ, Gurwitz JH. A systematic review of validated methods for identifying cerebrovascular accident or transient ischemic attack using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):100–28.

    Article  PubMed  PubMed Central  Google Scholar 

  8. McCormick N, Bhole V, Lacaille D, Avina-Zubieta JA. Validity of diagnostic codes for acute stroke in administrative databases: a systematic review. PLoS One. 2015;10(8):e0135834.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Benesch C, Witter DM Jr, Wilder AL, Duncan PW, Samsa GP, Matchar DB. Inaccuracy of the international classification of diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease. Neurology. 1997;49(3):660–4.

    Article  CAS  PubMed  Google Scholar 

  10. Kokotailo RA, Hill MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005;36(8):1776–81.

    Article  PubMed  Google Scholar 

  11. Ives DG, Fitzpatrick AL, Bild DE, Psaty BM, Kuller LH, Crowley PM, Cruise RG, Theroux S. Surveillance and ascertainment of cardiovascular events. The cardiovascular health study. Ann Epidemiol. 1995;5(4):278–85.

    Article  CAS  PubMed  Google Scholar 

  12. Hall R, Mondor L, Porter J, Fang J, Kapral MK. Accuracy of administrative data for the coding of acute stroke and TIAs. Can J Neurol Sci. 2016:1–9.

  13. Ranta A, Barber PA. Transient ischemic attack service provision: a review of available service models. Neurology. 2016;86(10):947–53.

    Article  PubMed  Google Scholar 

  14. Rothwell PM, Giles MF, Chandratheva A, Marquardt L, Geraghty O, Redgrave JN, Lovelock CE, Binney LE, Bull LM, Cuthbertson FC, et al. Effect of urgent treatment of transient ischaemic attack and minor stroke on early recurrent stroke (EXPRESS study): a prospective population-based sequential comparison. Lancet. 2007;370(9596):1432–42.

    Article  PubMed  Google Scholar 

  15. Lavallee PC, Meseguer E, Abboud H, Cabrejo L, Olivot JM, Simon O, Mazighi M, Nifle C, Niclot P, Lapergue B, et al. A transient ischaemic attack clinic with round-the-clock access (SOS-TIA): feasibility and effects. Lancet Neurol. 2007;6(11):953–60.

    Article  CAS  PubMed  Google Scholar 

  16. Piriyawat P, Smajsova M, Smith MA, Pallegar S, Al-Wabil A, Garcia NM, Risser JM, Moye LA, Morgenstern LB. Comparison of active and passive surveillance for cerebrovascular disease: the brain attack surveillance in Corpus Christi (BASIC) project. Am J Epidemiol. 2002;156(11):1062–9.

    Article  PubMed  Google Scholar 

  17. Brown DL, Lisabeth LD, Garcia NM, Smith MA, Morgenstern LB. Emergency department evaluation of ischemic stroke and TIA: the BASIC project. Neurology. 2004;63(12):2250–4.

    Article  CAS  PubMed  Google Scholar 

  18. Ferro JM, Falcao I, Rodrigues G, Canhao P, Melo TP, Oliveira V, Pinto AN, Crespo M, Salgado AV. Diagnosis of transient ischemic attack by the nonneurologist. A validation study. Stroke. 1996;27(12):2225–9.

    Article  CAS  PubMed  Google Scholar 

  19. Prabhakaran S, Silver AJ, Warrior L, McClenathan B, Lee VH. Misdiagnosis of transient ischemic attacks in the emergency room. Cerebrovasc Dis. 2008;26(6):630–5.

    Article  PubMed  Google Scholar 

  20. Castle J, Mlynash M, Lee K, Caulfield AF, Wolford C, Kemp S, Hamilton S, Albers GW, Olivot JM. Agreement regarding diagnosis of transient ischemic attack fairly low among stroke-trained neurologists. Stroke. 2010;41(7):1367–70.

    Article  PubMed  Google Scholar 

  21. Ghia D, Thomas PR, Cordato DJ, Worthington JM, Cappelen-Smith C, Griffith N, Hanna I, Hodgkinson SJ, McDougall A, Beran RG. Validation of emergency and final diagnosis coding in transient ischemic attack: south western Sydney transient ischemic attack study. Neuroepidemiology. 2010;35(1):53–8.

    Article  PubMed  Google Scholar 

  22. Goldstein LB. Accuracy of ICD-9-CM coding for the identification of patients with acute ischemic stroke: effect of modifier codes. Stroke. 1998;29(8):1602–4.

    Article  CAS  PubMed  Google Scholar 

  23. Leone MA, Gaviani P, Ciccone G. Inter-coder agreement for ICD-9-CM coding of stroke. Neurol Sci. 2006;27(6):445–8.

    Article  CAS  PubMed  Google Scholar 

  24. National Ambulatory Care Reporting System (NACRS) Metadata [https://www.cihi.ca/en/national-ambulatory-care-reporting-system-metadata].

  25. Canadian Coding Standards for Version 2015 ICD-10-CA and CCI [https://secure.cihi.ca/estore/productSeries.htm?pc=PCC189].

  26. Investigators WMPP. The World Health Organization MONICA project (monitoring trends and determinants in cardiovascular disease): a major international collaboration. J Clin Epidemiol. 1988;41(2):105–14.

    Article  Google Scholar 

  27. Austin PC, Daly PA, Tu JV. A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario. Am Heart J. 2002;144(2):290–6.

    Article  PubMed  Google Scholar 

  28. Nagurney JT, Brown DF, Chae C, Chang Y, Chung WG, Cranmer H, Dan L, Fisher J, Grossman S, Tedrow U, et al. Disagreement between formal and medical record criteria for the diagnosis of acute coronary syndrome. Acad Emerg Med. 2005;12(5):446–52.

    Article  PubMed  Google Scholar 

  29. Johnsen SP, Overvad K, Sorensen HT, Tjonneland A, Husted SE. Predictive value of stroke and transient ischemic attack discharge diagnoses in the Danish National Registry of patients. J Clin Epidemiol. 2002;55(6):602–7.

    Article  PubMed  Google Scholar 

  30. Goldstein LB, Jones MR, Matchar DB, Edwards LJ, Hoff J, Chilukuri V, Armstrong SB, Horner RD. Improving the reliability of stroke subgroup classification using the trial of ORG 10172 in acute stroke treatment (TOAST) criteria. Stroke. 2001;32(5):1091–8.

    Article  CAS  PubMed  Google Scholar 

  31. Tirschwell DL, Longstreth WT Jr. Validating administrative data in stroke research. Stroke. 2002;33(10):2465–70.

    Article  PubMed  Google Scholar 

  32. Chen G, Faris P, Hemmelgarn B, Walker RL, Quan H. Measuring agreement of administrative data with chart data using prevalence unadjusted and adjusted kappa. BMC Med Res Methodol. 2009;9:5–2288. 2289-2285

    Article  PubMed  PubMed Central  Google Scholar 

  33. Brenner H, Gefeller O. Variation of sensitivity, specificity, likelihood ratios and predictive values with disease prevalence. Stat Med. 1997;16(9):981–91.

    Article  CAS  PubMed  Google Scholar 

  34. Newton KM, Wagner EH, Ramsey SD, McCulloch D, Evans R, Sandhu N, Davis C. The use of automated data to identify complications and comorbidities of diabetes: a validation study. J Clin Epidemiol. 1999;52(3):199–207.

    Article  CAS  PubMed  Google Scholar 

  35. Humphries KH, Rankin JM, Carere RG, Buller CE, Kiely FM, Spinelli JJ. Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review? J Clin Epidemiol. 2000;53(4):343–9.

    Article  CAS  PubMed  Google Scholar 

  36. Appelros P, Terent A. Validation of the Swedish inpatient and cause-of-death registers in the context of stroke. Acta Neurol Scand. 2011;123(4):289–93.

    Article  CAS  PubMed  Google Scholar 

  37. Calanchini PR, Swanson PD, Gotshall RA, Haerer AF, Poskanzer DC, Price TR, Conneally PM, Dyken ML, Futty DE. Cooperative study of hospital frequency and character of transient ischemic attacks. IV. The reliability of diagnosis. JAMA. 1977;238(19):2029–33.

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

AYXY holds a Clinician Fellowship Award from Alberta Innovates Health Solutions. MDH reports funding from Heart & Stroke Foundation of Alberta, Northwest Territories, Nunavut and from Alberta Innovates Health Solutions. SPECTRA was funded by Genome Canada and Genome Alberta and DOUBT was funded by the Canadian Institutes of Health Research.

Availability of data and materials

The dataset supporting the conclusions of this article is available upon request.

Author information

Authors and Affiliations

  1. Department of Clinical Neurosciences, Community Health Sciences, Cumming School of Medicine, University of Calgary, Health Sciences Centre, Office 2935-B, 3300 Hospital Drive NW, Calgary, AB, T2N 4N1, Canada

    Amy Y. X. Yu & Gabrielle O. Wagner

  2. Department of Community Health Sciences, O’Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, Heritage Medical Research Building 3330 Hospital Drive NW, Calgary, AB, T2N 4N1, Canada

    Hude Quan

  3. Department of Emergency Medicine, Community Health Sciences, Cumming School of Medicine, University of Calgary, Foothills Campus, 3330 Hospital Drive NW, Calgary, AB, T2N 4N1, Canada

    Andrew McRae

  4. Departments of Clinical Neurosciences, Community Health Sciences, Medicine, Radiology, and Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Health Sciences Centre, Office 2939, 3300 Hospital Drive NW, Calgary, AB, T2N 4N1, Canada

    Michael D. Hill

  5. Department of Clinical Neurosciences, Radiology, Community Health Sciences, Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, C1242A, Foothills Medical Centre, 1403 29th St NW, Calgary, AB, T2N 2T9, Canada

    Shelagh B. Coutts

Authors
  1. Amy Y. X. Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hude Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew McRae
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gabrielle O. Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael D. Hill
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shelagh B. Coutts
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AYXY contributed to the study concept and design, acquisition, analysis, and interpretation of the data, as well as drafting and revising the manuscript. HQ contributed to the study concept and design, data acquisition and interpretation, critical revision of the manuscript, and supervision. AM contributed to the study concept and design, data interpretation, critical revision of the manuscript, and supervision. GOW contributed to the data acquisition, administrative support, and manuscript revision. MDH contributed to the study concept and design, data analysis and interpretation, critical revision of the manuscript, and supervision. SBC contributed to the study concept and design, data interpretation, critical revision of the manuscript, and supervision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Amy Y. X. Yu.

Ethics declarations

Ethics approval and consent to participate

The SPECTRA and DOUBT studies received approval from the University of Calgary institutional review board for research using human subjects and written informed consent was obtained from all patients enrolled. The current study received approval from the University of Calgary institutional review board for research and a waiver of consent was granted (REB15–2943).

Consent for publication

N/A

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1: Table S1.

Inclusion and exclusion criteria for the DOUBT and SPECTRA studies. Table listing the inclusion and exclusion criteria for the DOUBT and SPECTRA studies. (DOCX 22 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, A.Y.X., Quan, H., McRae, A. et al. Moderate sensitivity and high specificity of emergency department administrative data for transient ischemic attacks. BMC Health Serv Res 17, 666 (2017). https://doi.org/10.1186/s12913-017-2612-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-017-2612-6

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us