Our objective was to determine the proportion of adverse drug events to outpatient medications resulting in emergency department visits that were reported in administrative data. We found adverse drug events to be underreported in the administrative data of two large Canadian university hospitals, despite using an extensive list of ICD-10 codes. Even when a broad set of ICD-10 codes was used to include diagnoses indicating a very likely, likely or possible relationship with a medication, we were only able to identify 28% of adverse drug events, and 39% of adverse drug reactions. Among patients who required hospital admission, we were able to identify 55% of adverse drug events.
Prescribing medication is the most common medical intervention performed by physicians. Yet, many prescribing decisions are informed by incomplete or conflicting evidence, or by the results of randomized trials that may not be transferrable to clinical practice [7, 34, 35]. In addition, off-label use of medications and varying compliance behavior of patients contribute to suboptimal treatment outcomes, leading to a growing interest in developing improved methods to capture adverse drug event data from the real-world to generate more robust estimates about the comparative safety and effectiveness of medications, and to develop interventions to improve patient care [6–8, 17, 18].
In North America, emergency departments offer the majority of healthcare delivered for acute and unexpected medical conditions, including adverse drug events . Therefore, emergency department administrative data may offer unique opportunities to capture data on clinically significant adverse drug events that result from outpatient medication use [9, 11–13]. However, before such data are considered for this purpose, they should be evaluated for their completeness.
Our study is the first in the peer-reviewed literature to compare adverse drug event reports in administrative records of emergency department patients with adverse drug events diagnosed at the point-of-care. Prior studies have attempted to validate adverse drug event codes in administrative data by comparing adverse drug event reports in administrative data to events identified by chart review, using electronic trigger methods or between administrative databases [36–41]. Our study differs from these studies in the premise that all clinically significant adverse drug events are recorded in the medical record or identifiable using trigger methods. Indeed, two prior studies support the assumption that 40% of adverse drug events may not be documented in emergency department records [33, 42]. Therefore, in order to understand the sensitivity of administrative data and the ICD-10 code set, we derived our criterion standard at the point-of care using a pre-defined algorithm that included assessment by a pharmacist and a physician. We believe that this led to more precise estimates of adverse drug events. We disclosed all adverse drug events suspected in the emergency department to treating physicians (required by Ethics to ensure optimal patient care) prior to coding, thus optimizing the chances of their documentation in the medical chart.
A few studies have examined the sensitivity of adverse drug event codes within the ICD-9 coding system for events that occurred as a result of inpatient medications [38, 39]. Hougland et al. found that their code set detected more events than the hospital’s computerized adverse drug event surveillance system, and estimated that 55% of adverse drug events causing hospitalization, and 10% of adverse drug events occurring during the course of hospitalization were identified when compared to medical record review . Leonard et al. found that the sensitivity of their ICD-9 code set varied substantially by the type of adverse drug reaction they searched for, and estimated that the sensitivity of the codes for digoxin and phenytoin related events may be 84% and 86.7% respectively . However, the authors determined the criterion standard retrospectively by chart review in only 19-40% of records, all of which had been were pre-screened using an ICD-9 code set that included the adverse drug reaction codes . This may have falsely elevated the sensitivity of their code set, because the determination of the criterion standard was not independent of the code set they used to identify events.
Only one previous study has evaluated the sensitivity of emergency department data coded in ICD-10 for adverse drug reactions . Wu et al. used CIHI data to compare the emergency department discharge diagnosis with the admitting diagnosis among patients who were admitted to the hospital through emergency departments. The authors’ premise was that in patients admitted to hospital through the emergency department for a diagnosis of an adverse drug reaction, the patient’s emergency department discharge diagnosis and hospital admitting diagnosis should be the same, if adverse drug reactions are appropriately identified, recorded and coded. Using an ICD-10 code set containing 245 codes including the external cause codes Y40-59, Wu et al. found that 15% of emergency department visits for adverse drug reactions leading to hospital admission were coded with the corresponding admitting diagnosis in CIHI. In comparison, in our study including all emergency department patients (not just those admitted to hospital), we were only able to identify 3.4% of adverse drug reactions in the administrative data using our “narrower” code set containing ICD-10 codes categorized as A1, A2, B1 and B2. We believe that this large difference in our estimate of the degree of underreporting may be due to Wu et al.’s comparison of adverse drug reactions coded within one set of administrative data (NACRS) to another (the Discharge Abstract Database), as opposed to our comparison with a prospective standard. This indicates that adverse drug event reporting may be overestimated when reporting is evaluated by comparing between two administrative databases.
The strengths of our study include a rigorous assessment of adverse drug events at the point-of-care before any administrative coding occurred. Both a clinical pharmacist and a treating emergency physician assessed all patients in our cohort. The clinical pharmacists in our study evaluated patients independently from physicians, and took their own medical histories, contributing to the accuracy of the available medication information rather than relying on retrospective chart review. Pharmacists documented any suspected adverse drug events in the patients’ records and informed physicians of all potentially missed cases. All cases in which the pharmacists’ and physicians’ assessments of adverse drug events were discordant or uncertain were reviewed and adjudicated by an independent committee consisting of a clinical pharmacist and a medical toxicologist. Another strength of our study includes having conducted a literature review to identify adverse drug event codes in the ICD-10 coding system, reducing the possibility that we underestimated the capacity of the ICD-10 coding system to identify adverse drug events by using too narrow of a code set .
The operational definition of adverse drug events remains problematic, as several interpretations of its most common definition “harm caused by the use of a drug” exist [30, 31]. We approached our case definition of adverse drug events from the health services research perspective, in which the utilization of the emergency department leading to bed occupancy and incurring cost was the primary end point. Thus, all our cases were associated with an emergency department visit, and we did not capture any “harm” or injury” from illnesses not associated with an emergency department visit. Despite this, not all of the events captured in our study will be of interest from a pharmacovigilance or regulatory body perspective. From the latter perspective, adverse drug reactions, a subset of adverse drug events, are most relevant. Examples of events falling into our case definition that may not be relevant from a pharmacovigilance or regulatory body perspective were the following: the need to add a drug/untreated indication (e.g., lack of anticoagulation therapy leading to stroke in a patient with a previously established diagnosis of atrial fibrillation, a high CHADS2 score and previous documentation of the need for anticoagulation), too high or too low dose (e.g., a reduction in furosemide dose leading to pulmonary edema in a patient with previously controlled congestive heart failure and no alternate explanation), noncompliance/failure to receive a drug (e.g., noncompliance with insulin leading to diabetic ketoacidosis) or wrong drug (e.g., a patient with type II diabetes mellitus with recurrent episodes of hypoglycemia on glyburide). We deemed the inclusion of these types of events important from a health services research perspective, as these types of events have previously been associated with increased health services utilization and cost,  and many were classified as preventable . From a patient, clinician and system perspective, the development of methods to identify and monitor these types of events is desirable to generate a factual basis for generating hypotheses about their prevention and to inform health policies to reduce their occurrence. These may include specific actions related to prescribing, administering or monitoring of high-risk medications, or actions targeting specific patient groups. Data on the extent of occurrence and associated burden of events can be used to prioritize actions in a resource-constrained environment to target commonly occurring preventable and costly events. For example, through a recently implemented adverse drug event screening program in the Vancouver Costal Health Authority, through which detailed regional adverse drug event data are collected, our group identified that a large proportion of emergency department visits can be attributed to supratherapeutic/high warfarin dose without any associated bleeding. Identifying the etiologic cause of these visits is informing the development of specific preventative policies within the Health Authority, as well as the discourse between primary care and acute care in terms of the etiology of outpatient adverse drug events and measures for prevention.
In order to ensure that we did not apply too broad of a case definition of adverse drug events, we put mechanisms in place to ensure that events that could be explained by the exacerbation of the patient’s underlying disease or by alternate diagnoses were excluded. These mechanisms included capturing the physician’s working diagnosis, mandating the use of causality algorithms and using an independent adjudication committee. We did not consider the failure to use drug in the first place as an adverse drug event, unless the drug had clearly been documented as being indicated in the patient’s medical record. We put these safeguards in place, as adopting too broad of a case definition and overcalling cases as adverse drug events that might not be, may risk promoting inappropriate use of non-medicinal therapies, and may not serve to promote prudent and rational medication use.
Our study is not without limitations. First, we considered all adverse drug events that had been reported in either primary or secondary diagnostic codes, because some patients presented to the emergency department with more than one adverse drug event, one of which may have been coded under a secondary diagnosis field. Also, some patients may have been diagnosed with more than one diagnosis in the emergency department, one of which was deemed the primary reason for presentation or admission. Therefore, we did not exclude adverse drug event codes that were coded in secondary diagnosis fields in order to avoid underestimating the sensitivity of the administrative dataset and the ICD-10 code set. However, this means that we may have picked up adverse drug events that resulted from in-hospital treatment rather than from outpatient medications. This would have resulted in an overestimation of the sensitivity of the ICD-10 codes. Second, because of the cost and labor involved in establishing a prospective standard for adverse drug events, our sample size is limited. Thus, our study should be regarded as preliminary. Third, our results reflect two Canadian institutions and may not be generalizable to other institutions. Fourth, we expanded our code set to include possible adverse drug events (code categories C, D and E), to allow for better ICD-10 data capture which resulted in a greater proportion of false positives. Fifth, our results may have been influenced by the existing variation in the use of the terminology surrounding adverse drug events . It is possible that physicians were less likely to record (and coders less likely to code) events that they personally felt should not be considered drug-related, even though the presentation met our outcome definition. Finally, we wish to clarify why the number of events listed in this study differs from its parent study : The prospective data used for this study was derived from a prospective observational clinical decision rule derivation study in which we collected data on all outcomes (n = 221). The purpose of the parent study was to derive clinical decision rules to aid health care workers at the point-of-care to identify patients with a broad range of adverse drug events. Yet, this was not possible, likely due to the heterogeneity of the events. Therefore, as stated a priori in the protocol of our parent study, we proceeded to derive clinical decision rules for two narrower categorizations of adverse drug events. Thus, the clinical decision rule derivation study represents a subset of the events analyzed in the present study.