Skip to main content
Download PDF

Detection of adverse drug events in e-prescribing and administrative health data: a validation study

BMC Health Services Research volume 21, Article number: 376 (2021) Cite this article

Abstract

Background

Administrative health data are increasingly used to detect adverse drug events (ADEs). However, the few studies evaluating diagnostic codes for ADE detection demonstrated low sensitivity, likely due to narrow code sets, physician under-recognition of ADEs, and underreporting in administrative data. The objective of this study was to determine if combining an expanded ICD code set in administrative data with e-prescribing data improves ADE detection.

Methods

We conducted a prospective cohort study among patients newly prescribed antidepressant or antihypertensive medication in primary care and followed for 2 months. Gold standard ADEs were defined as patient-reported symptoms adjudicated as medication-related by a clinical expert. Potential ADEs in administrative data were defined as physician, ED, or hospital visits during follow-up for known adverse effects of the study medication, as identified by ICD codes. Potential ADEs in e-prescribing data were defined as study drug discontinuations or dose changes made during follow-up for safety or effectiveness reasons.

Results

Of 688 study participants, 445 (64.7%) were female and mean age was 64.2 (SD 13.9). The study drug for 386 (56.1%) patients was an antihypertensive, and for 302 (43.9%) an antidepressant. Using the gold standard definition, 114 (16.6%) patients experienced an ADE, with 40 (10.4%) among antihypertensive users and 74 (24.5%) among antidepressant users. The sensitivity of the expanded ICD code set was 7.0%, of e-prescribing data 9.7%, and of the two combined 14.0%. Specificities were high (86.0–95.0%). The sensitivity of the combined approach increased to 25.8% when analysis was restricted to the 27% of patients who indicated having reported symptoms to a physician.

Conclusion

Combining an expanded diagnostic code set with e-prescribing data improves ADE detection. As few patients report symptoms to their physician, higher detection rates may be achieved by collecting patient-reported outcomes via emerging digital technologies such as patient portals and mHealth applications.

Peer Review reports

Background

Drug expenditures have been growing at a faster rate than all other costs in health care [1, 2]. While modern drug therapy plays an important role in managing health problems, adverse drug events (ADEs) are frequent and costly, with up to 16% of emergency department (ED) visits and 7% of hospital admissions being medication-related [3,4,5,6]. The frequency of these events can be explained in part by established processes for drug approval, which test drugs in tightly controlled settings and in a limited number of patients who infrequently represent those typically prescribed the drug after approval. In addition, once drugs have entered the market, they are often prescribed off-label for conditions for which they have not been approved [7,8,9,10], which previous research has shown increases the risk of ADEs [11]. Consequently, robust methods of post-marketing surveillance have emerged as a requirement to monitor the safety and effectiveness of medications after they have been approved for sale [12,13,14].

Methods of post-marketing drug surveillance include voluntary systems for spontaneous reporting of ADEs [15] and prescription event monitoring [16], both of which suffer from systematic under-reporting [16,17,18,19]. The data gaps in these reporting systems have been increasingly supplemented with computerized administrative health data, which are timelier, more reliable, and easier to collect. Diagnostic codes from physician billing data are often used to detect ADEs, with International Classification of Disease (ICD) code sets developed for this purpose [20,21,22,23,24,25,26,27,28,29,30]. However, most of these codes have only been validated in the in-patient hospital setting [21, 22, 24,25,26, 31]. Few studies have evaluated the accuracy of ICD code sets for ADE detection in ED and outpatient settings, where the majority of medications are prescribed and resulting ADEs diagnosed [32]. Moreover, code sets evaluated in ED and outpatient settings have not been very helpful, with a reported sensitivity of only up to 28% in detecting ADEs [20, 23, 27, 28]. Many codes are external cause codes that describe precise causes of ADEs (e.g. ICD-10 Y40-Y59 drugs, medication, and biological substances causing adverse effects in therapeutic use) or codes that relate to drug-induced diagnoses (e.g. ICD-10 G44.4 drug-induced headache). ADE detection which relies on these codes requires that the ADE is both recognized by the busy clinician and coded and recorded in administrative health data, activities which are sub-optimally achieved [4, 20, 33, 34].

In contrast, automated systems that search electronic medical notes for phrases representing known adverse effects of specific drug classes are much better at detecting ADEs [23, 35]. This approach relies on the documentation of patient symptoms, which is routinely done, rather than on a clinician’s recognition of an ADE. To develop an enhanced ADE detection system for use in administrative health data, we adapted the strategy of searching for drug side effects by creating an expanded, therapeutic class-targeted ICD code set. This code set includes side effects of specific therapeutic drug classes (e.g. sexual dysfunction for antidepressants), in addition to the conventional external cause codes and drug-induced diagnoses.

One missed opportunity for ADE detection is the use of data from computerized prescribing systems. These systems are ideal for identifying when new drugs are started, when doses are changed, or when drugs are discontinued because of adverse effects or lack of therapeutic effectiveness [36, 37]. Moreover, electronic prescribing systems are commonly used, if not required, in many jurisdictions and the majority of these systems provide highly structured drug data. To further capture ADEs recognized by physicians but underreported in administrative data, we took advantage of an e-prescribing system that requires physicians to record the reasons for changing dose or discontinuing a drug – data that have previously been shown to be accurately recorded – to determine if e-prescribing data could increase ADE detection.

We assessed the accuracy of an expanded administrative data ICD code set and e-prescribing data, individually and combined, for ADE detection among patients receiving a newly prescribed medication in primary care.

Methods

Research design

We conducted a prospective cohort study to assess the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of using treatment change orders in e-prescribing data and diagnostic codes in administrative health data to detect adverse events attributable to medications prescribed in a primary care setting. We assessed the accuracy of ADE detection among new users of antidepressant and antihypertensive medications, as these are among the mostly frequently prescribed medication classes and account for a significant proportion of adverse events [8, 23, 38,39,40,41]. New users were identified through the e-prescribing system, which had information on all drugs prescribed and dispensed in the past year. New users were then followed for a period of 2 months, starting from the date the medication was first dispensed. Gold standard ADEs were defined as new or worsening symptoms reported by patients and adjudicated as medication-related by a clinical expert using the Naranjo criteria [42].

Study context

This study was conducted in Quebec, Canada, where the healthcare system provides complete coverage for medical services. Prescription drug coverage in Quebec is mandatory and provided by the provincial health insurance agency (Régie de l’assurance maladie du Québec – RAMQ) to individuals who are above the age of 65, are welfare recipients, or are not covered by their employer.

Patients were recruited into the study between 2007 and 2015 through MOXXI (Medical Office of the XXIst Century), an experimental electronic health record platform used by 110 primary care physicians for approximately 90,000 of their patients [43]. The system, which integrates information drawn in real-time from administrative health databases into patient profiles (e.g. medications dispensed, current and past health problems, recent ED visits and hospital admissions), also functions as an e-prescribing tool. Physicians using MOXXI must enter a treatment indication for every new medication prescribed by selecting an item from a drop-down listing of all known on- and off-label conditions for that drug. If a medication is discontinued or its dose is changed, the prescriber is required to record a reason for their action by selecting an option from a drop-down listing that includes: adverse drug reaction, ineffective treatment, drug interaction, allergic response, adjusting dose to optimize treatment, error in prescribing, incorrect medication dispensed, end of treatment, and substitution for less expensive drug (Fig. 1). The medication change and associated reason are also printed on the prescription and recorded in the treatment history of the health problem for which the drug was prescribed. A previous study showed that MOXXI had a specificity of 99.7% and sensitivity of 67.0% in documenting treatment discontinuations and dose changes, as well as high concordance between treatment change reasons recorded in the application and those obtained from chart-facilitated physician interview [37].

Fig. 1
figure 1

MOXXI stop/change option: indicating the reason for changing or discontinuing a medication

Full size image

Study population

Patients participating in MOXXI were eligible to be included in this study if they 1) had prescription drug insurance through the RAMQ, 2) were prescribed and dispensed an antidepressant or antihypertensive medication by a MOXXI physician between 2007 and 2015, and 3) had no prior prescription or record of dispensed medication for that same drug in the preceding 12 months. We chose to focus on incident users as adverse events are more likely to occur shortly after starting or stopping therapy [44,45,46,47].

After enrollment in the study, patients were followed for 2 months, starting from the date the antidepressant or antihypertensive medication was first dispensed. During this period, data were collected on medical services received, dose changes or discontinuations of the study drug, and patient-reported outcomes.

Data sources

Administrative health data

Real-time patient demographic, drug, and clinical information was gathered using MOXXI’s dedicated, secure interface to RAMQ databases, with records linked using patients’ unique health insurance numbers. Patient demographic information included date of birth, sex, and date of death (if applicable). Drug information included pharmacy claims data for all drugs dispensed to a patient following the visit in which they received the new prescription, as well as in the preceding 12 months. This included the date of dispensing, drug identification number, quantity dispensed, and prescription duration. The validity of these drug data has been previously demonstrated [48].

Medical fee-for-service billing data for services provided by any licensed physician in the province to study patients during the 2-month follow-up period and the 12 months prior to enrollment were also obtained via the RAMQ interface. These data included the date of service delivery, the service location (critical care, inpatient ward, ED, outpatient clinic, private office), and the ICD-9 diagnosis for the visit. In addition, data from the provincial hospitalization database were retrieved and linked to study patients by unique medicare number. For each hospitalization during the follow-up period and the 12 months prior, we retrieved admission and discharge dates, ICD-10 codes of the principal discharge diagnosis, length of stay, and the discharge destination (e.g. death, home, rehabilitation).

MOXXI electronic prescribing data

Data from the MOXXI e-prescribing tool were used to identify patients with new prescriptions of antihypertensive and antidepressant medications between 2007 and 2015. Prescription information included the patient’s unique health insurance number, date of the prescription, drug identification number, treatment indication, quantity, duration, directives, and number of refills. For each patient with a new prescription for one of the study drugs, we also retrieved data on subsequent discontinuations or dose changes made during the 2 months following first dispensation of the drug, along with the documented reason(s) for the change.

Patient interview data

A modified version of the Australian adverse reaction and drug event report was used to collect patient feedback on adverse events (Additional file 1: Patient Questionnaire) [49]. The interview was administered via telephone by a trained research assistant within 3 weeks following the first dispensation of the study drug. Patients were first asked to report any new or worsening symptom or health problem experienced in the past 3 weeks. Standardized system-related questions were then used to identify any body system changes patients may have experienced since starting the drug, and drug-specific standardized questions were used to determine if patients had experienced any of the known effects of the study drug.

Patient characteristics

Demographic and clinical characteristics used to characterize the study population included age, sex, the therapeutic class of the study drug, the treatment indication of the study drug, the number of concurrent medications patients were taking, and the presence or absence of the 17 conditions included in the Charlson Comorbidity Index [50, 51]. The latter was determined using diagnostic codes for medical services that patients received in the 12 months prior to enrollment in this study.

Adverse drug event (ADE) definitions

ADEs in administrative health data: Expanded therapeutic class-targeted code set & standard code set

The expanded, therapeutic class-targeted ICD code set was composed of ICD codes for known adverse effects of antihypertensive and antidepressant medications (e.g. sexual dysfunction for antidepressants), as well as external cause codes specifying an adverse reaction to those medication classes. Adverse effects for every drug group have been defined by the Oregon Drug Effectiveness Review Project [52] and those relevant to antidepressants and antihypertensives were mapped to corresponding ICD-9 and ICD-10 codes (Additional file 2: Appendices 1–2).

The narrower, standard code set consisted of diagnostic codes included in previously reported ADE code sets [20, 23, 27, 28]. This code set was comprised of external cause codes and drug-induced diagnoses specifying an adverse reaction to antihypertensive and antidepressant medications (Additional file 2: Appendix 3).

To assess ADEs using the standard and expanded code sets, ICD codes recorded in medical services claims for all physician visits (office, ED, outpatient, inpatient) and hospital admissions during the follow-up period were retrieved. Patients were classified as having a potential ADE if one or more ICD codes from the a) standard code set or b) expanded code set were documented as a reason for the physician visit or hospital admission.

ADEs in MOXXI electronic prescribing data

Potential ADEs in MOXXI e-prescribing data were defined as study drug discontinuations or dose changes made by the study physician during the follow-up period for safety (adverse drug reaction, allergy, drug interaction) or effectiveness reasons (treatment ineffective). Treatment changes made for ‘ineffectiveness’ reasons could indicate underlying problems related to medication safety, such as interactions, and because patients often discontinue medication due to adverse reactions [53,54,55].

Gold standard assessment of ADEs

New or worsening symptoms and health problems reported during patient interviews were reviewed by a clinical expert. The expert was also presented with information regarding other medications the patient was prescribed and dispensed, the patient’s medical problem list as documented in MOXXI, and medical services received by the patient (ED visits and hospital admissions). The Naranjo criteria for ADE assessment were used to assess the likelihood that the symptom(s) reported during the interview were related to the newly started therapy [42]. The Naranjo algorithm assesses the presence or absence of ten criteria related to the adverse event, each of which is given a specific weight based on its relative importance. The sum of weights is then used to determine an overall score, which places the probability that the event was medication-related into one of four categories: doubtful, possible, probable, or definite. Gold standard ADEs were defined as patient-reported adverse events whose Naranjo score was classified as probable or definite.

Outcome definitions

Sensitivity was defined as the proportion of true ADEs, as defined by the gold standard, which were correctly identified using e-prescribing and/or administrative health data. Specificity was defined as the proportion of true negatives that were identified as such using e-prescribing and/or administrative health data. The PPV was defined as the proportion of potential ADEs identified using e-prescribing and/or administrative health data that were true ADEs, and the NPV as the proportion of patients who did not have an ADE based on e-prescribing and/or administrative health data who were true negatives.

Statistical analysis

Descriptive statistics were used to characterize the study population and to estimate ADE rates, overall and by therapeutic class, using the gold standard definition, standard and expanded ICD code sets, and MOXXI e-prescribing data. The sensitivity, specificity, PPV and NPV of using administrative health data and e-prescribing data, individually and combined, to detect ADEs were also estimated, overall and by therapeutic class. 95% confidence intervals (CIs) around these estimates were constructed using the exact method for binomial proportions. All analyses were conducted using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA).

Results

Study cohort

Between July 2007 and June 2015, 5012 patients had a new prescription for an antihypertensive or antidepressant medication (Fig. 2). Of these, 1133 (22.6%) had insufficient RAMQ prescription drug coverage during the study period, 1393 (27.8%) declined to participate, and 1215 (24.2%) were refused enrollment by the MOXXI physician. A total of 1271 patients consented and were enrolled in the study. Compared to patients who declined to participate, these patients were younger (mean age 64.1 vs 67.0), a larger proportion of them were male (37.2% vs 34.2%), and a smaller proportion had an indication of depression if prescribed an antidepressant (40.5% vs 44.6%) (Additional file 2: Appendix 4). Among enrolled patients, 803 (63.2%) had the study drug dispensed and a subsequent 688 (85.7%) patients completed the interview within 3 weeks of medication dispensation and were included in the analysis.

Fig. 2
figure 2

CONSORT diagram of eligible, enrolled, and interviewed patients

Full size image

Patient characteristics

The majority (N = 445, 64.7%) of study participants were female and the mean age was 64.2 years (SD 13.9 years) (Table 1). The study drug for 386 (56.1%) patients was an antihypertensive medication, and for 302 (43.9%) patients it was an antidepressant. The most frequently prescribed antihypertensive drug classes were calcium channel blockers (30.1%) and angiotensin II receptor antagonists (29.3%), and the most common antidepressant classes were selective serotonin reuptake inhibitors (SSRIs, 38.7%) and serotonin-norepinephrine reuptake inhibitors (SNRIs, 34.4%). The most common recorded treatment indications for antihypertensive prescriptions were hypertension (79.3%) and edema (4.7%). For antidepressants, depression (40.1%), generalized anxiety disorder (20.5%), and insomnia (17.2%) were the most common treatment indications.

Table 1 Characteristics of Study Patients
Full size table

More than half (51.5%) of patients in this study had at least one comorbidity, with a larger proportion of antihypertensive users having a comorbidity (56.2%) compared to antidepressant users (45.4%) (Table 1). The most common comorbidities included chronic pulmonary disease (21.7%), diabetes (without complications) (15.3%), and renal disease (9.6%). In the one month following the visit in which the study drug was prescribed, patients were dispensed an average of 3.7 (SD 2.9) medications other than the study drug (Table 1). There was no difference between antidepressant and antihypertensive users in the number of medications dispensed.

ADE incidence

Of the 688 study participants, 318 (46.2%) reported at least one new or worsening symptom or health problem during the interview. Among patients prescribed an antihypertensive, 150 (38.9%) reported symptoms, the most common of which were headache (17.3%), dizziness (17.3%) and fatigue (16.0%). More than half (N = 168, 55.6%) of patients prescribed an antidepressant reported symptoms, among which dry mouth or throat (20.2%), headache (18.5%), and nausea or vomiting (17.9%) were the most common (Additional file 2: Appendix 5).

Using the gold standard definition, 114 (16.6%) study patients experienced an adverse event attributable to the study drug during the study follow-up period (Table 2). Among patients prescribed an antihypertensive, 40 (10.4%) experienced an ADE, compared with 74 (24.5%) patients prescribed an antidepressant who experienced an ADE.

Table 2 Adverse Drug Event (ADE) Rates, by Definition and Data Source
Full size table

Using our expanded, therapeutic class-targeted ICD code set applied to administrative health data, 62 (9.0%) patients experienced a potential ADE (51 (13.2%) patients on antihypertensives and 11 (3.6%) patients on antidepressants). Using the standard code set, 19 (2.8%) patients experienced a potential ADE (16 (4.1%) patients on antihypertensives and 3 (1.0%) patients on antidepressants). In patients who had a potential ADE based on our expanded code set, the most common recorded diagnoses were hyperglycemia (N = 21, 35.6%) and dyspnea (N = 10, 17.0%) among antihypertensive users and insomnia (N = 4, 36.4%) among antidepressant users (Additional file 2: Appendix 6).

Using MOXXI e-prescribing data, 40 (5.8%) patients experienced a potential ADE (20 (5.2%) patients on antihypertensives and 20 (6.6%) patients on antidepressants). Overall, 95% of potential ADEs were identified by drug discontinuations (90% antihypertensives, 100% antidepressants), and 5% by dose changes (10% antihypertensives, 0% antidepressants) (Additional file 2: Appendix 7). Drug discontinuation and dose changes identified as potential ADEs were more frequently made for safety (70%) than effectiveness reasons (30%). On average, treatment changes occurred within 30 days (SD 11 days) of the drug being dispensed.

Sensitivity, specificity, PPV, and NPV

The sensitivity of our expanded, therapeutic class-targeted ICD code set was 7.0% (95% CI 3.1–13.4%) and the specificity was 90.6% (95% CI 87.9–92.9%) (Table 3). Treatment discontinuation or dose change orders in e-prescribing data had a sensitivity of 9.7% (95% CI 4.9–16.6%) and specificity of 95.0% (95% CI 92.8–96.6%) in detecting ADEs. For the standard code set, the sensitivity was 0%. Supplementing our expanded ICD code set with e-prescribing data doubled the sensitivity to 14.0% (95% CI 8.2–21.8%) and decreased specificity to 86.0% (95% CI 83.0–88.8%). The PPVs of the expanded ICD code set, e-prescribing data, and the two combined for ADE detection were 12.9% (95% CI 5.7–23.9%), 27.5% (95% CI 14.6–43.9%), and 16.7% (95% CI 9.8–25.7%), respectively. The NPVs of these approaches were 83.1% (95% CI 79.9–85.9%), 84.1% (95% CI 81.1–86.8%), and 83.5% (95% CI 80.2–86.4%), respectively.

Table 3 Accuracy of Electronic Prescribing Data and Diagnostic Codes in Administrative Health Data in Detecting Adverse Drug Events, Overall and by Medication Class
Full size table

Assessing the accuracy of these approaches by therapeutic class revealed that while the sensitivity of e-prescribing data was higher among antidepressant users compared with antihypertensive users (13.5% vs 2.5%, respectively), the reverse was true regarding the sensitivity of our expanded ICD code set (5.4% vs 10.0%, respectively) (Table 3).

Symptoms reported to physicians

Interview data indicate that 73% of study patients who reported new or worsening symptoms and health problems during their interview did not inform their doctor of these issues. Compared to patients who reported all their symptoms to their physician, patients who did not report symptoms experienced less severe symptoms (27.0% of symptoms were rated severe or very severe vs 51.9% of symptoms), were younger (mean age 61.5 vs 66.3), and were more likely to be male (29.6% vs 25.3%) (Additional file 2: Appendices 8–9). We also found that the sensitivity of our combined approach was improved almost two-fold when the analysis was restricted to patients who had reported symptoms to their physician (25.8% vs 14.0%).

Discussion

In this validation study, we found that treatment change orders in e-prescribing data and diagnostic codes in administrative health data, individually and combined, enhanced the identification of ADEs caused by antidepressant and antihypertensive medications prescribed in a primary care setting compared to the standard code set. The higher sensitivity of our expanded ICD code set compared to the standard code set suggests that including diagnostic codes for known adverse effects of therapeutic drug classes improves ADE detection. However, the sensitivity of the expanded code set was still low, as even a comprehensive diagnostic code set will fail to capture ADEs if ICD codes are not reported in administrative health data. To fill this gap, we supplemented our expanded code set with e-prescribing data on change or discontinuation orders for safety or effectiveness reasons. Although this doubled the sensitivity, the sensitivity of our combined approach did not surpass 14.0%.

Our results can be explained, at least in part, by low rates of symptom reporting by patients to physicians. Indeed, 73% of study patients indicated in their interviews that they had not reported any of their symptoms to a physician. This suggests that e-prescribing and administrative health data alone may not detect mild ADEs for which patients do not seek medical attention. Even when patients do seek medical attention, there is discordance between the symptoms patients report and those documented in the electronic health record [56], which suggests that physician-reported data may not provide an accurate picture of patients’ symptoms. This could explain why, even when restricting analysis to patients who had reported symptoms to their physicians, the sensitivity of our combined approached remained relatively low. Thus, a potential alternative for ADE detection may lie in patient-reported outcomes (PROs), which have emerged as a necessary and increasingly promoted means of obtaining information from patients regarding their health status, including adverse events [57]. PROs have potential as a feasible and effective approach for ADE detection [58, 59], particularly if integrated into patient-centered digital health tools such as patient portals, mobile health (mHealth) applications, or even online reporting sites [60].

Interestingly, our subgroup analyses indicated that the sensitivity of treatment change orders in e-prescribing data was higher among antidepressant users compared with antihypertensive users. One potential explanation is that patients presenting to primary care physicians with an ADE related to new antidepressant use may be more likely to have their medication discontinued or changed. In contrast, patients with ADEs related to antihypertensive use may be prescribed an additional medication that targets the ADE, creating a medication cascade [61]. This is supported by a closer look at diagnostic codes of potential ADEs in administrative health data, the most common of which among antihypertensive users were for hyperglycemia. Physicians faced with a hyperglycemic patient may be more likely to prescribe a hypoglycemic medication than discontinue the antihypertensive [62]. This could explain the lower sensitivity of e-prescribing data among antihypertensive users.

Although our expanded, therapeutic class-targeted ICD code set was more sensitive than the standard code set in detecting ADEs in our cohort, its accuracy was similar to previously reported estimates for narrower code sets. In a 2001 study conducted in an outpatient setting, Honigman reported a PPV of 2% for the use of ICD-9 codes in identifying ADEs [23]. Similarly, Field found that computer-generated signals that included ICD-9 codes flagged true ADEs 7% of the time [27]. Results of both studies suggested low sensitivities of diagnostic codes in detecting ADEs, as did an investigation by Hohl in 2013 of the sensitivities of two ICD-10 code sets in detecting ADEs in the emergency department (6.8 and 28.1%) [20]. Despite using codes similar to those in our narrow standard code set, the sensitivities reported in these studies are more similar to that of our expanded code set. This may be explained by the data sources from which ICD codes were obtained in these studies, which differ from those used in our investigation. Electronic medical record data were used by the Field and Honigman studies, whereas Hohl used ED data from the National Ambulatory Care Reporting System (NACRS). The latter assigns ICD codes to ED discharge diagnoses by mapping selections made from a shortlist of 800 common term diagnoses to ICD-10-CA codes. Compared to the billing data used in our study, this process likely assigns ICD codes that more accurately reflect patients’ diagnoses. This could explain the higher sensitivity reported in Hohl despite their use of a narrower code set. In addition, as part of their process to identify gold standard ADEs, Hohl interviewed treating emergency physicians using a standardized questionnaire to determine patients’ working diagnoses. This may have made it more likely that treating physicians recognized and documented ADEs in the ED visit note, which in turn is used to assign ICD codes in NACRS, thus potentially overestimating the sensitivity of their code set.

Our study has several limitations. First, gold standard ADEs were often adjudicated based on incomplete information since lab test results and information on whether symptoms improved after medication discontinuation were often unavailable. These missing data impact on the Naranjo score calculation since item responses are recorded as “don’t know”. This uncertainty with missing information is a well-documented issue with the Naranjo algorithm, which nevertheless remains a frequently used method for ADE adjudication [63]. However, the ADE rates produced by our gold standard are similar to previously reported rates of adverse events attributable to antidepressant and antihypertensive medications [64,65,66], suggesting that the Naranjo algorithm produced accurate estimates in our sample. Second, the 3-week recall period used in patient interviews may have led to an underestimation of true ADE rates as patients may not have recalled mild symptoms that occurred earlier in this period. However, we expect such bias to be minimal as adverse effects of medications typically occur days or even weeks following the start of medication use [67, 68]. The 3-week window we used is meant to allow time for adverse effects to occur prior to the interview while minimizing poor recall. It is similar to (and in some cases even shorter than) recall periods used in previously published studies [45, 69,70,71]. In addition, the mean age of the study population was 64.2 years, of whom a low proportion would be expected to have significant cognitive problems impacting recall, although this was not specifically addressed. Third, administrative health data from the RAMQ record only the first 4 digits of ICD-9 codes, whereas many of the codes we identified as potential adverse effects of study drugs had up to 5 digits (and thus had to be truncated when applied to our data). This could have led to false positive ADEs detected using this approach. However, given the high specificity of administrative health data in detecting ADEs, this did not seem to be an issue in our particular sample. Finally, the generalizability of our results is unknown, as the accuracy of our ICD code set in detecting ADEs is largely dependent on how consistently and accurately ICD codes are recorded in billing data (since there are no systematic checks for data quality), which may vary between jurisdictions.

Conclusion

The results of this validation study suggest that supplementing diagnostic codes in administrative health data with treatment change orders in electronic prescribing data doubles sensitivity. However, even when combined, these approaches do not possess sufficient accuracy to detect most ADEs resulting from medications prescribed in the community. Further research should be conducted to investigate the utility of patient-reported outcomes in detecting potential ADEs via emerging digital health technologies such as patient portals and mobile health applications.

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available due to privacy reasons, nor is a de-identified dataset available upon request.

Abbreviations

ADE:

Adverse Drug Event

ED:

Emergency Department

ICD:

Intenational Classification of Disease

MOXXI:

Medical Office of the XXIst Century

RAMQ:

Régie de l’assurance maladie du Québec

PPV:

Positive Predictive Value

NPV:

Negative Predictive Value

CI:

Confidence Interval

SD:

Standard deviation

SSRI:

Selective serotonin reuptake inhibitor

SNRI:

Serotonin-norepinephrine reuptake inhibitors

SRI:

Serotonin reuptake inhibitor

NACRS:

National Ambulatory Care Reporting System

References

  1. Canadian Institute for Health Information(CIHI). National Health Expenditure Trends, 1975 to 2017. Ottawa: Canadian Institute for Health Informatin (CIHI); 2017.

  2. Martin AB, Hartman M, Benson J, Catlin A. National Health Spending in 2014: faster growth driven by coverage expansion and prescription drug spending. Health Aff. 2016;35(1):150–60. https://doi.org/10.1377/hlthaff.2015.1194.

    Article  Google Scholar 

  3. Zed PJ, Abu-Laban RB, Balen RM, Loewen PS, Hohl CM, Brubacher JR, et al. Incidence, severity and preventability of medication-related visits to the emergency department: a prospective study. Can Med Assoc J. 2008;178(12):1563–9. https://doi.org/10.1503/cmaj.071594.

  4. Hohl CM, Robitaille C, Lord V, Dankoff J, Colacone A, Pham L, et al. Emergency physician recognition of adverse drug-related events in Elder patients presenting to an emergency department. Acad Emerg Med. 2005;12(3):197–205. https://doi.org/10.1197/j.aem.2004.08.056.

  5. Kongkaew C, Noyce PR, Ashcroft DM. Hospital admissions associated with adverse drug reactions: a systematic review of prospective observational studies. Ann Pharmacother. 2008;42(7):1017–25. https://doi.org/10.1345/aph.1L037.

    Article  PubMed  Google Scholar 

  6. Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE prevention study group. JAMA. 1995;274(1):29–34. https://doi.org/10.1001/jama.1995.03530010043033.

    Article  CAS  PubMed  Google Scholar 

  7. Wong J, Motulsky A, Abrahamowicz M, Eguale T, Buckeridge DL, Tamblyn R. Off-label indications for antidepressants in primary care: descriptive study of prescriptions from an indication based electronic prescribing system. BMJ. 2017;356:j603.

  8. Wong J, Motulsky A, Eguale T, Buckeridge DL, Abrahamowicz M, Tamblyn R. Treatment indications for antidepressants prescribed in primary care in Quebec, Canada, 2006-2015. JAMA. 2016;315(20):2230–2. https://doi.org/10.1001/jama.2016.3445.

    Article  PubMed  Google Scholar 

  9. Radley DC, Finkelstein SN, Stafford RS. Off-label prescribing among office-based physicians. Arch Intern Med. 2006;166(9):1021–6. https://doi.org/10.1001/archinte.166.9.1021.

    Article  PubMed  Google Scholar 

  10. Carton L, Cottencin O, Lapeyre-Mestre M, Geoffroy PA, Favre J, Simon N, et al. Off-label prescribing of antipsychotics in adults, children and elderly individuals: a systematic review of recent prescription trends. Curr Pharm Des. 2015;21(23):3280–97. https://doi.org/10.2174/1381612821666150619092903.

    Article  CAS  PubMed  Google Scholar 

  11. Eguale T, Buckeridge DL, Verma A, Winslade NE, Benedetti A, Hanley JA, et al. Association of off-label drug use and adverse drug events in an adult population. JAMA Intern Med. 2016;176(1):55–63. https://doi.org/10.1001/jamainternmed.2015.6058.

  12. Group IEW. ICH harmonized tripartite guideline: Pharmacovigilance planning; 2004.

  13. Meyboom RHB. The case for good pharmacovigilance practice. Pharmacoepidemiol Drug Saf. 2000;9(4):335–6. https://doi.org/10.1002/1099-1557(200007/08)9:4<335::AID-PDS501>3.0.CO;2-S.

    Article  CAS  PubMed  Google Scholar 

  14. Borden EK. Post-marketing surveillance: drug epidemiology. J Int Med Res. 1981;9(6):401–7. https://doi.org/10.1177/030006058100900602.

    Article  CAS  PubMed  Google Scholar 

  15. Wiholm B, Olsson S, Moore N, Waller P. Spontaneous reporting systems outside the US. In: Strom BL, editor. Pharmacoepidemiology; 2000.

    Google Scholar 

  16. Mann RD. Prescription-event monitoring--recent progress and future horizons. Br J Clin Pharmacol. 1998;46(3):195–201. https://doi.org/10.1046/j.1365-2125.1998.00774.x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Hazell L, Shakir SAW. Under-reporting of adverse drug reactions. Drug Saf. 2006;29(5):385–96. https://doi.org/10.2165/00002018-200629050-00003.

    Article  PubMed  Google Scholar 

  18. Bégaud B, Martin K, Haramburu F, Moore N. Rates of spontaneous reporting of adverse drug reactions in france. JAMA. 2002;288(13):1588.

    Article  PubMed  Google Scholar 

  19. Key C, Layton D, Shakir SAW. Results of a postal survey of the reasons for non-response by doctors in a prescription event monitoring study of drug safety. Pharmacoepidemiol Drug Saf. 2002;11(2):143–8. https://doi.org/10.1002/pds.690.

    Article  PubMed  Google Scholar 

  20. Hohl CM, Kuramoto L, Yu E, Rogula B, Stausberg J, Sobolev B. Evaluating adverse drug event reporting in administrative data from emergency departments: a validation study. BMC Health Serv Res. 2013;13(1):473. https://doi.org/10.1186/1472-6963-13-473.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Hougland P, Nebeker J, Pickard S, Tuinen M, Masheter C, Elder S, et al. Using ICD-9-CM codes in hospital claims data to detect adverse events in patient safety surveillance. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Advances in patient safety: new directions and alternative approaches. Rockville (MD): Agency for Healthcare Research and Quality; 2008.

    Google Scholar 

  22. Hougland P, Xu W, Pickard S, Masheter C, Williams SD. Performance of international classification of diseases, 9th revision, clinical modification codes as an adverse drug event surveillance system. Med Care. 2006;44(7):629–36. https://doi.org/10.1097/01.mlr.0000215859.06051.77.

    Article  PubMed  Google Scholar 

  23. Honigman B, Lee J, Rothschild J, Light P, Pulling RM, Yu T, et al. Using computerized data to identify adverse drug events in outpatients. J Am Med Inform Assoc. 2001;8(3):254–66. https://doi.org/10.1136/jamia.2001.0080254.

  24. Miguel A, Azevedo LF, Lopes F, Freitas A, Pereira AC. Methodologies for the detection of adverse drug reactions: comparison of hospital databases, chart review and spontaneous reporting. Pharmacoepidemiol Drug Saf. 2013;22(1):98–102. https://doi.org/10.1002/pds.3348.

    Article  PubMed  Google Scholar 

  25. Kuklik N, Stausberg J, Jöckel K-H. Adverse drug events in German hospital routine data: a validation of international classification of diseases, 10th revision (ICD-10) diagnostic codes. PLoS One. 2017;12(11):e0187510. https://doi.org/10.1371/journal.pone.0187510.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Hodgkinson MR, Dirnbauer NJ, Larmour I. Identification of adverse drug reactions using the ICD-10 Australian modification clinical coding surveillance. J Pharm Pract Res. 2009;39(1):19–23. https://doi.org/10.1002/j.2055-2335.2009.tb00698.x.

    Article  Google Scholar 

  27. Field TS, Gurwitz JH, Harrold LR, Rothschild JM, Debellis K, Seger AC, et al. Strategies for detecting adverse drug events among older persons in the ambulatory setting. J Am Med Inform Assoc. 2004;11(6):492–8. https://doi.org/10.1197/jamia.M1586.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Wu C. Adverse drug reactions in the emergency department population in Ontario: analysis of National Ambulatory Care Reporting System and discharge abstract database 2003–2007. Toronto (ON): University of Toronto; 2010.

    Google Scholar 

  29. Stausberg J, Hasford J. Identification of adverse drug events: the use of ICD-10 coded diagnoses in routine hospital data. Dtsch Arztebl Int. 2010;107(3):23–9. https://doi.org/10.3238/arztebl.2010.0023.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Hohl CM, Karpov A, Reddekopp L, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc. 2013;21(3):547–57. https://doi.org/10.1136/amiajnl-2013-002116.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Geraci JM, Ashton CM, Kuykendall DH, Johnson ML, Wu L. International classification of diseases, 9th revision, clinical modification codes in discharge abstracts are poor measures of complication occurrence in medical inpatients. Med Care. 1997;35(6):589–602. https://doi.org/10.1097/00005650-199706000-00005.

    Article  CAS  PubMed  Google Scholar 

  32. Martys CR. Adverse reactions to drugs in general practice. Br Med J. 1979;2(6199):1194–7. https://doi.org/10.1136/bmj.2.6199.1194.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Dormann H, Criegee-Rieck M, Neubert A, Egger T, Geise A, Krebs S, et al. Lack of awareness of community-acquired adverse drug reactions upon hospital admission. Drug Saf. 2003;26(5):353–62. https://doi.org/10.2165/00002018-200326050-00004.

    Article  PubMed  Google Scholar 

  34. Gandhi TK, Burstin HR, Cook EF, Puopolo AL, Haas JS, Brennan TA, et al. Drug complications in outpatients. J Gen Intern Med. 2000;15(3):149–54. https://doi.org/10.1046/j.1525-1497.2000.04199.x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Field TS, Gilman BH, Subramanian S, Fuller JC, Bates DW, Gurwitz JH. The costs associated with adverse drug events among older adults in the ambulatory setting. Med Care. 2005;43(12):1171–6. https://doi.org/10.1097/01.mlr.0000185690.10336.70.

    Article  PubMed  Google Scholar 

  36. Schiff GD, Rucker TD. Computerized PrescribingBuilding the electronic infrastructure for better medication usage. JAMA. 1998;279(13):1024–9. https://doi.org/10.1001/jama.279.13.1024.

    Article  CAS  PubMed  Google Scholar 

  37. Eguale T, Tamblyn R, Winslade N, Buckeridge D. Detection of adverse drug events and other treatment outcomes using an electronic prescribing system. Drug Saf. 2008;31(11):1005–16. https://doi.org/10.2165/00002018-200831110-00005.

    Article  PubMed  Google Scholar 

  38. Gandhi TK, Weingart SN, Borus J, Seger AC, Peterson J, Burdick E, et al. Adverse drug events in ambulatory care. N Engl J Med. 2003;348(16):1556–64. https://doi.org/10.1056/NEJMsa020703.

    Article  PubMed  Google Scholar 

  39. Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003;289(9):1107–16. https://doi.org/10.1001/jama.289.9.1107.

    Article  PubMed  Google Scholar 

  40. Canadian Institute for Health Information (CIHI). Drug use among seniors in Canada, 2016. Otttawa, ON: Canadian Institute for Health Information (CIHI); 2018.

    Google Scholar 

  41. Beck CA, Patten SB, Williams JVA, Wang JL, Currie SR, Maxwell CJ, et al. Antidepressant utilization in Canada. Soc Psychiatry Psychiatr Epidemiol. 2005;40(10):799–807. https://doi.org/10.1007/s00127-005-0968-0.

    Article  PubMed  Google Scholar 

  42. Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981;30(2):239–45. https://doi.org/10.1038/clpt.1981.154.

    Article  CAS  PubMed  Google Scholar 

  43. Tamblyn R, Huang A, Kawasumi Y, Bartlett G, Grad R, Jacques A, et al. The development and evaluation of an integrated electronic prescribing and drug management system for primary care. J Am Med Inform Assoc. 2006;13(2):148–59. https://doi.org/10.1197/jamia.M1887.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Tamblyn R, Abrahamowicz M, Buckeridge DL, Bustillo M, Forster AJ, Girard N, et al. Effect of an Electronic Medication Reconciliation Intervention on Adverse Drug Events: A Cluster Randomized Trial. JAMA Network Open. 2019;2(9):e1910756.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Adverse drug events occurring following hospital discharge. J Gen Intern Med. 2005;20(4):317–23. https://doi.org/10.1111/j.1525-1497.2005.30390.x.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Boockvar K, Fishman E, Kyriacou CK, Monias A, Gavi S, Cortes T. Adverse events due to discontinuations in drug use and dose changes in patients transferred between acute and long-term care facilities. Arch Intern Med. 2004;164(5):545–50. https://doi.org/10.1001/archinte.164.5.545.

    Article  PubMed  Google Scholar 

  47. van den Bemt PM, Egberts AC, Lenderink AW, Verzijl JM, Simons KA, van der Pol WS, et al. Risk factors for the development of adverse drug events in hospitalized patients. Pharm World Sci. 2000;22(2):62–6. https://doi.org/10.1023/A:1008721321016.

    Article  PubMed  Google Scholar 

  48. Tamblyn R, Lavoie G, Petrella L, Monette J. The use of prescription claims databases in pharmacoepidemiological research: the accuracy and comprehensiveness of the prescription claims database in Quebec. J Clin Epidemiol. 1995;48(8):999–1009. https://doi.org/10.1016/0895-4356(94)00234-H.

    Article  CAS  PubMed  Google Scholar 

  49. Mitchell AS, Henry DA, Hennrikus D, O'Connell DL. Adverse drug reactions: can consumers provide early warning? Pharmacoepidemiol Drug Saf. 1994;3(5):257–64. https://doi.org/10.1002/pds.2630030504.

    Article  Google Scholar 

  50. Charlson M, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol. 1994;47(11):1245–51. https://doi.org/10.1016/0895-4356(94)90129-5.

    Article  CAS  PubMed  Google Scholar 

  51. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. https://doi.org/10.1016/0021-9681(87)90171-8.

    Article  CAS  PubMed  Google Scholar 

  52. McDonagh MS, Jonas DE, Gartlehner G, Little A, Peterson K, Carson S, et al. Methods for the drug effectiveness review project. BMC Med Res Methodol. 2012;12(1):140. https://doi.org/10.1186/1471-2288-12-140.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Wysowski DK, Swartz L. Adverse drug event surveillance and drug withdrawals in the United States, 1969-2002: the importance of reporting suspected reactions. Arch Intern Med. 2005;165(12):1363–9. https://doi.org/10.1001/archinte.165.12.1363.

    Article  PubMed  Google Scholar 

  54. Meyboom RHB, Lindquist M, Flygare A-K, Biriell C, Edwards IR. The value of reporting therapeutic ineffectiveness as an adverse drug reaction. Drug Saf. 2000;23(2):95–9. https://doi.org/10.2165/00002018-200023020-00001.

    Article  CAS  PubMed  Google Scholar 

  55. Hartigan-Go KY, Wong JQ. Inclusion of therapeutic failures as adverse drug reactions. In: Aronson JK, editor. Side Effects of Drugs Annual. 23: Elsevier; 2000. p. xxvii–xxxiii.

  56. Pakhomov SV, Jacobsen SJ, Chute CG, Roger VL. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care. 2008;14(8):530–9.

    PubMed  PubMed Central  Google Scholar 

  57. Basch E. New Frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med. 2014;65(1):307–17. https://doi.org/10.1146/annurev-med-010713-141500.

    Article  CAS  PubMed  Google Scholar 

  58. Blenkinsopp A, Wilkie P, Wang M, Routledge PA. Patient reporting of suspected adverse drug reactions: a review of published literature and international experience. Br J Clin Pharmacol. 2007;63(2):148–56. https://doi.org/10.1111/j.1365-2125.2006.02746.x.

    Article  CAS  PubMed  Google Scholar 

  59. Basch E. Systematic collection of patient-reported adverse drug reactions: a path to patient-Centred Pharmacovigilance. Drug Saf. 2013;36(4):277–8. https://doi.org/10.1007/s40264-013-0026-x.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Banerjee AK, Ingate S. Web-based patient-reported outcomes in drug safety and risk management. Drug Saf. 2012;35(6):437–46. https://doi.org/10.2165/11632390-000000000-00000.

    Article  PubMed  Google Scholar 

  61. Rochon PA, Gurwitz JH. The prescribing cascade revisited. Lancet. 2017;389(10081):1778–80. https://doi.org/10.1016/S0140-6736(17)31188-1.

    Article  PubMed  Google Scholar 

  62. Alderman MH. New onset diabetes during antihypertensive therapy. Am J Hypertens. 2008;21(5):493–9. https://doi.org/10.1038/ajh.2008.17.

    Article  CAS  PubMed  Google Scholar 

  63. Ross M, Sydney B, Wu PE, Anika AM, Papillon-Ferland L, Tamblyn R, et al. Adverse Drug Events in Older Adults: Review of Adjudication Methods in Deprescribing Studies. J Am Geriatrics Soc. 2020;68(7):1594-602.

  64. Grohmann R, Hippius H, Helmchen H, Ruther E, Schmidt LG. The AMUP study for drug surveillance in psychiatry - a summary of inpatient data. Pharmacopsychiatry. 2004;37(Suppl 1):S16–26.

    PubMed  Google Scholar 

  65. Alhawassi TM, Krass I, Pont LG. Antihypertensive-related adverse drug reactions among older hospitalized adults. Int J Clin Pharm. 2018;40(2):428–35. https://doi.org/10.1007/s11096-017-0583-7.

    Article  CAS  PubMed  Google Scholar 

  66. Thomopoulos C, Parati G, Zanchetti A. Effects of blood pressure lowering treatment in hypertension: 8. Outcome reductions vs. discontinuations because of adverse drug events – meta-analyses of randomized trials. J Hypertens. 2016;34(8):1451–63. https://doi.org/10.1097/HJH.0000000000000972.

    Article  CAS  PubMed  Google Scholar 

  67. Crawford AA, Lewis S, Nutt D, Peters TJ, Cowen P, O'Donovan MC, et al. Adverse effects from antidepressant treatment: randomised controlled trial of 601 depressed individuals. Psychopharmacology. 2014;231(15):2921–31. https://doi.org/10.1007/s00213-014-3467-8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Loga-Zec S, Asceric M, Loga-Andrijic N, Kapetanovic B, Zerem E. The incidence of antihypertensive drug-induced side effects in patients with diabetes mellitus type 2 and hypertension. Med Arch. 2014;68(6):372–5. https://doi.org/10.5455/medarh.2014.68.372-375.

    Article  PubMed  PubMed Central  Google Scholar 

  69. McDonald EG, Wu PE, Rashidi B, Forster AJ, Huang A, Pilote L, et al. The MedSafer study: a controlled trial of an electronic decision support tool for Deprescribing in acute care. J Am Geriatr Soc. 2019;67(9):1843–50. https://doi.org/10.1111/jgs.16040.

    Article  PubMed  Google Scholar 

  70. de Vries ST, Mol PGM, de Zeeuw D, Haaijer-Ruskamp FM, Denig P. Development and initial validation of a patient-reported adverse drug event questionnaire. Drug Saf. 2013;36(9):765–77. https://doi.org/10.1007/s40264-013-0036-8.

    Article  PubMed  Google Scholar 

  71. Vanderkooy JD, Kennedy SH, Bagby RM. Antidepressant side effects in depression patients treated in a naturalistic setting: a study of bupropion, moclobemide, paroxetine, sertraline, and venlafaxine. Can J Psychiatr. 2002;47(2):174–80. https://doi.org/10.1177/070674370204700208.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

Not applicable.

Funding

This study was funded by a Canadian Institutes for Health Research (CIHR) Operating Grant, funding reference number 93528. The funder had no role in the design of the study, data collection, analysis, interpretation, or manuscript preparation.

Author information

Authors and Affiliations

  1. Clinical and Health Informatics Research Group, McGill University, 1140 Pine Avenue West, Montreal, QC, H3A 1A3, Canada

    Bettina Habib, Robyn Tamblyn & Nadyne Girard

  2. Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada

    Robyn Tamblyn

  3. Department of Medicine, McGill University Health Centre, Montreal, Canada

    Robyn Tamblyn & Tewodros Eguale

  4. School of Pharmacy, Massachusetts College of Pharmacy and Health Sciences, Boston, MA, USA

    Tewodros Eguale

  5. Division of Geriatric Medicine, University of Ottawa, Ottawa, Ontario, Canada

    Allen Huang

Authors
  1. Bettina Habib
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robyn Tamblyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadyne Girard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tewodros Eguale
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Allen Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RT and TE contributed to study concept and design. AH adjudicated adverse drug events. BH and NG contributed to the statistical analysis. BH drafted the manuscript. BH, RT, NG, TE, and AH critically reviewed the manuscript for intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Bettina Habib.

Ethics declarations

Ethics approval and consent to participate

This study received ethics approval from the McGill University Institutional Review Board (IRB), study number A01-B02-02A. All study participants provided written informed consent to participate in this study. Authorization to access administrative health data of consenting patients was granted by the Quebec privacy commission (Commission d’accès à l’information du Québec).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12913_2021_6346_MOESM1_ESM.pdf

Additional file 1. Patient Questionnaire

Additional file 2: Appendix 1

. Documented adverse effects of antidepressant and antihypertensive medications. Appendix 2. Expanded, Therapeutic Class-Targeted ICD Code Set. Appendix 3. Standard ICD Code Set. Appendix 4. Characteristics of consented patients, patients who declined consent, patients who were not dispensed the study drug, patients who were dispensed the study drug, and patients who were included in the final analysis. Appendix 5. Symptoms reported during interview, overall and by study drug therapeutic class, and among patients who reported all, none, or some of their symptoms to their physician. Appendix 6. Potential medication-related adverse effects experienced by antidepressant and antihypertensive users based on expanded, therapeutic class-targeted codeset. Appendix 7. Details of treatment changes among patients with a potential ADE based on e-prescribing data. Appendix 8. Characteristics of patients who reported all, none, or some of the symptoms they indicated in the interview to their physician. Appendix 9. Severity of symptoms experienced, as reported during interviews, by patients who reported all, none, or some of their symptoms to their physician.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Habib, B., Tamblyn, R., Girard, N. et al. Detection of adverse drug events in e-prescribing and administrative health data: a validation study. BMC Health Serv Res 21, 376 (2021). https://doi.org/10.1186/s12913-021-06346-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-06346-y

Keywords

  • Adverse drug event
  • Administrative health data
  • Electronic prescribing data
  • Validation

BMC Health Services Research

ISSN: 1472-6963

Contact us