Predictive values of diagnostic codes for identifying serious hypocalcemia and dermatologic adverse events among women with postmenopausal osteoporosis in a commercial health plan database

Osteoporosis, a condition characterized by loss of bone mass and increased risk of fracture, affects approximately 10 million individuals in the United States (US) [1]. Approximately 30% of postmenopausal Caucasian women have osteoporosis; their lifetime fracture risk is estimated as 40% [2]. Treatment includes abaloparatide, bisphosphonates, calcitonin, denosumab, raloxifene, and teriparatide, some of which have been linked with increased risk of hypocalcemia and adverse dermatologic events [3–7]. Post-marketing safety studies of medicines often rely on administrative claims databases to identify adverse outcomes following drug exposure [8–11]. Valid ascertainment of outcomes is essential for accurate results. Yet, the accuracy of identifying hypocalcemia and adverse dermatologic events, especially the more serious events, using claims-identified codes is not well described.

We conducted a study to assess the validity of diagnostic codes in claims data for hypocalcemia and dermatologic events, as compared with medical record confirmation of events, among a population of women with postmenopausal osteoporosis (PMO).

Medical records were sought for a random 265 of 441 potential dermatologic adverse events identified from the database (6 bullous, 15 erythematous, 247 urticaria/rash and 3 with multiple codes); 184 (69%) charts were received (all had sufficient information for adjudication). The physician confirmed 128 as dermatologic events (PPV 69.6%, 95% CI 62.4–76.1%) (Table 2). PPVs varied across subtypes of dermatologic events (highest for urticaria/rash [PPV 70.5%, 95% CI 63.1–77.2%]) and healthcare setting (highest for ED [PPV 77.0%, 95% CI 69.4–83.5%]). Additionally, the PPV was higher for claims associated with emergency medicine (PPV 78.1%, 95% CI 70.2–84.7%) relative to other provider specialties. The inclusion of incidental dermatologic adverse events had little impact (PPV 74.5%, 95% CI 67.7–80.2%).

In this nationwide, observational study of women with PMO, the performance of diagnosis codes in identifying hypocalcemia and dermatologic adverse events from health insurance claims data varied across settings, and by provider specialty. Our definition of hypocalcemia (as the primary reason for obtaining ED or inpatient care) yielded a PPV of 40%, and for dermatologic adverse events, a PPV of 70%. The inclusion of incidental cases increased the PPV of hypocalcemia appreciably, suggesting that secondary hypocalcemia may frequently be recorded in the primary position on claims. Incidental cases were infrequent for dermatologic adverse events, possibly because these events generally represent the true primary reason for the patients’ care. With both outcomes, the diagnosis codes from ED claims were more accurate than inpatient claims. Serious hypocalcemia and dermatologic adverse events may be treated and resolved within the ED without requiring hospital admission, and if hospitalization does occur, these outcomes—hypocalcemia in particular—may be considered a secondary concern.

There are few published data for comparison. Strom et al. reported that within Medicaid claims, 60.9% of the erythematous events captured through presence of ICD-9 695.1 (erythema multiforme, Stevens-Johnson syndrome, and toxic epidermal necrolysis) were later confirmed as true cases [15]. Within a health plan database, Chan et al. reported that the presence of a discharge diagnosis of erythema multiforme yielded a PPV of 60.7% [16]. These are similar to our PPV finding of 56 to 67% (including incidental cases) for serious erythematous events, which also included ICD-9 695.5 (exfoliation due to erythematous conditions).

Historically, 70 to 80% of medical records requested by our research group (and similar institutions) are obtained [17, 18]. In this study, as expected, our retrieval rate was at the lower end of this spectrum as we sought medical records only from the principal site of care. This choice arose from the study objective to validate outcomes associated with a specific medical claim, rather than to confirm the presence of an outcome. While our lower retrieval rate decreased the precision of the PPVs, leading to broader confidence intervals, it likely did not bias the PPVs estimates, unless the chart retrieval rate was somehow differential with respect to the true case status. For example, if hospitals were more likely to provide charts for true cases of hypocalcemia, our PPV estimates would be biased toward 100%. However, this scenario seems unlikely. With other study objectives, it is generally feasible to seek charts from multiple providers or institutions (e.g., a dermatologist and a hospital) to increase the fraction of events for which at least one medical record is available.

In this study, we had expected that limiting our algorithm to the first-position diagnoses on claims would increase the PPV for capturing serious occurrences of adverse events that were the primary reason for seeking care. However, we found that clinically incidental or secondary events are also captured through diagnosis codes recorded in the primary position. Further, it is important to note that outcomes leading to hospitalization or ED visits may have had ICD-9 codes recorded in a secondary position on claims. These cases were not counted in this study, and thus, incidence derived with these code sets will be underestimated.

This study was conducted in a US commercially-insured population which, on average, tend to be slightly younger than the US general population. While we expect the results of this study to be generalizable to other insured populations, caution must be taken if there are differences in coding standards for reimbursement for hypocalcemia or dermatologic adverse events across insurers. Further, as PPVs vary according to disease prevalence, our PPVs may underestimate those observed in populations with a higher prevalence of hypocalcemia and/or dermatologic events than our study population, and overestimate those observed in populations with lower prevalence of these conditions than our study population. This highlights the need to assess the performance of case-identification algorithms within specific populations of interest. Lastly, we recognize that additional work is needed to assess the performance of algorithms for identifying other outcomes of interest that are associated with the use of osteoporosis medication, including osteonecrosis of the jaw, and atypical femur fractures.

Our results suggest that the current algorithms to identify serious hypocalcemia and dermatologic adverse events are moderately accurate for events leading to an ED visit (PPV 81.8% for hypocalcemia and PPV 77.0% for dermatologic adverse events) and has lower accuracy for events leading to hospitalization. In certain scenarios, estimates derived from the current claims definitions may be insufficient, and algorithms that include other components of insurance claims data should be explored to further refine the algorithm. Alternatively, the outcome definitions could be widened to include all occurrences of the events that result in healthcare services.