Utilizing over 9500 VA patients with postbronchodilator spirometry we determined that ICD-9 codes have a moderate to good ability to discriminate patients who have fixed airflow obstruction from those who do not with outpatient codes offering better performance than inpatient codes. The addition of a patient's age and pharmacy data including the number of MDIs of albuterol and ipratropium bromide to outpatient and inpatient ICD-9 codes improves the sensitivity and specificity and the overall discriminative performance of a model used to identify patients with airflow obstruction. These variables showed similar performance when utilizing GOLD criteria for airflow obstruction compared to the LLN standard for airflow obstruction.
The use of ICD-9 codes to identify cohorts of patient with COPD using administrative data is common [11–18]. Investigators and payers have utilized these codes to describe the epidemiology of COPD[12, 14, 15, 17, 24, 34–36], to evaluate the effectiveness and safety of treatments in COPD[11, 13, 18, 37, 38], and more recently, as a means to assess the quality of care provided to patients with COPD . In fact, the National Committee for Quality Assurance (NCQA) and the Agency for Health Research and Quality (AHRQ) both advocate for use of quality measures relying on ICD-9 code-based COPD case-identification [39, 40]. It is therefore surprising that the validity of both outpatient and inpatient ICD-9 codes for identifying patients with COPD has not been rigorously studied in large populations.
Most prior efforts to establish the validity of ICD-9 codes for COPD utilize chart review or physician consensus as the gold standard. One of the most widely referenced studies, conducted by Rawson and colleagues, utilized the 1987 Saskatchewan health care data files to assess the validity of inpatient COPD ICD-9 codes compared to both the patient's inpatient medical chart and provider service data . Two hundred patient charts were randomly selected from the 4613 hospitalized patients with a primary ICD-9 code for COPD (n = 496). The charted discharge diagnosis from the patient's medical record showed exact agreement for 94.2% of these patients. However, overall concordance between physician documentation of COPD related care and hospital discharge COPD-related ICD-9 codes (490-493, 496) was 68%. An analysis by Ginde and colleagues utilized a similar approach to determine the positive predictive value for principle ICD-9 codes to identify acute exacerbations of COPD in the emergency department . A random sample of 200 patients was taken from all 644 patients with a code for COPD (491.2x, 492.8, 496) at two academic medical centers between 2005 and 2006. Chart review for these patients was used to establish the gold standard for COPD exacerbation which was defined as: 1) the presence of a respiratory infection, 2) change in cough or 3) change in sputum with known physician diagnosed COPD. The overall positive predictive value for the presence of any of the specified codes was 97%. The positive predictive value for a code of 496 alone was 60% (95% CI 32-84%).
Finally, a more recent study using claims in Ontario, Canada examined the combination of ICD-9 outpatient codes and ICD-10 inpatient codes to identify patients with COPD cared for by community providers . The combination of one or more outpatient ICD-9 codes (491.xx, 492.xx, 496.xx) or one or more inpatient ICD-10 codes (J41, J43, J44) had a sensitivity of 85% and specificity of 78.4% among 113 patients with COPD and 329 patients without COPD. An expert panel reviewed each patient's medical record to determine the gold standard for COPD. Spirometry was available in only 180 patients and details about its collection were not reported in the study. The study was further limited by employing ICD-10 codes which have yet to be universally adopted by many countries around the world.
While these studies outlined above suggest that ICD-9 codes can be used to accurately identify physician defined COPD, none universally employed spirometry to define the criterion standard for COPD. Physician diagnosed COPD may not be the optimal gold standard to define COPD. A number of previous studies highlight the difficulty physicians have in correctly indentifying COPD in the absence of spirometry. In North America only 20-30% of patients billed for a COPD-related visit have had spirometry to confirm or refute the diagnosis of COPD [12, 41–43]. Up to 20% of physicians confronted with a standardized patient in a COPD exacerbation fail to correctly identify COPD as the cause of respiratory complaints . These data raise concerns about the validity of the COPD gold standard used in prior studies examining the use of ICD-9 codes to identify patients.
The only study utilizing primarily spirometry to define COPD compared discrimination between patients with asthma versus patients with COPD. The accuracy of ICD-9 codes demonstrated excellent performance (AUC 0.98) for the calculated ratio of total COPD ICD-9 codes to total respiratory ICD-9 codes to differentiate patients with asthma from patients with COPD; however, this comparison cannot develop models to predict patients with COPD as the comparator was patients with asthma. Finally, unlike our study, which included over 9500 patients, this study was limited by its inclusion of only 151 patients with COPD .
Our study has several strengths. Our gold standard for COPD used the most rigorous definition possible - fixed airflow obstruction on spirometry and captures a large number of patients who had clinical indication for spirometry. This is contrast to many of the previous studies highlighted above.
Our results also have important implications for clinical investigators and health services and health policy analysts. We present the coefficients for a model incorporating administrative variables that can be used to accurately identify patients with COPD. This equation can be used by investigators to calculate the predicted probability of airflow obstruction within novel cohorts. The sensitivity, specificity, positive and negative predictive values for cut points in the model-based predicted probability of airflow obstruction will allow an investigator to maximize sensitivity or specificity depending on the needs of the study practice. For example, one might select a lower cut point (0.25) in the model-based predicted probability of airflow obstruction if utilizing this model to screen a clinical database to identify candidates for a COPD clinical trial. In this situation, maximizing sensitivity would capture the majority of patients with true COPD but at the cost of a large number of false positives. Study staff could access the medical records of these patients to eliminate people without airflow obstruction on spirometry.
We recognize several limitations to our analysis. First, we did not externally validate our model in alternative cohorts of patients. Model performance will likely drop when our model is applied to different patients as a result of geographic and temporal changes, differences in data definitions and case-mix. We assessed the optimism in the estimated AUC for our model utilizing the bootstrap which resulted in no appreciable change in the AUC, but recognize that external validation is a necessary step prior to widespread use . Second, our model was derived on US veterans that were mostly older white men. This may limit the generalizability of our models if applied outside of the VA. In addition, the primary reason for collection of ICD-9 codes in VA patients is not for billing purposes. Differences in coding practice between the VA and other organizations capturing ICD-9 codes primarily for billing purposes may alter the performance of our models if applied outside the VA. Third, some degree of ascertainment bias is likely present, as we were unable to assess clinic visits and hospital admissions to non-VA facilities. Fourth, we collected ICD-9 codes from the one year pre- and one year post the date of spirometry, a time interval that may have reduced the sensitivity and specificity of the codes for COPD. For example, a provider may provide a COPD code on initial evaluation only to learn that spirometry rules out the diagnosis of COPD. Nevertheless, we believe the time interval we used is appropriate because it approximates how ICD-9 codes are screened in observational research and provides a conservative estimate of their performance.
Finally, we limited our cohort to patients referred for spirometry who received a bronchodilator during their test. This was done to ensure that we had a rigorous gold standard by which we defined COPD, but may limit the applicability of our model to only patients who are clinically referred for spirometry. Given the high prevalence of COPD in this population, and the VA more generally , the positive predictive value of our model will decrease if applied to a broader population. Several studies suggest that the prevalence of physiologically determined COPD is closer to 10-20%[7, 9, 25], which is considerably lower than the 48% prevalence observed in our sample. By limiting our analysis to only patients referred to spirometry we provide a conservative estimate of the models performance if applied to a general population. Discriminating patients with COPD from those without COPD among patients who are ill enough to be referred to spirometry is likely a more difficult task than discriminating COPD patients from those without COPD among all patients in a general population. Nevertheless, the estimates of the positive and negative predictive values will change when applying our model to cohorts with different COPD prevalence. Additional testing of our model in broader populations should be done prior to widespread use.