Disparities in preventive procedures: comparisons of self-report and Medicare claims data

Racial and ethnic disparities in preventive procedures have been widely documented [1–9]. These findings are based primarily on either patient self-report or the use of claims data. Each has limitations. Self-report is associated with significant overestimation of rates of preventive procedures in most [10–17], but not all studies [18, 19]. In contrast, claims data may underestimate procedures [20]. The size of disparities in mammography differs depending on survey question wording [21, 22], but more importantly, on whether self-report or claims data are used [23]. Such discrepancies could represent either greater over-reporting by minorities of preventive procedures relative to majorities or less efficient Medicare billing procedures by providers servicing minority patients.

Determining whether different data sources yield disparate estimates of racial and ethnic disparities in preventive procedures is relevant to national monitoring of these disparities. A finding of similar disparities regardless of data source reinforces use of self-report data by the National Healthcare Disparities Report or for tracking progress towards Health People 2010 objectives. Discrepant findings suggest the need for further research to determine which is the more reliable data source for assessing disparities. Assessment of potential discrepancies is also critical to understanding disparities in health. Self-report data from the National Health Interview show small racial disparities in mammography [24], but analysis of Medicare claims show significantly larger racial disparities [23]. Significantly, racial disparities in mammography based on Medicare claims have been linked to racial differences in stage at diagnosis [25].

Thus, the primary aim of this study is to determine whether estimates of racial and racial disparities in receipt of six different types of largely preventive procedures differ between self-report and Medicare claims data. To do so, we determine whether minority status, defined as African American or Hispanic (compared to non-Hispanic White) is associated with self-report of procedures in the absence of a claim and vice versa. We also examine whether any such associations are accounted for by other patient characteristics.

Throughout this paper we use the term preventive procedures rather than screening to indicate that while the majority of procedures are likely performed for screening purposes, some are performed for diagnostic or other reasons. Most published studies analyzing self-report data do not distinguish the reason for the procedure, and typically assume the procedure is for screening. Most national self-report surveys (including the Medicare Current Beneficiary Survey used here, the Medical Expenditure Panel Survey, and the Behavioral Risk Factor Surveillance System) do not have a question on the reason for testing. While the National Health Interview Survey currently does include a question on the reason for testing, the information is not used in the published procedure rates used to track the Healthy People 2010 goals [24], or in publications derived from the National Health Interview Survey [3, 26, 27]. Our analyses examine discrepancies between self-report and claims data, so we use an inclusive definition of preventive procedures to enable comparison to other self-report literature.

The comparisons of self-reported and claims-based preventive procedures, by minority status, are shown in Figure 1. Rates of preventive procedures based on self-report are significantly higher than those based on claims, for both minorities and majorities. Absolute differences by minority status (majority vs. minority) for self-report ranged from -2.4% for cholesterol testing to 17.9 % for influenza vaccination. In contrast, differences based on claims data ranged from 5.1% for cholesterol testing to 19.9 % for influenza vaccination.

Overall agreement between self-report and claims was generally lower for minorities. The kappa statistic measures the level of agreement two values with values ranging from "0" for no agreement to "1.0" for perfect agreement. A kappa between self-report and claims data for minorities ranged from 0.19 (for colorectal cancer testing) to 0.58 (for mammography). For majorities, these statistics ranged from 0.37 (for colorectal cancer testing) to 0.70 (for mammography). The different ranges of kappa between majorities and minorities are largely due to higher rates of self-reported preventive procedures in the absence of a claim for minorities.

Rates of a claim in the presence of self-report of non-receipt were appreciably lower than self-report in the absence of a claim. Again, the crude and adjusted odds ratios were similar (Table 3). Minorities were more likely than Majorities to have a PSA test claim in the absence of a self-report (1.55 95% CI: 1.17 – 2.06), but were less likely to have an influenza vaccination claim in the absence of a self-report (0.69 95% CI: 0.51 – 0.93). The remaining odds ratios did not reach statistical significance.

To address the differences in report of true preventive services versus diagnostic services, analyses for PSA testing, mammography, and colorectal cancer testing were recomputed using only true preventive codes [20]. Those models produced results very similar to models described above: similar minority effects were apparent for mammography and colorectal cancer testing; no minority effects were observed for PSA testing.

Previous research has shown larger disparities in mammography between African Americans and majorities when receipt of mammography is based on claims data rather than on self-report [23, 32]. The current study extends these findings to other preventive procedures. With the exception of PSA testing, greater discrepancies were observed between self-reported receipt of preventive care and documentation by Medicare claims among minorities. These effects persisted after controlling for a range of patient characteristics and were observed across different types of preventive procedures. These discrepancies resulted in differing estimates of racial/ethnic disparities in preventive care with generally larger disparities observed using claims than self-report data.

Unfortunately, there is no clear gold standard in this study; we cannot determine whether self-report or a Medicare claim is a more accurate reflection of procedures received. It is not clear whether higher rates of self-report in the absence of a claim represent a greater tendency among minorities to "over-report" across many procedures or whether these findings represent suboptimal Medicare billing by providers who serve minorities.

Each of these explanations has some plausibility. Based on medical record documentation, self-report tends to overestimate actual receipt of preventive procedures partly through underestimation of the time interval since the previous test [12, 14, 15, 17, 33, 34]. Some studies suggest this reporting bias may be greater among minorities than among majorities [13, 14, 35, 36]. Moreover, reports of preventive care by minorities seem to be more sensitive than non-minority reports to survey question wording [21, 22]. Potential explanations for these examples of over-reporting include cultural differences in perception of time elapsed, effects of cross racial/ethnic interviewer-respondent effects [37–39], and social desirability [40–43].

While self-reports overestimate receipt of preventive procedures, administrative billing data may underestimate actual use [16]. Minorities are more likely to been seen by safety net providers such as hospital clinics and community health centers [44]. These providers often receive substantial revenue through Medicaid or Medicare prospective payment system [45, 46]. As a result, there may be less financial incentive among these providers to optimize patient Medicare encounter billing since much of their patient revenue is not as strongly dependent on billing coding for specific visits [47]. Furthermore, greater use of paper submission of claims or slower adoption of Medicare prevention codes [20] could result in higher claim rejection rates by Medicare. This, in turn, could affect data sources that rely on the presence of claims to document receipt of procedures.

There are several ways these different explanations could be distinguished in future research. Rates of self-reported receipt of preventive procedures and Medicare claims data by race or ethnicity could be compared based on chart validation or through other sources of validation. For example, Medicare claims for mammography were recently compared to a mammography registry showing claims data to be moderately sensitive [48]. In addition, studies are needed to compare Medicare billing patterns including payment denial according to the racial composition of the physician's practice. For example, a finding of higher preventive procedure claim denial rates for those physicians' practices having large numbers of minorities would suggest that the use of claims data may be biased for those practitioners.

Distinguishing among these competing explanations has important policy implications. If, on one hand, minorities are more likely to "over-report" receipt of preventive procedures, estimates of racial/ethnic disparities in those procedures based on survey self-report will underestimate the actual magnitude of disparities. This could result in the premature conclusion that disparities in preventive care have been eliminated. For example, self-report survey data have shown that African American women have caught up with majority women in most areas of preventive procedures and actually exceed them in others [8, 49]. Differences in survey wording appear to reduce disparities by reducing reporting bias associated with minority status [21]. Increasing use of electronic health records may eventually minimize reliance on billing and survey data to assess disparities in preventive care.

If, on the other hand, if these discrepancies are primarily driven by inconsistencies in billing, this suggests caution in the use of claims data, not only as a proxy for procedures performed, but for comparisons among providers who serve a greater proportion of minority patients. That is, differential billing efficiency may bias comparisons among different types of providers [50]. Such bias could penalize minority care providers, as many current pay-for-performance systems rely on claims data to assess care and reward performance. Further, failure to successfully bill for procedures could result in decreased payments, and thus compromise resources available to provide quality care.

The limitations of our analyses should be noted. First, the sample was confined to community dwelling, Medicare beneficiaries aged 65 and older, and those not enrolled in managed care. The extent to which these findings generalize to other groups cannot be assessed. Second, colorectal cancer testing survey questions were only asked in one year. As a result, the small sample size limited the power of these analyses. Third, although the findings summarized in Figure 1 suggest a general pattern of larger disparities based on claims compared to self report data, the discrepancies are relatively modest and beyond the power of the study to demonstrate statistically significant difference in differences. Fourth, we used a broad definition of preventive procedures because these data do not allow us to clearly distinguish between screening or diagnostic tests. However, comparable effects to those presented above were found when analyses were repeated using preventive codes only. Fifth, and finally, our selection of procedures might be differentially affected by the systematic lack of claims. For some procedures and selected years, Medicare did not reimburse for screening, e.g., Pap smear testing is reimbursed biennially and reimbursement for PSA screening and colonoscopy did not begin in 2001. As a result, physicians may have billed for screening services using diagnostic codes (which is why we included both codes). However, limitation of analyses to 2001 and/or exclusion of colonoscopy did not appreciably alter the findings.

Embedded in the problem of distinguishing screening and diagnostic testing is a critical issue for monitoring disparities in screening. Some claims-based evidence suggests that disparities for diagnostic testing are narrower than those for screening [5]. Such evidence is consistent with the observation that minority patients present later in the course of cancer [51] (and thus obtain "diagnostic" rather than "screening" tests). Thus, self-report studies that, in general, do not distinguish screening and diagnostic testing may underestimate disparities in screening.

This study shows that estimates of racial/ethnic disparities, across a variety of preventive care procedures, vary depending on whether self-report or claims are used to assess them. Whether these differences reflect biases in participant report or in billing claims is unclear. These competing explanations have profoundly different policy implications, and thus warrant careful study. Future monitoring of disparities in screening will require more careful distinction of screening from diagnostic uses of preventive procedures.