Comparing administrative and survey data for ascertaining cases of irritable bowel syndrome: a population-based investigation

Both administrative and survey data are used to conduct population-based health services and outcomes research about chronic conditions such as diabetes, hypertension, arthritis, and osteoporosis [1–5]. A primary concern when using administrative data for studies of chronic conditions is the accuracy and completeness of diagnostic information [6, 7], while recall bias is a key concern for self-report data. These limitations of both data sources may result in discrepant research findings, yet only a few studies have compared administrative and survey data for ascertaining disease cases.

Okura et al. [8] observed good agreement between survey and administrative data for ascertaining cases of diabetes, hypertension, myocardial infarction, and stroke. Estimates of agreement were substantially lower for heart failure. Rector et al. [9] found that the sensitivity of administrative data, when compared to survey data, was very good for identifying cases of hypertension and diabetes, but lower for arthritis and heart failure. Other studies have found moderate to good agreement between administrative and survey data [4, 10–12]. While agreement tends to be highest for well-defined chronic conditions, such as diabetes, it also depends on the case-ascertainment methodology applied to administrative data [11, 12]. For example, Rector et al. [9] found that the number of years of data used to identify chronic disease cases could have a substantial impact on the sensitivity of administrative data.

Recently, administrative data have been investigated as a potential data source for health services and outcomes research about irritable bowel syndrome (IBS) [13]. Population-based investigations about IBS have primarily been conducted using survey data [14–16]. IBS is a common non-inflammatory gastrointestinal condition; the prevalence in North America is estimated to be between 7% and 15% [15–17]. Individuals with IBS have higher health care use and costs than their non-IBS counterparts [18, 19]. IBS is a difficult condition to investigate in population-based research because there is no single diagnostic test to confirm disease presence and the symptoms of IBS may also be associated with other conditions, including infections. However, when compared with clinical data or medical charts, diagnoses in administrative data have shown good specificity for ascertaining IBS cases [20, 21].

This study compares administrative and survey data for ascertaining cases of IBS. The objectives are to: (1) estimate agreement between the two data sources for IBS and compare the results to those for inflammatory bowel disease (IBD), a gastrointestinal condition with well-defined diagnostic criteria in comparison with IBS; (2) compare the frequency of IBS-related diagnoses in administrative data for survey respondents with and without self-reported IBS, and (3) estimate IBS prevalence from both sources.

A retrospective cohort study was undertaken using administrative and survey data from the province of Manitoba, Canada, which has a population of approximately 1.2 million. The Research Data Repository housed at Manitoba Centre for Health Policy (MCHP) contains administrative data provided by the provincial health ministry. The Repository also houses population-based survey data from a national health survey, the Canadian Community Health Survey (CCHS; http://www.statcan.gc.ca/concepts/health-sante/cycle3_1/index-eng.htm) and the two sources can be directly linked via a unique, anonymized personal health identification number (PHIN). The University of Manitoba Health Research Ethics Board approved the conduct of this research and the Manitoba Health Information Privacy Committee approved the data linkage.

The survey data were collected between January and December 2005. The CCHS was developed to provide cross-sectional estimates of health status, health determinants, and health system use for a target population of individuals 12 years of age and older living in private dwellings. The survey does not include individuals living on Indian Reserves and other government-owned land, institutional residents, and full-time members of the Canadian Forces; these groups represent approximately 2% of the Manitoba population. The Manitoba response rate was 83.3%; the national response rate was 78.9%.

Sources of administrative data were records of inpatient hospitalizations and outpatient physician billing claims. Manitoba, like other Canadian provinces, has a universal health system; almost all Manitoba residents are covered under the Manitoba Health Services Insurance Plan (MHSIP). A hospitalization record is completed upon patient discharge. Each record includes up to 16 diagnosis codes from the International Classification of Diseases, 9^th Revision, Clinical Modification (i.e., ICD-9-CM) prior to April 1, 2004 and up to 25 diagnosis codes from the 10th Revision for this date onward (i.e., ICD-10-CA). Procedure codes are also captured in hospitalization records. Physicians paid on a fee-for-service basis submit billing claims to the provincial health ministry; these claims capture virtually all outpatient services, including those for hospital emergency departments and outpatient departments. While some physicians are salaried (e.g., approximately 7% of family physicians) [22], it has been estimated that approximately 90% of these physicians submit parallel billing claims for administrative purposes. Each visit results in one billing claim that contains a single ICD-9-CM code. ICD-9 codes in physician data are recorded to the third digit, while codes in hospital data are recorded using up to five digits.

Overall, 152 (3.0%) members of the study cohort reported having IBS, another 0.8% reported having CD or UC, and 95.3% reported having no bowel disorder. Another 49 respondents, who indicated they had bowel incontinence or another, unspecified bowel disorder, were excluded from the analysis.

Almost one-quarter of IBS cases had a diagnosis of IBD in physician claims data up to three years before the survey interview date, but in the year immediately before or after the interview date the number of cases with this diagnosis was too small to analyze. Among IBS cases, the most frequent IBS-related diagnoses recorded in physician billing claims in the three-year period before the survey interview date were anxiety disorders (34.4%), symptoms involving the abdomen and pelvis (26.9%), and diverticulitis of the intestine (10.6%). In the one-year period following the interview date the most common IBS-related diagnoses were anxiety disorders (20.7%), symptoms involving the abdomen and pelvis (10.2%), and symptoms involving the urinary system (4.2%). These were also the most common IBS-related diagnoses among non-cases, but the percentages were significantly different (p < 0.001 for all tests) for the two groups.

The crude IBS prevalence estimate from survey data was 3.00 per 100 (95% CI: 2.37, 3.76). Using administrative data, the crude prevalence estimate was 1.71 per 100 (95% CI: 1.69, 1.74) for a one-year case-ascertainment algorithm and 4.23 per 100 (95% CI: 4.19, 4.28) for a three-year algorithm. The number of IBS cases identified from hospitalization data was small; for the former algorithm, 1.74% of cases were identified solely from hospital data and for the latter algorithm the corresponding figure was 0.43% of cases.

Both administrative and survey data resulted in higher prevalence estimates for females than for males (Figure 1); the ratio was 4.27 for survey data, 2.14 for the one-year algorithm and 2.08 for the three-year algorithm. The analysis by age group (Figure 2) revealed that for the two youngest age groups, prevalence estimates from administrative and survey data were not significantly different (p ≤ .10 for both administrative data algorithms). For the oldest age group, they were significantly higher using administrative data than survey data.

This study compared population-based administrative and survey data for IBS diagnoses. The results show that agreement between administrative and survey data for IBS was low, but it was much higher for IBD. Agreement was investigated for two periods of time both before and after the survey interview date. Agreement between administrative and survey data remained low regardless of the size or direction (i.e., pre-interview versus post-interview) of the case-ascertainment window.

Compared with survey respondents who reported having no bowel disorder, respondents with a self-reported diagnosis of IBS were more likely to have diagnoses in administrative data for symptoms of the digestive system and abdomen, other gastrointestinal conditions including IBD, selected procedures, and an anxiety disorder. These findings are consistent with previous research, which has shown that that IBS patients have an increased likelihood of diagnosis for a variety of gastrointestinal, genitourinary, and psychological conditions when compared with the general population [21, 27]. Given that IBD and IBS may have similar symptoms, an increased frequency of diagnosis for IBD may be indicative of physicians attempting to "rule out" the presence of this condition in IBS patients [23]. A recent Canadian study [28] found that physicians are accurate in their use of ICD-9 diagnosis codes for Crohn's disease and ulcerative colitis in billing claims. Thus, the increased frequency of IBD diagnosis among IBS patients does not appear to be due to inaccuracies in diagnoses.

IBS prevalence estimates have been reported to be two to three times higher among females than males [17], which is consistent with the findings observed in this study for both data sources. However, crude estimates in this study were still lower than those reported in previous North American research [14, 16, 17]. This may be due to a number of factors, including the lack of specificity of the survey questions about bowel disorders or the unfamiliarity of Manitoba physicians with diagnostic criteria for IBS. However, there is substantial international variation in IBS prevalence estimates, with some non-North American counties reporting values as low as 3 and 5 per 100 [29]. The estimates obtained from administrative data were higher for the oldest age group than for the younger two age groups, which is not consistent with the estimates obtained from survey data nor with the estimates reported in previous research, which show declining IBS prevalence with age [15]. This may be a result of some loss of specificity for ascertaining IBS cases in physician data because diagnoses in Manitoba's physician data are only recorded to the third digit; very few IBS cases were identified from hospital data, in which diagnoses are recorded with greater specificity. This finding may also be a result of increased reporting among older adults of a variety of symptoms and/or an increased likelihood of inaccurate assignment of an IBS diagnosis in older adults [30].

Well-defined chronic conditions are more likely to result in good agreement between administrative and survey data [8, 31]. The Rome criteria have been developed to provide physicians with a systematic methodology to classify individuals with functional gastrointestinal disorders, including IBS [32]. However, based on the results of medical chart review, Goff et al. [20] reported that the criteria are infrequently used by physicians to establish a diagnosis of IBS. Wilson et al.[16] suggest that the use of the Rome criteria by physicians may result in underestimation of IBS prevalence in population-based research that uses medical records.

One limitation of this study is that it was not possible to compare clinical data with the administrative data or survey data. As well, the survey data do not contain information about date of diagnosis, which would have been useful for decisions about the number of years of administrative data needed to ascertain disease cases. Some diagnoses occurred infrequently in administrative data and therefore it was not possible to test for differences between survey respondents with and without IBS. Finally, as noted previously, only three-digit ICD-9 codes are available in physician claims data in Manitoba, resulting in loss of specificity to ascertain IBS cases. This situation is not unique in Canada; physician data in Ontario, the largest province in Canada, is also limited to three-digit diagnosis codes [33]. However, there is a lack of consistency across jurisdictions in the way diagnoses are recorded in physician data. For example, ICD-9 codes in Medicare physician data from the United States may be recorded using up to five digits [9].

Given the low agreement between survey and administrative data and evidence that this disease appears to be under-reported in both data sources, it is important to undertake further research that can improve IBS case ascertainment in population-based data. Thompson et al. [32] suggest that imprecision in the wording of survey questions contributes to inaccuracies in case ascertainment. Wilson et al. [16] recommended that survey questions based on the Rome criteria be supplemented with additional questions to identify patients formerly diagnosed with IBS but with symptoms currently under control. Thus, future research could focus on comparing agreement between administrative and survey data when different question wording methods are adopted. For administrative data, techniques that do not rely exclusively on a single diagnosis code could be used to improve ascertainment results for IBS. Machine-learning and statistical classification models, including latent class analysis and neural networks, which have been applied to chronic conditions where there may be low sensitivity associated with using a single diagnosis code to ascertain disease cases [12, 34, 35], might be applied to this problem.

Population-based chronic disease research can provide important information about the effectiveness of disease treatment and management initiatives and health promotion and disease prevention strategies. The quality of this research depends, in part, on the accuracy and validity of chronic disease case ascertainment methods. In both data sources, the use of a single methodology for identifying disease cases may result in missed cases. Finally, researchers who rely on either administrative data or survey data to conduct population-based studies about IBS should recognize that lack of comparability between the two data sources will be an important confounder of their results.