Skip to main content
Download PDF

The impact of multimorbidity level and functional limitations on the accuracy of using self-reported survey data compared to administrative data to measure general practitioner and specialist visits in community-living adults

BMC Health Services Research volume 21, Article number: 1123 (2021) Cite this article

Abstract

Background

Researchers often use survey data to study the effect of health and social variables on physician use, but how self-reported physician use compares to administrative data, the gold standard, in particular within the context of multimorbidity and functional limitations remains unclear. We examine whether multimorbidity and functional limitations are related to agreement between self-reported and administrative data for physician use.

Methods

Cross-sectional data from 52,854 Ontario participants of the Canadian Community Health Survey linked to administrative data were used to assess agreement on physician use. The number of general practitioner (GP) and specialist visits in the previous year was assessed using both data sources; multimorbidity and functional limitation were from self-report.

Results

Fewer participants self-reported GP visits (84.8%) compared to administrative data (89.1%), but more self-reported specialist visits (69.2% vs. 64.9%). Sensitivity was higher for GP visits (≥90% for all multimorbidity levels) compared to specialist visits (approximately 75% for 0 to 90% for 4+ chronic conditions). Specificity started higher for GP than specialist visits but decreased more swiftly with multimorbidity level; in both cases, specificity levels fell below 50%. Functional limitations, age and sex did not impact the patterns of sensitivity and specificity seen across level of multimorbidity.

Conclusions

Countries around the world collect health surveys to inform health policy and planning, but the extent to which these are linked with administrative, or similar, data are limited. Our study illustrates the potential for misclassification of physician use in self-report data and the need for sensitivity analyses or other corrections.

Peer Review reports

Background

Accurate healthcare utilization data are essential for making evidence-informed decisions for planning, prioritization, and health policy development. Data on physician visits, the most frequently used health service [1], commonly come from claims-based administrative and self-report sources [2]. Administrative data, particularly in jurisdictions with a single or large payer, are generally considered the reference standard for measuring health service use since they are typically complete and not vulnerable to concerns like recall bias. However, there are limitations in terms of access and population coverage. For example, in Canada most administrative data are available only at the province-level. In the United States, Medicare data are national but include only a subgroup of the population. Further, administrative data have limited information on the social determinants of health, physical function, symptoms, and other factors associated with healthcare utilization [3]. Large population-based health surveys, (such as the Canadian Longitudinal Study on Aging, Health and Retirement Study, and others collected in over 100 countries [4]) often include in-depth measures of these factors as well as self-report healthcare use data, which means they can be used to study the impact of a larger variety of variables on service use.

Multimorbidity has emerged as one of the greatest challenges facing healthcare [5, 6] and reliable estimates of its impact on physician use are key for planning. While authors have examined how well self-reported physician visits predict use based on administrative data, the results have been mixed [7]. This is not unexpected given the complexity of operationalizing multimorbidity and the potential for different kinds of conditions, such as symptomatic ones, to differentially impact on patient-important outcomes [8]. A systematic review demonstrated that people increasingly under-report their physician use relative to administrative data as their actual use increases [7] but research looking at the effect of multimorbidity, which is strongly associated with use, have found both under- and over-reporting [9,10,11,12]. No studies have examined the influence of multimorbidity on the accuracy of self-reported physician use, or if this might differ by the impact of those conditions on functional limitations. Globally, few of the over 100 population-based surveys collected worldwide are linked to administrative data [4, 13] meaning that efforts to study the effect of the socio-demographic, social, functional, or other variables on physician use most often rely solely on self-reported visits. However, if there is disagreement between data sources, and that disagreement is related to level of multimorbidity and/or functional limitations this could lead to biased estimates [14]. We undertook this study to: 1) Estimate agreement between self-reported physician visits and administrative data, 2) Examine whether agreement differs by level of multimorbidity and functional limitation, and 3) Examine whether any differences in agreement by level of multimorbidity and functional limitation may be explained by age and sex.

Methods

Study design and setting

This is a cross-sectional study in which we used population-based self-reported questionnaire and administrative data from Ontario, Canada’s most populous province [15]. Retrospective administrative data are used to match the timeframe (in the past 12 months) of the self-reported physician use questions. We followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline and the Reporting of Studies Conducted using Observational Routinely-Collected Health Data (RECORD) statement guidelines [16, 17].

Data sources

General practitioner (GP) and specialist physician visits were obtained from provincial physician billing claims, which are generated for physician reimbursement, but also are used regularly for research and have been studied extensively for their validity [18].

Self-report data GP and specialist physician visits, chronic conditions, and functional limitations were obtained from the population-based Canadian Community Health Survey (CCHS). The CCHS collects information on health-related data at 2-year intervals from a random sample of community-living individuals 12 years or older. It covers approximately 97% of the target population in Canada and typically has response rates of > 70% [19]. Excluded from the CCHS sampling frame are persons living on First Nations reserves or Crown lands, in institutions, full-time members of the Canadian Forces, and residents of some remote regions. We pooled data from three CCHS cycles with consistent questions (2005–2006, 2007–2008, and 2009–2010) to increase the sample size.

For participants who consented (approximately 78% across the three cycles), CCHS data were linked to administrative data using unique encoded identifiers based on their CCHS participation date and analyzed at ICES. ICES is an independent non-profit research institute whose legal status under Ontario’s health information privacy laws allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement. The study received approval from the Hamilton Integrated Research Ethics Board at McMaster University (certificate # 13–590).

Study participants

Of the 101,749 Ontario CCHS participants who agreed to linkage with administrative data, 54,893 were aged 45 years or older. We focussed on middle-aged and older adults as they have a higher prevalence of multimorbidity and are the focus of most multimorbidity research. We excluded participants receiving palliative care (n = 223) or residing in long-term care (124), had no healthcare contact in the previous 5 years (n = 68), were non-residents (n = 64), or ineligible for the Ontario Health Insurance Plan (n = 50). We further excluded participants with missing CCHS data on physician utilization, chronic conditions, or functional limitations (n = 1100), and non-matching age information between data sources (n = 136). If an individual was included in more than one CCHS cycle, we chose the first (n = 274). The final sample included 52,854 individuals.

Measures of healthcare utilization

CCHS participants were asked if they had seen a family doctor or GP about their physical, emotional, or mental health in the past 12 months. If they responded “yes”, they were asked how many times. They were asked questions regarding specialists which were described as “any other medical doctor such as a surgeon, allergist, orthopaedist, or psychiatrist”. The 12-month timeframe is used in many population-based surveys in Canada, the United States (Health and Retirement Study and National Health Interview Survey) as well as the Survey of Health, Ageing and Retirement in Europe (SHARE) studies which were conducted in 28 European countries and Israel.

We compared the self-reported GP and specialist visits with those recorded in administrative data in the 12 months preceding the CCHS interview. To make the administrative data sources more comparable to the self-reported, we excluded billing for specialists who typically do not meet with patients (e.g., diagnostic radiologists). For both GP and specialist visits, we counted multiple billings within the same day as a single visit. We did not know if CCHS respondents reported their physician visit count as only those in outpatient clinic visits or if they included inpatient care. To assess this, we conducted sensitivity analyses in which we excluded specialist visits that occurred during hospital stays. We did not look at agreement on hospital stays, even though they are of high interest. CCHS participants were asked about overnight stays in hospital in the past 12 months, but the question does not allow us to distinguish between true hospital admissions and overnight stays in emergency departments or observation units.

Multimorbidity and functional limitations

Multimorbidity was operationalized using 12 self-reported chronic conditions: Alzheimer’s diseases/dementia, anxiety/depression, arthritis, asthma, cancer, chronic obstructive pulmonary disease, diabetes, heart disease, hypertension, inflammatory bowel disease, stomach or intestinal ulcers, and stroke. These conditions were chosen because they are available in both data sources, prevalent in middle-aged and older adults [20], frequently reported in the multimorbidity literature [21], and consistent with our previous work [22,23,24]. In the CCHS, respondents were asked: “Has a doctor ever told you that you have [condition]?” and to consider conditions that lasted or were expected to last at least 6 months. Multimorbidity was defined as the sum of chronic conditions (0, 1, 2, 3, 4+). Having a functional limitation (yes or no) was defined as needing help with any of the following activities of daily living: preparing meals, appointments and errands, housework, personal care, moving inside house, and personal finances. While functional limitations data were only available in the CCHS, we chose to use multimorbidity based on self-report because most surveys are not linked to administrative data and we wanted to examine how in these cases measurable factors were associated with agreement.

Covariates

Sex and age at CCHS interview date (categorized as: 45–54, 55–64, 65–74, and 75+) were identified using administrative data.

Statistical analysis

We described demographics, multimorbidity, functional limitations, the average number of visits and the percent of participants reporting any GP and any specialist visit in the past 12 months based on self-report and administrative data both overall and by level of multimorbidity. For both GP and specialist visits, agreement between the two data sources on “any physician visit in the past 12 months” was measured by overall agreement (the percent whose self-report utilization in the past 12 months matched the administrative data), sensitivity (the percent with at least one physician visit in the past 12 months in administrative data who were correctly identified as having at least one physician visit based on self-report), and specificity (the percent of participants without a physician visit in the past 12 months who also self-reported no physician visits); administrative data was the reference. Sensitivity and specificity, with 95% confidence intervals, were calculated by level of multimorbidity and for those with and without functional limitations. We stratified by age and sex to examine for confounding and effect modification of observed patterns, given that both variables are associated with healthcare use [25], multimorbidity [26], and functional limitations [27]. A complete-case unweighted analysis was conducted using SAS 9.4 [28] as there was missing self-reported data for < 2% of eligible CCHS participants and CCHS weights are not available for the linkable subset of participants in Ontario .

Results

We included 52,854 individuals aged 45 years or older who met our inclusion criteria (Supplemental Fig. 1). Of these, 29,593 (56.0%) were female and 22,839 (43.2%) were over age 65 (Table 1). Overall, 13,206 (25.0%) participants had no chronic conditions, 15,027 (28.4%) had one, and 24,621 (46.5%) had 2 or more; 8438 (16.0%) had functional limitations. The distribution of all characteristics was similar across the three CCHS cycles.

Table 1 Prevalence of demographic characteristics, number of chronic conditions, and functional limitations based on self-report for 52,854 Ontario Participants 45 years or older of the Canadian Community Health Survey Cycles 3–5
Full size table

Table 2 presents the data on GP and specialist visits in the past year based on self-report and administrative data. Compared to administrative data, participants under-reported both GP (mean 3.65 vs. 6.25) and specialist visits (mean 2.00 vs. 3.65) and the magnitude of the difference increased with the level of multimorbidity. Although the magnitude of the difference was smaller, a similar trend of increased under-reporting of specialist visits was still found when we excluded specialist visits that occurred during a hospital stay (data not shown). Compared to administrative data, participants under-reported having any GP visits (84.8% vs. 89.1%) but over-reported specialist visits (69.2% vs. 64.9%). Overall agreement, sensitivity and specificity were higher for GP visits compared to specialist visits. The percent visiting a GP and specialist increased with level of morbidity and the absolute difference between self-report and administrative data decreased. The overall agreement and sensitivity for both service types increased with level of multimorbidity but specificity decreased to a greater degree, especially for GP visits (0 vs. 4+ CCs: sensitivity 84.4% (83.7, 85.1%) vs. 95.7% (95.1–96.2%); specificity 71.7% (70.2, 73.3%) vs. 27.3% (19.3, 35.2%)).

Table 2 Number of GP and Specialist Visits Based on Self-Report and Administrative Data Sources by Number of Chronic Conditions
Full size table

In Fig. 1, we present the sensitivity and specificity of any GP and specialist visits based on self-report compared to administrative data by level of multimorbidity. The two lines represent participants with (squares) and without (circles) functional limitations. Regardless of the health service type, sensitivity increases and specificity decreases with level of multimorbidity with a similar pattern for people with and without functional limitations. There was some indication that people with functional limitations had slightly higher sensitivities for GP visits but the relationship was not consistent for specialist visits. There were no consistent differences in specificity for either GP or specialist visits.

Fig. 1
figure 1

Sensitivity and Specificity of Any GP and Specialist Visit in the Previous 12 Months Based on Self-Report Compared to Administrative Data Stratified by Functional Limitation Status

Full size image

We present sensitivity and specificity by level of morbidity stratified by age (Fig. 2a) and sex (Fig. 2b) with the overall line in red. Overall sensitivity was higher for GP visits (90% or higher for all levels of multimorbidity) compared to specialist visits (ranging from approximately 75% for 0 to 90% for 4+ chronic conditions). Specificity levels for self-report tended to start higher for GP visits than specialist visits but decreased more swiftly with multimorbidity level; in both cases specificity levels fell below 50%. There was some evidence that younger age groups and males had higher specificity for specialist visits, but the patterns across level of multimorbidity persisted indicating that the relationship is not likely due to confounding and that age and sex are not strong effect modifiers.

Fig. 2
figure 2

a b. Sensitivity and Specificity of Any GP and Specialist Visit in the Previous 12 Months Based on Self-Report Compared to Administrative Data Stratified by Age (2a) and Sex (2b)

Full size image

Discussion

We sought to understand agreement between self-report and administrative data by level of multimorbidity. We found the average number of physician visits based on both data sources increased with level of multimorbidity, but self-report tended to underestimate physician use compared to administrative data. For both GP and specialist visits, the percent of participants with at least one encounter increased with the level of multimorbidity as did the sensitivity of self-reported data. In contrast, the specificity of self-reported data decreased with level of multimorbidity, especially for GP visits. The pattern in sensitivity and specificity did not differ greatly by functional limitations, but sensitivity was slightly higher for GP visits in those with functional limitations. Sex and age did not appear to be strong confounders or effect modifiers.

Agreement on the number of physician visits

Prior studies [9, 11, 12, 29,30,31] generally found physician visits were under-reported compared to administrative data, but most combined GP and specialist visits. Of those that examined GPs and specialists separately, Raina et al. [29], who looked at individuals ≥ 65 years, found under-reporting for both GP and specialist visits while Peersman et al. [12], who looked at individuals ≥25 years, found under-reporting of GP visits but no difference in specialist visits. In a systematic review, Bhandari and Wagner [7] found under-reporting utilization was positively associated with increased utilization, thus under-reporting could be smaller in studies with younger adults, who tend to have fewer specialist visits. Our finding that under-reporting increased with multimorbidity, which is associated with age, supports this.

Agreement on any physician visits

Our results for overall agreement and the relative under-reporting and over-reporting of “any” GP and “any” specialist visit mirrored Raina’s [29] and Peersman’s [12] results; however, Raina did not find a relationship between multimorbidity and discrepant reporting, which may have been because under- and over-reporting were treated as single “any disagreement” category due to sample size. While we found that sensitivity increased with the level of multimorbidity and specificity decreased to a greater extent, we did not find that these patterns differed by presence of functional limitations. Previous studies on agreement between healthcare utilization and functional limitations report mixed results [9, 11, 12]. We found some evidence for higher sensitivity (fewer false negatives) for self-reported GP visits in participants with functional limitations compared to those without, however this is likely no clinically meaningful, especially for those with two or more chronic conditions since sensitivity was greater than 90% in both groups.

Older age is one of the few sociodemographic factors that is consistently associated with under-reporting [7]. Sex, although not consistently found to impact on agreement, is associated with both the prevalence and nature of multimorbidity and the healthcare use experience [3]. We did not observe attenuation or consistent differences in the patterns of sensitivity and specificity across level of multimorbidity by age or sex suggesting that neither factor explains our findings.

Limitations

We included 12 conditions in our definition of multimorbidity because we restricted to those available across all 3 CCHS cycles. Furthermore, our measure of functional limitations is fairly crude (any vs. none) and mainly focussed on instrumental rather than basic activities of daily living. Our finding of similar patterns in agreement between data sources for those with and without functional limitations could reflect that our measure did not adequately capture the phenomenon. Although we found differences in the relationship between chronic conditions and functional limitations using this operationalization in our previous work [32] future studies could examine more nuanced definitions of functional limitations when examining patterns of agreement. As well, we report only on the number of GP and specialist visits and not the need for these visits, whether health issues were addressed, or continuity of care. While much healthcare utilization research focuses on acute care, physician visits are the most common way in which people with multiple chronic conditions interact with the healthcare system. Finally, although there appears to be an association with increased levels of multimorbidity and agreement on healthcare utilization, we can not speculate as to the mechanism based on these data.

In conducting our study, a number of issues emerged that suggest that comparing administrative and self-report data is far more complex than would appear on the surface. For example, there is billing by physicians for services that patients do not see like diagnostic radiologists. In our analyses we also did not include hospitalizations because the language in the survey does not reflect how the information is captured in administrative data. This highlights the need for conceptual clarity when comparing across data sources to ensure that we are really asking the same thing from each. Finally, much of what we seek to understand is the burden of multiple chronic conditions – whether from the patient, provider, or system perspective – but frequently the full scope of care providers are not captured in administrative data (e.g. social workers or physiotherapists) while surveys rarely define these provider roles or capture the intensity of use. Neither data source on its own represents the totality of experience and, as is, can often be difficult to reconcile. Future surveys intended to link with or complement administrative data should consider how to ask questions that can be harmonized.

Conclusions

Our agreement results may be generalizable to other population-based studies of community-living older adults as many have similar questions to assess chronic conditions and healthcare utilization, however patterns of utilization may vary among different healthcare settings. Both self-reported population-based data and administrative data are essential to understand healthcare utilization and its drivers, but globally few countries have access to population-based survey data linked with administrative data. While these data sources should complement one another, we found that they were not necessarily well-aligned. The results of our study can be used to better understand the potential impact of misclassification when using self-report data to measure physician visits. We found that among individuals with higher levels of multimorbidity who had no administrative record of seeing a physician in the prior year, a higher proportion self-reported having a seen a GP or specialist (false positive) than not (true negatives). This could attenuate estimates of association between multimorbidity and healthcare utilization based on self-report. Yet, large population-based surveys, like the CCHS, are vital for understanding drivers of service use because they capture a depth and breadth of data not available in administrative data. Although our data reflect the Canadian context, our findings illustrate the potential impact of misclassification in studies using self-reported physician utilization data and may guide strategies to address measurement error, as recommended by the STRengthening Analytical Thinking for Observational Studies (STRATOS) guidelines [33, 34] and used as a starting point for sensitivity analyses in other studies.

Availability of data and materials

ICES is a prescribed entity under the Ontario Personal Health Information Protection Act. As such, ICES policies and procedures are approved by Ontario’s Information and Privacy Commissioner. These policies require that access to data be limited to persons who require such access to perform their role on an approved ICES Project or Third-Party Project. Thus, we are prohibited from making ICES data publicly available. Only the results of analysis of ICES data may be made available. Researchers can contact das@ices.on.ca for data access.

Abbreviations

CCHS:

Canadian Community Health Survey

GP:

General Practitioner

STRATOS:

STRengthening Analytical Thinking for Observational Studies

References

  1. Griffith LE, Gruneir A, Fisher K, Panjwani D, Gafni A, Patterson C, et al. Insights on multimorbidity and associated health service use and costs from three population-based studies of older adults in Ontario with diabetes, dementia and stroke. BMC Health Serv Res. 2019;19(1):313. https://doi.org/10.1186/s12913-019-4149-3.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Lehnert T, Heider D, Leicht H, Heinrich S, Corrieri S, Luppa M, et al. Health care utilization and costs of elderly persons with multiple chronic conditions. Med Care Res Rev. 2011;68(4):387–420. https://doi.org/10.1177/1077558711399580.

    Article  PubMed  Google Scholar 

  3. Vegda K, Nie JX, Wang L, Tracy CS, Moineddin R, Upshur RE. Trends in health services utilization, medication use, and health conditions among older adults: a 2-year retrospective chart review in a primary care practice. BMC Health Serv Res. 2009;9(1):217. https://doi.org/10.1186/1472-6963-9-217.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Macfarlane SB. National Household Surveys: Collecting Data Where People Live. In: Macfarlane SB, AbouZahr C, editors. The Palgrave Handbook of Global Health Data Methods for Policy and Practice. London: Springer; 2019. p. 145–64.

    Chapter  Google Scholar 

  5. Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380(9836):37–43. https://doi.org/10.1016/S0140-6736(12)60240-2.

    Article  PubMed  Google Scholar 

  6. Kone Pefoyo AJ, Bronskill SE, Gruneir A, Calzavara A, Thavorn K, Petrosyan Y, et al. The increasing burden and complexity of multimorbidity. BMC Public Health. 2015;15(1):415. https://doi.org/10.1186/s12889-015-1733-2.

    Article  PubMed Central  Google Scholar 

  7. Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Medical Care Research & Review. 2006;63(2):217–35. https://doi.org/10.1177/1077558705285298.

    Article  Google Scholar 

  8. Griffith LE, Gilsing A, Mangin D, Patterson C, van den Heuvel E, Sohel N, et al. Multimorbidity frameworks impact prevalence and relationships with patient-important outcomes. J Am Geriatr Soc. 2019;67(8):1632–40. https://doi.org/10.1111/jgs.15921.

    Article  PubMed  Google Scholar 

  9. Ritter PL, Stewart AL, Kaymaz H, Sobel DS, Block DA, Lorig KR. Self-reports of health care utilization compared to provider records. J Clin Epidemiol. 2001;54(2):136–41. https://doi.org/10.1016/S0895-4356(00)00261-4.

    Article  CAS  PubMed  Google Scholar 

  10. Petrou S, Murray L, Cooper P, Davidson LL. The accuracy of self-reported healthcare resource utilization in health economic studies. Int J Technol Assess Health Care. 2002;18(3):705–10. https://doi.org/10.1017/S026646230200051X.

    Article  PubMed  Google Scholar 

  11. Wolinsky FD, Miller TR, An H, Geweke JF, Wallace RB, Wright KB, et al. Hospital episodes and physician visits: the concordance between self-reports and Medicare claims. Med Care. 2007;45(4):300–7. https://doi.org/10.1097/01.mlr.0000254576.26353.09.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Peersman W, Pasteels I, Cambier D, De MJ, Willems S. Validity of self-reported utilization of physician services: a population study. Eur J Pub Health. 2014;24(1):91–7. https://doi.org/10.1093/eurpub/ckt079.

    Article  Google Scholar 

  13. Manuel DG, Perez R, Sanmartin C, Taljaard M, Hennessy D, Wilson K, et al. Measuring burden of unhealthy behaviours using a multivariable predictive approach: life expectancy lost in Canada attributable to smoking, alcohol, physical inactivity, and diet. PLoS Med. 2016;13(8):e1002082. https://doi.org/10.1371/journal.pmed.1002082.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Shaw PA, Deffner V, Keogh RH, Tooze JA, Dodd KW, Kuchenhoff H, et al. Epidemiologic analyses with error-prone exposures: review of current practice and recommendations. Ann Epidemiol. 2018;28(11):821–8. https://doi.org/10.1016/j.annepidem.2018.09.001.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Canada S. Population estimates on July 1st, by age and sex. Statistics Canada. 2019; https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000501&pickMembers%5B0%5D=1.7&pickMembers%5B1%5D=2.1.

  16. Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, et al. The REporting of studies conducted using observational routinely-collected health data (RECORD) statement. PLoS Medicine / Public Library of Science. 2015;12:e1001885.

    Google Scholar 

  17. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, et al. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–8. https://doi.org/10.1136/bmj.39335.541782.AD.

    Article  Google Scholar 

  18. Glazier RH, Gozdyra P, Kim M, Bai L, Kopp A, Schultz SE. Geographic variation in primary care need, service use and providers in Ontario, 2015/16. Toronto, ON: Institute for Clinical Evaluative Sciences; 2018.

    Google Scholar 

  19. Beland Y. Canadian community. Health Rep. 2002;13.

  20. Canadian Institute For Health Information. Seniors And The Health Care System: What Is The Impact Of Multiple Chronic Conditions? https://secure.cihi.ca/estore/productFamily.htm?locale=en&pf=PFC1575. 2011. Ottawa, Ontario, CIHI.

  21. Diederichs C, Berger K, Bartels DB. The measurement of multiple chronic diseases--a systematic review on existing multimorbidity indices. Journals of Gerontology Series A-Biological Sciences & Medical Sciences. 2011;66(3):301–11. https://doi.org/10.1093/gerona/glq208.

    Article  Google Scholar 

  22. Gruneir A, Fisher K, Perez R, Favotto L, Patterson C, Markle-Reid M, et al. Measuring multimorbidity series: an overlooked complexity—comparison of self-report vs. administrative data in community-living adults—paper 1: introduction. J Clin Epidemiol. 2020;124:160–2. https://doi.org/10.1016/j.jclinepi.2020.04.020.

    Article  PubMed  Google Scholar 

  23. Griffith LE, Gruneir A, Fisher K, Upshur R, Patterson C, Perez R, et al. Measuring multimorbidity series—an overlooked complexity comparison of self-report vs. administrative data in community-living adults: paper 2. Prevalence estimates depend on the data source. J Clin Epidemiol. 2020;124:163–72. https://doi.org/10.1016/j.jclinepi.2020.04.019.

    Article  PubMed  Google Scholar 

  24. Gruneir A, Griffith LE, Fisher K, Perez R, Favotto L, Patterson C, et al. Measuring multimorbidity series. An overlooked complexity - Comparison of self-report vs. administrative data in community-living adults: Paper 3. Agreement across data sources and implications for estimating associations with health service use. J Clin Epidemiol. 2020;124:173–82. https://doi.org/10.1016/j.jclinepi.2020.04.018.

    Article  PubMed  Google Scholar 

  25. Violan C, Foguet-Boreu Q, Flores-Mateo G, Salisbury C, Blom J, Freitag M, et al. Prevalence, determinants and patterns of multimorbidity in primary care: a systematic review of observational studies. PLoS One. 2014;9(7):e102149. https://doi.org/10.1371/journal.pone.0102149.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Jindai K, Nielson CM, Vorderstrasse BA, Quinones AR. Multimorbidity and functional limitations among adults 65 or Older, NHANES 2005-2012. Prev Chronic Dis. 2016;13:E151. https://doi.org/10.5888/pcd13.160174.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Rochon PA, Bronskill SE, Gruneir A, Liu B, Johns A, Lo AT et al.. Ontario Women's Health Equity Report. Older Women's Health. Edited by Rochon PA. 2011. Toronto, St. Michael's Hospital and the Institute for Clinical Evaluative Sciences.

  28. SAS/STAT software. [14.1]. 2017. Cary, NC, SAS Institute Inc.

  29. Raina P, Torrance-Rynard V, Wong M, Woodward C. Agreement between self-reported and routinely collected health-care utilization data among seniors. Health Serv Res. 2002;37(3):751–74. https://doi.org/10.1111/1475-6773.00047.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Roberts RO, Bergstralh EJ, Schmidt L, Jacobsen SJ. Comparison of self-reported and medical record health care utilization measures. J Clin Epidemiol. 1996;49(9):989–95. https://doi.org/10.1016/0895-4356(96)00143-6.

    Article  CAS  PubMed  Google Scholar 

  31. Cleary PD, Jette AM. The validity of self-reported physician utilization measures. Med Care. 1984;22(9):1984–803. https://doi.org/10.1097/00005650-198409000-00003.

    Article  Google Scholar 

  32. Griffith LE, Raina P, Levasseur M, Sohel N, Payette H, Tuokko H, et al. Functional disability and social participation restriction associated with chronic conditions in middle-aged and older adults. J Epidemiol Community Health. 2017;71(4):381–9. https://doi.org/10.1136/jech-2016-207982.

    Article  PubMed  Google Scholar 

  33. Keogh RH, Shaw PA, Gustafson P, Carroll RJ, Deffner V, Dodd KW, et al. STRATOS guidance document on measurement error and misclassification of variables in observational epidemiology: part 1-basic theory and simple methods of adjustment. Stat Med. 2020;39(16):2197–231. https://doi.org/10.1002/sim.8532.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Shaw PA, Gustafson P, Carroll RJ, Deffner V, Dodd KW, Keogh RH, et al. STRATOS guidance document on measurement error and misclassification of variables in observational epidemiology: part 2-more complex methods of adjustment and advanced topics. Stat Med. 2020;39(16):2232–63. https://doi.org/10.1002/sim.8531.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We state that we have no one to thank because everyone who meaningfully contributed is included as an author.

Funding

This project was funded through an operating grant from the Canadian Institutes of Health Research (CIHR) (funding reference number: FRN 130546). Dr. Griffith is supported by the McLaughlin Foundation Professorship in Population and Public Health. Dr. Gruneir is supported by a CIHR New Investigator Award. Dr. Markle-Reid is funded through the Canada Research Chairs program. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). Parts of this material are based on data and information compiled and provided by the MOH, the Canadian Institute for Health Information (CIHI), and Cancer Care Ontario (CCO). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding or data sources. No endorsement is intended or should be inferred. We thank IQVIA Solutions Canada Inc., for use of their Drug Information File.

Author information

Authors and Affiliations

  1. Department of Health Research Methods, Evidence, and Impact, McMaster University, MIP-Suite 309A, 1280 Main Street West Hamilton, Hamilton, Ontario, L8S 4K1, Canada

    Lauren E. Griffith & Rumaisa Aljied

  2. Department of Family Medicine, University of Alberta, Edmonton, Alberta, Canada

    Andrea Gruneir

  3. ICES, Toronto, Ontario, Canada

    Andrea Gruneir

  4. Women’s College Research Institute, Women’s College Hospital, Toronto, Ontario, Canada

    Andrea Gruneir

  5. School of Nursing, McMaster University, Hamilton, Ontario, Canada

    Kathryn A. Fisher, Maureen Markle-Reid & Jenny Ploeg

  6. ICES, McMaster University, Hamilton, Ontario, Canada

    Richard Perez & Francis Nguyen

  7. Department of Medicine, McMaster University, Hamilton, Ontario, Canada

    Christopher Patterson

  8. Division of Clinical Public Health, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada

    Ross Upshur

  9. Bridgepoint Collaboratory for Research and Innovation, Sinai Health System, Toronto, Ontario, Canada

    Ross Upshur

Authors
  1. Lauren E. Griffith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Gruneir
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kathryn A. Fisher
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rumaisa Aljied
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Francis Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Christopher Patterson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Maureen Markle-Reid
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jenny Ploeg
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ross Upshur
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LG was involved in all aspects of this study including study conception and design, data analysis and interpretation, and drafting and critically revising the manuscript. AG and KF were involved in study conception and design, data analysis and interpretation, and critically revising the manuscript. RA, RP, FN were involved with study conception, data analysis and interpretation, and assisted in critically revising the manuscript. CP, RU, MMR and JP were involved in study conception and design, data analysis and interpretation, critically revising the manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Lauren E. Griffith.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Hamilton Integrated Research Ethics Boards at McMaster University (Ethics certificate #: 13–590). All methods were carried out in accordance with relevant guidelines and regulations. ICES is a prescribed entity under Ontario’s Personal Health Information Protection Act (PHIPA). Section 45 of PHIPA authorizes ICES to collect personal health information, without consent, for the purpose of analysis or compiling statistical information with respect to the management of, evaluation or monitoring of, the allocation of resources to or planning for all or part of the health system. Projects that use data collected by ICES under section 45 of PHIPA, and use no other data, are exempt from REB review. The use of the data in this project is authorized under section 45 and approved by ICES’ Privacy and Legal Office.

Consent for publication

Not applicable.

Competing interests

The authors have no competing interests to declare.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplemental Fig. 1.

Study cohort creation including Ontario participants of the Canadian Community Health Survey cycles 3–5 who consented to administrative data linkage; CCHS, Canadian Community Health Survey, CC, chronic conditions

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Griffith, L.E., Gruneir, A., Fisher, K.A. et al. The impact of multimorbidity level and functional limitations on the accuracy of using self-reported survey data compared to administrative data to measure general practitioner and specialist visits in community-living adults. BMC Health Serv Res 21, 1123 (2021). https://doi.org/10.1186/s12913-021-07160-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-07160-2

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us