Skip to main content

Comparing mental and physical health of U.S. veterans by VA healthcare use: implications for generalizability of research in the VA electronic health records

Abstract

Objective

The Department of Veterans Affairs’ (VA) electronic health records (EHR) offer a rich source of big data to study medical and health care questions, but patient eligibility and preferences may limit generalizability of findings. We therefore examined the representativeness of VA veterans by comparing veterans using VA healthcare services to those who do not.

Methods

We analyzed data on 3051 veteran participants age ≥ 18 years in the 2019 National Health Interview Survey. Weighted logistic regression was used to model participant characteristics, health conditions, pain, and self-reported health by past year VA healthcare use and generate predicted marginal prevalences, which were used to calculate Cohen’s d of group differences in absolute risk by past-year VA healthcare use.

Results

Among veterans, 30.4% had past-year VA healthcare use. Veterans with lower income and members of racial/ethnic minority groups were more likely to report past-year VA healthcare use. Health conditions overrepresented in past-year VA healthcare users included chronic medical conditions (80.6% vs. 69.4%, d = 0.36), pain (78.9% vs. 65.9%; d = 0.35), mental distress (11.6% vs. 5.9%; d = 0.47), anxiety (10.8% vs. 4.1%; d = 0.67), and fair/poor self-reported health (27.9% vs. 18.0%; d = 0.40).

Conclusions

Heterogeneity in veteran sociodemographic and health characteristics was observed by past-year VA healthcare use. Researchers working with VA EHR data should consider how the patient selection process may relate to the exposures and outcomes under study. Statistical reweighting may be needed to generalize risk estimates from the VA EHR data to the overall veteran population.

Peer Review reports

Introduction

The Veterans Affairs (VA) is the largest integrated health care delivery system in the United States (US), providing care to over 6 million eligible veterans each year within 1200 geographically dispersed health care facilities [1,2,3]. With the size and scope of the VA health care delivery system comes a vast quantity of data on diagnoses, medication history, and sociodemographics, as well as provider and facility characteristics—all of which are stored in the VA electronic health records (EHR). Integration of data from several outside sources into the VA EHR (e.g., data for patients receiving care through the VA’s Community Care Program, Medicare data for patients aged ≥65) provide a nearly complete record of health care at both the patient- and organizational-level and an ideal data source for studying clinically important questions in veterans health. However, a major concern for the generalizability of EHR-based studies is selection bias, which is a systematic error of effect estimates introduced if the association between exposure and disease differs between those who contribute data to an EHR and those who do not [4,5,6]. Risk of selection bias is particularly high in cases when a subset of a population with a specific risk profile is strongly underrepresented in the study sample. As such, the VA beneficiary characteristics derived from the EHR raise concerns about selection bias and the generalizability of VA EHR-based research to the larger US veteran population. Nevertheless, the vast quantity of health-related data captured in the VA’s EHR represents an important resource for conducting health services research when properly interpreted.

The US Department of Veteran Affairs is a cabinet-level department tasked with providing services and benefits to US military veterans. The VA EHR data offer a critical tool for achieving these aims: they can be used to inform the development of prevention strategies tailored to the unique characteristics and needs of US veterans and to evaluate the efficacy of clinical and public health interventions. For example, veteran are recognized as a population at elevated risk of suicide [7, 8] and suicide prevention is an explicit focus of care improvement in the VA health system [9,10,11]. Recent research has used VA EHR data and machine learning technology to predict suicidal behavior in VA patients [12,13,14]. Using the information generated from this research, the VHA began national implementation of the Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment (REACH VET) program, which applied the algorithm to identify patients in the highest suicide risk [12]. Using VA EHR data, subsequent studies evaluated the impact of the REACH VET program, finding that it was associated with greater treatment engagement and fewer mental health admissions, emergency department visits, and suicide attempts [15].

In addition to providing useful knowledge for improving the public health and medical care of US veterans, VA EHR data can be used to study the efficacy of clinical interventions, which can then be used to improve clinical care in the general US population. For example, although randomized controlled trials are generally considered the gold standard to determine the efficacy of medications and health care interventions, they are expensive, time-prohibitive, difficult to implement, and ill-suited for the study of high-risk interventions, rare outcomes, and the consequences of harmful exposures (e.g., exposure to potentially traumatic events) [16, 17]. Electronic health record data can provide information about the efficacy of health interventions in instances when randomized controlled trials are unfeasible or undesirable. For example, EHR data have been used to provide rapid results during the COVID-19 pandemic about the potential protective effect of some antihistamines on risk of SARS-CoV-2 infection [18]. However, demonstrating the efficacy of a treatment in one study sample (e.g., VA patients) does not necessarily provide evidence of its efficacy in other populations, whether that is US veterans in general or the total US adult population. For example, caution is warranted when interpreting results from EHR based studies due to selection bias. One particular type of selection bias of concern that is present in EHR based studies is called collider bias, which arises from the exposure of interest being associated with the likelihood of being observed and can result in spurious associations when none exists [19]. As such, no matter how rigorous or carefully executed an EHR-based research study, the results depend on the setting in which they were derived (e.g., VA patient population), and often depend on factors that might be constant within the studied population but different elsewhere. Because any given association between an exposure and an outcome will vary across settings and populations as a function of how different the study sample (e.g., VA patient population) and the target population (e.g., non-VA veteran population) are from one another, information on the distribution of covariates in the VA patient population and non-VA veteran population must be considered to use the knowledge generated from research conducted in VA EHR data to inform policy for populations outside the VA patient population.

Users of VA healthcare represent a population with greater physical, mental, and social challenges than the general US adult population [20,21,22] as well as the overall US veteran population [23,24,25,26,27,28]. The higher burden of health and social challenges present in the VA versus non-VA veteran population may be a consequence of the VA healthcare benefits eligibility criteria, which is based on each veteran’s military service history, disability rating, income level, and other benefits applicants receive (e.g., VA pension benefits).

Although prior research has provided insight into the sociodemographic and health characteristics that may vary between veterans who use the VA for their healthcare and non-VA veterans, at least two important gaps in the literature remain. First, most studies have focused on VA enrollees, a population that differs from veterans who use the VA for their healthcare [23,24,25,26,27,28], which is the patient population captured in the VA EHR. Veterans who use VA healthcare services live closer to VA facilities [29,30,31,32], and are more likely to have a psychiatric or substance use disorder diagnosis [29, 30], and greater healthcare needs [29, 31] than VA-enrolled veterans who do not. Because not all VA-enrolled veterans utilize VA health care services each year, prior research documenting sociodemographic and health differences in veterans by VA enrollment status may not adequately capture important differences between veterans overall and the VA patient population captured in the VA EHR. Second, the demographic profile of veterans, generally, and of the VA patient population in particular is changing: over the last two decades the age distribution has become younger, and the share of women veterans and racial/ethnic minorities has increased over the last two decades [33]; however, only two published studies have analyzed data that were collected within the past 10 years [26, 34], one of which limited its analysis to veterans with service-connected conditions [34, 35], and another that was focused on examining sociodemographic and health differences in veterans with versus without health coverage [26]. Therefore, the results of previous studies that have examined the differences between VA enrollees and non-VA veterans do not reflect the changing demographic profile of veterans, the representativeness of the VA patient population as contained in the VA EHR data remains unknown.

To address these limitations, we leveraged data from the 2019 National Health Interview Survey (NHIS) to characterize differences in the distribution of sociodemographic characteristics, physical and mental health, and health behaviors in US military veterans who did and did not use VA healthcare services during the past year. For this analysis, we selected variables that if they (a) have been previously shown to vary between VA and non-VA veterans or (b) are factors measured and available for study in the VA EHR data. The 2019 NHIS data are particularly well-suited for this analysis because of their large sample and ability to differentiate between veterans receiving VA healthcare services and veterans not receiving any past-year VA care. As such, this study provides the most current description of 1 year of VA use and non-use among non-institutionalized veterans.

Methods

Study population

We analyzed data on US veterans from the 2019 NHIS, a nationally representative household survey of the civilian noninstitutionalized US population. The investigation was carried out in accordance with the latest version of the Declaration of Helsinki and informed consent was obtained from all survey participants. The 2019 NHIS Sample Adult component included 31,997 adults, aged ≥18 years, of which 3061 (9.6%) were veterans, defined as adults who had ever served on active duty in the US Armed Forces, military Reserves, or National Guard and were not currently on active duty [36]. After excluding 10 respondents with missing age information, the analytic sample included 3051 veterans. These publicly available data are exempt from IRB review.

Measures

Past year VA healthcare use

Our primary predictor variable was past-year use of VA healthcare services. This variable captures all participants whose data would be included in the VA EHR. Past year VA healthcare use was assessed using the question “During the past 12 months, did you receive any care at a Veteran’s Health Administration facility or receive any other healthcare paid for by the VA?” A dichotomous variable assessed whether veterans did or did not endorse past year use of VA healthcare services, regardless of whether they also utilized a different type of healthcare coverage (labeled hereafter as “VA patients” and “non-VA veterans”, respectively).

Sociodemographic characteristics

Sociodemographic variables for this study included age (18–34, 35–44, 45–54, 55–64, 65+), gender (male, female), ethnicity and race (Hispanic, non-Hispanic: White, Black, Asian or Pacific Islander, other [Native American, Alaska Native, Other Race]), sexual orientation (heterosexual, sexual minority), education level (<high school, high school or equivalent, some college or more), and family income relative to the federal poverty line (FPL; < 100% FPL, 100–199% FPL, 200–399% FPL, or ≥ 400% FPL).

Chronic health conditions

Participants reported whether a doctor or other healthcare professional had ever diagnosed them with high blood pressure, heart disease, diabetes, cancer (excluding non-melanoma skin cancer), arthritis, asthma, or chronic lung disease (i.e., chronic obstructive pulmonary disease, emphysema, or chronic bronchitis). In addition to considering the 7 chronic health conditions individually, we also created a composite variable, coded yes if a participant reported having ever being diagnosed with 1 or more of the 7 selected chronic conditions. Self-reported physician-diagnosed medical conditions have been found to have high validity [37].

Pain

Pain frequency, severity, and specific pain conditions were assessed using questions developed by the Washington Group on Disability Statistics [38]. Respondents were first asked “In the past 3 months, how often did you have pain? Would you say never, some days, most days, or every day?” For those who had pain at least some days, a follow-up question assessing bothersomeness was asked: “Thinking about the last time you had pain, how much pain did you have—a little, between a little and a lot, or a lot?” Participants who reported pain at least some days in the past 3 months were considered to have any pain. Participants who reported pain on “most days” or “every day” during the past 3 months were considered to have frequent pain. Participants who reported pain on “most days” or “every day” in the past 3 months and that the pain bothered them “a lot” were considered to have severe pain. Finally, participants were asked separate questions about pain in specific areas of the body (back; hands, arms, or shoulder; hips, knees, or feet; abdominal, pelvic, or genitals; migraines or headaches; and tooth or jaw) in the past 3 months, and whether they had symptoms of arthritis-related joint pain in the past 30 days. All pain measures have been extensively validated in the US and internationally [38].

Mental health status

Depressive symptom severity was assessed using the Patient Health Questionnaire—version 8 (PHQ-8), with a value of ≥10 used to identify adults experiencing depression [39]. Generalized Anxiety Disorder scale—version 7 (GAD-7) was used to assess anxiety, with moderate/severe anxiety symptoms indicated by GAD-7 scores ≥10 [40].

Combustible and electronic cigarette use

Participants were categorized into three mutually exclusive groups based on whether they had smoked ≥100 cigarettes in their lifetime and smoked at least some days in the past 30 days: current smokers (≥100 lifetime cigarettes and past 30-day use), former smokers (≥100 lifetime cigarettes and no past 30-day use), and never smokers (smoked < 100 cigarettes in their lifetime). Current electronic cigarette use or “vaping” was based on respondents endorsing they now use electronic cigarettes either every day or some days.

Self-reported health status, disability, and obesity

An indicator variable for fair or poor self-reported health was constructed based on responses to the question “Would you say your health in general is excellent, very good, good, fair, or poor?”; coded as 1 if a participant endorsed fair or poor health and coded as 0 if they endorsed excellent, very good, or good. This dichotomous measure is a reliable and valid measure of general physical well-being and highly correlated with objective measures of functional impairment, morbidity, and mortality [41, 42]. Disability was assessed using the Washington Group Composite Disability indicator. Participants who reported having serious difficulty in either seeing, hearing, mobility, communication, cognition, or self-care were classified as having a disability [38]. Obesity was defined as current body mass index ≥30 kg/m2 [43].

Statistical analysis

Veterans were stratified by past-year VA healthcare use, and Pearson’s χ2 tests were used to evaluate differences between VA patients and non-VA veterans on sociodemographic characteristics, chronic health conditions, pain, mental health status, combustible cigarette use and vaping, and self-reported health. The χ2 test assumes the data were obtained through random selection, the data are frequencies or counts, with mutually exclusive levels of the variable, the study groups are independent, and the value of the cell expected should be 5 or more in at least 80% of the cells, with no cell having an expected count of less than one [44, 45]. In accordance with the American Statistical Association, we reported the actual P values, rather than expressing a statement of inequality (P < .05), to avoid the potential problem of incorrectly interpreting a P value as significant or not based on a pre-determined threshold value [46, 47]. All percentages and standard errors were calculated with SAS-callable SUDAAN 11.0.1 and NHIS sample weights were used to account for the complex survey design and survey nonresponse to produce estimates nationally representative of the non-institutionalized population of veterans residing in the US. Multivariable logistic regression models (SAS-callable SUDAAN 11.0.1) using sample weights generated weighted predicted marginal prevalence estimates (back-transformed from marginal log-odds) of sociodemographic and medical profiles in each US veteran group (VA patients and non-VA veterans), both unadjusted and adjusted for sociodemographic factors related to VA healthcare use, including age, gender, race and ethnicity, education, and family income. Predicted marginal prevalences were then used to calculate risk differences (RD) and adjusted risk differences (aRD) with 95% confidence intervals (CIs), which estimate group differences in absolute risk between VA patients and non-VA veterans. We estimated unadjusted and adjusted odds ratios (presented in the online appendix), which we transformed into Cohen’s d by \(d={L}_{OR}\frac{\sqrt{3}}{\pi }\), where π =3.14159 and LOR is the natural logarithm of the odds ratio to provide information on the magnitude of the effects [48], with effect sizes of d = 0.2, 0.5, and 0.8 indicating “small”, “medium”, and “large” effects, respectively [49].

Results

Most veterans were male (89.2%), non-Hispanic White (79.0%), heterosexual (97.4%), completed some college or more (65.4%); 47.9% aged 65 and above and 45.1% reported a family income > 400% the federal poverty line (Table 1). Approximately 32% of veterans reported receiving past-year VA healthcare services. Non-VA veterans were more likely to have higher incomes and to be non-Hispanic White than VA patients.

Table 1 Sociodemographic characteristics of US military veterans overall and by past-year use of Veterans Administration (VA) healthcare: NHIS 2019

VA patients had a higher burden of any chronic health condition (aRD = 11.94; 95%CI = 8.08–15.80), high blood pressure (aRD = 12.46; 95%CI = 8.25–16.67), diabetes (aRD = 7.73; 95%CI = 4.31–11.15), arthritis (aRD = 15.17; 95%CI = 10.90–19.45), and chronic lung disease (aRD = 6.65; 95%CI = 3.84–9.45) than non-VA veterans (Table 2). Pain was more prevalent among VA patients than non-VA veterans. The highest differences in pain prevalence were for any pain (aRD = 13.02; 95%CI = 9.01–17.03), frequent pain (aRD = 19.89; 95%CI = 15.64–24.14) and severe pain (aRD = 7.78; 95%CI = 4.24–10.71). Both depressive symptoms (aRD = 5.70; 95%CI = 2.91–8.49) and anxiety symptoms (aRD = 6.72; 95%CI = 4.28–9.16) were also more prevalent in VA patients than non-VA veterans. Although group differences in current cigarette use and current electronic cigarette use were negligible, VA patients were more likely to be former smokers (aRD = 8.73; 95%CI = 4.55–12.92) than non-VA veterans. Moderate differences in overall measures of general health (aRD = 12.87; 95%CI = 9.18–16.55) and disability (aRD = 10.58; 95%CI = 7.21–13.94) were observed between the two groups, with 27.9% of VA patients reporting fair or poor health and 20.2% reporting disability, compared to 18.0 and 11% of non-VA veterans.

Table 2 Health conditions and behaviors in US military veterans by past-year use of VA health care: NHIS, 2019

Figure 1 shows the magnitude of the sociodemographic and health differences between VA patients and non-VA veterans. The difference between VA patients and non-VA veterans in the prevalence of non-Hispanic Blacks were moderate (d ≥ 0.50), although differences between VA patients and non-VA veterans were small for all other race and ethnicity groups and income. For health conditions, we observed the largest group differences between VA patients and non-VA veterans for frequent pain (d = 0.49), severe pain (d = 0.40), depressive symptoms (d = 0.67), and anxiety symptoms (d = 0.47).

Fig. 1
figure 1

Love plot displaying unadjusted and adjusted effect sizes of 31 characteristics between US military veterans with past-year VA care and veterans without past-year VA care in the National Health Interview Survey (NHIS), 2019. Logistic models were used to generate unadjusted predicted marginal prevalences and adjusted predicted marginal prevalences, standardized to the distribution of sociodemographic characteristics of the sample. Regressions adjusted for age category, gender, race/ethnicity, education, and poverty status based on Federal Poverty Level. The unstandardized regression coefficients and pooled variance from the unadjusted and adjusted regression models were then used to calculate the Cohen’s d

Discussion

Using data from the nationally representative 2019 National Health Interview Survey, we documented important differences in the distribution of socioeconomic and health characteristics of veterans who use and who do not use VA services. There were several important findings from this study. First, consistent with prior work, the sociodemographic composition of VA patients in 2019 differed from non-VA veterans, the primary population of interest for many VA EHR-based studies [23]. Our finding that members of disadvantaged racial and ethnic minority groups and low-income veterans were overrepresented in the VA patient population is consistent with the prior research that defined VA healthcare use by VA enrollment status rather than VA healthcare use [23,24,25,26,27]. However, we observed relatively minimal differences in the age and gender distribution of VA patients and non-VA veterans, in contrast to these previously published studies that found women and younger veterans overrepresented in VA enrollees relative to non-enrolled veterans [23,24,25,26,27]. Although we cannot explain why VA enrollees, but not VA patients, are more likely to be younger and female than other veterans, women and younger VA enrollees may prefer to receive their care outside of the VA system; these groups may have greater access to non-VA healthcare (e.g., as part of their employment benefits) or have better health and lower healthcare needs than their peers.

Second, VA patients were disproportionately burdened by physical and psychological morbidity and disability, including higher prevalences of high blood pressure, diabetes, arthritis, chronic lung disease, frequent and severe pain, depression and anxiety symptoms, and fair/poor self-reported health. Although the over-representation of high-risk health conditions may be expected in a patient population accessing outpatient medical and hospital services [24, 27], the over-representation of physical and psychological morbidity and disability in the VA patient population may be exacerbated by the eligibility criteria for VA services, which prioritizes veterans with severe income limitations and service-connected disability [34, 35]. Veterans with service-connected conditions, particularly those with psychiatric disorders such as depression and PTSD, depend heavily upon the VA for health care. For example, Maynard et al. [34] found that veterans with service-connected psychiatric disorders accounted for most hospitalizations in the VA system, and almost half of VA enrollees with PTSD and/or major depression had one or more mental health visits in 2016. As such, we would expect VA patients to have greater physical and psychological morbidity and disability than non-VA veterans.

The over-representation of high-risk sociodemographic and health conditions in the VA patient population indicate that VA EHR-based studies may yield estimates that are not generalizable to the overall veteran population. However, statistical methods have been proposed to improve the generalizability of EHR results to populations of clinical and policy interest [50, 51]. For example, the substantial body of literature on suicide and its potential causes among veterans has relied heavily on data from the VA’s EHR databases [13, 52, 53], which will result in gaps in knowledge about those who do not receive care within the VA. Using information on the differences in the distribution of socioeconomic and health characteristics of veterans who use and who do not use VA services, future VA EHR-based studies could apply selection probabilities with model-based standardizations to estimate the results in the total US veteran population. The same approach can be applied to estimate the treatment effect in a population distinct from the study sample. For example, given that the factors that determine whether a person receives healthcare through the VA versus Medicare are well documented [54,55,56], similar methods could be applied to generalize results from the VA EHR data to the Medicare population to estimate the expected effect, for example, of implementing a VA program in the Medicare population. Although the specific set of characteristics that must be included in a selection model will depend on the research question being investigated, our study provides critical information on the variables that differentiate VA patients from non-VA veterans that future studies require to accurately estimate the conditional probability of being selected.

Study limitations are noted. First, the NHIS sample does not include homeless individuals or those in institutional settings. Although homeless veterans are disproportionately affected by physical and mental illness, they compromise a very small fraction of the VA patient population (~ 37,000 veterans were homeless in 2020 [57]), and thus their effect on the overall findings would be limited. Second, self-report measures of chronic medical conditions, mental distress, and anxiety in the NHIS are not confirmed with medical diagnosis or collateral information. Social desirability could lead to underreporting of stigmatized conditions, although there is no reason to believe this would vary by past-year VA healthcare use. Third, given the documented changes in the underlying VA patient population over time, our findings may not generalize to earlier years. Fourth, neither VA patients nor non-VA veterans were engaged as stakeholder partners in the planning, conduct, or dissemination phases of this study. However, our research team was comprised of a diverse set of experts, including VA research scientists and VA clinician/researchers, which are considered patient partners and stakeholder partners by the PCORI Engagement Rubric [58].

Our study provides valuable results on the representativeness of the VA patient population to the overall US veteran population. These findings will be useful for both hypothesizing about how inferences derived from VA EHR data will generalize to the overall US veteran population and minimizing the effect of bias in the context of differential patient population selection that affect both exposures and outcomes. Differences between the VA patient population and overall US veteran population should be continuously monitored to identify potential influential changes in their sociodemographic and clinical profile over time. Future research should investigate how to best use VA EHR data to better understand and meet the needs of all US veterans, including VA enrollees who might leave the VA for other public insurance options (e.g., Medicaid, Medicare) or those who choose community providers.

Availability of data and materials

The datasets generated and analyzed during the current study are publicly available in the National Health Interview Survey, 2019 at https://www.cdc.gov/nchs/nhis/2019nhis.htm.

References

  1. Rose L, Tran LD, Asch SM, Vashi A. Assessment of changes in US Veterans health Administration care delivery methods during the COVID-19 pandemic. JAMA Netw Open. 2021;4(10):e2129139-e.

    Article  Google Scholar 

  2. US Department of Veterans Affairs. VA utilization profile FY 2017 2020 Available from: https://www.va.gov/vetdata/docs/Quickfacts/VA_Utilization_Profile_2017.pdf.

    Google Scholar 

  3. US Department of Veterans Affairs. About VHA 2022 Available from: https://www.va.gov/health/aboutvha.asp.

    Google Scholar 

  4. Thompson LA, Gurka MJ, Filipp SL, Schatz DA, Mercado RE, Ostrov DA, et al. The influence of selection bias on identifying an association between allergy medication use and SARS-CoV-2 infection. eClinicalMedicine. 2021;37:100936.

    Article  Google Scholar 

  5. Haneuse S, Arterburn D, Daniels MJ. Assessing missing data assumptions in EHR-based studies: a complex and underappreciated task. JAMA Netw Open. 2021;4(2):e210184-e.

    Article  Google Scholar 

  6. Beesley LJ, Mukherjee B. Statistical inference for association studies using electronic health records: handling both selection bias and outcome misclassification. Biometrics. 2022;78(1):214–26.

    Article  Google Scholar 

  7. General OotS, Prevention NAAfS. 2012 national strategy for suicide prevention: goals and objectives for action: a report of the US surgeon General and of the National Action Alliance for suicide prevention. 2012.

    Google Scholar 

  8. Affairs UDoV. National veteran suicide prevention annual report. Office of Mental Health and Suicide Prevention, vol. 2019; 2019.

    Google Scholar 

  9. Affairs DoV. National strategy for preventing veteran suicide: 2018–2028. Washington, DC: US Department of Veterans Affairs; 2018.

    Google Scholar 

  10. Kemp J, Bossarte R. Suicide data report: 2012: Department of Veterans Affairs, mental health services, Suicide Prevention; 2013.

    Google Scholar 

  11. Ikeda R, Hedegaard H, Bossarte R, Crosby AE, Hanzlick R, Roesler J, Seider R, Smith P, Warner M. Improving national data systems for surveillance of suicide-related events. Am J Prev Med. 2014;47(3 Suppl 2):S122-9.

  12. U.S. Veterans Administration. REACH VET, predictive analytics for suicide prevention. 2017.

    Google Scholar 

  13. McCarthy JF, Bossarte RM, Katz IR, Thompson C, Kemp J, Hannemann CM, et al. Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of Veterans affairs. Am J Public Health. 2015;105(9):1935–42.

    Article  Google Scholar 

  14. Kessler RC, Hwang I, Hoffmire CA, McCarthy JF, Petukhova MV, Rosellini AJ, et al. Developing a practical suicide risk prediction model for targeting high-risk patients in the Veterans health Administration. Int J Methods Psychiatr Res. 2017;26(3).

  15. McCarthy JF, Cooper SA, Dent KR, Eagan AE, Matarazzo BB, Hannemann CM, et al. Evaluation of the recovery engagement and coordination for health–veterans enhanced treatment suicide risk modeling clinical program in the veterans health administration. JAMA Netw Open. 2021;4(10):e2129900-e.

    Article  Google Scholar 

  16. Mulder R, Singh AB, Hamilton A, Das P, Outhred T, Morris G, et al. The limitations of using randomised controlled trials as a basis for developing treatment guidelines. Evid Based Mental Health. 2018;21(1):4–6.

    Article  Google Scholar 

  17. Frieden TR. Evidence for health decision making — beyond randomized, controlled trials. N Engl J Med. 2017;377(5):465–75.

    Article  Google Scholar 

  18. Reznikov LR, Norris MH, Vashisht R, Bluhm AP, Li D, Liao Y-SJ, et al. Identification of antiviral antihistamines for COVID-19 repurposing. Biochem Biophys Res Commun. 2021;538:173–9.

    Article  CAS  Google Scholar 

  19. Griffith GJ, Morris TT, Tudball MJ, Herbert A, Mancano G, Pike L, et al. Collider bias undermines our understanding of COVID-19 disease risk and severity. Nat Commun. 2020;11(1):5749.

    Article  CAS  Google Scholar 

  20. Rogers WH, Kazis LE, Miller DR, Skinner KM, Clark JA, Spiro A III, et al. Comparing the health status of VA and non-VA ambulatory patients: the Veterans’ health and medical outcomes studies. J Ambul Care Manage. 2004:27(3).

  21. Agha Z, Lofgren RP, VanRuiswyk JV, Layde PM. Are patients at Veterans affairs medical centers sicker? A comparative analysis of health status and medical resource use. Arch Intern Med. 2000;160(21):3252–7.

    Article  CAS  Google Scholar 

  22. Kazis LE, Miller DR, Clark J, Skinner K, Lee A, Rogers W, et al. Health-related quality of life in patients served by the Department of Veterans Affairs: results from the Veterans health study. Arch Intern Med. 1998;158(6):626–32.

    Article  CAS  Google Scholar 

  23. Meffert BN, Morabito DM, Sawicki DA, Hausman C, Southwick SM, Pietrzak RH, et al. US Veterans who do and do not utilize Veterans affairs health care services: demographic, military, medical, and psychosocial characteristics. Prim Care Companion CNS Disord. 2019;21(1):18m02350.

    Article  Google Scholar 

  24. Nelson KM, Starkebaum GA, Reiber GE. Veterans using and uninsured veterans not using Veterans affairs (VA) health care. Public Health Rep. 2007;122(1):93–100.

    Article  Google Scholar 

  25. Reiber GE, Koepsell TD, Maynard C, Haas LB, Boyko EJ. Diabetes in nonveterans, veterans, and veterans receiving Department of Veterans Affairs health care. Diabetes Care. 2004;27(Suppl 2):B3–9.

    Article  Google Scholar 

  26. Weissman JD, Russell D, Haghighi F, Dixon L, Goodman M. Health coverage types and their relationship to mental and physical health in U.S. veterans. Prev Med Rep. 2019;13:85–92.

    Article  Google Scholar 

  27. Wong ES, Wang V, Liu CF, Hebert PL, Maciejewski ML. Do Veterans health Administration enrollees generalize to other populations? Med Care Res Rev. 2016;73(4):493–507.

    Article  Google Scholar 

  28. Vespa JE. Those who served: America’s Veterans From World War II to the War on Terror. American Community Survey Report Washington, D.C.: U.S. Census Bureau; 2020 Available from: https://www.census.gov/content/dam/Census/library/publications/2020/demo/acs-43.pdf.

  29. Petersen LA, Byrne MM, Daw CN, Hasche J, Reis B, Pietz K. Relationship between clinical conditions and use of Veterans affairs health care among Medicare-enrolled veterans. Health Serv Res. 2010;45(3):762–91.

    Article  Google Scholar 

  30. Carey K, Montez-Rath ME, Rosen AK, Christiansen CL, Loveland S, Ettner SL. Use of VA and Medicare services by dually eligible veterans with psychiatric problems. Health Serv Res. 2008;43(4):1164–83.

    Article  Google Scholar 

  31. Hynes DM, Koelling K, Stroupe K, Arnold N, Mallin K, Sohn MW, et al. Veterans’ access to and use of Medicare and Veterans affairs health care. Med Care. 2007;45(3):214–23.

    Article  Google Scholar 

  32. McCarthy JF, Blow FC, Valenstein M, Fischer EP, Owen RR, Barry KL, et al. Veterans affairs health system and mental health treatment retention among patients with serious mental illness: evaluating accessibility and availability barriers. Health Serv Res. 2007;42(3 Pt 1):1042–60.

    Article  Google Scholar 

  33. National Center for Veterans Analysis and Statistics. Table 2L: VETPOP 2018 LIVING VETERANS BY PERIOD: Veterans affairs; 2018 updated June 22, 2021. Available from: https://www.va.gov/vetdata/docs/Demographics/New_Vetpop_Model/2L_VetPop2018_POS_National.xlsx.

    Google Scholar 

  34. Maynard C, Batten A, Liu CF, Nelson K, Fihn SD. The burden of mental illness among Veterans: use of VHA health care services by those with service-connected conditions. Med Care. 2017;55(11):965–9.

    Article  Google Scholar 

  35. Maynard C, Nelson K. Compensation for Veterans with service connected disabilities: current findings and future implications. J Disabil Policy Stud. 2020;31(1):57–62.

    Article  Google Scholar 

  36. National Center for Health Statistics. Survey description, National Health Interview Survey, 2019. Hyattsville: US Department of Health and Human Services, CDC, National Center for Health Statistics; 2020. Available from: https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2019/srvydesc-508.pdf

    Google Scholar 

  37. Martin LM, Leff M, Calonge N, Garrett C, Nelson DE. Validation of self-reported chronic conditions and health services in a managed care population. Am J Prev Med. 2000;18(3):215–8.

    Article  CAS  Google Scholar 

  38. Nahin RL. Estimates of pain prevalence and severity in adults: United States, 2012. J Pain. 2015;16(8):769–80.

    Article  Google Scholar 

  39. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009;114(1–3):163–73.

    Article  Google Scholar 

  40. Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.

    Article  Google Scholar 

  41. McGee DL, Liao Y, Cao G, Cooper RS. Self-reported health status and mortality in a multiethnic US cohort. Am J Epidemiol. 1999;149(1):41–6.

    Article  CAS  Google Scholar 

  42. Idler EL, Kasl SV, Lemke JH. Self-evaluated health and mortality among the elderly in New Haven, Connecticut, and Iowa and Washington counties, Iowa, 1982-1986. Am J Epidemiol. 1990;131(1):91–103.

    Article  CAS  Google Scholar 

  43. Pi-Sunyer FX, Becker DM, Bouchard C, Carleton RA, Colditz GA, Dietz WH, Foreyt JP, Garrison RJ, Grundy SM, Hansen BC, Higgins M. Clinical guidelines on the identification, evaluation, and treatment of overweight and obesity in adults: Executive summary. Am J Clin Nut. 1998;68(4):899-917.

  44. Bewick V, Cheek L, Ball J. Statistics review 8: qualitative data - tests of association. Crit Care. 2004;8(1):46–53.

    Article  Google Scholar 

  45. McHugh ML. The chi-square test of independence. Biochem Med (Zagreb). 2013;23(2):143–9.

    Article  CAS  Google Scholar 

  46. Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129–33.

    Article  Google Scholar 

  47. Wasserstein RL, Schirm AL, Lazar NA. Moving to a World Beyond “p < 0.05”. Am Stat. 2019;73(1):1–19.

    Article  Google Scholar 

  48. Sánchez-Meca J, Marín-Martínez F, Chacón-Moscoso S. Effect-size indices for dichotomized outcomes in meta-analysis. Psychol Methods. 2003;8(4):448–67.

    Article  Google Scholar 

  49. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed: Lawrence Erlbaum Associates; 1988.

    Google Scholar 

  50. Rothman KJ, Greenland S, Lash TL. Modern Epidemiology: Lippincott Williams & Wilkins; 2008.

    Google Scholar 

  51. Stuart EA, Bradshaw CP, Leaf PJ. Assessing the generalizability of randomized trial results to target populations. Prev Sci. 2015;16(3):475–85.

    Article  Google Scholar 

  52. Alemi F, Avramovic S, Renshaw KD, Kanchi R, Schwartz M. Relative accuracy of social and medical determinants of suicide in electronic health records. Health Serv Res. 2020;55(S2):833–40.

    Article  Google Scholar 

  53. Lynch KE, Gatsby E, Viernes B, Schliep KC, Whitcomb BW, Alba PR, et al. Evaluation of suicide mortality among sexual minority US Veterans from 2000 to 2017. JAMA Netw Open. 2020;3(12):e2031357-e.

    Article  Google Scholar 

  54. Hebert PL, Wong ES, Reddy A, Batten A, Gunnink E, Wagner TH, et al. Events associated with changes in reliance on the Veterans health Administration among Medicare-eligible Veterans. Med Care. 2020;58(8):710–6.

    Article  Google Scholar 

  55. Liu CF, Manning WG, Burgess JF Jr, Hebert PL, Bryson CL, Fortney J, et al. Reliance on Veterans affairs outpatient care by Medicare-eligible veterans. Med Care. 2011;49(10):911–7.

    Article  Google Scholar 

  56. Liu CF, Batten A, Wong ES, Fihn SD, Hebert PL. Fee-for-service Medicare-enrolled elderly Veterans are increasingly voting with their feet to use more VA and less Medicare, 2003-2014. Health Serv Res. 2018;53(Suppl 3):5140–58.

    Article  Google Scholar 

  57. Henry M, de Sousa T, Roddey C, Gayen S, Bedner TJ, Abt Associates. The 2020 Annual Homeless Assessment Report (AHAR) to Congress: Part 1: Point-In-Time Estimates of Homelessness Washington, D.C.2021 Available from: https://www.huduser.gov/portal/sites/default/files/pdf/2020-AHAR-Part-1.pdf.

    Google Scholar 

  58. Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI). Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. JAMA. 2012;307(15):1636–40.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This study was supported in part by grants no. R01DA048860 (D.H.) and T32DA031099 (D.H.) from the NIDA. D.H. and M.W. have received financial support for this research from the New York State Psychiatric Institute.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the conception and design of the study, provided critical feedback during the analysis and interpretation of the results, reviewed and revised the paper, approved the final version of the paper being submitted here for review, and have agreed to be accountable for all aspects of the paper. David S. Fink - Design of the study, interpretation of the data, and drafted the paper. Malka Stohl – Analysis and interpretation of data, critical review and revising the paper for important intellectual content. Zachary L. Mannes - Interpretation of data, critical review and revising the paper for important intellectual content. Dvora Shmulewitz - Design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Melanie Wall - Design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Sarah Gutkind - Interpretation of data, critical review and revising the paper for important intellectual content. Scott Sherman - Interpretation of data, critical review and revising the paper for important intellectual content. Silvia Martins - Design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Andrew J. Saxon - Design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Deborah S. Hasin – Design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Mark Olfson – design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Jaimie Gradus – design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Charles Maynard – design of the work, interpretation of data, critical review and revising the paper for important intellectual content. Katherine M. Keyes – design of the work, interpretation of data, critical review and revising the paper for important intellectual content.

Corresponding author

Correspondence to Deborah S. Hasin.

Ethics declarations

Ethics approval and consent to participate

The study was approved by Institutional Review Boards at the VA Puget Sound, New York Harbor Healthcare Systems, and New York State Psychiatric Institute. The investigation was carried out in accordance with the latest version of the Declaration of Helsinki and informed consent was obtained from all survey participants.

Consent for publication

Not applicable.

Competing interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix Table 1.

Health conditions and behaviors in US military veterans by past-year use of VA health care: NHIS, 2019.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fink, D.S., Stohl, M., Mannes, Z.L. et al. Comparing mental and physical health of U.S. veterans by VA healthcare use: implications for generalizability of research in the VA electronic health records. BMC Health Serv Res 22, 1500 (2022). https://doi.org/10.1186/s12913-022-08899-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-08899-y

Keywords