Skip to main content
Download PDF

Prospective evaluation of social risks, physical function, and cognitive function in prediction of non-elective rehospitalization and post-discharge mortality

BMC Health Services Research volume 22, Article number: 574 (2022) Cite this article

Abstract

Background

Increasing evidence suggests that social factors and problems with physical and cognitive function may contribute to patients’ rehospitalization risk. Understanding a patient’s readmission risk may help healthcare providers develop tailored treatment and post-discharge care plans to reduce readmission and mortality. This study aimed to evaluate whether including patient-reported data on social factors; cognitive status; and physical function improves on a predictive model based on electronic health record (EHR) data alone.

Methods

We conducted a prospective study of 1,547 hospitalized adult patients in 3 Kaiser Permanente Northern California hospitals. The main outcomes were non-elective rehospitalization or death within 30 days post-discharge. Exposures included patient-reported social factors and cognitive and physical function (obtained in a pre-discharge interview) and EHR–derived data for comorbidity burden, acute physiology, care directives, prior utilization, and hospital length of stay. We performed bivariate comparisons using Chi-square, t-tests, and Wilcoxon rank-sum tests and assessed correlations between continuous variables using Spearman’s rho statistic. For all models, the results reported were obtained after fivefold cross validation.

Results

The 1,547 adult patients interviewed were younger (age, p = 0.03) and sicker (COPS2, p < 0.0001) than the rest of the hospitalized population. Of the 6 patient-reported social factors measured, 3 (not living with a spouse/partner, transportation difficulties, health or disability-related limitations in daily activities) were significantly associated (p < 0.05) with the main outcomes, while 3 (living situation concerns, problems with food availability, financial problems) were not. Patient-reported cognitive (p = 0.027) and physical function (p = 0.01) were significantly lower in patients with the main outcomes. None of the patient-reported variables, singly or in combination, improved predictive performance of a model that included acute physiology and longitudinal comorbidity burden (area under the receiver operator characteristic curve was 0.716 for both the EHR model and maximal performance of a random forest model including all predictors).

Conclusions

In this insured population, incorporating patient-reported social factors and measures of cognitive and physical function did not improve performance of an EHR-based model predicting 30-day non-elective rehospitalization or mortality. While incorporating patient-reported social and functional status data did not improve ability to predict these outcomes, such data may still be important for improving patient outcomes.

Peer Review reports

Introduction

Increasing evidence suggests social factors like unstable housing [1]; food insecurity [2,3,4,5,6,7]; transportation difficulties [8]; chronic stress [9]; and ability to get help with activities of daily living [10,11,12,13] influence various health and healthcare utilization outcomes, including risk of hospitalization and frequent use of the emergency department (ED) [14]. Similarly, problems with physical and cognitive function may contribute to patients’ rehospitalization risk [15, 16]. Understanding a patient’s risk for readmission may help healthcare providers develop appropriate tailored treatment and post-discharge care plans to reduce potentially avoidable readmissions and mortality. Teasing out the independent contributions of these factors to health outcomes is complicated since they may coexist and correlate with illness or disability burden.

In 2015, we described the development of a real-time score – referred to as the Transitions Support Level (TSL) score – to predict rehospitalization and post-discharge mortality risk [17]. Subsequently, Kaiser Permanente Northern California (KPNC) hospitals instantiated this model in the electronic health record (EHR). The TSL employs longitudinal comorbidity, acute physiology, discharge care directives, and recent utilization [17] and generates a predicted percent risk for a composite outcome (non-elective hospitalization and/or death within 30 days following hospital discharge) ranging from 0 to 100%. Risk estimates based on the TSL were integrated with discharge and follow-up workflows, which has resulted in decreased rehospitalizations [18]. In that same year Kaiser Permanente’s (KP)’s Care Management Institute developed the Your Current Life Situation (YCLS) questionnaire and item bank to screen members for social factors that could affect health and healthcare access [19]. YCLS items have been used in a variety of pilot projects in different programs and settings but are not part of routine outpatient or inpatient assessment. During this time period, KPNC also started to evaluate the use of brief patient-reported measures of functional status.

To test the hypothesis that patient-reported information could enhance the TLS’s prediction of post-hospital outcomes over EHR data alone, we employed a prospective cohort study design. The aim of our study was to test the hypothesis that adding patient-reported data about social factors and physical and cognitive function to the TLS data would significantly enhance the ability to predict 30-day rehospitalization and mortality. A secondary aim was to determine what specific patient-reported information might be recommended for routine ascertainment and entry into the EHR.

Materials and methods

Setting

KPNC is an integrated healthcare delivery system that includes 21 hospitals and 257 medical offices [17, 20,21,22,23,24] serving ~ 4.5 million Kaiser Foundation Health Plan members in Northern California. The program completed deployment of the Epic EHR (www.epicsystems.com) in all its hospitals and clinics in mid-2010. For this study, we recruited patients at 3 KPNC hospitals (Oakland, San Leandro, Walnut Creek). The study was approved by the KPNC Institutional Review Board for the Protection of Human Subjects.

Eligibility criteria

Patients were required to meet the following eligibility criteria: age ≥ 18 years; English speaker; insurance coverage other than Medicaid (Medi-Cal); inpatient or admitted for observation, other than Labor & Delivery service; current hospitalization began inside a KPNC hospital; discharge without further hospitalization elsewhere; not on infection isolation precautions; no “Comfort Care Only” order in effect at the time of discharge; and cognitively and functionally able to provide informed consent and answer questionnaires. The latter was determined in several ways: no diagnosis of dementia in EHR; ability to answer questions and provide informed consent without a proxy; and was alert, oriented, and approachable as verbally confirmed by the patient’s nurse or other hospital care provider. Since we were trying to interview patients as close to discharge date as possible, only patients with planned discharge that day or the following day were approached.

All hospitalized adults in KPNC are assigned an automated daily Transition Support Level (TSL) score every morning at 0600. The TSL score is based on a patient’s admission acute severity of illness, longitudinal comorbidity score, whether the patient experienced any hospitalizations in the 7 and 30 days preceding the index hospitalization, length of stay (truncated at 30 days), and discharge care directive (“full code” or not) [17]. Patients with a TSL of ≥ 25% on the day of discharge are automatically enrolled in the KPNC Transitions Program, and they receive additional in-person discharge services and extra follow-up calls.

We oversampled patients whose predicted risk was between 15–44% (low-middle range) for two reasons. First, based on published [17, 24] and internal analyses, we knew that most outcomes occurred above the 15% threshold (i.e., higher risk patients are more likely to experience adverse outcomes). Second, we reasoned that the greatest potential benefit would be to reclassify mid-level risk patients into either a higher or lower risk band, since very high-risk patients would be unlikely to have their risk estimates changed with new information and lower risk patients may not require additional intervention or follow-up.

Recruitment and interview procedures (Fig. 1 and Appendix 2)

Fig. 1
figure 1

CONSORT diagram

Full size image

Prior to recruitment, we obtained permission to approach patients from the study hospitals’ leadership and individual hospital attending physicians. Physician approvals permitted research staff to approach any patient who was under their care for the duration of the study, given that all eligibility and screening criteria were met. Research staff obtained confirmation from the patient’s nurse that the patient was alert, oriented, and approachable. Recruitment was conducted Monday-Friday between 9:00 AM and 5:00 PM from May 7, 2018 to October 31, 2019.

Research staff approached eligible patients and described the study. Patients who agreed to participate confirmed their identity using an authentication process, provided informed consent, signed an authorization form for use and disclosure of their information, and signed an acknowledgment of receipt of the KPNC Research Participants’ Bill of Rights. Patients then completed a staff-administered interview covering basic demographics, social risk factors, and physical and cognitive functioning. Staff used tablets to enter responses into a secure online data management tool separate from the EHR system, and data for the Patient Reported Outcomes Measurement Information System (PROMIS) questionnaires were entered directly into the PROMIS online Assessment Center (www.assessmentcenter.net). Participants received a $10 gift card/code as a token of our appreciation.

Study measures

Research staff administered interview questionnaires

Our interview questionnaire included the National Institutes of Health’s PROMIS Physical Function and Cognitive Function Abilities Subset questionnaires [25] and social factors items derived from the KP YCLS item bank. PROMIS measures have a mean of 50 and a standard deviation (SD) of 10 in a referent population. Means above 50 and means within 0.5 SDs below 50 are considered within normal limits for function. For our final analyses, we created six social factor predictor variables based on the YCLS items: (1) relationship/marital status (married or living with partner); (2) food insecure (had worried about running out of food sometimes in the prior 3 months or anticipates having trouble paying for food in next 3 months); (3) housing-related concerns (in a temporary housing situation or homeless, concerned about housing conditions, or anticipates having trouble paying for housing or utilities in next 3 months); (4) financial strain (anticipates having trouble paying for ≥ 1 of 9 basic expenses in next 3 months); (5) transportation difficulties (anticipates problems with transportation during next 3 months); and (6) disability and help status (3-level variable: no health problem or disability that limits normal daily activities, limited but has ready access to help with medical needs and daily activities; limited but lacks ready access to help) (see Appendix 1: Interview Instruments).

Electronic health record data

In addition to the EHR information needed to create the composite outcome (non-elective rehospitalization or 30-day mortality post-discharge), we confirmed that patients had Kaiser Foundation Health Plan membership, and extracted the TSL and its individual components (patients’ longitudinal comorbidity burden, severity of illness, length of stay, past healthcare utilization, and code status [17, 23]). As previously described [26], all adults with a KPNC medical record number are assigned a monthly COmorbidity Point Score, version 2 (COPS2, based on Centers for Medicare and Medicaid Services Hierarchical Condition Categories), with increasing COPS2 scores associated with increasing mortality risk [23]. Additionally, patients were assigned a Laboratory-based Acute Physiology Score, version 2 (LAPS2) [23] on admission and every hour after hospitalization. Increasing LAPS2 scores reflect worsening instability – for example, in July 2018, the median hourly LAPS2 among all KPNC intensive care unit patients was 110, whereas the median ward score was 52. It is not possible to admit a patient to KPNC hospitals without specifying code status, which can be subsequently updated. We classified each patient’s care directive as “full code” or “other” (which included “partial code,” “do not resuscitate,” and “comfort care only”) [23]. To compare, we assigned each hospitalization a Charlson Comorbidity Index score (CCI) using the methodology of Deyo et al. [27].

We extracted age at hospitalization; sex; self-reported race, hospitalization venue (via the ED or not); total index hospital length of stay (LOS); whether a patient experienced any overnight inpatient hospitalization in the first 7 days and separately in the 8 to 30 days preceding the index hospitalization [17]; discharge disposition (home; regular or custodial skilled nursing facility, SNF; and Home Health services); and referral to hospice. We classified the principal diagnosis codes using the Health Care Utilization Project (www.ahrq.gov/data/hcup) single-level diagnosis clinical classification software categories and clustered them into 30 groups called Primary Conditions [17, 23].

Statistical methods

All analyses were done in SAS GRID, version 9.04.01M5P091317 and RStudio, version 1.3.1073. We performed bivariate comparisons using Chi-square, t-tests, and Wilcoxon rank-sum tests, as appropriate. We assessed correlations between continuous variables using Spearman’s rho statistic.

For our principal outcome analyses, we employed logistic regression. The dependent outcome was the same composite outcome (non-elective rehospitalization and/or death within 30 days of hospital discharge) that was used to calibrate the TSL score [17]. The independent variables included age, sex, the 6 above-mentioned social factors predictors, the 2 PROMIS scores, the aggregate TSL score, and all individual components of the TSL. In addition to testing various specific combinations (e.g., TSL + social factors items), we tested a random forest model that included all the individual above-mentioned predictors. We assessed model performance by measuring the area under the receiver operator characteristic curve (c statistic), Nagelkerke pseudo-R [2], and Brier score [28,29,30]. For all models, the results we report are those obtained after fivefold cross validation.

Results

During the study recruitment period, 36,086 patients were hospitalized at the three participating sites. Of these patients, 4,811 met all initial screening and eligibility criteria as determined by the research staff. Of these, 2,263 patients were approached by research staff. Of these patients, 1,551 agreed to be interviewed and 712 declined to participate. The remaining 2,548 patients were excluded or not selected (i.e., not approached by research staff). See Fig. 1 for additional context.

During the recruitment period there were 36,086 adult patients hospitalized at the three Kaiser Permanente Northern California participating Medical Centers with an inpatient or observation designation. 2,548 patients not approached for various reasons were categorized as either “excluded” (N = 1,171) or “not selected” (N = 1,377). Excluded patients met most in-hospital eligibility criteria, but were not approached for another reason (i.e., nurse disapproved approaching patient, patient was already discharged, time constraints, or other as noted by research staff – such as “patient sleeping” or “family member visiting”). Patients “not selected” included those determined to be ineligible when research staff reviewed the patient’s electronic medical records on the day of recruitment. Not selected patients included: patients with discharge location unknown; patient continuing care at another facility after discharge; patient began their hospitalization at another non-KP facility and was transferred in; non-KP insurance coverage; comfort care only order in effect, could not confirm eligibility; or could not speak English. There were 1,551 patients interviewed and enrolled in the study. Four patients died prior to discharge and were removed from the analysis.

Table 1 shows the characteristics of patients during their selected hospitalization who were interviewed, excluded or not selected, and all other hospitalized patients (additional comparisons, Appendix 4).

Table 1 COHORT characteristics and unadjusted outcomesa
Full size table

The interviewed and enrolled cohort had more African American patients but fewer Hispanics. When compared to all other hospitalizations, interviewed patients were sicker, with higher CCI, COPS2, and LAPS2 scores. However, the interviewed cohort was healthier, with lower CCI, COPS2, and LAPS2 scores, than those who were excluded or not selected. Only 0.5% of the interviewed cohort was admitted for observation, and LOS was longer, but the distribution of discharge diagnoses was similar to that of the rest of the cohort. Four (0.3%) patients died in the hospital after their interview. In the remaining 1,547 interviewed patients, post-discharge mortality was somewhat lower than in the remaining patients. Compared to the rest of the hospitalized cohort, excluded patients’ TSL scores were higher (mean, 16.2, 11.4, p < 0.0001). Interviewed patients’ TSL scores were somewhat higher than the rest of the hospitalized cohort (12.8 vs. 11.4, p < 0.0001). Table 2 shows that, due to logistic difficulties, we were not successful in oversampling patients with TSL scores ≥ 15%. However, the table shows that the TSL score distribution of the interviewed patients is similar to that of all adult hospitalizations prior to the study and that weighting the sample was not necessary.

Table 2 Targeted and achieved sampling fractionsa
Full size table

Of the 1,547 interviewed patients who left the hospital alive, 212 (13.7%) experienced the adverse composite outcome. Table 3 shows that interviewed patients’ cognitive function (mean, 32.4) was outside the normalized PROMIS range, but their physical function (mean 53.6) was within the normalized PROMIS range.

Table 3 Relationship between predictors and composite outcomea
Full size table

Table 3 shows the two most common social risk factors as presence of a health problem or disability (51.3%) and not living with a partner (47.1%); the prevalence of the remaining social risk factors ranged from 8.4%-15.8%. Table 3 also compares the 212 patients who experienced the composite outcome with the 1335 patients who did not. Patients with the composite outcome had significantly higher CCI, COPS2, LAPS2, and TSL scores (all comparisons, p < 0.0001), and their cognitive function (30.8 vs. 31.8, p = 0.027) and physical function (52.3 vs. 53.9, p = 0.01) scores were lower than those who did not experience the outcome. Patients with the composite outcome were significantly more likely to have transportation difficulties (22.8% vs. 14.8%, p = 0.008) and to have a health problem or disability that affected their activities of daily living (66.0% vs. 49.0%, p < 0.001). Among patients with a health or disability issue, those with the composite outcome were more likely to report having help readily available.

Table 4 shows the performance of different multivariable models for predicting the composite outcome (Appendix 5 shows model coefficients). We found that incorporating the social factors and PROMIS variables did not improve model discrimination or explanatory power over the health and demographic data routinely captured in the EHR. Importantly, a random forest model that included all available predictors did not result in improved performance as compared to the logistic model with TSL. Analyses restricted to patients with mid-range (15.0–44.9%) TSL scores showed similar results.

Table 4 Performance of multivariable predictive models for composite outcomea
Full size table

Correlations across continuous variables (age, LAPS2, COPS2, TSL, and PROMIS) and cross tabulations between these variables and individual social factor variables are provided in Appendix 3. Correlations between the 2 PROMIS scales and the other continuous variables were statistically significant but not necessarily clinically significant, with the highest correlation involving the PROMIS scales found for cognitive function and age (-0.15, p < 0.001). Age was positively correlated with severity of illness (LAPS2), comorbidity burden (COPS2), and TSL score. Cross tabulations between the continuous variables and individual social factors variables did not show a consistent pattern except with respect to the PROMIS physical function scale, where all patients with unfavorable social factors had significantly lower physical function (all 6 comparisons with p < 0.001).

Discussion

Our prospective study with hospitalized patients found that adding patient-reported information about social factors, physical function, and cognitive status did not improve the statistical performance of a predictive model of rehospitalization or death within 30 days post-discharge compared to a model based on EHR clinical data alone. However, we did find that presence of unfavorable patient-reported social risks, lower physical function, and lower cognitive function were independently associated with a higher rate of the composite outcome.

Although we cannot compare our findings to other studies directly, our results are consistent with the findings of Bhavsar et al., who studied 8 outcomes (ED use, hospitalization, outpatient visit use, occurrence of myocardial infarction, stroke, asthma, accidents, and influenza). Their team found that adding neighborhood socioeconomic status variables did not improve upon the statistical performance of predictive models using routinely available clinical data from an EHR [31]. Our findings are also consistent with our own recent analyses, where we found that adding neighborhood socioeconomic status variables did not improve prediction models for mortality and 6 health care utilization variables [32].

It is important to highlight key limitations of our study. First, our cohort was restricted to an insured population in a health system with a relatively high degree of vertical outpatient-inpatient service integration. This integration is manifest in the availability of services aimed at preventing hospitalization, such as chronic condition management programs, a central call center, electronic patient portals, and automated risk detection, which have resulted in a steady decrease in rehospitalization and post-discharge mortality [24]. Second, due to resource limitations, we could not interview patients on weekends, nor could we interview non-English speakers, which latter effect is most evident in the reduced proportion of Hispanic patients in our cohort. Because of state of California regulatory requirements, we could not interview very low- income Kaiser Foundation Health Plan members covered through Medi-Cal (California’s Medicaid program). Thus, it is possible that, because of our setting and selection process, our study sample may not be generalizable to other settings. Third, since social factors and measures of physical and cognitive function were all patient-reported, it is possible that people’s responses did not accurately reflect their true level of vulnerability. These aforementioned exclusions and limitations may have resulted in an overall lower risk hospitalized cohort, thereby reducing our ability for our measures to impact the score. Fourth, it is possible that confounding or redundancy factors were occurring. For example, patients reporting health or disability problems who also had readily available help from others were more likely to experience the adverse composite outcome. This could be because people with health or disability problems that limit their ability to take care of their daily needs are more likely to have help availability than those not so limited. Lastly, our sample might consist of very similar patients for several reasons making it more difficult for a model to discriminate patients by their future outcome. The sample comes from a capitated staff model managed care insured population where patients at the extremes of the social and functional scales might not be represented. We also sampled only low-mid risk patients, and the sample of interviewed patients appeared to be healthier and lower risk than those excluded from the study.

It is important to consider, however, that the most important factor affecting our results is that the outcome time frame (30 days after discharge) is very narrow. Most comorbid conditions develop over many years, and it seems likely that by the time a patient has been hospitalized, the effect of social factors and lower physical and cognitive function has already manifested as a set of comorbid conditions. Not surprisingly then, predictive models that include quantification of comorbidities and acute physiology are unlikely to be affected by including factors that may have existed for years prior to the hospitalization and predisposing patients to develop conditions resulting in hospitalization.

The fact that the patient-reported social factors and PROMIS cognitive and physical function status variables we employed failed to enhance prediction for a 30-day outcome does not mean that ascertaining this information is not valuable. Having this information available in the EHR at time of discharge planning could be of significant value in designing tailored interventions and in assessing which patients might benefit most from specific interventions [33]. For example, patients with social risks or current social needs could be triaged to KPNC Continuum of Care programs such as Hospital-to-Home Transitions and Complex Needs Programs for follow-up in the outpatient setting. Additionally, it is likely that identifying and addressing social risks such as access to healthy food, transportation, a stable and supportive living situation, instrumental and emotional social support, and health literacy will have a positive impact on the recovery process, including reduction in medical complications and potentially preventable ED and outpatient visits. We have shown that, when information on benefit from an intervention is available, this information can be used to develop more accurate rehospitalization predictive models [34]. These variables could enhance the predictive ability of other outcomes in different populations, so it is important to continue to evaluate and quantify the value of collecting these data in studies such as this one.

It is important to recognize another key strength of this study. We were able to successfully collect social factors, physical function, and cognitive status patient-reported information during an inpatient stay using standardized questionnaires administered via tablet. It is feasible to collect this data and knowing this information about a patient at discharge may allow for more complex and tailored interventions aimed at improving outcomes and reducing readmissions [35].

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to their being the property of Kaiser Foundation Health Plan, Inc., but are available to interested collaborators in the context of a formal collaboration approved by the Kaiser Permanente Northern California Institutional Review Board for the Protection of Human Subjects. Dataset requests can be sent to either Dr. Patricia Kipnis, corresponding author, or Ms. Heather A. Clancy, first author.

Abbreviations

CCI:

Charlson Comorbidity Index score

COPS2:

COmorbidity Point Score, version 2 (based on Centers for Medicare and Medicaid Services Hierarchical Condition Categories)

ED:

Emergency department

EHR:

Electronic health record

KPNC:

Kaiser Permanente Northern California

LAPS2:

Laboratory-based Acute Physiology Score, version 2

LOS:

Length of stay

PROMIS:

Patient Reported Outcomes Measurement Information System

SD:

Standard deviation

TSL:

Transition Support Level

YCLS:

Your Current Life Situation

References

  1. Kushel MB, Gupta R, Gee L, Haas JS. Housing instability and food insecurity as barriers to health care among low-income Americans. Research Support, Non-U.S. Gov’t. Research Support, U.S. Gov’t, P.H.S. J Gen Intern Med. 2006;21(1):71–7. https://doi.org/10.1111/j.1525-1497.2005.00278.x.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Seligman HK, Jacobs EA, Lopez A, Sarkar U, Tschann J, Fernandez A. Food insecurity and hypoglycemia among safety net patients with diabetes. Arch Intern Med. 2011;171(13):1204–6. https://doi.org/10.1001/archinternmed.2011.287.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Berkowitz SA, Seligman HK, Choudhry NK. Treat or eat: food insecurity, cost-related medication underuse, and unmet needs. Am J Med. 2014;127(4):303-310e3. https://doi.org/10.1016/j.amjmed.2014.01.002.

    Article  PubMed  Google Scholar 

  4. Herman D, Afulani P, Coleman-Jensen A, Harrison GG. Food Insecurity and cost-related medication underuse among nonelderly adults in a nationally representative sample. Am J Public Health. 2015;105(10):e48-59. https://doi.org/10.2105/ajph.2015.302712.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Gundersen C, Ziliak JP. Food insecurity and health outcomes. Health Aff (Millwood). 2015;34(11):1830–9. https://doi.org/10.1377/hlthaff.2015.0645.

    Article  Google Scholar 

  6. Silverman J, Krieger J, Kiefer M, Hebert P, Robinson J, Nelson K. The relationship between food insecurity and depression, diabetes distress and medication adherence among low-income patients with poorly-controlled diabetes. J Gen Intern Med. 2015;30(10):1476–80. https://doi.org/10.1007/s11606-015-3351-1.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Phipps EJ, Singletary SB, Cooblall CA, Hares HD, Braitman LE. Food insecurity in patients with high hospital utilization. Popul Health Manag. 2016;19(6):414–20. https://doi.org/10.1089/pop.2015.0127.

    Article  PubMed  Google Scholar 

  8. Syed ST, Gerber BS, Sharp LK. Traveling towards disease: transportation barriers to health care access. J Community Health. 2013;38(5):976–93. https://doi.org/10.1007/s10900-013-9681-1.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Keller A, Litzelman K, Wisk LE, et al. Does the perception that stress affects health matter? The association with health and mortality. Health Psychol. 2012;31(5):677–84. https://doi.org/10.1037/a0026743.

    Article  PubMed  Google Scholar 

  10. Verbrugge LM, Balaban DJ. Patterns of change in disability and well-being. Med Care. 1989;27(3 Suppl):S128–47.

    Article  CAS  Google Scholar 

  11. Wilcox VL, Kasl SV, Berkman LF. Social support and physical disability in older people after hospitalization: a prospective study. Health Psychol. 1994;13(2):170–9.

    Article  CAS  Google Scholar 

  12. Oxman TE, Hull JG. Social support, depression, and activities of daily living in older heart surgery patients. J Gerontol B Psychol Sci Soc Sci. 1997;52B(1):P1-14.

    Article  CAS  Google Scholar 

  13. Xu H, Covinsky KE, Stallard E, Thomas J 3rd, Sands LP. Insufficient help for activity of daily living disabilities and risk of all-cause hospitalization. J Am Geriatr Soc. 2012;60(5):927–33. https://doi.org/10.1111/j.1532-5415.2012.03926.x.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Sheikh S. Risk factors associated with emergency department recidivism in the older adult. West J Emerg Med. 2019;20(6):931–8. https://doi.org/10.5811/westjem.2019.7.43073.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hoyer EH, Needham DM, Miller J, Deutschendorf A, Friedman M, Brotman DJ. Functional status impairment is associated with unplanned readmissions. Arch Phys Med Rehabil. 2013;94(10):1951–8. https://doi.org/10.1016/j.apmr.2013.05.028.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Hoyer EH, Needham DM, Atanelov L, Knox B, Friedman M, Brotman DJ. Association of impaired functional status at hospital discharge and subsequent rehospitalization. J Hosp Med. 2014;9(5):277–82. https://doi.org/10.1002/jhm.2152.

    Article  PubMed  Google Scholar 

  17. Escobar GJ, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and postdischarge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–23. https://doi.org/10.1097/MLR.0000000000000435.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Marafino BJ, Escobar GJ, Baiocchi MT, Liu VX, Plimier CC, Schuler A. Evaluation of an intervention targeted with predictive analytics to prevent readmissions in an integrated health system: observational study. BMJ. 2021;374:n1747. https://doi.org/10.1136/bmj.n1747.

    Article  PubMed  PubMed Central  Google Scholar 

  19. LaForge K, Gold R, Cottrell E, et al. How 6 organizations developed tools and processes for social determinants of health screening in primary care: an overview. J Ambul Care Manage Jan/Mar. 2018;41(1):2–14. https://doi.org/10.1097/jac.0000000000000221.

    Article  Google Scholar 

  20. Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–9.

    Article  Google Scholar 

  21. Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra-hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80. https://doi.org/10.1002/jhm.817.

    Article  PubMed  Google Scholar 

  22. Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–30. https://doi.org/10.1002/jhm.964.

    Article  PubMed  Google Scholar 

  23. Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–53. https://doi.org/10.1097/MLR.0b013e3182881c8e.

    Article  PubMed  Google Scholar 

  24. Escobar GJ, Plimier C, Greene JD, Liu V, Kipnis P. Multiyear rehospitalization rates and hospital outcomes in an integrated health care system. JAMA Netw Open. 2019;2(12):e1916769. https://doi.org/10.1001/jamanetworkopen.2019.16769.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Ader DN. Developing the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5):S1–2. https://doi.org/10.1097/01.mlr.0000260537.45076.74.

    Article  Google Scholar 

  26. Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med. 2020;383(20):1951–60. https://doi.org/10.1056/NEJMsa2001090.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613–9. https://doi.org/10.1016/0895-4356(92)90133-8.

    Article  CAS  PubMed  Google Scholar 

  28. Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;78(1-3). https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

  29. Nagelkerke N. A note on a general definition of the coefficient of determination. Biometrika. 1991;78:691–2.

    Article  Google Scholar 

  30. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–35.

    Article  Google Scholar 

  31. Bhavsar NA, Gao A, Phelan M, Pagidipati NJ, Goldstein BA. Value of neighborhood socioeconomic status in predicting risk of outcomes in studies that use electronic health record data. JAMA Netw Open. 2018;1(5):e182716–e182716.

    Article  Google Scholar 

  32. Schuler A, O’Suilleabhain L, Rinetti-Vargas G, et al. Assessment of value of neighborhood socioeconomic status in models that use electronic health record data to predict health care use rates and mortality. JAMA Netw Open. 2020;3(10):e2017109. https://doi.org/10.1001/jamanetworkopen.2020.17109.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Calvillo-King L, Arnold D, Eubank KJ, et al. Impact of social factors on risk of readmission or mortality in pneumonia and heart failure: systematic review. J Gen Intern Med. 2013;28(2):269–82. https://doi.org/10.1007/s11606-012-2235-x.

    Article  PubMed  Google Scholar 

  34. Marafino BJ, Schuler A, Liu VX, Escobar GJ, Baiocchi M. Predicting preventable hospital readmissions with causal machine learning. Health Serv Res. 2020;55(6):993–1002. https://doi.org/10.1111/1475-6773.13586.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Leppin AL, Gionfriddo MR, Kessler M, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–107. https://doi.org/10.1001/jamainternmed.2014.1608.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This project was funded by Kaiser Foundation Hospitals, Inc., and The Permanente Medical Group, Inc. Dr. Vincent Liu was also funded by grant R35GM128672 from the National Institutes of Health. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. We would like to thank Jasmine Deras, Cecilia Doan, Jenna Hoyt, Caitlin Lydon, and Dana Morales for conducting the interviews, Adriana Martinez for helping to coordinate the study, Dr. Lawrence Gerstley for technical assistance in patient selection and identification. We would also like to thank Angela Wahleithner, Cates De Castro, and Kristina Flanagan for administrative guidance as well as Anessa Hileman, Anna Liza Azaula, and Maria Lani Malicdem for operational guidance and support at the Oakland, San Leandro, and Walnut Creek hospitals. We would like to thank the clinical, nursing, and care coordination staff at the 3 participating medical centers including Dr. Yu-Te Lee, Dr Rajeeva Ranga, Dr. Anna Bavykina, Ed Chan, Bri Dinoso, Miraslava Harter, Bernadette Yee, and Rachelle-ann Neitte among many others who supported these efforts and/or worked with our team to make conducting the study as seamless as possible. We wish to thank Drs. Tracy Lieu, Stephen Parodi, and Diane Brown for reviewing the manuscript and Ms. Kathleen Daly for editing and formatting the manuscript.

Funding

This project was funded by Kaiser Foundation Hospitals, Inc., and The Permanente Medical Group, Inc. Dr. Vincent Liu was also funded by grant R35GM128672 from the National Institutes of Health.

Author information

Authors and Affiliations

  1. Systems Research Initiative, Kaiser Permanente Division of Research, 2000 Broadway Avenue, Oakland, CA, 94612, USA

    Heather A. Clancy, Zheng Zhu, Nancy P. Gordon, Patricia Kipnis, Vincent X. Liu & Gabriel J. Escobar

  2. Intensive Care Unit, Kaiser Permanente Medical Center, 700 Lawrence Expressway, Santa Clara, CA, 95051, USA

    Vincent X. Liu

Authors
  1. Heather A. Clancy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nancy P. Gordon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patricia Kipnis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vincent X. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gabriel J. Escobar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Each author made substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data; or drafted the work or substantively revised it. Each author approved the submitted version and agrees both to be personally accountable for their own contribution and ensures that questions related to the accuracy or integrity of any part of the work will be appropriately investigated, resolved, and the resolution documented in the literature. The corresponding authors ensure that all listed authors have approved the manuscript.

Corresponding author

Correspondence to Patricia Kipnis.

Ethics declarations

Ethics approval and consent to participate

A designated member of the Kaiser Permanente Northern California Institutional Review Board conducted an expedited review and approved the study, # CN-17–3119.

Informed Consent & Authorization Waiver Determinations:

  • 1. The IRB waived the Privacy Rule authorization requirement for the purposes of, and only for the purposes of, recruitment. This waiver was granted to allow the research team to access Protected Health Information prior to obtaining written participant authorization.

  • 2. The IRB waived the requirement that Privacy Rule authorization be obtained from patients who were not enrolled in the study (i.e., patients in the comparison group) for the medical records review (i.e., data only) portion of the study.

  • 3. The IRB waived the requirement that informed consent be obtained from patients who were not enrolled in the study (i.e., patients in the comparison group) for the medical records review (i.e., data only) portion of the study.

Physician approvals permitted research staff to approach any patient who was under their care for the duration of the study, given that all eligibility and screening criteria were met. All methods were performed in accordance with the relevant guidelines and regulations.

Research staff approached eligible patients and described the study. Patients who agreed to participate confirmed their identity using an authentication process, provided informed consent, signed an authorization form for use and disclosure of their information, and signed an acknowledgment of receipt of the KPNC Research Participants’ Bill of Rights. All signatures, including for the informed consent, were collected electronically via tablet.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Appendix 1

. Interview instruments. Appendix 2. Expanded detail on sampling & recruitment. Appendix 3. Supplemental analyses in the interviewed population. Appendix 4. Table 1 expanded cohort characteristics and unadjusted outcomes. Appendix 5. Logistic regression beta coefficients and odds ratios for the predictive models for composite outcome.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Clancy, H.A., Zhu, Z., Gordon, N.P. et al. Prospective evaluation of social risks, physical function, and cognitive function in prediction of non-elective rehospitalization and post-discharge mortality. BMC Health Serv Res 22, 574 (2022). https://doi.org/10.1186/s12913-022-07910-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-07910-w

Keywords

  • Social factors
  • Social determinants of health
  • Cognitive function
  • Physical function
  • Post-discharge outcomes
  • Readmission risk
  • Predictive modeling

BMC Health Services Research

ISSN: 1472-6963

Contact us