A study to reduce readmissions after surgery in the Veterans Health Administration: design and methodology

Background Hospital readmissions are associated with higher resource utilization and worse patient outcomes. Causes of unplanned readmission to the hospital are multiple with some being better targets for intervention than others. To understand risk factors for surgical readmission and their incremental contribution to current Veterans Health Administration (VA) surgical quality assessment, the study, Improving Surgical Quality: Readmission (ISQ-R), is being conducted to develop a readmission risk prediction tool, explore predisposing and enabling factors, and identify and rank reasons for readmission in terms of salience and mutability. Methods Harnessing the rich VA enterprise data, predictive readmission models are being developed in data from patients who underwent surgical procedures within the VA 2007–2012. Prospective assessment of psychosocial determinants of readmission including patient self-efficacy, cognitive, affective and caregiver status are being obtained from a cohort having colorectal, thoracic or vascular procedures at four VA hospitals in 2015–2017. Using these two data sources, ISQ-R will develop readmission categories and validate the readmission risk prediction model. A modified Delphi process will convene surgeons, non-surgeon clinicians and quality improvement nurses to rank proposed readmission categories vis-à-vis potential preventability. Discussion ISQ-R will identify promising avenues for interventions to facilitate improvements in surgical quality, informing specifications for surgical workflow managers seeking to improve care and reduce cost. ISQ-R will work with Veterans Affairs Surgical Quality Improvement Program (VASQIP) to recommend potential new elements VASQIP might collect to monitor surgical complications and readmissions which might be preventable and ultimately improve surgical care.


Background
Readmissions following hospital discharge result in increased costs, resource utilization and worse patient outcomes [1,2]. Reducing avoidable readmission is therefore a quality improvement goal. As part of the Affordable Care Act (ACA), excessive rates of readmissions now result in reduced hospital Medicare reimbursement, affecting up to two-thirds of US hospitals in the private and federal sectors [3]. The Veterans Health Administration (VA) is not yet directly penalized for readmissions by ACA or VAspecific payment rules. However, it is under increased pressure to become more transparent in its processes of care. VA makes public both facility and nationwide readmission rates each quarter. Similar to the private sector, the VA now reports outcome measures, including readmissions, on the Centers for Medicare and Medicaid Services Hospital Compare website.
Readmission following admission for specific medical conditions has been the focus of many studies, but much less is known about factors associated with surgical readmission. In contrast to medical admissions, index surgical admissions are usually planned, patients are screened for risk factors prior to admission, and surgical procedures have well-known and expected complication rates. Thus, correlates of unplanned readmission after surgery may differ markedly from those of medical readmissions. Primary among these are older age and comorbidity burden, [4][5][6] poor social or caregiver support, [7] more numerous prescription medications at discharge, [6] previous hospitalizations, [8] and functional disability [9]. Predisposition to readmission also varies by surgical procedure type: colorectal surgeries have relatively high rates of surgical site infection and gastrointestinal complications, [10] abdominal procedures are associated with dehydration and ileus, [4] and vascular surgeries are associated with readmission for bleeding, groin wound complication and wound debridement [11].
To date, much of our knowledge regarding the quality of surgical care in the VA comes from the Veterans Affairs Surgical Quality Improvement Program (VASQIP) which has been adopted by portions of the private sector [12]. VASQIP assesses complications that occur after surgery but is not an exhaustive source of post-operative complications; readmission may be a proxy for other post-operative factors not currently assessed.
To address the knowledge gap we have initiated a study, "Improving Surgical Quality: Readmission (ISQ-R)", a mixed methods study comprising retrospective analysis and prospective surveys regarding surgeries with consistently high rates of unplanned readmissions: colorectal, thoracic and vascular procedures [13]. ISQ-R will (a) develop risk prediction models of 30-day readmission and classify reasons for readmission in retrospective data; (b) prospectively collect patient psychosocial factors at discharge and correlate with 30-day readmission; and (c) assemble a Delphi panel of experts to rank the identified reasons for readmission with respect to their potential preventability and suitability for use as care quality measures.

Study design and aims
An overview of the study aims and methods is presented graphically in Table 1. The first ISQ-R aim is to evaluate the contribution of patient, procedure, postoperative and system factors to unplanned readmission within 30 days of hospital discharge following major surgery (open procedures for definitive treatment, with at least 2-day stay), [14] and use these data to develop and validate a  Fig. 1). We will develop a classification of readmission reasons and explore processes of care linked with readmission using a cohort of VASQIP-assessed major surgeries from 2007 to 2012. Reasons for readmission will be identified by International Classification of Disease, 9 th Edition (ICD9) diagnosis codes and categorized by etiology [15]. Our second aim is to assess potential patient factors not currently collected by VASQIP and analyze their association with readmission using surveys at four VA facilities. The readmissions literature on surgical populations hypothesizes that social support and caregiver factors may contribute to such events [16,17]. To frame our study we used the Andersen Behavioral Model for Healthcare Utilization, a model frequently used to explain variation in health services utilization [18]. Andersen proposes three main classes of factors: (1) Need, (2) Predisposing and (3) Enabling. Andersen's Behavioral Model was designed to describe factors explaining healthcare use and equitable access. It has been applied to both acute care and emergency department utilization [19,20]. Need is the primary and most proximate predictor of healthcare utilization, operationalized by surgery and comorbidity. Predisposing factors (i.e., age, education, self-efficacy, self-management, cognitive function, mood) and enabling factors (i.e., transportation, caregiver status, access to primary care) affect the likelihood of utilizing healthcare and further explain or predict variation in utilization. In the case of 30-day readmissions, predisposing factors expected to correlate most strongly with post-operative readmissions include poor self-efficacy and poor self-management skills while predominant enabling factors include insufficient caregiver help.
Our third aim is to rank reasons for readmission developed from Aims 1 and 2 for (a) potential preventability and (b) appropriateness as a measure of surgical quality. Using a Delphi process, our panel will rate reasons by potential preventability and potential to serve as a measure of surgical quality beyond the current VASQIP measures. The study was approved by the VA Central Institutional Review Board prior to initiation.

Cohort
We will develop two study cohorts, one retrospective and one prospective. The retrospective cohort for Aim 1 will consist of patients with VASQIP-assessed major procedures in 2007-2012 with post-operative length of stay (LOS) of 2-30 days and alive at discharge [21]. This cohort will be linked with VA administrative data to examine predictors of readmission within VA [22,23]. The prospective cohort will consist of patients in four VA facilities undergoing colorectal, thoracic or vascular surgeries over a 12-month period. We chose these surgeries as they have high rates of readmission; however, the reasons for readmission differ across groups. Colorectal cases are associated with more infectious and electrolyte disturbance reasons, whereas vascular and thoracic cases are associated with more cardiovascular and bleeding reasons. We will use this prospective cohort to validate the surgical readmission risk prediction model and model reasons for readmissions.
We addressed two major considerations when constructing our cohorts. First, what constitutes major Major surgical procedures are more likely to have postoperative complications, consume more resources and have a higher rate of readmissions directly linked to the surgical intervention. For the readmission risk prediction model, we are interested in modeling readmission after surgery rather than unplanned admission after an outpatient procedure. To remove most outpatient cases, we require a minimum of two postoperative hospital days. Most procedures that can be performed on an outpatient basis will have a LOS of zero or one (overnight observation). The 2-day requirement also excludes most diagnostic procedures, such as intra-operative arteriograms and cystoscopies. Furthermore, our cohort is derived from VASQIP, which preferentially assesses major surgeries. The 30-day timeframe is commonly used; we will explore shorter and longer periods for readmission for our patients undergoing colorectal, thoracic or vascular surgery.

Data source: VASQIP
VASQIP began in 1991 to analyze risk-adjusted 30-day morbidity and mortality within VA [14,21]. Of approximately 375,000 surgical procedures in the VA yearly, 150,000 are major surgeries [24]. Complete assessment of perioperative data is documented on~100,000 cases per year using a defined sampling algorithm to ensure adequate representation of the breadth and complexity of procedures. Data are abstracted from computerized sources. On the 30 th postoperative day, outcomes are collected from charts, morbidity and mortality conferences and communication with patients.
VASQIP currently assesses 21 complications within 30 days postoperatively and monitors 62 pre-surgical clinical and laboratory and 12 intraoperative risk factors. The risk-adjusted data obtained from this program have been reported back to VA medical centers and led to improvements in morbidity and mortality rates [25]. We will use selected variables from VASQIP (see Table 2).

VA administrative data
Administrative data in the VA's centralized data repositories reliably capture all inpatient and outpatient healthcare encounters [26,27]. Inpatient files include details of the stay: admission, discharge, diagnoses, procedures, transitions between bed sections (e.g., from surgical intensive care unit (ICU) to general medical), and provider types. Outpatient files document post-discharge follow-up care and emergency department visits. Files include age, gender, marital status, race/ethnicity, laboratory results, vital signs and pharmacy fills. Dates of death come from the VA Mini-Vitals, derived via validated algorithm with estimated 98% sensitivity from four sources: VA inpatient records, Veteran death benefits claims, Social Security and Medicare files [28].
VA patients may use non-VA healthcare. To examine the contribution of non-VA data to assessing readmissions, we used the Cardiac Stent Risk study [29] and identified 30-day readmissions within VA and Medicare/ Medicaid files following non-cardiac surgery (see Table 3). Medicare/Medicaid Inpatient Institutional Claims identified readmissions to a non-VA facility following a VA surgery. While 30-day readmission rates were higher for veterans with Medicare/Medicaid versus without, the majority of patients (92%) were readmitted to a VA facility. Therefore, we will not use Medicare/Medicaid data in ISQ-R.

Survey data
To address the predisposing and enabling factors targeted in Aim 2, the medical record alone would be insufficient. Therefore surveys will collect psychosocial factors from patients and caregivers. To improve the generalizability of our results, we will collect prospective data in four geographically distinct VA facilities: Birmingham, Boston, Milwaukee and Palo Alto.
The construct "discharge complexity" is one we felt was important although it is not routinely measured. Unable to identify any validated tool, we developed one specific to our study in preliminary work. The tool quantifies the complexity of the patient's discharge instructions in the following domains: a) wound care; b) indwelling catheter/drain care; c) new ostomy (colorectal) or amputation (vascular); d) durable medical equipment; e) number of medications; and f ) number of new medications. We pilot tested this tool in 30 patients, finalized the measure and will employ it in Aim 2 with 800 patients and their caregivers.
Other factors to be collected include activities of daily living, pain, perceived stress, depression, cognition and caregiver accessibility (see Table 4).

Study outcomes
The primary outcome of unplanned readmission will be reviewed for 30-day vs other possible time frames. In preliminary work, we estimated the median time to readmission as 10 days (IQR 4-18 days; unpublished data). Thus, 14 days is too short to capture many readmissions, while readmissions beyond 30 days are less likely to be linked to the index surgery. We will perform analyses to determine how shorter or longer periods may inform readmissions attributable to the surgical episode of care.

Analysis plan
Model building for Aim 1 Logistic regression will be performed to investigate the predictors of 30-day readmission in VASQIP-assessed cases, evaluating the contribution of patient, procedure and system factors regardless of post-discharge mortality. To consider the relationship between death and readmission, sensitivity analyses will proceed in parallel to the main analyses using logistic regression to model the composite outcome of readmission or death, and Cox regression to model time to readmission considering death as a censoring event.
In building the predictive models we will consider surgical admissions to be nested within facilities and will use Generalized Estimating Equations (GEE) to account for clustering. Analyses will be repeated using generalized linear mixed models considering patients nested within facilities and surgeries within patients to check for consistency. We will estimate the intra-class correlation resulting from multiple observations per patient, and repeat models limiting cases to one surgery per patient to understand the impact of repeated surgeries.
The risk prediction models to identify at-risk patients in real time will operate at two distinct time points: prior to admission for surgery, and prior to discharge after surgery. Building the pre-admission model will proceed in stages. Three separate models will be constructed corresponding to pre-operative patient factors, procedure factors and hospital factors (see Table 5). After trimming non-significant terms, models will be compared on their residual deviance. The model with the lowest deviance will serve as the foundation, adding remaining covariates and retaining only those attaining significance at p < 0.05; this approach ensures retention of a reasonably small set of predictors, consistent with potential realtime incorporation into system operations when a quick result is requisite. Results will be compared to a full model containing all predictors. The readmission predictive model will be constructed by adding postoperative covariates to the pre-operative model, incorporating statistically significant post-operative variables into the final model. In addition, a full model will be estimated for comparison.

Risk prediction tool
The starting point for constructing the risk prediction tool will be the final parsimonious pre-operative and time-of-discharge predictive models described. A second  pair of pre-operative and time-of-discharge models will be developed using classification and regression trees (CART) including all available covariates. The betterperforming model at each time point will be identified using split-sample validation and bootstrap replication. Each bootstrap replication will be randomly partitioned into training (80%) and testing (20%) subsets. The regression and tree models will each be fit on the training subset, and their predictions on the testing subset will be used to calculate separate c-statistics for each model. T-tests will assess differences in mean c-statistics between models across replications. In the event the performance of the logistic regression models is superior, the corresponding tree models will be retained for predicting readmission in cases involving missing data.

Development of readmission reason categories
We will use the primary diagnosis for each readmission to classify readmission reason and develop readmission categories separately for colorectal, thoracic and vascular cases. To refine categories and develop subcategories, we will calculate frequency of readmission categories for the overall cohort and by surgery. Our experience with colorectal surgery is illustrative. In analyses preparatoryto-research, 1161 patients (14%) were readmitted within 30 days following 8180 elective admissions for colorectal surgery. The top 10 primary diagnoses for readmission accounted for 58% of the overall readmissions (Table 6). After identifying the top diagnoses in our ISQ-R cohorts, we will develop broad categories, e.g., (1) surgical site infection, peritoneal abscess, wound disruption; (2) electrolyte disturbance (dehydration, acute kidney failure); (3) non-surgical site infection (urinary tract infection, Clostridium difficile); (4) ileus/obstruction. We will also consider secondary diagnoses as these may reveal patterns not evident from the primary diagnoses alone. The goal is to develop clinically meaningful categories that can be linked to processes of care and are potentially actionable. We base this iterative process and vetting of categories on our prior work with the National Surgery Office in identifying reasons for surgical case cancellations. As part of a technical assistance project, we summarized 9528 reasons into six actionable categories that are now used in tracking and reporting surgical case cancellations [30]. Once the investigative team has developed readmission reason categories and data definitions, these will be vetted and revised with the National Surgery Office Advisory Board. The readmission reason categories will be tested in our prospective assessment of surgery patients in Aim 2. Table 4 Measures collected from patients undergoing colorectal, thoracic and vascular operations in four VA medical centers

RUG-ADL Assessment
The Resource Utilization Group -Activities of Daily Living Assessment measures functional status at discharge, readmission and at 30-day follow-up. This four-item validated questionnaire assesses a patient's independence with mobility, toileting, transfer and eating [39].

Pain
The National Institutes of Health Numeric Rating Pain Scale assesses pain intensity at the time of discharge, readmission and 30-day follow-up [40].
Pain Meds Total dose of pain medication administered 24 h (7 am-7 am) before discharge Perceived Stress Scale Cohen's Perceived Stress Scale (PSS) is a 10-item scale that quantifies patient's stress. The PSS has been shown to correlate with health behavior and health services utilization [41].

CES-D4
The Center for Epidemiologic Studies Depression Screen, 4-item version, assesses psychological distress at the time of discharge. The CES-D4 has been shown to be have a positive predictive value of 85% for depression in an older adult population [31].

MoCA
The Montreal Cognitive Assessment is a validated tool to assess a patient's cognitive function. This tool has a positive predictive value of 89% for mild cognitive impairment (90% sensitivity; 87% specificity) when compared to clinical criteria supported by psychometric measures [42].
Caregiver Accessibility These questions were guided by the literature around the immediacy and availability of the designated caregiver (i.e., does caregiver live with patient) [43,44].
Transportation Patient access to transportation and burden of transportation (number of post-operative visits and travel distance).

CTM-15, adapted
The Care Transition Measure is a 15-item scale that addresses the hospital's efforts at care coordination at discharge. The survey also assesses patient self-efficacy in implementing the discharge plan. The tool was designed as a post-discharge, recall assessment [45]. We will adapt the tool: a) to assess these items on the day of discharge; b) to assess patient self-efficacy with wound, indwelling device, new ostomy or durable medical equipment, as applicable; and c) to assess patient understanding of whom and when to contact regarding warning signs or symptoms that may arise.

Institute for Healthcare Improvement Readmission Tool
This tool was developed by IHI as part of a conceptual roadmap to reduce avoidable re-hospitalizations by intervening at the system level. The tool will be adapted for surveying patients at readmission [46].
Brief Survey of Post-operative Care Queries patient on unplanned emergency visit or readmission at an outside hospital; keeping post-operative appointments; difficulty getting medications filled (costs) and refilled (especially pain medication); receipt of home health or durable medical equipment.

Processes of care and surgical readmission for Aim 2
We developed two exploratory hypotheses to analyze our prospective cohort. Regression models will examine whether readmissions attributed to electrolyte disturbances are associated with increases in serum creatinine greater than 0.5 mg/dl from admission to discharge, and whether mean and median pain score in the 24 h prior to discharge (scale 1-10) are associated with 30-day readmission. We will assess whether additional predisposing and enabling factors are associated with: 1) 30-day readmission, 2) unplanned emergency/physician visit within 30 days, and 3) composite 30-day readmission and/or unplanned visit. Sub-analyses will examine the associations between study outcomes and depressive symptoms (as measured by the Center for Epidemiologic Studies Depression Scale, 4-item version or CESD4) [31]. As an exploratory analysis, we will also test whether a discharge prescription for antidepressants moderates the association between CESD4 score and readmission/unplanned visit. The parsimonious model will assess odds of readmission/unplanned visit as a function of CESD4 by antidepressant (interaction term).
We will carefully examine the relationships between readmission/unplanned visit outcomes and enabling/predisposing factors in the context of the Andersen model, adding each factor to the at-discharge model from Aim 1. A final parsimonious model will result from forwardstepwise selection augmenting the at-discharge model with significant predisposing and enabling factors. The final prospective models will be developed and tested using split sample validation and bootstrapping as above, retaining the model with the best split-validation performance. Then we will determine whether the final prospective model has greater discrimination than the final retrospective at-discharge model from Aim 1.

Reasons for readmission -Aim 3
We will convene a Delphi panel of surgeons (general, vascular and thoracic), non-surgeon clinicians (hospitalist, general internist, emergency department physician) and quality improvement nurses to review the reasons for readmission among VASQIP-assessed colorectal, thoracic and vascular cases. The panel will rank the degree to which the reasons for readmission developed in Aims 1 and 2 are (a) potentially preventable and (b) appropriate measures of surgical quality. We will use the RAND Appropriateness Method (RAM), [32] widely used in the development of patient safety and quality improvement tools [33][34][35]. Our team has employed the modified RAM to identify the potential preventability of inpatient readmissions for acute myocardial infarction, heart failure and pneumonia [36,37]. Table 5 Specific variables to be included in the three separate models described in aim 1 Our modified RAM consensus-building process entails these steps: 1) Define the goal of the quality measure, 2) Review the literature for similar measures, 3) Collect clinical input and data analysis to develop the measure, and 4) Consensus process -convene a panel of experts to rank each measure on a scale of appropriateness, ranging from 1 (extremely inappropriate) to 9 (extremely appropriate) [32]. We will consider each category for colorectal, thoracic and vascular cases and determine whether it incorporates preventability and poor quality of care.
We define consensus according to the RAM: all but one Delphi panelist scores within a range of 3 points around the median rank. To determine which reasons for readmission were most preventable or useful, we will use the following criteria: 1) consensus among participants, and 2) a median rating for "appropriateness as a measure of preventability for readmission" or "appropriateness as a measure of surgical quality" of 7 or higher.
At the completion of the modified Delphi process, we will have established consensus around which reasons for readmission are most likely to be potentially preventable and which reasons indicate poor surgical quality for colorectal, thoracic and vascular surgery patients.

Discussion
The study described herein is underway. Some of the large data analyses have been completed, quantifying preoperative, intraoperative and postoperative factors as important but limited predictors of readmission (10% of variance explained) [38]. Collection of psychosocial data from the prospective cohort is underway, as is the process of understanding reasons for readmission per the panel of experts. The psychosocial predictors may prove valuable for identifying and improving management of at-risk surgical patients, and possibly discharge planning may be tailored to the patient's medical and psychosocial status. For patients not yet readmitted prior to the follow-up outpatient appointment typically scheduled for 2-3 weeks post-discharge, failure to attend the scheduled appointment could trigger immediate outreach, as those patients could be coping with situations that could lead to long-term poor outcomes, such as inadequate transportation or other social supports impacted by their changed physical status.

Limitations
The types of surgery studied are limited to threecolorectal, thoracic and vascular, while other types were excluded, therefore the factors identified and conclusions drawn may not be applicable to arthroscopic, orthopaedic, gynecologic, obstetric, head-and-neck, spinal, trauma-related and other types of procedures. This initial choice seemed reasonable given the higher rates of unplanned readmissions following these categories of procedures. The Delphi panel does not include non-hospital clinicians and laypeople, and therefore may fail to identify reasons that are salient to those groups or less apparent to hospital clinicians.

Future directions
We expect that this study will lead to improved understanding of the reasons for readmission following colorectal, thoracic and vascular surgical procedures. This may allow us to develop novel quality measures based on readmissions in certain categories. Moreover, we will be able to develop and test interventions that address factors associated with preventable readmission. Future work should expand the Delphi panel to include community providers, caregivers and patients to deepen our understanding of social context, and allow clinicians to evolve a patient-centred approach to interventions for reducing readmissions. Depending on our findings, these interventions may focus on enhancing structural factors, peri-discharge practices or patient and caregiver support.
Identifying the factors that predict readmission will transition the field from the current narrow focus on operative and hospital care to an approach that pays equal attention to addressing social determinants of health and patient and caregiver knowledge, attitudes and skills. Incorporating veteran-centric care while helping our patients stay out of the hospital furthers VA's goal to provide the highest quality care possible.