Comparison of measures of comorbidity for predicting disability 12-months post-injury

Background Understanding the factors that impact on disability is necessary to inform trauma care and enable adequate risk adjustment for benchmarking and monitoring. A key consideration is how to adjust for pre-existing conditions when assessing injury outcomes, and whether the inclusion of comorbidity is needed in addition to adjustment for age. This study compared different approaches to modelling the impact of comorbidity, collected as part of the routine hospital episode data, on disability outcomes following orthopaedic injury. Methods 12-month Glasgow Outcome Scale – Extended (GOS-E) outcomes for 13,519 survivors to discharge were drawn from the Victorian Orthopaedic Trauma Outcomes Registry, a prospective cohort study of admitted orthopaedic injury patients. ICD-10-AM comorbidity codes were mapped to four comorbidity indices. Cases with a GOS-E score of 7–8 were considered “recovered”. A split dataset approach was used with cases randomly assigned to development or test datasets. Logistic regression models were fitted with “recovery” as the outcome and the performance of the models based on each comorbidity index (adjusted for injury and age) measured using calibration (Hosmer-Lemshow (H-L) statistics and calibration curves) and discrimination (Area under the Receiver Operating Characteristic (AUC)) statistics. Results All comorbidity indices improved model fit over models with age and injuries sustained alone. None of the models demonstrated acceptable model calibration (H-L statistic p < 0.05 for all models). There was little difference between the discrimination of the indices for predicting recovery: Charlson Comorbidity Index (AUC 0.70, 95% CI: 0.68, 0.71); number of ICD-10 chapters represented (AUC 0.70, 95% CI: 0.69, 0.72); number of six frequent chronic conditions represented (AUC 0.70, 95% CI: 0.69, 0.71); and the Functional Comorbidity Index (AUC 0.69, 95% CI: 0.68, 0.71). Conclusions The presence of ICD-10 recorded comorbid conditions is an important predictor of long term functional outcome following orthopaedic injury and adjustment for comorbidity is indicated when assessing risk-adjusted functional outcomes over time or across jurisdictions.

This study compared different approaches to modelling the impact of comorbidity, collected as part of the routine hospital discharge data, on 12-month disability outcomes in an orthopaedic injury population to inform the development of risk adjustment models for disability outcomes. A key additional aim was to establish whether the inclusion of comorbidity provides additional important predictive value in addition to adjustment for age alone.

Data
The Victorian Orthopaedic Trauma Outcomes Registry (VOTOR) monitors the management and outcomes following admission to hospital for adults with acute orthopaedic injury [17]. This sentinel site registry collects data about all orthopaedic trauma admissions to four participating hospitals in the state of Victoria, Australia. The contributing hospitals include two adult major trauma services (Level 1 trauma centre equivalent), one large regional (rural) hospital, and one large metropolitan hospital, to ensure a wide representation of orthopaedic trauma patients in the registry. All eligible cases are included on the registry using an opt-off consent process [18]. The registry is approved by the Human Research Ethics Committee at each participating hospital and Monash University.

Procedures
Cases included survivors to hospital discharge, admitted between March 2007 and July 2010. Demographic, injury event, International Classification of Diseases 10th revision Australian Modification (ICD-10-AM) diagnosis codes (up to 40 per admission), and 12-month functional outcomes data were extracted for analysis.
ICD-10-AM orthopaedic injury diagnosis codes were mapped to 10 orthopaedic injury groups. Indicator variables were generated to represent important nonorthopaedic injuries, including variables for intracranial injury and/or skull fracture, multiple rib fractures, intraabdominal or intra-thoracic organ injury, and burns.

Comorbidity measures
ICD-10-AM diagnosis codes are the source of comorbidities for VOTOR. In Victoria, each code contains a prefix with "P" representing principal diagnosis requiring treatment during the stay, "A" representing additional diagnoses and "C" representing in-hospital complications.
Australian Coding Standards specify that most conditions should be coded as additional diagnoses if they affect treatment, require investigation, or use resources during the admission. However, the standards require coding of certain conditions whenever they are present; some communicable diseases (e.g. HIV, viral hepatitis), diabetes and pregnancy [19].
For the purposes of this study, the following were excluded from mapping to comorbidity measures: i. All in-hospital complications ("C" prefix codes); ii. Chapter XIX (Injury, poisoning and certain other consequences of external causes) with a "P" prefix (indicating an injury principal diagnosis); iii. All Chapter XVIII (Symptoms, signs and abnormal clinical and laboratory findings, not elsewhere classified) as these are not comorbidities; iv. Chapters XX (External causes of morbidity and mortality) to XXII (Codes for special purposes) as these are not comorbidities.
Remaining "P" and "A" prefixed codes were checked by the authors (PAC, RAL, BJG) to remove erroneously coded complications. These included acute posthaemmorhagic anaemia, acute subendocardial myocardial infarction, acute renal failure, acute respiratory infections, fever and post-traumatic amnesia. Codes remaining were assumed to represent comorbidities.
Four methods for classifying comorbidity were used and the ICD-10-AM codes mapped to the four indices. These were: (i) Charlson Comorbidity Index (CCI); (ii) Functional Comorbidity Index (FCI); (iii) categorisation by ICD-10 chapter; and (iv) the six conditions described by Haagsma et al. [7].

Charlson comorbidity index (CCI)
The 19 Charlson conditions were mapped to the CCI from the ICD-10-AM codes (Table 1), resulting in a weight of 1, 2, 3 or 6 in accordance with Charlson's recommendations, and zero if no CCI codes were recorded [5]. The CCI is used widely for outcome adjustment in the injury literature [10,11,14].

ICD-10 chapters
Indicator variables were generated for the presence or absence of conditions in Chapters I to XVII, and Chapter XIX. A variable indicating the number of chapters represented was generated, an approach used by Cameron et al. [10].
Six frequent chronic conditions described by Haagsma et al.
Haagsma et al. investigated the impact of comorbidity on disability weight estimates in a sample of 2,295 injured patients in the Netherlands [7]. The six most common self-reported chronic diseases were: (i) chronic non-specific lung disease; (ii) heart disease; (iii) diabetes; (iv) backache; (v) osteoarthritis; and (vi) rheumatoid arthritis. All other comorbidities are considered as "other". Indicator variables for these conditions were mapped from the ICD-10 codes ( Table 1).
The key differences between the comorbidity indices used relate to the number of conditions represented. The ICD chapter approach maps all available ICD-10 comorbidity codes into 18 chapter-related groups, but does not specifically identify individual conditions. For example, diabetes is included in the "Endocrine, nutritional and metabolic disorders" chapter. The six frequent chronic conditions described by Haagsma et al. [7] also uses all available ICD-10 comorbidity codes but only six specific diagnoses are included, with all remaining comorbidities grouped together in an "other" category. The FCI includes only 18 conditions, with patients' ICD-10 coded comorbidities not included in this list considered as having no comorbidities. The ICD-10 chapter, six frequent chronic conditions and FCI do not weight the severity of comorbidities. The CCI includes 19 conditions, but weights the severity of these conditions. For example, the presence of diabetes is given either a weighting of 1 or 2 depending on whether there is end-organ disease, and mild liver disease is differentiated from moderate/severe liver disease by CCI weightings.
The relationship between the conditions included in each comorbidity measure is complex. The ICD-10 chapter approach includes all conditions specified by the six frequent chronic condition, FCI and CCI comorbidity measures. All remaining measures include specific categories for heart disease, chronic pulmonary disease, and diabetes, although how this is represented varies ( Table 1). For example, the six frequent common conditions approach bundles all diagnoses related to heart disease into a single category, while the FCI separates heart disease into three categories; angina, congestive heart failure (CHF)/heart disease, and heart attack. The CCI uses two categories for heart disease; myocardial infarction, and CHF. Arthritis is common to the six frequent chronic conditions and FCI approach, although the six frequent chronic conditions approach separates rheumatoid arthritis and osteoarthritis into individual categories. Similarly, the six frequent common conditions approach includes a category for "backache" while the FCI includes only a subset of relevant ICD-10 codes in its degenerative disc disease category (Table 1). Both the FCI and CCI include a specific category for peripheral vascular disease. Gastrointestinal and neurological disease are included in both the FCI and CCI but the specificity of categories differs between the two measures ( Table 1). Only the FCI includes osteoporosis, obesity, depression, anxiety disorders, vision impairment and hearing impairment as individual categories, while only the CCI includes connective tissue disease, dementia, renal disease, metastatic disease, AIDS, leukemia, lymphoma, and other tumours as specific categories. Overall, the common groups of conditions represented in all comorbidity measures are diabetes, heart disease, and chronic pulmonary disease.

Outcome measures
Survivors to discharge in VOTOR are followed-up by standardised telephone interview at 12-months post-injury to collect disability outcomes [18]. The outcome of interest for this study was the Glasgow Outcome Scale -Extended (GOS-E) which rates a person's function on an 8-level scale from 1 (death) to 8 (upper good recovery) [21] .The GOS-E is recommended for measurement of trauma patient outcomes [22,23], has demonstrated high levels of responsiveness to change in patients without head injury [24], and considers the patient's pre-injury function in the scoring process [21]. The GOS-E was dichotomised with a score <7 representing ongoing disability and 7-8 "recovery".

Data analysis
A split dataset approach was used with the dataset randomly divided into two equal parts [25]. Models were developed using the "training" dataset and internally validated using the "test" dataset. Means and standard deviations (SD), or medians and interquartile range (IQR), were used to summarise continuous variables. Categorical variables were summarised using counts and percentages. Logistic regression models were fitted with "recovery" as the outcome. Model performance was measured using calibration and discrimination statistics. Age was categorised into eight groups, as age in its continuous form was not linearly related to the log odds of recovery. Calibration measures how well the models predict over the entire range and was assessed using the Hosmer-Lemeshow (H-L) statistic and calibration curves. A higher H-L statistic and significant p-value correspond to poorer calibration [26]. Calibration curves plot the observed proportion of events against the predicted probabilities of events, with perfect agreement between observed and predicted probabilities forming a 45°line ("line of best fit") [25].
The area under the receiver operating characteristic (ROC) curve (AUC) measures the capacity of the model to discriminate between those who do and do not experience the outcome of interest [25]. ROC plots sensitivity against 1-specificity over the range of probabilities. Discrimination is generally classified as Acceptable (AUC 0.7 -<0.8), Excellent (AUC 0.8 -<0.9) and Outstanding (AUC ≥0.9) [26].
In the training dataset, a model was fitted with injury variables and age as independent variables, and "recovery" as the dependent variable. Each measure of comorbidity was added and the models were compared using likelihood ratio (LR) tests, AUC (95% CI), and the H-L statistic. The following comorbidity adjustment methods were used.
i. CCI weight categorised as 0, 1, ≥ 2 ii. Number of ICD-10 chapters represented categorised as 0, 1, ≥ 2 iii. FCI score categorised as 0, 1, ≥ 2 iv. Number of the six frequent chronic conditions described by Haagsma et al. [7] represented categorised as 0, 1, ≥ 2 v. All indicator variables for ICD-10 chapters included vi. All FCI condition indicator variables included vii. All of the six frequent chronic condition indicator variables included.
The number of conditions/weighted index was not linearly related to the log odds of recovery, requiring categorisation. The individual conditions of the CCI were not modeled as the weighted index is the most commonly used form of the index. Models were then fitted to the test dataset and fit assessed using the AUC, H-L statistics, and calibration curves. Data were complete for all data items included in the models, ensuring that comparison between models was based on the same cases. All analyses were performed using Stata Version 11.0 (Stata Corp, College Station, Texas). A p-value <0.05 was considered significant.

Overview of the dataset
There were 15,471 survivors to discharge, and 13,519 (87.4%) had a valid GOSE-E at 12-months. Cases lost to follow-up at 12-months included a higher proportion of patients less than 45 years of age, male, and injured in motor vehicle crashes ( Table 2). The overall distribution of orthopaedic injuries sustained was comparable between the groups, but the proportion without documented comorbidities able to be coded to the indices of interest was higher in the cases lost to follow-up (Table 2).
Most cases followed-up at 12-months post-injury had no comorbidity recorded in the ICD-10-AM diagnoses ( Table 3). Mental and behavioural disorders (14%), diseases of the circulatory system (10%), and endocrine, nutritional and metabolic disorders (7%) were most prevalent when using the ICD-10 chapters to classify comorbidity. Using the FCI, diabetes (6%), heart disease (5%) and neurological disease (5%) were most prevalent, while "other" conditions (34%) were most common using the six frequent chronic conditions described by Haagsma et al. (Table 3).
The random split resulted in 6,798 cases in the training dataset and 6,792 cases in the test dataset. Cases in the two datasets were comparable. The percentage of cases who had "recovered" by 12-months post-injury was 42% in both datasets.

Contribution of age and comorbid conditions to prediction of 12-month disability Training dataset
Adding age resulted in improved model fit over adjustment for injuries alone ( Table 4). The addition of comorbid status, irrespective of method of comorbid measurement, improved model fit further (Table 4). All comorbidity adjustment methods resulted in acceptable calibration (as tested using the H-L statistic), but use of the ICD-10 chapters and the six frequent chronic conditions approaches demonstrated higher AUC than adjustment using the FCI or CCI (Table 4). Adjustment for the number of comorbid conditions compared to adjusting for the presence or absence of each condition/chapter did not result in improved discrimination for the ICD-10 chapters (Χ 2 1 = 0.11, p = 0.743), FCI (Χ 2 1 = 0.37, p = 0.544) or the six frequent chronic conditions (Χ 2 1 = 1.21, p = 0.271). The AUC was higher for the number of ICD-10 chapters represented when compared to the number of the six frequent chronic conditions represented (Χ 2 1 = 8.75, p = 0.003). However, the overall range of the AUC for the models adjusting for comorbidity ranged from 0.716 to 0.729 (Table 4).

Test dataset
Fitting the models to the test dataset resulted in a similar pattern of results but poorer model fit with none of the models demonstrating acceptable calibration and lower AUC than the training dataset models ( Table 5). The AUC ranged from 0.691 to 0.704 with the number of ICD-10 chapters represented demonstrating the highest discrimination (Table 5). Despite poor calibration of models as measured using the H-L statistic, calibration curves tracked close to the line of best fit with the ICD-10 and six frequent common condition curves showing better calibration at lower prediction percentiles and all models over-estimating recovery at higher prediction percentiles (Figure 1).

Discussion
The potential for comorbidity to impact on the long term disability experienced by injury survivors is clear but how best to adjust for comorbidity has not been well explored. This study of 13,519 injury survivors found that comorbidity, mapped to commonly used indices from routinely collected ICD-10 diagnosis codes, is an important predictor of functional recovery, providing additional predictive value over adjustment for age alone.
The findings confirm that comorbidity impairs patient recovery after injury. However, measurement of comorbidity remains a challenge. A lack of defined criteria for what constitutes a comorbidity, and absence of an established gold standard for measuring comorbidity [3,9], have led to a variety of methods and indices being used in injury research. Routinely collected discharge data, patient self-report and medical record review are common sources of comorbidity information [27,28]. Each source has strengths and weaknesses with self-report   criticised due to the potential for recall bias, difficulty in data collection with cognitive deficits, and prohibitive costs in large studies [28]. Medical record review is considered the most comprehensive method, with documented conditions corresponding well to established comorbidity indices, but is resource intensive [28]. In the current study, routinely collected discharge data were used to map ICD-10 diagnoses to comorbidity indices. Routinely collected discharge data have been described as "inevitably imperfect" [28] due to the coding of only a subset of recorded conditions and a focus on coding for maximising reimbursement in many settings. However, routinely collected discharge data provide a less resource intensive method for capturing comorbidities and have been shown to agree with medical record review for key conditions such as diabetes, cancer, chronic lung disease and alcohol abuse [29]. Studies directly comparing various data sources in trauma are absent, but Fan et al., in their study of >10,000 Veterans Affairs patients, found comparable prediction of health-related quality of life when comparing routinely collected discharge data with record review and self-report [27]. Our findings are consistent with previous studies that have found comorbidity based on patient or proxy self-report [7,12,13,15,16] and registry data, [11] to be an important predictor of longer term functional or healthrelated quality of life outcome following injury. In contrast to previous studies, comorbidity provided additional predictive value over adjustment only for age [3][4][5]. Adjustment for the specific conditions showed little improvement over adjustment based on the number of conditions represented for the ICD-10 chapters, FCI and the six frequent chronic conditions reported by Haagsma et al., supporting methods of adjustment previously used [10,11,15,16].
Discrimination and calibration statistics of the various models revealed relatively little difference in model prediction of functional recovery, with ICD-10 chapter and the six frequent chronic condition-based models demonstrating slightly better performance than FCI and CCI models. A potential explanation for the improved performance of the ICD-10 and the six frequent chronic condition models is that they used all of the available ICD-10 codes, and therefore all of the comorbidity information available, while the FCI and CCI restricted the conditions included in the model to a specific subset. In data not shown, only diabetes, heart disease, stroke, neurological disease, PVD and visual impairment were significant predictors of recovery in  the FCI models. For the six frequent chronic condition models, diabetes, heart disease and "other" conditions remained significant. The ICD-10 chapters related to ear, respiratory, skin, musculoskeletal, digestive, genitourinary and congenital conditions failed to reach significance in the model. The prevalence of osteoarthritis, rheumatoid arthritis and obesity in the study population was 1%, 0.1% and 1% ( Table 2). The reported prevalence of these diseases in the Australian population is 15%, [30] 2%, [31] and 19-22% [32], respectively. In contrast, the prevalence of heart disease in the study population was 6%, slightly higher than the reported Australian population prevalence for heart, stroke and vascular conditions of 4% [33]. The prevalence of diabetes in the study population was 6%, also slightly higher than the reported 4% national prevalence [34]. The differences in sample prevalence of diseases relative to population prevalence could be explained by the demographic profile of injury patients. In general, the trauma population tends to be younger and healthier than the general population. A number of prospective studies have shown pre-injury quality of life to be higher than age and gender matched population norms [35][36][37] and hence would be expected to have a lower prevalence of chronic diseases. However, the prevalence of some of these conditions (e.g. arthritis) is substantially lower than expected, particularly given the prevalence of elderly patients in the study sample and this is likely to reflect the ICD-10 coding practices and directives.
As noted in the methods, coders are directed to code conditions that have impacted on care provided during the patient's hospital stay, and this will underestimate the true prevalence of chronic conditions. The finding that the models performed similarly is more likely to reflect the types of conditions coded from the medical record and the importance of these conditions to patient care. All indices included conditions such as diabetes and heart disease which require ongoing clinical management during a patient's admission.
The overarching purpose of this study was to explore the relative contribution of each comorbidity measurement approach, and its contribution over age alone, to prediction of functional outcome. Importantly, despite models including age, injuries sustained and comorbidity, the capacity to discriminate between recovered/nonrecovered patients was only in the acceptable range (AUC ≈ 0.70), confirming the importance of other factors. Numerous injury, personal and environmental factors have the capacity to influence the recovery of an individual. This is well demonstrated in the literature where factors such as compensable status [10,11,15,16,38,39], level of education [11,13,16], the intent of injury [11], gender [11,13,16,40,41], social circumstances [13,16,18], intent [11], mechanism of injury [11], and the level of designation of the trauma centre of management [11,42] have been shown to be important. While adjustment for comorbidity is important, the contribution of other factors cannot be under-estimated and should be part of any riskadjustment processes.
The strengths of this study are the large number of cases (>13,000), high follow-up rate at 12-months (87% of all survivors to hospital discharge), and the use of a hospital discharge data to identify comorbidities which has been shown to have low error rates in diagnoses audited [43]. However, there were study limitations.  The study focused on orthopaedic trauma cases rather than all admitted trauma cases. While many cases had also sustained non-orthopaedic injuries, the results may not reflect cases without orthopaedic injury and this should be considered when interpreting the findings. Nevertheless, the orthopaedic registry was selected as the data source specifically for this study because this population includes a higher proportion of elderly patients with comorbidities than other injury populations such as major trauma. There were differences between included cases and those lost to follow-up at 12-months with a bias towards older patients with comorbidity in the group followed-up and included in this study. Secondly, there are challenges in collecting comorbidity information from patients from large populations and particularly where cognitive deficit (e.g. head injury and pre-existing dementia) are prevalent. Therefore, we were not able to assess the agreement between self-report and ICD-10 coding, or fully evaluate the relationship between the FCI and the six frequent chronic conditions not prevalent in the ICD-10 codes, highlighting issues with adapting self-report based indices (e.g. the six frequent chronic conditions described by Haagsma et al.) to ICD-10 based datasets. In future, the development of privacy protecting record linkage systems may enable primary care data to be linked to trauma and hospital discharge datasets to permit evaluation of the impact of conditions not deemed to influence in-hospital care.
Each of the participating hospitals' coders used the same coding standard but the potential for variable interpretation of the medical record and the coding directives remains, despite regular auditing of the hospital discharge data. Additionally, ICD-10-AM coding is done in Australia for reimbursement purposes. In other jurisdictions, where ICD-10 coding is not used for reimbursement or where systems limit the number of codes recorded, the number and distribution of codes may differ. Whether this would impact on ICD-10 based comorbidity adjustment is unknown but warrants consideration in the interpretation of the findings and for further research. Finally, this study focused on functional outcome. The possibility that the relationship between comorbidity and other outcomes such as health-related quality of life, return to work and pain differs from the relationship between comorbidity and functional status will be examined in further research using data from VOTOR.

Conclusions
Mapping of ICD-10 codes to comorbidity indices showed that comorbidity is an important predictor of long term functional outcome following orthopaedic trauma, independent of age and injuries sustained. Adjustment for comorbidity is indicated when assessing risk-adjusted functional outcomes over time or across jurisdictions.