Functional status decline as a measure of adverse events in home health care: an observational study

Research that examines the quality of home health care is complex because no gold standard exists for measuring adverse events, and because the patient and clinician populations are highly heterogeneous. In 1999, the Center for Medicare and Medicaid Services (CMS) instituted the use of the Outcomes and Assessment Information Set (OASIS) for all home health care agencies to be used for internal and external quality assessment. In October of 2000, OASIS measures also became the basis for case-mix adjustment in a new home health care prospective payment system. Shaughnessy et al. developed a comprehensive framework using a two-stage process for assessing the quality of home health care which separates the effects of natural disease progression from the effects of substandard care. The process and the measures they developed were used by CMS as the model on which to base home health care quality improvement [1, 2]. As part of its outcomes based quality initiative, CMS identifies 13 potential adverse events for patients in home health care. Adverse events are considered to be both rare and potentially preventable through provision of appropriate skilled care. The adverse events cover four main areas: emergency care, serious unexpected events, unmet care needs, and declines in health or physical functioning [3].

In this paper, we evaluate three indices of ADL decline among home health patient episodes of care at a large metropolitan home health care agency. One index is calculated in a comparable fashion to the one currently used by CMS, described above. Based on empirical and theoretical considerations, two additional indices are developed. The main question is whether the CMS definition is the most appropriate ADL index for defining adverse events in the context of home health care. A secondary question is the degree to which the indices may be affected by floor effects that may lead to biased results [6].

The data consist of episodes of patient care derived from two separate six-month periods, the first of which is used to derive the analysis sample, with the second providing a follow-up sample for evaluating consistency over time. The same six months from adjacent years are used to minimize any seasonal bias across samples. This study is based on deidentified data and as such was granted exemption from ethical review by Case Western Reserve University.

The start of any given episode of care is defined as the first recorded OASIS assessment during the sample time frame for start of care or resumption of care. Resumption of care occurs when patients are returning to home health care after a transfer to an institutional setting. The end of a given episode of care is defined as the first discharge OASIS assessment occurring on or before 60 days following the start of the episode; if no discharge has occurred by 60 days, the closest OASIS assessment available at or before 60 days is used. Following the implementation of the home health prospective payment system for Medicare home health, CMS required agencies to conduct OASIS assessments every 60 days for patients still in care. Thus, the maximum length for an individual episode is 60 days. Because the time frames for each sample is 6 months, patients may experience multiple episodes within a sample.

Sample

There were a total of 54,732 and 51,560 patient episodes for the prognostic and follow-up datasets, respectively. Of these, 5,295 were excluded from the prognostic dataset and 3,876 were excluded from the follow-up dataset because they did not have outcome data (e.g. died, hospitalized) yielding a total of 49,437 and 47,684 episodes of care, respectively for the prognostic and follow-up datasets (see Figure 1).

For the prognostic data, the population of episodes averages 71 years of age with 68% being female. The population is highly diverse: 36% white, 26% black, 26% Hispanic, 5% Asian, and 7% some other ethnicity. Clinical diagnoses cover a wide spectrum of conditions, with the most common being dyspnea (48%), circulatory diseases (28%), and skin lesions (24%). In general, compared to the entire dataset, the subsets of episodes experiencing substantial functional decline tend to have a larger proportion of the following: women, older patients, Black patients, low physician availability, low income, more comorbidities, and episodes covered by both Medicare and Medicaid health insurance. The follow-up data has similar demographic characteristics.

We conclude that we have no 'floor' effects for gt1_ADLs or gt2_ADLs because the results are comparable with and without 'floor' covariates. Interaction is assessed and determined not to be present. Gt3_ADLs models do not converge, resulting in spurious results that are not shown.

The c-statistics from models of gt1_ADLs are substantially lower at 0.679 and 0.683 for the prognostic and follow-up datasets, respectively (see Table 3). The Hosmer-Lemeshow χ² is 9.781 with a p-value of 0.281, indicating reasonable model fit. The follow-up data results are comparable (χ²statistic with 8 degrees of freedom is 10.090 with a p-value of 0.259). The models for gt1_ADLs have 32 and 26 significant covariates for the prognostic and follow-up datasets, respectively.

Across the prognostic and follow-up datasets, simple frequencies of ADL adverse events were similar for all three indices. All three indices are most influenced by the substantial decline in the ADL for bathing. Declines in bathing may be most frequent due to the difficulty of independent use of showers and bathtubs for elderly, infirm patients.

Gt3_ADLs, which is based on the current CMS definition of an ADL adverse event, was too conservative, yielding few episodes experiencing functional decline and numerous 'floor' episodes as compared to the alternative indices. However, there are also benefits to using gt3_ADLs. Continuing to use the implementation of the CMS defined ADL index promotes consistency in process of care investigations across the home health care industry. Furthermore, this index is infrequent enough to be considered rare, a caveat of being an adverse event. However, when events are very rare, determining the root cause of problems is difficult. Likewise, it is difficult to evaluate if implemented solutions have been effective with rare events.

The proposed indices gt1_ADLs and gt2_ADLs offer advantages of their own. The gt1_ADLs models adhere to the '×10' guideline and is consistent with the high part of the range (4–8%) often found in the hospital literature [23–26]. However, gt1_ADLs may be too frequent to be considered rare, a necessary attribute of adverse events with ~10% of patients experiencing such an event. Gt2_ADLs adheres to the '×10 rule' [27], a statistical guideline advising a dichotomous outcome to be, at minimum, ten times as large as the number of covariates being analyzed. The entire dataset has 112 covariates; prognostic and follow-up data result in 783 and 763 episodes respectively, just slightly under this recommendation.

There may be several important policy and clinical implications of implementing the use of gt2_ADLs as an ADL index for defining adverse events in the context of home health care. While still considered rare (less than 2% of the entire population), gt2_ADLs is more frequent and by definition, less restrictive. Requiring the decline of only two ADLs compared to three (e.g. gt3_ADLs), gt2_ADLs encompasses both a larger frequency of episodes and a population experiencing slightly less functional decline and hence, one less likely to have episodes at the "floor" at baseline. A slightly larger and less incapacitated population may provide more valid results for both case-mix and for evaluating changes in clinical and/or policy practices.

The models for gt2_ADLs provide the most valid index for predicting patients that will experience functional decline. This conclusion is based on criteria of c-statistics above 0.70; non-significant Hosmer-Lemeshow χ² indicating reasonable model fit, and the requirement of fewer covariates to produce valid models indicative of greater efficiency as compared to gt1_ADLs. These results are proven reliable with comparable follow-up data results.

The use of gt2_ADLs for defining ADL adverse events in home health care results in a slightly larger and less incapacitated population which may be more appropriate for case-mix adjustment and evaluating changes implemented in clinical and/or policy practices. However, this study provides only a first step in evaluating these adverse event indicators. Further research is necessary to assess the ability of these indices to be used in case-mix adjustment for evaluating quality in home health care. As well, the larger question of the quality of care in home health care using these measures remains unanswered–comparisons of these functional status measures with other external appraisals of quality would contribute valuable information in home health care quality research.