Observed-predicted length of stay for an acute psychiatric department, as an indicator of inpatient care inefficiencies. Retrospective case-series study.

Abstract

Background

Length of stay (LOS) is an important indicator of efficiency for inpatient care but it does not achieve an adequate performance if it is not adjusted for the case mix of the patients hospitalized during the period considered. After two similar studies for Internal Medicine and Surgery respectively, the aims of the present study were to search for Length of Stay (LOS) predictors in an acute psychiatric department and to assess the performance of the difference: observed-predicted length of stay, as an indicator of inpatient care inefficiencies.

Methods

Retrospective case-series of patients discharged during 1999 from the Psychiatric Department from General Hospital "Hermanos Ameijeiras" in Havana, Cuba. The 374 eligible medical records were randomly split into two groups of 187 each. We derived the function for estimating the predicted LOS within the first group. Possible predictors were: age; sex; place of residence; diagnosis, use of electroconvulsive therapy; co morbidities; symptoms at admission, medications, marital status, and response to treatment. LOS was the dependent variable. A thorough exam of the patients' records was the basis to assess the capacity of the function for detecting inefficiency problems, within the second group.

Results

The function explained 37% of LOS variation. The strongest influence on LOS came from: age (p = 0.002), response to treatment (p < 0.0001), the dummy for personality disorders (p = 0.01), ECT therapy (p = 0.003), factor for sexual and/or eating symptoms (p = 0.003) and factor for psychotic symptoms (p = 0.025). Mean observed LOS is 2 days higher than predicted for the group of records with inefficient care, whereas for the group with acceptable efficiency, observed mean LOS was 4 days lower than predicted. The area under the ROC curve for detecting inefficiencies was 69%

Conclusions

This study demonstrates the importance of possible predictors of LOS, in an acute care Psychiatric department. The proposed indicator can be readily used to detect inefficiencies.

Background

Today there is a growing interest in improving quality and efficiency of health care to the maximum, a fact which highlights the necessity of good indicators of quality and efficiency of health care.

Length of stay (LOS) has been repeatedly used as an indicator of efficiency for inpatient care, probably due to its clear meaning as one of the main sources of hospital costs and because LOS can be also deemed an indicator of quality [1–3]. However, LOS for a certain period and facility, is not a useful basis for meaningful comparisons unless it is adjusted for the case mix of patients hospitalized during the period considered. This process is called "risk adjustment" and is thoroughly described and discussed in the book edited by Iezzoni [4].

Allegedly the ideal way of adjustment should be based on the difference between LOS a patient should require–provided the attention received was efficient–, predicted LOS, and the actual one, observed LOS. As a continuous variable, LOS (or a proper transformation) may be modeled by means of a linear regression approach [5]. If an adequate model is found, the difference between observed LOS (OLOS) and predicted LOS (PLOS) could be a proper efficiency indicator. We already evaluated such an indicator for Internal Medicine and Surgery departments with fair results [6].

As a matter of fact, several studies, some of them recently performed, have looked for LOS predictors in psychiatric departments with similar aims [7–19].

Diagnosis, severity of illness, age, sex, physical co morbidities, treatment issues and psychosocial characteristics have been already confirmed as LOS determinants with more or less strength across the above referred studies.

The present work entails two related and successive aims. Firstly, the search for an appropriate function to predict the optimal LOS for an inpatient in an acute psychiatric ward, according to his or her characteristics, and, secondly, explore the ability of the difference Observed-Predicted LOS to detect inefficient care.

Methods

The study is basically a retrospective case-series study. It is mainly descriptive although some hypothesis testing has been performed during the derivation of the function.

Setting

Information was collected from the clinical records of patients discharged from the Psychiatric Department in "Hermanos Ameijeiras" General Hospital in Havana, during 1999. The hospital is a government funded and public facility, it provides secondary and tertiary medical attention within all clinical and surgical specialities for adults except Obstetrics. The Psychiatry department comprises 46 beds, seven of which are reserved for a one week anti alcoholic addiction treatment. The remaining 39 beds are used for regular hospitalized patients admitted from three sources: 1) outpatient attention in the hospital, 2) the outpatient facilities within the hospital's catchment area or 3) the emergency department in the hospital.

Data retrieval

Included records belonged to new patients or known psychiatric patients in an acute phase of their illness. Excluded records were from: 3 self requested discharges, 2 patients included in research protocols affecting LOS, 2 patients transferred from other hospitals, 4 patients admitted for alcoholism treatment with pre-established LOS, 4 not concluded for unknown reasons (possibly self requests not stated), 2 patients escaped from the ward, and 2 patients in which the final main diagnosis was not psychiatric. For 20 patients who had more than one admission within the period only the last one was considered.

The 374 clinical records left available for our study were split randomly into two groups of 187 each. The first group was employed to derive the optimal function to estimate LOS. In the second group, we evaluated the capacity of the function to detect inefficiency problems during their stay. Thus, in both groups we obtained information from each patient record about the following variables allegedly affecting LOS: age, sex; place of residence, marital status, main diagnosis, administered medications, use of electroconvulsive therapy, co morbidities, response to treatment and symptoms at admission. Symptoms were included to account for the patient's severity of illness at admission since there is no regular Severity Index recorded for patients in this Department. Categories of all the variables, except symptoms, are displayed and detailed in Table 1. The list of all symptoms and their categories are displayed in Table 2. LOS was expressed in days from admission to discharge.

Table 1 Distribution of patients and length of stay summaries for different variable categories.

Table 2 Psychiatric symptoms and syndromes considered for Principal Component Analysis

Data for validation

In the second group each record was thoroughly examined looking for sources of inefficient care that could be retrieved from the record, namely delays due to: a) more than 2 days between the indication and the realization of laboratory tests, b) more than 4 days between the realization of laboratory tests and results return from the corresponding laboratory, c) more than 2 days between admission and diagnosis discussion (a feature of all clinical records in the hospital that should be done within 48 hours after admission), d) more than 2 days for interconsultations with another specialist within the hospital, e) more than 3 weekend leaves and e) more than 4 days for prescribed leaves. Records were then classified as: reflecting acceptable efficient care if none of the mentioned situations were found in the record, or otherwise as care with efficiency problems. This assessment and classification was made by one of the authors (RML) blindly regarding the difference Observed-Predicted LOS (OLOS-PLOS). Doubts were discussed with another author (REJ) until agreement.

Predicted LOS was obtained for each patient via the function derived with the first study group and the differences OLOS–PLOS were obtained at the end of the study, when all the information was ready for statistical processing.

Statistical Analysis

The whole group of 374 records was firstly described (mean, standard deviation and median of LOS) within the categories of the different variables. With the first group of 187 records, a Multiple Linear Regression model was applied for assessing the independent influence of each variable on LOS and appraising the possibility of obtaining the predicted LOS. An initial exploration of LOS distribution in this group showed a right asymmetry suggesting the natural logarithm of LOS as the dependent variable. Principal Component Analysis was applied to reduce 19 symptoms to 8 factors that explained 61% of symptom variation (Table 3). The regression function was derived with the variables and the 8 factors–after a Varimax rotation–in place of symptoms [20]. Thus, the following explanatory (independent) variables were included in the function: age as quantitative; sex, marital status, response to treatment and electroconvulsive therapy, as binary; co morbidities, administered drugs, diagnosis and place of residence as dummy variables; and the 8 factors (principal components) accounting for symptoms at admission. The final function was thus adjusted with 185 patients (after eliminating two outliers with standardized residuals higher than 3) and 24 variables. A determination coefficient (R²) of 0.374 was obtained and considered acceptable for the next step.

Table 3 Principal components for psychiatric symptoms. Rotated component matrix¹

The estimated function was then used to obtain the predicted LOS for each patient in the second group (187 patients). We calculated for each patient in this group its score for each of the principal components with the Factor Score Coefficient Matrix obtained with the first group of records. The difference OLOS-PLOS was also obtained for each patient in this group and the association between these differences and the classification group, according to type of attention, evaluated with one way ANOVA. Finally, an ROC curve was obtained to evaluate the capacity of the new indicator (OLOS-PLOS) to detect records with inefficiency problems. The area under the curve was the global measure of the indicator performance. Statistical analysis was performed using SPSS Version 10.0.

The Ethics for Research Committee of Hospital "Hermanos Ameijeiras" approved the research protocol provided the authors maintain the confidentiality of data retrieved from clinical records. Only two of the authors (RML and MM, both medical doctors) worked directly with the records. The identity of the patients could not be identified in the database for statistical analysis.

Results

Sample description

Table 1 shows the main description of all variables in the whole group of medical records. The number of patients is fairly high for all categories. Higher mean LOS was found for patients receiving ECT during their stay and those with a delayed response to treatment.

Symptoms principal components

Table 3 displays the rotated component matrix for the symptoms. Each number in the table represents the correlation between the particular symptom and the rotated factor. Though it is not the aim of the study to deepen into the internal structure of the group of symptoms, it can be considered a fine factor solution since each symptom is only highly correlated with one of the factors. Factors are also easy to interpret since each factor correlates highly to one, two or three symptoms. Eight factors account for 61% of the variation of 19 original symptoms, a fact considered satisfactory.

Multiple Linear Regression results

Table 4 displays the results of the definitive multiple linear regression model which explains 37.4% of LOS variation in the sample. Residual analysis showed a Normal distribution and no need for quadratic terms. According to standardized regression coefficients (SRC) and statistical significance, the strongest influence on LOS came from six variables: age (SRC = 0.254), response to treatment (SRC = 0.246), the dummy for personality disorders (SRC = -0.236), ECT therapy (SRC = 0.215), factor 3, sexual and eating disorders (SRC = 0.203) and factor 1. psychotic symptoms (SRC = 0.174). The coefficients for the dummy variables standing for diagnoses indicate adjusted mean LOS for patients with a diagnosis of a personality disorder is the lowest and adjusted mean LOS for those included in the category of other disorders (reference category) is the highest. Thus, it is not possible to identify the specific diagnosis with the largest adjusted LOS in our series.

Table 4 Multiple linear regression results¹. Optimum predicting function for logarithm of length of stay.

Validation

Table 5 shows means and standard deviations of the indicator (OLOS–PLOS) for both groups of medical attention according to efficiency. In the group with efficient care observed LOS is in average 4 days lower than predicted meanwhile in the group with efficiency problems observed LOS is in average 2 days higher than predicted. Figure 1 provides a general view of the expected positive relation between observed and predicted LOS; however, it is apparent from the table and the scatter diagram that some patients have an observed LOS rather high or low according to their predicted LOS.

Table 5 Mean differences OLOS-PLOS¹ according to efficiency of care

Figure 1

Relationship between observed and predicted LOS. Scatter diagram.

Figure 2 shows the ROC Curve for detecting inefficient care with the indicator. The area under the curve is 0.695. (95% CI = 0.603 – 0.786).

Figure 2

ROC Curve evaluating performance of OLOS-PLOS in detecting inefficiencies. Area = 0.695 95% CI = (0.603 – 0.787)

Table 6 shows sensitivity and specificity, as well as predictive values for different cut-off points of the indicator. A reasonably high specificity (87%) will be obtained with a cut-off point of 5 days, and a high sensitivity with a cut-off point of -6 days (80%). There is no optimal cut-off point with high sensitivity and specificity. With a prevalence of 24% for records with inefficient care, positive predictive values are low but negative predictive values are very high for almost any point.

Table 6 Sensitivity, specificity and predictive values¹ for different cut-off points in the indicator OLOS-PLOS.

Discussion

Our results focus on the plausibility of obtaining a function that fairly estimates the LOS a given patient, admitted in a Psychiatric Department for acute patients, should have had according to his or her characteristics.

Age and gender relationship to psychiatric LOS have been reported in several studies [9, 12, 14, 16, 21]. Oiesvold et al [14] report longer LOS for patients in the older ages and for females in psychiatric patients in hospitals of Sweden and Finland. Huntley et al [12] classified age as one of the five variables significantly predicting LOS steadily over time. Barnow et al [9] found a correlation coefficient as high as 0.73 for describing the univariate association between age and LOS for depressed patients. Richter [21] found diagnosis and age were responsible of 10.5% of the LOS variations. Our results agree with these authors with regard to age but not to gender. Age is a natural determinant of LOS since it is closely related with all vital events; some authors (vg. Kiesler et al [7]), mix it up with another demographic variables while others like Tucker and Brems [8] just include it as a covariate.

Diagnosis is also a variable related to LOS in Psychiatric patients but how to include it with the aim of predicting LOS, is a challenge. Diagnosis Related Groups (DRG's) have been deeply explored [7, 22, 23]; they have the advantage of being just a few for Psychiatry though some authors have alleged they are not relevant for predicting LOS [24, 25]. The Diagnostic and Statistical Manual of Mental Disorders (DSM) classifications (III and IV, lately) [8, 13] or International Classification of Diseases (ICD 9^th or 10^th) [11] are also used in this context. Perhaps a broad classification system would achieve the best predictions but it would imply a huge number of patients for deriving the prediction function. We used an ad hoc classification based on DSM IV that yielded differences in LOS when analyzed univariately and when it was adjusted for other variables as well.

Most of the authors find an association between LOS and diagnosis [11–14, 26] but there are some discrepancies, most authors report psychoses as responsible for the highest LOS [12, 14, 16] but others find major depression [13] as more important predictor of LOS.

It is recognized that the patient's severity of illness influence LOS independently from diagnosis [9, 17, 27] but finding a valid, reliable and useful way to measure it, with the aim of adjusting quality indicators, has always been and continues to be a challenge [28]). Various scales for measuring severity of illness in Psychiatric inpatients have arisen in the last two decades. The Brief Psychiatric Rating Scale (BPRS) in its expanded version [29], the Psychiatric Severity of Illness Index (PSII) [27], the Computerized Psychiatric Severity Index (CPSI) [30] and the Health of the Nation Outcome Scale (HoNOS) [31, 32] are probably the most mentioned ones. However, a low reliability is to expect during their use since appraisers must categorize symptoms in various levels of severity according to their opinion. Thus, implementation of any of these scales implies a period of special training and/or detailed instructions, a fact that prevents their use in daily practice. For instance, Durbin et al [33], in an attempt to introduce CPSI for predicting LOS from clinical records, had evaluators participate in a 3 day training program.

Our principal component solution is a real possibility since we collected information from all 19 symptoms described in the psychiatric record routinely used in our wards and afterward converted them in 8 factors. However, we still had to implement 3 categories for 8 of the symptoms (see Appendix), a feature that should be changed in the near future for the sake of gaining reliability.

We have not found any other study that employs PCA to reduce dimension with the aim of predicting LOS but the method has already been used to reduce symptoms' dimension in the field of Psychiatry and Psychology [34–36].

ECT was also an important LOS predictor in our series. It has been included by other authors in LOS prediction models [11] or mentioned as a cause for longer stays [13, 37]. The use of ECT during hospitalization is a severity indicator, but is also a cause of complications.

Response to treatment turned out to be a variable with strong influence on LOS. Among reviewed literature only Draper and Luscombe [13] recognize the role of this issue in LOS prolongation. We understand it is a difficult aspect to assess and include in information systems unless the physician in charge of the patient completes the discharge form, a claim that should be evaluated in future research.

Other variables included in our study have been also explored by other authors. Marital status (or as living alone) has been acknowledged as an important LOS determinant in different studies [10, 11, 13, 16, 17, 38]; physical co morbidities have also been analyzed [13, 33, 39] and found fairly relevant. Place of residence, as distance from home to hospital was also included in a Brazilian study [40]; medications were also examined by Parks [10] and found "polypharmacy" as a LOS predictor in geropsychiatric patients.

About the goodness of fit of the regression function, 0.37 is not an encouraging determination coefficient but the difficulty to find functions that explain more than 50% of LOS variation, a variable of complex nature, is also true. Among the psychiatric domain, Creed et al were able to explain up to 49% of LOS variation including demographic data, clinical features, social measures and behavioural issues [11]. Richter found an R2 of only 0.11 including in its function: age, diagnosis and other clinical and sociodemographic variables [21]. Stoskopf and Horn found coefficients in the range of 0.10–0.14 including only diagnosis and severity [30]. Huntley et al achieved to explain 17% of total variance in LOS including five variables in a stepwise regression analysis [12]. Regarding the fitted model, we chose the logarithmic transformation of LOS since its original distribution was right tailed. LOS distribution has been explored by various authors. Priest et al analyzed LOS distribution in an acute Psychiatric department in London; he found the exponential model yielded the best fit [41]. Stevens et al fit the exponential model and explore the influence of several factors by means of a Cox Regression model, an approach that would not allow LOS prediction [16]. However, several authors [11, 13, 18] choose the logarithmic transformation for the search of predictors via a regression function, and perhaps most authors fit the regression model with the original LOS observations [10, 12, 17, 23, 33]. Marazzi in a huge European study found Lognormal, Weibull, or Gamma models were fine for describing the distribution of length of stay [42].

Regarding the proposed indicator (OLOS-PLOS), we confirmed the tendency of observed LOS to be higher than predicted LOS when there are inefficiency problems. However, we did not achieve a highly sensitive and specific cut-off point for detecting inefficiencies, a fact that emphasizes the necessity of refining the method with more variables and larger samples. A control method for efficient care similar to the proposed here is reported in some studies but not for psychiatric areas [43, 44].

It is fair to recognize that the process of detecting inefficiencies in the records was somewhat arbitrary; first of all we almost identify inefficiencies as delays though it could be argued inefficient care can be provided without any delay. However, it would be out of the scope of the study to search for another kind of inefficiencies, as, for instance the ones arising from a wrong management of the patient. In second place, some cut offs for deeming a record "inefficient" are also arbitrarily chosen; we chose the time intervals considered normal for the hospital usual performance in all departments. Perhaps the main limitation of the present approach is that the prediction function must be estimated in the same setting where allegedly inefficiencies exist. This handicap is partly solved with the elimination of outliers during the function development. Finally, a practical limitation of our method ensues from the many variables that must be reported by assistant physicians or retrieved from the records. We believe this issue can be solved with the introduction of computers and friendly computer programs at the wards.

Conclusions

Our work supports the importance of a series of variables as LOS predictors in a Psychiatry department. The observed-predicted length of stay can be implemented as an indicator of inefficiencies provided the appropriate cut-off point is chosen. The approach showed its validity and is adaptable to other settings although there is an obvious need to continue the effort in the search of more explanatory functions.