Identifying patients at risk of nursing home admission: The Leeds Elderly Assessment Dependency Screening tool (LEADS)
© Slade et al; licensee BioMed Central Ltd. 2006
Received: 01 August 2005
Accepted: 13 March 2006
Published: 13 March 2006
Discharge from hospital to a nursing home represents a major event in the life of an older person and should only follow a comprehensive functional and medical assessment. A previous study identified 3 dependency scales able to discriminate across outcomes for older people admitted to an acute setting. We wished to determine if a single dependency scale derived from the 3 scales could be created. In addition could this new scale with other predictors be used as a comprehensive tool to identify patients at risk of nursing home admission.
Items from the 3 scales were combined and analysed using Rasch Analysis. Sensitivity and specificity analysis and ROC curves were applied to identify the most appropriate cut score. Binary logistic regression using this cut-off, and other predictive variables, were used to create a predictive algorithm score. Sensitivity, specificity and likelihood ratio scores of the algorithm scores were used to identify the best predictive score for risk of nursing home placement.
A 17-item (LEADS) scale was derived, which together with four other indicators, had a sensitivity of 88% for patients at risk of nursing home placement, and a specificity of 85% for not needing a nursing home placement, within 2 weeks of admission.
A combined short 17-item scale of dependency plus other predictive variables can assess the risk of nursing home placement for older people in an acute care setting within 2 weeks of admission. This gives an opportunity for either early discharge planning, or therapeutic intervention to offset the risk of placement.
The National Service Framework for older people within the UK highlighted the need for a single assessment process to determine the most appropriate setting for ongoing care . To date, professionals are often faced with a large assortment of scales to choose from, identifying measures to aid them in this decision was seen as a priority. As the setting of care is largely determined by the extent of dependency and, for example nursing needs, then clearly measures of dependency will be important in this process. A previous study examined the use of 7 outcome scales and other predictive factors (e.g. presence of pressure sores) in order to identify which scales were predictive of outcome when the patient had recently entered an acute hospital setting. Out of the 7 scales examined only four scales, The Modified Barthel Index (MBI) the Abbreviated Mental Test (AMTS), the Northwick Park Dependency Scale (NPDS), and the Caregiver Strain Index (CSI) were found to discriminate across outcomes as defined by placement, for example to a nursing home [3–7].
Mindful of the fact that using several different scales can be time consuming for staff and stressful for the patient we wished to determine whether it was possible to create a screening tool to identify dependency consistent with a nursing home placement from three of these health status scales (MBI, AMTS, NPDS). The CSI was not included as not all patients have a carer, and the three other scales were, in theory, measuring an underlying construct of dependency. If combining some of the original items, together with other key predictive variables into a new scale, could create an algorithm to act as a screen for such risk, this may provide an economical way of assessing the likely need for a nursing home admission.
To assess if a screening tool (the Leeds Elderly Assessment Dependency Scale (LEADS) could be developed from three previously identified scales which discriminate for nursing home placement . To test if this scale, along with other key predictive variables (identified in previous study), would be sensitive and specific to predicting the need for a nursing home placement within two weeks of admission to acute wards for the Care of the Elderly.
Patients were recruited on admission to the Care of the Elderly wards in a multi-site acute hospital trust. A random sample of every fourth patient admitted to three such wards, together with all patients requiring a comprehensive assessment were included in the study. Full details of recruitment and patient characteristics are given elsewhere .
Outcome was defined in terms of discharge destination i.e. whether the person was discharged to home, home with family/carer, sheltered housing, residential or nursing home care (in the initial study). In this paper we are primarily concerned with differentiating between nursing home placement and the 'other' placements, as nursing home placement has considerable impact on patients, their families, and on service provision.
Stage 1: Rasch analysis, developing the LEADS
The three scales were combined and the development of a single shorter scale was explored using Rasch analysis . The Rasch model is the current standard for the development of unidimensional scales (e.g. of impairment or dependency) delivering metric quality outcomes in health care . Briefly, data collected from scales completed by clinical staff, which are intended to be summated into an overall score are tested against the expectations of the Rasch measurement model. The model defines how responses to items should be if measurement (at the metric level) is to be achieved. This was considered the most appropriate model for identifying items that could measure the underlying latent trait (level of disability) into a short assessment tool as it is the only measurement model delivering a metric transformation of ordinal scale [10, 11]. A previous study constructed short form scales using this method without any loss of validity, and found it to be a better method for item reduction than more classical forms of item reduction e.g. principal components analysis . Wolfe in his paper discussed different mechanisms for combining scales . The method chosen for the current analysis was the common person equating method. That is the same patient is assessed, at the same time, and assigned values (by clinicians) to each item on the three scales. In looking for potential items to discard, items that were identified as redundant by the Rasch analysis (that is they had high negative residuals, equating to high item-total correlations in classical analysis), were discarded. Also, items that failed to follow the expected probabilistic relationship of a valid scale (misfitting items), were also discarded. Finally, items that showed bias for external factors such as age and gender (Differential Item Functioning were also removed) .
The most appropriate cut off point was determined by the sensitivity, specificity, likelihood ratio and ROC curves . Deeks and Altman suggest that likelihood ratios have more powerful properties making them more appropriate in clinical use than sensitivity and specificity alone . Likelihood ratios is the ratio of the probability of finding people who will need nursing home placement to the probability of predicting those patients who will not be at risk of nursing home placement using a defined cut-off score. For comparison, ROC curves are also presented .
Stage 2: Binary Logistic Regression
Following the construction of the LEADS, a binary logistic regression analysis was used to identify the screening cut-off score derived from the LEADS in combination with the other variables . As the numbers going into a nursing home were expected to be relatively small, it was expected that this would cause some problems in interpretation. King and Zeng identified the difficulties in analysing rare event analysis in logistic regression and advocate that all the rare event cases are used and random sample of the remainder . Consequently this strategy was adopted here, with repeated random samples (in the event of 50 cases) taken from the remainder (the non-nursing home cases) in a one-to -two ratio of those placed in nursing home and non-nursing home cases. The exp (B) values from the binary logistic regression for the indicator variables, together with the LEADS cut-off score were used in an algorithm to create an overall algorithm score for the risk of nursing home placement . (See Appendix 1)
Stage 3: Sensitivity, specificity and likelihood ratios
The predictive score for nursing home, identified by the algorithm was then examined for sensitivity and specificity to identify the best predictive cut-off score for the total algorithm. The aim was to maximise sensitivity, and minimise false positives and again, for comparison, ROC curves are presented .
Ethics and Consent
Ethical committee approval was obtained from the Leeds Teaching Hospital NHS Trust. Patients were asked to sign a written consent form. If they were unable to give consent due to cognitive, visual or communication problems, a relative or carer was asked to consent on their behalf.
Five hundred and forty nine patients were recruited into the full study, of whom 258 were assessed on each of the three scales to be analysed as potential contributions to the screening tool, and discharged to various destinations . The mean age of these 258 patients was 83.8 years (SD 5.5) and their mean length of stay was 31.2 days (SD 31.6). Seven out of ten (70.2%) were female.
Stage 1: Reducing the item set and producing a single scale: using Rasch Analysis
Items and their location in the LEADS scale
ORIGINAL SCALE & ITEM NO.
Know current month
Orientated to place
Enteral feeding (NPDS 8.3) was the item with the highest negative location (-5.799 logits). This means that the majority of people did not require enteral feeding. In contrast the stairs item (MBI 3) had the highest positive location (+5.031 logits), suggesting that the majority of people found stairs difficult, and independence in this activity was difficult for this group to achieve (Table 1).
Mean LEADS score by outcome with confidence intervals
Mean (95% CI)
Home without carer
Home with carer
Stage 2: Predicting the need for nursing home placement: Binary logistic regression
Using the nursing home and the combined 'other' groups as a dependent variable, a binary logistic regression was used to identify predictors for patients at risk of a nursing home placement. There were a disproportionate number of cases between these groups (233:25) and thus, five random samples were selected from the 'other' group and added to the nursing home group to create repeated samples for analysis. The results were consistent for all samples and thus the results presented are the sample that gave the best predictive model.
Binary logistic regression fit statistics
-2 LOG LIKELIHOOD
NAGELKERKE R SQUARE
COX & SNELL R SQUARE
% CORRECTLY IDENTIFIED
All variables including LEADS cut-off 19
Minimum Exp B Variables from logistic regression and range over 5 samples
CUT-OFF 19 LEADS Range
Respite care on admission
Communication difficulties on admission
Family/patient expressed wish for placement
Grade 1 plus pressure sore
Stage 3: Sensitivity, specificity of the final algorithm
Specificity and sensitivity of the cut scores from the LEADS algorithm full cohort
Appendix 1 - Algorithm
The algorithm for predicting nursing home placement using SPSS based on minimum Exp B
Compute predscrn = 358
if (LEADS score le 19) predscrn = predscrn -65
if (familypat wish = 1(no)) predscrn = predscrn -34
if (communication difficuties = 1 (yes)) predscrn = predscrn -112
if (grade 1+ pressure sore = 1 (yes)) predscrn = predscrn -18
if (respite care on admission = 1 (yes)) predscrn = prdescrn -128
Excel spreadsheet with the algorithm and scoring for the LEADS is available from:
Currently, people are faced with a bewildering variety of potential measures for use in assessing outcome in an acute elderly setting. Having previously identified three scales that discriminated between people going to a nursing home as against other outcomes, we have now shown that it is possible, through Rasch analysis, to extract items that work well together and measure the underlying dependency trait. Clinicians may still wish to use the original scales for clinical purposes but in terms of measurement, the 17-item LEADS scale and associated algorithm has been shown to be a powerful tool in predicting patients at risk of nursing home placement and those likely to go to other types of care or home.
The false positive rate in the final analysis may be viewed as a major weakness in the approach. Some patients improved such that they could go home, or into other institutional settings. This is a valid comment and the majority of mismatch between the indicative and final placement was for those patients who went home. It is important to remember this data was collected within two weeks of admission. Thus the algorithm, as well as providing a common equitable means of assessment, can act as an early warning system for risk of institutional care. Early identification of those patients at risk enables interventions to be instigated early on in their admission, potentially reducing the risk of nursing home placement. Given the parsimony of the scale there is nothing to prevent repeated measurements during the patients stay in hospital, so providing a monitoring system for the continuing risk of institutional placement.
There are a number of weaknesses to the study. The low number of patients subsequently entering a nursing home was always going to be a cause of concern. However, we accommodated this, as best as possible, by sampling from the other group and comparing the results. The assessment was also only undertaken once within two weeks of admission. Additional work needs to be carried out using repeated assessments and to look at the changes in sensitivity and specificity over time in order too determine if there is an optimum time to maximise these parameters. Due to the low numbers of those entering a nursing home, we had to use this group in the development of the algorithm cut point, as well as its validation. This is likely to overestimate its predictive value, although we did try to offset this as far as possible by validating the algorithm on the full data set, rather than the developmental sample. Finally, as with all models developed on a particular set of data, these results need replicating on other elderly acute samples to support conclusions about the predictive validity of the screening tool.
Using selective items from three separate scales, previously shown to be discriminative for nursing home placement, together with other key indicators, enables those working in an acute setting, within two weeks of admission, to identify 85 % of patients at risk of needing nursing home placement. The resulting LEADS scale and four indicator variables can easily be administered by any health care professional and the risk algorithm lends itself to a simple spreadsheet calculation.
We would like to acknowledge and thank the Northern and Yorkshire Region NHS Executive for funding the project.
We would like to thank the staff and patients from Leeds Teaching Hospitals NHS Trust involved in the original study for their help and co-operation.
- Department of Health: The single assessment process for older people. Health Service Circular 2002/001. The NHS plan (Single assessment process). [http://www.dh.gov.uk/PolicyAndGuidance/HealthAndSocialCareTopics/SocialCare/SingleAssessmentProcess/fs/en]
- Slade A, Fear J, Tennant A: Predicting outcome for older people in a hospital setting: which scales are appropriate?. International Journal of Therapy and Rehabilitation. 2004, 11: 25-30.View ArticleGoogle Scholar
- Mahoney F, Barthel D: Functional Evaluation: The Barthel Index. Md State Med J. 1965, 14: 61-65.PubMedGoogle Scholar
- Shah S, Vanclay F, Cooper B: Improving the sensitivity of the Barthel Index for stroke rehabilitation. J Clin Epidemiol. 1989, 42: 703-709. 10.1016/0895-4356(89)90065-6.View ArticlePubMedGoogle Scholar
- Hodkinson HM: Mental impairment in the elderly. JR Coll Physicians Lond. 1973, 7: 305-317.Google Scholar
- Turner-Stokes L, Tonge P, Nyein K, Hunter M, Nielson S, Robinson I: The Northwick Park Dependency score (NPDS): a measure of nursing dependency in rehabilitation. Clin Rehabil. 1998, 12: 304-318. 10.1191/026921598669173600.View ArticlePubMedGoogle Scholar
- Robinson BC: Validation of a Caregiver Strain Index. J Gerontol. 1983, 3: 344-348.View ArticleGoogle Scholar
- Rasch G: Probabilistic models for some intelligence and achievement tests. 1960, Cophenhagen: Danish Institute for Educational Research, 1980 Chicago: University of Chicago Press, expandedGoogle Scholar
- Tennant A, McKenna SP, Hagell P: Application of Rasch Analysis in the development and application of quality of life instruments. Value Health. 2004, 7 Suppl 1: S22-26. 10.1111/j.1524-4733.2004.7s106.x. Sep-OctView ArticlePubMedGoogle Scholar
- Bond TG, Fox CM: Applying the Rasch Model, Fundamental measurement in the Human Sciences. New Jersey: Lawrence Erlbaum Associates
- Wolfe E: Equating and item banking with the Rasch model. J Appl Meas. 2000, 1: 409-434.PubMedGoogle Scholar
- Luquet C: A method for shortening instruments using the Rasch model: Validation on a hand functional measure. Rev Epidemol Sante Publique. 2001, 49: 273-286.Google Scholar
- Tennant A, Penta M, Tesio L, Grimby G, Thonnard JL, Slade A, Lawton G, Simone A, Carter J, Lundgren-Nilsson A, Tripolski M, Ring H, Biering-Sørensen F, Marincek C, Burger H, Phillips S: Assessing and adjusting for cross cultural validity of impairment and activity limitation scales through Differential Item Functioning within the framework of the Rasch model: the Pro-ESOR project. Medi Care. 2004, 42 (Suppl 1): 37-48.Google Scholar
- Ebell MH: 'Likelihood Ratios', Introduction to information mastery. Michigan State University, [http://www.poems.msu.edu/InfoMastery/Diagnosis/likelihood_ratios.htm]
- Deeks JJ, Altman DG: Diagnostic tests 4: Likelihood ratios. BMJ. 2004, 329: 168-169. 10.1136/bmj.329.7458.168.View ArticlePubMedPubMed CentralGoogle Scholar
- Altman DG, Bland JM: Statistics Notes: Diagnostic tests 3: receiver operating characteristic plots. BMJ. 1994, 309: 188.View ArticlePubMedPubMed CentralGoogle Scholar
- Field A: Logistic Regression. Discovering Statistics using SPSS for Windows. 2000, London: Sage Publications, 163-205.Google Scholar
- King G, Zeng LC: Explaining rare events in international relations. International Organization. 2001, 55: 697-715.Google Scholar
- Hosmer DW, Lemeshow S: Applied Logistic Regression. 2000, New York: Wiley, 2View ArticleGoogle Scholar
- Loong T: Understanding sensitivity and specificity with the right side of the brain. BMJ. 2003, 327: 716-719. 10.1136/bmj.327.7417.716.View ArticlePubMedPubMed CentralGoogle Scholar
- SPSS Version 11.5. 2003, SPSS Inc Chicago
- Andrich D, Lyne A, Sheridon B, Luo G: RUMM2010. Perth: RUMM Laboratory., [http://www.rummlab.com.au/]
- Bland JM, Altman DG: Multiple significance tests: the Bonferroni method. BMJ. 1995, 310: 170.View ArticlePubMedPubMed CentralGoogle Scholar
- Fisher WP: Reliability statistics. Rasch Measurement Transactions. 1992, 6: 238.Google Scholar
- Smith EV: Metric Development and Score Reporting in Rasch Measurement. J Appl Meas. 2000, 1: 303-326.PubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/6/31/prepub