Predicting hospital mortality among frequently readmitted patients: HSMR biased by readmission

In various countries in the world, risk adjusted in-hospital mortality ratios are currently calculated using routinely collected data. The variations seen in crude mortality for which adjustment is needed, may be attributed to various sources, including variations in: registration data, casemix, quality of care and chance [1]. After adjustment for casemix, the hospital standardised mortality ratio (HSMR) can be used as a tool for hospitals to analyse their death rates by comparing their riskadjusted mortality with the national average, establishing a starting point for improvement of hospital outcomes [2]. In some countries HSMRs are also made publicly available for the benefit of for example: the patient, the politician, the hospital manager and the clinician. This public reporting potentially may result in league tables, suggesting that the ranking reflects differences in quality of care. A somewhat more subtle approach is being applied in the UK by combining standardised mortality ratios with other safety indicators, in order to establish hospital safety rankings. These rankings are publicised in the "How Safe is Your Hospital" guide [3], compiled by researchers at the Dr Foster Intelligence Unit at Imperial College London. However, publicly comparing and judging hospital performances in this way is heavily disputed [4, 5] at the moment. Some publications mention the Netherlands as one of the countries where HSMRs are also made publicly available [6, 7], yet up to and including 2008 the HSMRs have not been publicised with the exception of a few hospitals that voluntarily put their values on the web. Mandatory publication was planned to start in 2010, based on admission data from 2009, and various articles suggest that the quality of the Dutch HSMRs is sufficiently high to justify publication. Heijink et al [8] for example state that they did not find evidence that the HSMR cannot be used as an indicator to monitor and compare hospital quality in the Netherlands. However there is some controversy around this subject and the Dutch Ministry of Health recently decided to postpone mandatory publication for at least one year, because there were doubts about the reliability and validity of the current figure [9]. This decision was based on the outcomes of a study [10] conducted by Santeon, a group of six large non-university teaching hospitals in the Netherlands (Additional file 1: Table S1), confirming indeed why publicly comparing HSMRs may not be a good idea [4].

One of the findings of this Dutch study [10], not being addressed in [4], showed large differences between the hospitals with respect to numbers of readmissions per patient, if measured over a prolonged period of time. Currently the HSMR in the Netherlands is not adjusted however for any form of readmission. In the UK the HSMR is adjusted for readmission for acute cases, but not for a prolonged period of time.

The association between readmissions and the HSMR is mentioned in a paper by Jarman [11]. The effects of adjusting for readmissions are described as follows: " ....There is also not much difference between normal HSMRs based on all admissions and those based on only one (e.g., the last) admission in a year. ....". We interpret this finding as follows: If a patient visits the hospital more than once that year, then any one of the admissions would represent the contribution of that patient to the HSMR ratio. Using a limited part of the patient's admission history, would, on average, not make a difference to the HSMR of that hospital. This may suggest that adjustment for readmission would not make sense. Using this method, we question whether a period of one-year is sufficiently long to embrace all the effects of readmission and indeed whether all the risk conditions surrounding readmissions can be properly addressed. A more recent article by Jarman [2] however, does address the need to investigate adjustment for readmissions as follows: " ... further improvements to the case-mix model are being evaluated. The numbers of previous admissions within a given time period, which requires the linking of admissions of the same patient, could be of potential use....". In our publication we will share how differences in numbers of previous admissions made an impact upon the HSMRs of the hospitals and what can be done to improve the HSMR model used.

During the five-year study period 240662 patients accounted for 418566 admissions in total. 164884 patients (69% of total patients) concerned 'first-and-only' admissions. The other 75778 patients (31% of total patients) who were admitted more than once accounted for 253682 admissions (61% of total admissions) of which 177904 were readmissions (43% of total admissions).

Variations in admission frequencies

We analysed the distribution of the admission frequency per patient class P(m) per hospital (table 2). For m = 1 hospital B has the highest percentage admissions (46% of total), hospital D the lowest (29% of total). For P(m > 7) hospital D has high scores, for example: 9% of the admissions concerned patients being admitted more than 20 times (P(m > 20)), whereas for hospitals E and F this percentage amounts to 0,6%. We also analysed the inter-hospital variation of the average admission frequency on the level of the 12 main CCS diagnostic groups (figure 1). The first three diagnostic groups shown - neoplasms, heart diseases and respiratory diseases - cover two thirds of all readmissions. For these diagnostic groups we calculated the ratio between the highest and the lowest observed hospital average of readmission frequencies. For neoplasms, hospital D (3.3 readmissions) and hospital B (1.1 readmissions) differed by a factor of 3. For heart diseases and respiratory diseases the ratio between highest and lowest amounted to 1.8 and 1.7 respectively. Overall (bottom line of figure 1) hospital D has the highest average readmission frequency (1.14) and hospital B and F have the lowest average (0.57); a factor of 2 difference between the highest and the lowest.

	Percentages of admissions of total admissions per hospital
Patient view class	A	B	C	D	E	F	All hospitals
P(m = 1)	39.1%	46.0%	37.7%	29.3%	38.9%	45.3%	39.4%
P(m = 2)	19.0%	20.8%	20.5%	17.3%	22.4%	21.2%	20.0%
P(m = 3)	10.2%	10.3%	12.3%	10.6%	12.6%	11.2%	11.0%
P(m = 4)	6.2%	6.3%	8.2%	6.6%	7.4%	6.8%	6.8%
P(m = 5, 6)	7.4%	6.5%	9.4%	8.1%	8.8%	7.3%	7.8%
P(m = 7-9)	5.8%	4.6%	5.8%	6.7%	5.2%	4.4%	5.5%
P(m = 10-20)	7.7%	3.9%	4.7%	12.2%	4.2%	3.2%	6.3%
P(m > 20)	4.6%	1.6%	1.5%	9.1%	0.6%	0.6%	3.3%
Total	100%	100%	100%	100%	100%	100%	100%

* Hospitals are presented in random order and labelled A - F.

Mortality per patient view class and per admission view class

In table 3 the total results of mortality calculations are shown from the perspective of the patient view classes. The row of class P(m = 1) shows the outcomes for all 164884 patients who were admitted exactly once, row P(m = 2) for 41876 patients admitted exactly twice, and so on. The crude mortality per patient for first-and-only admissions amounts to 5.4%. For m = 2 through to m = 9 we see an average growth in mortality of roughly 1% per patient view class increment. For m = 10 through to 21+ the crude mortality per patient class does not show growth anymore and stabilizes around 13%. Table 3 also shows that DHM-2008 predicts a decline in mortality per admission from 4.2% (P(m = 1)) to 1.1% (P(m > 20)).

	Observed number of			Crude mortality per		DHM-2008 predicted
Patient view class	patients	admissions	deaths	patient	admission	number of deaths	mortality per admission
P(m = 1)	164884	164884	8836	5.4%	5.4%	6971	4.2%
P(m = 2)	41876	83752	2868	6.8%	3.4%	3363	4.0%
P(m = 3)	15356	46068	1377	9.0%	3.0%	1899	4.1%
P(m = 4)	7064	28256	727	10.3%	2.6%	1153	4.1%
P(m = 5, 6)	6055	32490	695	11.5%	2.1%	1302	4.0%
P(m = 7-9)	2938	22795	404	13.8%	1.8%	820	3.6%
P(m = 10-20)	2063	26364	265	12.8%	1.0%	714	2.7%
P(m > 20)	426	13957	55	12.9%	0.4%	157	1.1%
Total	240662	418566	15227	6.3%	3.64%	16379	3.9%

* The numbers represent totals of the six hospitals. The crude mortality per patient stabilises around 13% after 10 (re)admissions. The predicted mortality per admission increasingly differs (higher) from observed mortality for increasing patient view classes. The statistical significance of this is shown in table 5 by means of corresponding standardised mortality ratios and 95% confidence intervals.

In table 4 the results of mortality calculations are shown from the perspective of the admission view classes. Row A(n = 1) shows the outcomes for all of the 240662 patients being admitted for the first time. From these patients, 75778 were admitted at least a second time, shown in row A(n = 2), etc. In this case, going from A(n = 1) to A(n > 20), a decline of mortality per admission from 4.1% to 1.1% is predicted by DHM-2008.

Admission view class	Observed number of		Crude mortality	DHM-2008 predicted
	admissions	deaths	per admission	number of deaths	mortality per admission
A(n = 1)	240662	8836	3.7%	9783	4.1%
A(n = 2)	75778	2868	3.8%	2981	3.9%
A(n = 3)	33902	1377	4.1%	1389	4.1%
A(n = 4)	18546	727	3.9%	740	4.0%
A(n = 5,6)	19124	695	3.6%	714	3.7%
A(n = 7-9)	12634	404	3.2%	421	3.3%
A(n = 10-20)	12483	265	2.1%	293	2.2%
A(n > 20)	5437	55	1.0%	57	1.1%
Total	418566	15227	3.6%	16379	3.9%

* The numbers represent totals of the six hospitals. Predicted mortality per admission is in line with observed mortality for all admission view classes. The statistical significance of this is shown in table 5 by means of corresponding standardised mortality ratios and 95% confidence intervals.

In the patient view the number of observed deaths for first-and-only admissions (P(m = 1)) is clearly higher than predicted and for P(m = 2) - P(m > 20) lower (table 3). In the admission view the differences are smaller (table 4).

From tables 4 and 5 we can calculate the standardised mortality ratios for both views via the ratio of observed deaths and predicted deaths per category. Table 5 shows the results, including 95% confidence intervals (p < 0.0001). Figure 2 presents a graphical representation of this. In the patient view the SMRs decline from 127 (P(m = 1)) to 35 (P(m > 20)) and none of the corresponding 95% confidence intervals includes the expected overall HSMR value 93.0 (95% CI: 91.5 - 94.5). This shows a significant association between patient view categories and SMRs indicating a lack of model fit. In the admission view however, the SMRs fluctuate between 90 and 99 and all corresponding 95% confidence intervals include the overall HSMR value 93.0 (95% CI: 91.5 - 94.5) indicating a good fit of the HSMR model for this view, which we will discuss later.

Patient View	SMR per Patient View Category	Lower 95% CI of SMR	Upper 95% CI of SMR	Admission View	SMR per Admission View Category	Lower 95% CI of SMR	Upper 95% CI of SMR
P(m = 1)	126.8	124.1	129.4	A(n = 1)	90.4	88.5	92.3
P(m = 2)	85.3	82.2	88.5	A(n = 2)	96.0	92.5	99.6
P(m = 3)	72.5	68.7	76.5	A(n = 3)	99.2	94.1	104.6
P(m = 4)	63.1	58.6	67.8	A(n = 4)	98.1	91.1	105.5
P(m = 5, 6)	53.4	49.5	57.5	A(n = 5, 6)	97.4	90.3	104.9
P(m = 7-9)	49.3	44.6	54.3	A(n = 7-9)	95.8	86.7	105.6
P(m = 10-20)	37.1	32.8	41.8	A(n = 10-20)	90.4	79.9	102.0
P(m > 20)	34.9	26.3	45.5	A(n > 20)	96.1	72.4	125.1

(Re)admission frequencies show substantial variations between the hospitals. For example: the overall value of the readmission frequency of hospital D equals double the value of hospital B and F. For the main CCS diagnostic group neoplasms, this value for hospital D equals even three times the value of B. We conclude that there are substantial differences in the numbers of readmissions within a given time period between the six hospitals.

Patients with higher admission frequencies bear lower predicted risks per admission, which can be explained by shifts in the casemix. For the patient view however, lowering of risks is only partly predicted by DHM-2008, since the amount of predicted deaths compared to observed deaths turned out to be relatively low for first-and-only admissions and high for readmissions. The corresponding standardised mortality ratio in the patient view is high for class P(m = 1) (127) and low for the other classes (gradually dropping to 35 for P(m > 20)) compared to the overall average of 93. In contrast to this we found that the standardised mortality ratios per n^th admission view class are approximately as predicted, fluctuating around 93. DHM-2008 demonstrates quite a fair goodness of fit for the n^th admission view classes. This result matches the findings of Jarman [11], where no differences in HSMR were detected by picking n^th admissions for any value of n. At the same time however, a comparison of predicted and observed deaths for the patient view classes does not demonstrate a good fit for the model. The question now arises why DHM-2008 should be suited to fit the n^th admission view and not be suited to fit the patient view? As we will explain below, different admission frequency classes of the patient view may incur risk differences, not detected by the known adjustment variables. In that case, comparing these classes would commit the constant risk fallacy and establishing a model fit along the lines of the patient view would be preferred to a fit along the lines of the admission view.

We will demonstrate this point by three examples that show mechanisms, lowering risks for higher admission frequencies and having nothing to do with higher quality of care:

Example 1: Admission policies may increase the number of readmissions without proportionally increasing real risks. One hospital may systematically combine the diagnosis and treatment into a single admission. Another hospital may have an admission for diagnosis and a second one for treatment, being granted a double predicted risk count, most likely without doubling of real risk, but doubling the expected risk.

Example 2: Hospitals may show different treatment practices in the frequency with which chronically ill patients are admitted for the same disease. For example readmission frequencies for the treatment of neoplasms in hospitals B and D differ on average by a factor of 3. This difference can be explained by differences in balance between inpatient versus outpatient treatments (late versus early adopters of a trend moving from inpatient toward outpatient treatment, for example [14]). The predicted risk contribution for hospital D may thus be tripled compared to B, but it is not likely that patients of D are being exposed to a tripled real risk. On the contrary, the mortality risk per patient stabilizes around 13% if admitted 10 or more times (table 3). Every incremental admission for chemotherapy further increases the denominator of the HSMR, but on average does not increase the observed mortality, the numerator, further.

Example 3: Patient referrals to tertiary care, frequently occurring in the Netherlands, may cause differences. The transfer of a patient back and forth most often is an advantage for the referring hospital, as they count a double admission, while the hospital to which the patient is being referred is at a disadvantage because they only count for one admission. On top of this the latter hospital has to deal with the risk of conducting a potentially complicated medical procedure. It is unlikely that a patient under these circumstances will experience a doubling of risk in the referring hospital.

Another plausible mechanism may be hidden in the physical condition of frequently readmitted patients. If these patients are unexpectedly resilient, they will dominate higher admission frequency groups through natural selection and consequently cause lower undetected risks, compared to lower admission frequency groups. If this hypothesis is valid, it might become visible as well in patient specific casemix properties that are known to us such as age and comorbidity (table 6). The average age of frequently admitted patients indeed is decreasing for the highest values of n, indicating a fitter population. The comorbidity is increasing however; a logical consequence of the fact that we applied the notion of readmission for any disease. So patients with more co-morbidity may be admitted more often for the various diseases they suffer from. It also indicates higher vulnerability and in that case would contradict the hypothesis. Although the hypothesis looks appealing, we cannot proof its correctness with the data currently available.

Looking back at the admission history of a frequently admitted patient, the additional risk-lowering factors just described, may have come into play already after the patient's first admission. The n^th admission view constitutes a cross-section of various patient view groups, each having their own additional risks. Consequently the n^th admission view cannot discriminate additional risk differences and is not suited to be used for readmission adjustment. Instead, we think the variable 'admission frequency' of the patient view should be used for this purpose. Patient views are showing why DHM-2008 predicts too high a risk for a frequently readmitted patient, as illustrated by the following case that we observed:

Patient × of hospital D contributed 3.1 predicted deaths to the denominator of the HSMR through seven successive admissions within six months. A single patient however can maximally contribute a value of 1 - in case of death - to the numerator of the HSMR.

Numerous examples alike became available in our study, for example: we found 174 patients in hospital D, each of whom contributed more than 1 predicted death to the denominator of the HSMR due to various readmissions. In total these contributions in hospital D added up to 232 predicted deaths, whereas 'only' 75 deaths were observed in this group. Clearly the number of deaths predicted by DHM-2008 for frequently admitted patients is being overestimated.

A final illustration of this phenomenon is shown in figure 3: a scatter diagram where we plotted the HSMRs as well as the SMRs of the three main CCS diagnostic groups that show the largest variations in readmission frequency - neoplasms, heart diseases and respiratory diseases - against the readmission frequency (see also figure 1). In all cases there is a downward trend: higher readmission frequencies corresponding with lower (H)SMRs.

We conclude that there is a significant association between HSMRs and numbers of readmissions per patient.

Since the study involved five consecutive years, we were not able to capture the complete patient view. Patient view sequences that started before 2003 and continued to emerge in the period 2003 - 2007 were truncated. The same happened with sequences that started during the period 2003 - 2007 and continued in the years thereafter. Consequently the picture will never be complete. For each patient being admitted at least twice, we calculated the time elapsed between the date of the first and the date of the last discharge. For 63% of these patients, the time elapsed amounted to less than one year and for 91% less than 3 years, suggesting that the larger part of the effect has been captured.

How can HSMRs be adjusted for the effects of readmissions? For that purpose, an additional adjustment variable 'admission frequency,' as used in this study in the patient view, may be applied. After adjustment, the SMRs of the patient view classes in our study will fluctuate around 93. As a consequence the goodness of fit for the admission view will be lost. However, this does not provide us with a principal problem since the division into admission frequency classes (patient view) is along the lines of identified and distinct risk classes for which adjustment is clearly needed. N^th admission classes turn out to be meaningless in terms of risk differentiation.

In particular for the higher admission frequencies, a prolonged measurement period of various years was needed in order to make visible the effects we described. For example the average time which elapsed between the first and last date of discharge for patient view class P(m > 20), amounted to 2.5 years. This does not mean however that having many patients in class P(m > 20), does not have an effect if measured for only one year. Ideally the adjustment for readmission would be based on an 'admission counter' in the file of each patient that is being increased by 1 after every new readmission. Since such counts are not being kept, an approach along the lines of this study, awkward as it might be, will be necessary.

A final remark concerns the following: (frequent) readmissions are sometimes taken to be a proxy indicator for poor quality of care. But instead of working against the hospitals with higher readmission frequencies the current HSMR seems in that case to work in favour of those hospitals, underlining the following statement: If HSMRs in the Netherlands ever will be publically reported and used to compare hospitals, then the issues raised in [4] need to be resolved and, on top of this, adjustment for readmission will be necessary. If HSMRs and particularly SMRs on diagnostic level are used by hospitals as a starting point for quality improvement [2] - a better idea for usage of the HSMR indicator - then adjustment for readmission will be necessary as well, in order to prevent misleading signals to be generated. Hospitals will in that case better be able to avoid falsely acting upon too high SMR values as well as to avoid falsely non-acting upon too low SMR values.

This study has shown that patients admitted more frequently experience a lower risk of dying per admission. The HSMR model used, detects lower risks for higher admission frequency groups. However the observed mortality of these groups demonstrate the real risks to be even lower, indicating differing risk conditions between the groups. Consequently, comparing admissions of patients from different admission frequency groups, as done by the current HSMR model, commits the constant risk fallacy. The study showed substantial variations in the overall distribution of readmission frequencies, up to a factor of 2, between the six hospitals. As a result hospitals with high admission frequencies experience unadjusted decrease of risks and hospitals with low admission frequencies experience unadjusted increase of risks. These misleading outcomes can only be demonstrated by analysis over at least three years, but is in effect every day. As 43% of all admissions in this study concerned readmissions, the impact on the HSMR of some hospitals may be substantial. This finding is relevant for all countries where hospitals use (H)SMRs for monitoring and improving hospital performance. Further research to identify how the effects of readmissions may impact the HSMR in other countries is clearly warranted. The accuracy of the HSMR may improve by taking admission frequency as an additional adjustment parameter.