- Research article
- Open Access
Hospital characteristics associated with low-value care in public hospitals in New South Wales, Australia
BMC Health Services Research volume 20, Article number: 750 (2020)
Rates of low-value care vary between hospitals in New South Wales, Australia. Understanding factors associated with this variation will help in understanding the drivers of low-value care and in planning initiatives to reduce low-value care.
For eight low-value procedures, we used Poisson regression of the number of low-value episodes at each hospital to assess the association between low-value care and hospital characteristics. We also used hierarchical clustering on the low-value procedures used and their rates at each hospital to try to identify groups of hospitals with higher or lower rates of low-value care across multiple procedures.
Some hospital characteristics, such as hospital peer group and proportion of total episodes that involve the specific procedure, showed associations for some procedures, but none were consistent across all eight procedures. We clustered hospitals into five groups, but low-value care rates did not differ much between these groups.
Available hospital variables show little association with rates of low-value care and no patterns across different low-value procedures. We need to investigate factors within hospitals, such as clinician knowledge and beliefs about low-value care.
Choosing Wisely  and similar movements have highlighted various tests and procedures that are low value and not expected to provide a net benefit in specific patient groups. Efforts have been made to measure how much these tests and procedures are used in practice, with measurement programs in various countries, including the United States [2,3,4], Canada [5, 6], Austria , and Australia [8,9,10,11].
The prevalence of low-value care varies by region [3, 6] and by hospital [10, 12]. Factors that may contribute to this variation include aspects of the health system and the population [3, 6]. For example, the ratio of specialist to primary care practitioners is a significant factor in the United States  and Alberta, Canada . For population factors, race and poverty rate are significant in the United States , while age, sex, and neighourhood income are significant in Alberta .
In New South Wales, Australia’s most populous state, there is considerable interhospital variation in the prevalence of low-value care . In a previous analysis, we used multilevel models to assess the contribution of Local Health District (LHD) and hospital of treatment and patient’s area of residence to the variation in low-value care prevalence . LHDs are administrative regions that make funding and policy decisions for all hospitals in the region. Hence, differences in policy could explain differences in low-value care prevalence. However, we found the LHD contributed little to the total variation . We also found the patient’s area of residence explained little of the variation; most variation is at the hospital level . Therefore, for this analysis we narrow the focus to further examine variation between hospitals.
A full understanding of interhospital variation in low-value care is likely to require research within hospitals to examine local policies, access to technological infrastructure, attitudes and knowledge of individual clinicians, and aspects of hospital “culture” , but such data are not yet available. This exploratory analysis was conducted to assess how much currently available administrative data can tell us about interhospital variation in low-value care.
Data and setting
Inpatient care in Australia is provided in public and private hospitals. In New South Wales, the most populous state, there are 221 public facilities and 210 private facilities , although many of these hospitals do not provide the services we are studying. We were not given access to private hospital data for this analysis.
Our data included all admissions to public hospitals in New South Wales for the period 1 July 2010 to 30 June 2017. The data include patient demographic information (e.g., age, sex, area of residence), clinical information (e.g., procedures performed and diagnoses assigned), and administrative information (e.g., private or public patient, emergency or elective care) for all inpatient episodes at public hospitals in NSW. Outpatient encounters are not included in this dataset.
Low-value care indicators
We have previously developed indicators based on admitted patient data to identify low-value care involving 27 procedures [9, 10]. Many of these procedures have very low numbers of low-value episodes in our setting . For this study, we selected eight procedures with annual low-value numbers ranging from about 80 (renal artery angioplasty) to 1762 (endoscopy for dyspepsia in people < 55 years old) (Table 1). The lower limit of about 80 was arbitrarily chosen to provide sufficient low-value episodes per hospital and per parameter to obtain reasonable model estimates.
Our analysis was in two parts. First, for each procedure, we modelled the number of low-value episodes at each hospital per year (so the unit of observation is the hospital-year). Using RStan  through the brms package , we estimated multilevel Poisson models with a random intercept term for hospitals to account for correlation within hospitals between years. Explanatory variables were year, the hospital peer group (a classification of similar hospitals based on total volume of episodes and types of service provided; see Additional file 1) , the proportion of female patients at the hospital in each year, the proportion of episodes involving patients aged over 65 years at the hospital in each year, whether the hospital was in a rural or metropolitan LHD, and the proportion of total episodes at the hospital that involved the procedure being modelled in each year. The age, sex, and rural variables were intended to capture differences in the hospital catchment populations. Rural status also reflects access to specialist care . The proportion of total episodes that involved the relevant procedure was included because hospitals that do a procedure more often may also be doing more low-value procedures . We included the number of episodes involving the procedure at each hospital in each year as an offset term. We fit models with each explanatory variable individually for unadjusted results, and then included all variables in a single model to produce estimates adjusted for all variables.
Secondly, we used hierarchical clustering to assess if there are groups of hospitals with high or low rates of low-value care across multiple procedures. Our approach was to build a typology of hospitals based on the low-value procedures they performed and the rates of those procedures, and then assess if the hospital groups differ on other characteristics. We created a series of indicators of whether each procedure was provided (0 if not provided) because not all procedures are provided at all hospitals. Because the proportion of low value services varied substantially between procedures, we standardised the observed proportions to the range 0–1, by subtracting the minimum and dividing by the range of observed proportions for each procedure. We used the indicators of whether the procedure was provided and the standardised proportions of each low-value procedure in the hierarchical clustering. That is, we grouped hospitals on the procedures they provided and the low-value rates of those procedures. We then compared these groups on hospital characteristics, including total volume, number of the procedures provided, percentage of female patients, percentage of patients aged > 65 years, and rural or metropolitan location.
Additional file 2 contains a summary of the hospital characteristics for each of the low-value procedures included in this study.
Hospital characteristics associated with low-value care
In general, the hospital characteristics available for this analysis showed little clear association with rates of low-value care, with no clear pattern across different low-value procedures (Fig. 1). Hospitals where the relevant procedure accounted for a higher proportion of all procedures at the hospital were more likely to do low-value carotid endarterectomy, knee arthroscopy, renal artery angioplasty, and percutaneous coronary intervention (PCI). In contrast, hospitals with more hysterectomies as a proportion of all procedures were less likely to perform abdominal (low-value) hysterectomies. Compared with principal referral hospitals, peer group B hospitals (major hospitals) were less likely to do low-value renal artery angioplasty and PCI. Low-value knee arthroscopy appeared more likely at smaller community hospitals than principal referral hospitals, but adjustment for other variables reduced or reversed this effect.
Low-value colonoscopy was less likely at smaller hospitals, but a similar pattern was not observed for endoscopy, although in our dataset about 40% of patients under 50 years who receive colonoscopy also receive endoscopy.
Rural hospitals had lower rates of low-value colonoscopy, endoscopy, and carotid endarterectomy, but higher rates of low-value knee arthroscopy, endovascular repair of abdominal aortic aneurysm (EVAR), and renal artery angioplasty.
Clustering hospitals by low-value care
After viewing the hierarchical clustering tree, we chose to group the hospitals into five clusters (Fig. 2). Table 2 summarises the characteristics of the hospitals in the five groups derived from hierarchical clustering. We did not observe a significant difference in the rates of low-value care between clusters; hence there is no clear association between hospital characteristics and low-value care across multiple procedures. However, the large hospitals in cluster 1 and the small hospitals in cluster 5 had the lower rates of low-value care than the other clusters.
Clusters 1 and 2 are predominantly large metropolitan hospitals; cluster 1 comprises only principal referral hospitals, while cluster 2 has both principal referral and major hospitals. Cluster 1 hospitals are larger and provide all the eight potentially low-value procedures, while cluster 2 hospitals each only provide six or seven of the procedures, and renal artery angioplasty is not provided by any of them. The low-value care rate was slightly higher in cluster 2 than in cluster 1.
Cluster 3 consists of two rural hospitals, a principal referral hospital, and a specialist women’s hospital. Each of these provides only one of the eight services. This group had the lowest rate of low-value care.
Clusters 4 and 5 are small hospitals. Cluster 4 includes smaller metropolitan hospitals, and rural community hospitals, while cluster 5 is primarily small rural hospitals. Cluster 5 hospitals do fewer of the procedures and have lower rates of low-value care compared with cluster 4.
Hospitals vary in their rates of low-value care in New South Wales , and internationally [19, 20], but there is little information about the factors that contribute to this variation. In this study, we examined several hospital characteristics but found no general associations across eight low-value procedures. Some characteristics, such as hospital peer group and proportion of total episodes that involve the specific procedure, showed associations for some procedures, but none were consistent across all eight procedures. We also clustered the hospitals by the procedures they perform and the rates of low-value care across the procedures to try to identify groups that have higher or lower rates of low-value care generally, with little success.
Our results imply low-value care is not a general property of a hospital in NSW, but varies by procedure within hospital. Low-value care needs to be reported and investigated by procedure type. Combining low-value estimates for multiple procedures at the hospital level will not convey a true picture of low-value care in NSW, especially as most hospitals do not provide all the procedures being measured. This is in contrast to the United States, where multiple low-value indicators have been combined to form the Johns Hopkins Overuse Index (JHOI) . The main reason for this difference is population. The JHOI uses 20 tests and procedures with > 800 events across the United States , while we have only eight procedures with 80–1800 events in NSW.
In general, NSW hospitals that have relatively higher volumes of a particular procedure tended to have higher low-value rates for that procedure. Direct measures of low-value care, such as ours, can be used to monitor actual rates of low-value care, as well as associated costs or downstream consequences . However, data limitations meant we could not develop measures for most recommendations . The results presented here suggest higher overall rates of a procedure may be a useful indirect indicator of hospitals where low-value care occurs for recommendations that cannot be adapted for direct measurement. It is important that such indirect measures only be used to target further investigation, as efforts to simply reduce rates may lead to undertreatment of patients for whom a procedure is appropriate .
As the hospital characteristics we examined here are not strongly associated with low-value care, we need to look elsewhere for explanations of the variation in rates between hospitals. A Norwegian study found important variation in patient safety culture at the ward level , and our results support this possibility that low-value care also varies between small units within hospitals. The most obvious factor to consider is individual treating doctors, as it is they who decide (in consultation with the patient) what treatment to provide. Group (by specialty or ward) level practice norms may also develop independently within a hospital, resulting in low-value care ‘hotspots’ that reflect a department within a hospital as opposed to the hospital itself. This is a critical insight for quality improvement initiatives. Whether a hospital has high or low rates of low-value care likely depends on the choices of the doctors who work there, and will depend on their knowledge of and agreement with the recommendations and underlying evidence .
NSW hospitals do not appear to have high or low rates of low-value care across multiple procedures. Instead, low-value rates of each procedure need to be examined separately. Basic characteristics of the hospitals and patient populations do not strongly associate with low-value care. Understanding the drivers of low-value care is likely to require mixed method studies within hospitals. In particular, examining the knowledge and attitudes of the clinicians who provide these procedures may be more likely to provide explanations of the hospital variation in rates of low-value care.
Availability of data and materials
The data that support the findings of this study are held by the NSW Ministry of Health and restrictions apply to the use of these data under the Health Records and Information Privacy Act 2002 (NSW). On reasonable request, the authors will assist in preparing a data request and ethics application to the NSW Population and Health Services Research Ethics Committee.
Endovascular repair of abdominal aortic aneurysm
Johns Hopkins Overuse Index
Local Health District
New South Wales
Percutaneous coronary intervention
Levinson W, Kallewaard M, Bhatia RS, Wolfson D, Shortt S, Kerr EA. “Choosing wisely”: a growing international campaign. BMJ Qual Saf. 2015;24:167–74.
Schwartz AL, Landon BE, Elshaug AG, Chernew ME, McWilliams JM. Measuring low-value care in Medicare. JAMA Intern Med. 2014;174:1067–76.
Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2014;30:221–8.
Segal JB, Bridges JFP, Chang H-Y, Chang E, Nassery N, Weiner J, et al. Identifying possible indicators of systematic overuse of health care procedures with claims data. Med Care. 2014;52:157–63.
Bhatia RS, Levinson W, Shortt S, Pendrith C, Fric-Shamji E, Kallewaard M, et al. Measuring the effect of choosing wisely: an integrated framework to assess campaign impact on low-value care. BMJ Qual Saf. 2015;24:523–31.
McAlister FA, Lin M, Bakal J, Dean S. Frequency of low-value care in Alberta, Canada: a retrospective cohort study. BMJ Qual Saf. 2018;27:340–6. https://doi.org/10.1136/bmjqs-2017-006778.
Sprenger M, Robausch M, Moser A. Quantifying low-value services by using routine data from Austrian primary care. Eur J Pub Health. 2016;26:912–6.
Brett J, Elshaug AG, Bhatia RS, Chalmers K, Badgery-Parker T, Pearson S-A. A methodological protocol for selecting and quantifying low-value prescribing practices in routinely collected data: an Australian case study. Implement Sci. 2017;12:58.
Chalmers K, Badgery-Parker T, Pearson S-A, Brett J, Scott IA, Elshaug AG. Developing indicators for measuring low-value care: mapping choosing wisely recommendations to hospital data. BMC Res Notes. 2018;11:163.
Badgery-Parker T, Pearson S-A, Chalmers K, Brett J, Scott IA, Dunn S, et al. Low-value care in Australian public hospitals: prevalence and trends over time. BMJ Qual Saf. 2019;28:205–14.
Chalmers K, Pearson S-A, Badgery-Parker T, Brett J, Scott IA, Elshaug AG. Measuring 21 low-value hospital procedures: claims analysis of Australian private health insurance data (2010 – 2014). BMJ Open. 2019;9:e024142.
Krimphove MJ, Cole AP, Friedlander DF, Nguyen DD, Lipsitz SR, Nguyen PL, et al. The current landscape of low-value care in men diagnosed with prostate cancer: what is the role of individual hospitals? Urol Oncol Semin Orig Investig. 2019;37:575.e9–575.e18. https://doi.org/10.1016/j.urolonc.2019.04.001.
Badgery-Parker T, Feng Y, Pearson S-A, Levesque J-F, Dunn S, Elshaug AG. Exploring variation in low-value care: a multilevel modelling study. BMC Health Serv Res. 2019;19:345.
Australian Institute of Health and Welfare. Hospital resources 2017–18: Australian hospital statistics. 2019. https://www.aihw.gov.au/reports/hospitals/hospital-resources-2017-18-ahs/report-editions. Accessed 29 Apr 2020.
Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, et al. Stan: a probabilistic programming language. J Stat Softw. 2017;76:1–32.
Burkner P-C. Brms: an R package for Bayesian multilevel models using Stan. J Stat Softw. 2017;80:1–28.
NSW Ministry of Health. NSW hospital peer groups 2016. Sydney: NSW Health; 2016. https://www1.health.nsw.gov.au/pds/ActivePDSDocuments/IB2016_013.pdf.
Tracey EA, Mccaughan B, Badgery-Parker T, Young JM, Armstrong BK. Patients with localized non-small cell lung cancer miss out on curative surgery with distance from specialist care. ANZ J Surg. 2015;85:658–63.
Kirkham KR, Wijeysundera DN, Pendrith C, Ng R, Tu JV, Laupacis A, et al. Preoperative testing before low-risk surgical procedures. CMAJ. 2015;187:E349–58.
Segal JB, Nassery N, Chang H-Y, Chang E, Chan KS, Bridges JFP. An index for measuring overuse of health care resources with Medicare claims. Med Care. 2015;53:230–6.
Badgery-Parker T, Pearson S-A, Dunn S, Elshaug AG. Measuring hospital-acquired complications associated with low-value care. JAMA Intern Med. 2019;179:499–505. https://doi.org/10.1001/jamainternmed.2018.7464.
Baker DW, Qaseem A, Reynolds PP, Gardner LA, Schneider EC. Design and use of performance measures to decrease low-value services and achieve cost-conscious care. Ann Intern Med. 2013;158:55–9.
Deilkås E, Hofoss D. Patient safety culture lives in departments and wards: multilevel partitioning of variance in patient safety culture. BMC Health Serv Res. 2010;10.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud P-AC, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458–65. https://doi.org/10.1001/jama.282.15.1458.
Consent for publication.
This project was funded through a partnership agreement between the Capital Markets Cooperative Research Centre, the NSW Ministry of Health, and the University of Sydney, with additional funding from the HCF Research Foundation and the National Health and Medical Research Council (NHMRC) (project grant ID 1109626). The funding organisations had no role in the design and analysis of the study, nor in the writing of the article, nor in the decision to publish the study.
Ethics approval and consent to participate
This study was approved by the NSW Population and Health Services Research Ethics Committee (CI 2015/09/607).
AGE receives salary support as the HCF Research Foundation Professorial Research Fellow, is a Ministerial appointee to the (Australian) Medicare Benefits Schedule (MBS) Review Taskforce, is a Board Member of the NSW Bureau of Health Information (BHI) and advisor to Cancer Australia. He has consulted on low-value care to Private Healthcare Australia, the Queensland, South Australia and Victoria state health departments, and the Department of Veterans Affairs. TBP receives salary support via a doctoral scholarship from the Capital Markets Cooperative Research Centre-Health Market Quality Program, and a University Postgraduate Award from the University of Sydney, and has received consulting fees from Queensland and Victorian state health departments. TBP is a member of the editorial board of BMC Health Services Research.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Badgery-Parker, T., Pearson, SA. & Elshaug, A.G. Hospital characteristics associated with low-value care in public hospitals in New South Wales, Australia. BMC Health Serv Res 20, 750 (2020). https://doi.org/10.1186/s12913-020-05625-4