Skip to main content
Download PDF

Implementing performance improvement in New Zealand emergency departments: the six hour time target policy national research project protocol

BMC Health Services Research volume 12, Article number: 45 (2012) Cite this article

Abstract

Background

In May 2009, the New Zealand government announced a new policy aimed at improving the quality of Emergency Department care and whole hospital performance. Governments have increasingly looked to time targets as a mechanism for improving hospital performance and from a whole system perspective, using the Emergency Department waiting time as a performance measure has the potential to see improvements in the wider health system. However, the imposition of targets may have significant adverse consequences. There is little empirical work examining how the performance of the wider hospital system is affected by such a target. This project aims to answer the following questions: How has the introduction of the target affected broader hospital performance over time, and what accounts for these changes? Which initiatives and strategies have been successful in moving hospitals towards the target without compromising the quality of other care processes and patient outcomes? Is there a difference in outcomes between different ethnic and age groups? Which initiatives and strategies have the greatest potential to be transferred across organisational contexts?

Methods/design

The study design is mixed methods; combining qualitative research into the behaviour and practices of specific case study hospitals with quantitative data on clinical outcomes and process measures of performance over the period 2006-2012. All research activity is guided by a Kaupapa Māori Research methodological approach. A dynamic systems model of acute patient flows was created to frame the study. Consequences of the target (positive and negative) will be explored by integrating analyses and insights gained from the quantitative and qualitative streams of the study.

Discussion

At the time of submission of this protocol, the project has been underway for 12 months. This time was necessary to finalise both the case study sites and the secondary outcomes through key stakeholder consultation. We believe that this is an appropriate juncture to publish the protocol, now that the sites and final outcomes to be measured have been determined.

Background

In May 2009, the New Zealand government announced a new policy aimed at improving the quality of Emergency Department (ED) care and whole hospital performance. The 'Shorter Stays in ED' target stipulates that "95% of patients will be admitted, discharged or transferred from an ED within six hours by July 2009" [1]. This target was supported by the National ED Advisory Group, made up of senior ED clinicians and managers [2]. They advised the policy initiative based on observational studies from overseas showing that overcrowding or long waits in ED were associated with poorer outcomes for patients such as increased mortality [3–5]. Similar ED time targets have been recommended by health reformers in Australia [6] and follow the lead of the English National Health Service which introduced time targets for ED in 2001 [7]. In the UK, official monitoring indicated that these targets were successfully met at the national level by 2004 [8]. This contrasted with the lack of improvement in ED waiting times in Scotland, Wales and Northern Ireland over the same time period, where such targets were not introduced [9].

Nevertheless, there has been widespread debate about whether targets in general, and the English ED targets in particular 'do more harm than good' [10]. Advocates of a target approach claim that ED targets can act as an effective catalyst for quality improvement across the whole hospital, and even the wider health system. Taking a systems view of a hospital, the ED is only one part of the patient journey. The pathway for ED patients into in-patient beds is determined by bed availability, inpatient care delivery and discharge practices. These have been shown to improve through attention to quality initiatives [11]. Therefore, from a whole system perspective, using the ED waiting time as a performance measure has the potential to see improvements in parts of the health system beyond ED. This has been demonstrated via application of dynamic system model simulations [12, 13] and such simulations have become a cogent influence on policy makers.

However, instead of acting as a catalyst for stimulating and improving broader health system performance, the imposition of targets may have significant adverse consequences. Two types of adverse consequences are typically identified. The first is gaming; or reactive subversion, such as 'hitting the target but missing the point'. Examples are ambulances being made to wait outside the ED to delay arrival time, [14] or re-designating patients as short stay admissions at or around the target time to avoid 'breaching' the target [15, 16]. The second is substitution effects; the potential diversion of attention from other important clinical areas and possible distortion of clinical and management priorities. Organisations, in their efforts to meet targets, turn resources and attention away from other important dimensions of performance and quality, and in doing so, shift systemic problems to other parts of the system or organisation [17]. For example, the pressure of admission from ED to an already stretched hospital service with close to 100% bed occupancy may serve to precipitate inappropriate discharge of patients. It is also possible that pressure to meet time targets for patient discharge may exacerbate inequities of care due to clinicians implicit biases [18].

As such, it is possible that the introduction of targets may ultimately detract from health service quality of care. Relevant measures of quality and performance include mortality, hospital length of stay (LOS), readmission rates, and more specific, condition-related measures such as time to reperfusion for those with acute myocardial infarction [19]. Some of these outcomes have been associated with ED overcrowding, [5, 20, 21] however the associations are not consistent [22–24] and there is little evidence to support the assumption that introduction of an explicit 'ED stay' performance measure will improve clinical markers of quality of care [25].

Whatever relationships there may be between the use of targets and broader clinical and hospital outcomes, these are almost certainly shaped by organisational factors. The literature on organisational studies has demonstrated that the application of performance targets drives organisational attention and resources towards achieving these targets. This has been illustrated in manufacturing, [26, 27] retail, [28] and education [29]. Relevant factors include the motivational role of the Chief Executive Officer, the structure of the senior management team and elements of organisational culture. However, it is not at all clear how targets affect organisational behaviour within the complex and complicated context of a health system. Much of the organisational studies literature has been based in contexts that are relatively less complex, more likely to have an agreed organisational purpose, and where the value system of employees is more aligned [30, 31].

The evidence regarding organisational responses to the English ED target does not yet add up to a coherent picture. A report commissioned by the UK Health Commission, identified a number of organisational factors contributing to delays in patient care, including the level of clinician involvement in ED and hospital management, levels of nurse absenteeism and lower proportion of non-salary expenditure [32]. This study, however, did not link ED performance with broader clinical outcomes. Another recent study sought to investigate the presence and extent of dysfunctional organisational response to the ED time target [17]. The authors undertook an aggregated analysis of the effects of the ED targets on English National Health Service trusts between 2002 and 2007. They concluded that there was no evidence of dysfunctional effects, and that if anything, the impact on quality was beneficial. However, this research has significant limitations. Firstly, only mortality and inpatient admission (itself a rather ambiguous indicator) were included in the study, and the results conflicted with other reports using the same dataset [33, 34]. Secondly, it is doubtful that an exclusively quantitative study of this type would be capable of detecting gaming and effort substitution. Gaming behaviour may be undetectable or statistically insignificant at the aggregate level because the nature and strategies of gaming most likely vary across health care organisations due to differences in organisational context.

The New Zealand health care system is divided into 20 regional District Health Boards (DHB), who manage an extensive system of publicly-owned hospitals that are directly subject to government policy priorities. This provides a context in which it is possible to investigate the effect of an explicit policy directive on the performance of EDs, hospitals and wider health systems.

The introduction of the ED target in New Zealand raises some fundamental questions which can be adequately answered only by undertaking a thorough investigation of implementation processes and impact on the quality of health care delivered. There is very little empirical work that examines how the performance of the wider hospital system is affected by the introduction of a performance target within ED. Both positive and negative consequences of time targets can be best detected by combining qualitative, case-study research into the behaviour and practices of specific hospitals with quantitative data on clinical outcomes and other dimensions of performance.

A priority for this research is to contribute to health equity in New Zealand Emergency Departments. Health inequalities in New Zealand are well documented. In particular, health inequalities by ethnicity have been described as "the most consistent and compelling of health inequalities in New Zealand" [35]. Importantly, evidence also suggests that changes in the political and economic environment, including setting of health targets, may have "unequal impact" and can increase disparities rather than reduce them [36].

Key results of the 2006/7 New Zealand Health Survey [37] describe current inequalities in ED use in New Zealand by ethnicity, gender, age and neighbourhood deprivation. We will therefore consider the impact on the Target via these variables, where data is adequate.

This research therefore aims to investigate the following questions:

  • How has the introduction of the ED target affected broader hospital performance over time, and what accounts for these changes?

  • Which initiatives and strategies have been successful in moving hospitals towards the target without compromising other quality and hospital outcomes?

  • Is there a difference in change in outcomes between different ethnic age and deprivation groups?

  • Which initiatives and strategies have the greatest potential to be transferred across organisational contexts?

Methods/design

This is a mixed methods study with three interdependent research streams. The streams employ a range of quantitative and qualitative methods to address a number of research questions. Each stream is conducted consecutively and concurrently over the study time frame and outputs from each inform the others. This material is united into a final, fourth stream to address the overall aims of the study (Figure 1).

Figure 1
figure 1

Content and relationships of the different study streams.

Full size image

Methodological approach with respect to health inequalities

This research project has considerable potential to contribute to the health needs of Māori. Robust examination of both clinical outcomes and the policy implementation process can reveal critical insights on how this policy will affect Māori patients receiving acute emergency care. Research findings will provide empirical support for identifying both the positive and negative effects of the policy for Māori health, and potentiate Māori health gain. Understanding whether the setting of ED targets introduces disparities for Māori (or vice versa) is a specific area of investigation for this research.

Given this context, this research is being guided by a Kaupapa Māori Research (KMR) methodology [38, 39]. Within the context of this study, the KMR framework includes senior Māori research expertise throughout all stages and streams of the research process to: ensure utility and cultural safety of the research process for Māori; Māori research kaitiakitanga (guardianship and protection) of Māori data; review and approval of Māori data collection, analysis and interpretation; analysis and interpretation of Māori data and comparisons with non-Māori and review of any manuscripts involving Māori data prior to submission for publication. Such an approach requires the researchers to aim for equal explanatory and analytical power for Māori compared to non-Māori, a rigorous process to the collection, evaluation and reporting of ethnicity data from available datasets, and interpretation of the data from a non-victim blame or cultural deficit positioning [40, 41]. The research team has agreed to conduct all research activity in accordance with the Tōmaiora Māori Health Research Centre research protocols [42].

Stream one

Research aims

The aim of this research stream is to identify initiatives that have been implemented in response to the performance target across the 20 DHBs, to explore the impact of these initiatives on patient flow into and out of EDs, and to identify four case study sites for in-depth analysis. The selection of hospitals was based on initial target performance results from the first quarter of 2009, and by geographic location and population density.

Design/method and rationale

The core research question for this stream is 'What impact do different initiatives aimed at the ED performance target have on measures of patient flow? We used a dynamic system modelling approach similar to that used in other studies internationally that have attempted to model the dynamics of an ED [12, 43, 44] to frame this stream. This approach had four sequential steps [45].

  1. 1.

    An initial mapping of patient flow.

  2. 2.

    Refining the initial model with an expert reference group.

  3. 3.

    Using the refined concept model to inform a survey of clinical directors (CD) and service managers within each ED that is subject to the national performance time-target. This survey was designed to collect information on the initiatives that have been developed in response to the target and where these initiatives sit on the concept model of patient flow (Additional file 1). Respondents were be asked to identify any changes in resource use (especially staff changes) associated with the interventions and to estimate any additional departmental expenditure associated with the intervention. Information from January 2006 onwards of measures of patient flow (ED admissions/discharges, bed occupancy, LOS elective admissions and elective surgical cancellations) were collected.

  4. 4.

    Survey data collected from each site will subsequently be used to develop a working system dynamic model of patient flow into and out of ED, and illustrate the impact of initiatives put in place in response to the performance target.

Recruitment and data collection

Reference group

This was convened in December 2010 and consisted of the representatives of expert groups relevant to this study, including members of: the national ED advisory group to the Ministry of Health, the Australasian College for Emergency Medicine (ACEM), the College of Emergency Nurses of New Zealand (CENNZ), the Royal New Zealand College of General Practitioners (RNZCGP), Māori representatives, Pacific peoples representatives, representatives from inpatient specialties covering medicine, surgery and paediatrics, representatives of inpatient and community older people's health and the CD and service managers from each case site.

Survey

The CD and/or service manager in each ED completed the survey in mid 2011. Follow-up will occur at the end of 2012 to capture any further interventions.

Data analysis

Further analysis of the data will follow the usual conventions for system modelling [45]. The concept model will be entered into a specialised software package, iThink [46] and differential equations will be calculated to estimate the patient flow, and impact of the interventions. This will allow us to examine whether the intervention has simply moved the 'access block' to other parts of the hospital. It will also allow us to identify any cost-shifting to other services and to make some estimates of any changes in down-stream costs (such as changes in LOS). Survey results will also be analysed to determine the areas of cost related to the interventions, and the perceived impacts the interventions have had on performance.

Stream two

Research aims and questions

This research stream aims to investigate the impact of the target on objective markers of quality of care. The core research questions for this stream are; 'Was there a change in clinically relevant outcomes after the target was introduced?' and 'Were there differential impacts of the target in at risk ethnic, age and deprivation groups?'

Design/method

We will investigate whether the rate of change of quality of care markers differs over time, pre and post introduction of the ED time targets in July 2009. The study will compare outcomes of interest using data collected from three years prior to and after the introduction of the target (1st January 2006 to 31st December 2008 and 1st January 2010 to 31st December 2012). A one year 'target settling' period six months either side of the target introduction in July 2009 will be excluded from analysis. The primary outcomes will be:

  • ED and Hospital length of stay (LOS) [47]

  • Re-attendance rates within 48 hours of discharge [48, 49]

  • Differences in these outcomes in different ethnic, age and deprivation groups

The secondary outcomes are other clinical process measures and clinical outcomes that reflect the quality of care delivery both in ED and wider hospital. The candidate indicators were determined by a systematic literature review and the final set was confirmed by an expert reference group (see Stream one, above).

The secondary outcomes will be;

  • all cause mortality [2, 3, 50, 51]

  • time to reperfusion for ST elevation myocardial infarction (STEMI) [49]

  • time to analgesia in ED [52]

  • time to theatre for fractured neck of femur [53]

  • time to antibiotics for severe infections [54]

  • time to treatment in acute asthma [55, 56]

  • proportion of patients who leave without being seen [48, 49]

  • time to CT for traumatic brain injury [57, 58]

  • time to appendectomy for acute appendicitis [59, 60]

  • appropriateness of discharge information provided to General Practitioners from ED [61, 62]

Outcome measures: definition, rationale and data collection

Primary

LOS starts from date/time of arrival and ends at time/date of discharge. Lack of capacity for acute admissions is a key factor driving the length of time spent in ED [63–65]. Increasing capacity by reducing LOS may have drawbacks. NZ has a shorter average LOS than both Australia and the UK [66] and LOS for Māori is two days shorter than for non-Māori [67, 68], with personal experiences in hospital a contributing factor to this [67]. A small change in LOS, for example a decrease of 0.25 days, is important, as MOH data suggests this would result in approximately 125000 more bed-days available per year nationwide. The distribution of times that patients spend in ED should be a smooth curve with a right skew when plotted graphically (most patients leave within a short time, some stay much longer). We will examine the distribution of ED LOS times pre and post target introduction, looking for a 'spike' of admissions or discharges at or near the target time (which may reflect gaming or patients moved elsewhere prior to completion of their care) [69, 70].

Re-attendance is defined as re-admission to hospital or representation to ED within 48 hours. This outcome is considered an adverse event [71, 72] and may result from inappropriate early discharge. Data on LOS and re-attendance, along with age, ethnicity, gender and NZ deprivation score is available from routinely collected hospital data and will be collected for all 20 DHBs. Ethnicity will be sourced from the National Health Index (a unique patient identifier in NZ) and reported according to the Ethnicity Data Protocols for the Health and Disability Sector, with appropriate adjustments made to account for known discrepancies in NHI data [73].

Secondary

While hospital mortality will be available from national datasets for all patients within the time periods, the other secondary outcomes occur within a subgroup of patients. Using the four case study sites selected in Stream One, a series of chart audits from patient hospital records, each with specific sampling frame and sample size calculations will be conducted. These records will be chosen at random throughout the time periods of interest to gain an in-depth understanding of within hospital variation before and after the introduction of the time target policy. The records will be identified by ICD-10 codes appropriate to each clinical outcome.

Statistical analyses

The data for each of the outcomes will be recorded by hospital by twelve month periods (to avoid modeling seasonal change). For continuous outcomes, in order to compare the rate of change in the measures pre to post intervention and whether any change is influenced by age, ethnicity or hospital, a general linear mixed model will be used with outcome transformed as necessary to overcome correlation of mean and variance in variables such as LOS. A random coefficients model will be used to allow the change to be modeled within hospital. Explanatory variables will be year within pre- and post-intervention time periods, hospital, age category, ethnicity and pre or post intervention. Appropriate interactions of year, intervention, ethnicity, age and hospital will be investigated with higher order interactions being removed if not important and the analysis being split where non-ignorable interactions exist. Binary outcomes such as readmission will be analysed similarly using a generalised linear mixed model. Estimates of least square mean values of outcomes, with 95% confidence intervals, will be calculated, within important subgroups. Data will be analysed using STATA version 10, StataCorp LP, 4905 Lakeway Drive, College Station, TX 77845 USA.

Sample size calculations

All sample size estimations below are based on having 80% power to demonstrate a difference at the five percent level of significance using the models described above. Expected distributions used for estimating variance were based on one tertiary hospital's data, which was the only data available to the researchers at that stage of the study. Expected proportions in each ethnic and age category were based on New Zealand hospital admission data. Māori proportions ranged from five to 18% in the different age groups. For outcomes retrieved from routine electronic data there are very large numbers (almost one million ED attendances annually) and it is possible to look for disparities in change in Māori and other at risk ethnic and age groups and differences in change within hospitals.

The sample size calculations for the primary outcomes are based on detecting interactions. The real difference in LOS considered important to be powered to detect is 0.25 days. To detect a difference in change for Māori compared to European/other, within age groups (< 65 and ≥ 65) requires a total sample of 12,000 in those aged < 65, 76,000 in ≥ 65 (or 4,000 in ≥ 65 if there was no ethnic interaction). These numbers would be present within individual hospitals except for investigating ethnic differences in change in the older age group, where Māori numbers are less. This will be investigated at the national level.

The re-attendance rate at the index hospital was approximately five percent, somewhat higher than reported elsewhere [30, 63, 65, 66]. To detect a real difference in change of one percent in re-attendance in Māori, or other at risk ethnic groups, compared to European/other, within age groups would require approximately 10,000 in the smallest group. Nationwide, there are about 9,000 Māori people older than 65 years admitted annually. Therefore, with three years data after the target was introduced we will have greater than 80% power to investigate this difference.

For secondary outcomes that require manual data extraction, random samples of records of the appropriate size from the six years of the study will be drawn. A sample size calculation for time to reperfusion is given here as this represents the smallest number of clinical events, is clinically very important and health outcomes are associated with disparities by age and ethnicity [74]. Across NZ approximately 1,500 patients per year with STEMI receive reperfusion therapy [74]. In order to detect a five minute change in time to reperfusion, based on a mean (SD) of the log transformed time of 3.61(.58) would require 160 patients per year over the time period of the study or 50 per year to detect a 10 minute change. For each confirmed clinical outcome, a power calculation to determine an appropriate sample size for our research question will be undertaken.

Stream three

Research aim and question

The specific aim of this research stream is to explore how organisations respond to the target by investigating the perspectives, experiences and actions of front line clinical and management personnel in the ED and wider hospital. A second and important aim is to identify the variations in organisational responses between case study sites and different informants. The core research question for this steam is 'How is the ED time target policy implemented?'

Design/method and rationale

This stream involves qualitative research into the four case study hospitals. Key strengths of qualitative research in the field of health service and policy research include the ability to enhance understanding of complex phenomenon in dynamic organizational contexts, and interpretation of experience from a variety of actors [75]. Qualitative design and methods have been recently applied to investigate the effect of performance improvement initiatives in the UK from the perspective of front line personnel, for example the emergency nursing experience of the 4 hour ED time target [76], the experience of pay for performance in primary health care providers [77, 78] and factors affecting the adoption of a "see and treat" model in emergency care [79]. The design for this research stream is qualitative multiple case study, using semi-structured interviews (Additional file 2) and policy documents within the case study sites to collect data. Comparison and contrast of findings can be made within and across different cases and different informant groups (clinical and management, inside the ED and in the wider hospital).

Recruitment and data collection

The first of two rounds of semi-structured interviews across the four case study hospitals, was conducted in early 2011, the second will be in mid 2012. These interviews explore the organisational response to the ED target and associated interventions. Two rounds of data collection will ensure that any changes over time can be captured. The choice of four case studies is based on achieving richness of data, variance in context and comparison of performance, but also recognises resource and time constraints within the research project overall. Between ten and twelve research informants at each site were purposively selected to achieve theoretical saturation, ensure spread across informant groups and requisite experience of policy implementation. Semi-structured interviews ensure focus and structure in the interview, whilst enabling flexibility to probe informant responses for detail, clarification of meaning or examples. Interviews were conducted face to face and audio taped. Strategic, operational, service and policy documents relevant to policy implementation, such as memoranda, guidelines and procedures, will also be collected from case study sites.

Data analysis

Transcribed interview data will be analysed using a general thematic inductive approach to identify the range of perspectives, experiences and actions across the hospitals and the respective groups of policy implementers. Documentary data will also be analysed using thematic inductive techniques. Interview data analysis will be analysed using MAXQDA (version 10), Verbi GmbH, Berlin, Germany.

Stream four

As indicated in Figure 1 above, this stream will draw together information from the other streams. In essence, Stream Two will provide a picture of the patterns of clinical and organisational performance over time (2006-2012), while Streams One and Three will provide the means to interpret and explain such patterns and provide information on costs and resources used to meet the target.

Data analysis

Using data from Stream Two we will classify the case study organisations according to degree of success in meeting ED target with/without adverse clinical and hospital outcomes. Themes developed in the analysis of Stream One and Stream Three data will be mapped against Stream Two results to construct explanations of success/failure. For example, information collected in Stream Two may raise a suspicion of gaming in a particular site. A 'spike' of admissions or discharges at or near the target time may reflect gaming; or that patients are moved out of ED prior to completion of their care. Such consequences may show up in data on clinical outcomes. The data collected in Stream Three will provide a more detailed interpretation of this phenomenon and its broader consequences, especially when such information is incorporated into Stream Three interview schedules.

There are some important precedents for using a mixed methods approach in order to account for ED and hospital responses to targets. Mason et al. were able to attribute some variation in ED mean waiting times to the management style of lead clinicians, with data on management style initially collected through in-depth interviews [32]. Mannion et al., using a multiple case-study approach, identified key attributes of organisational culture and management style that accounted for differences between 'high performing' from 'low performing' trusts according to official performance measures [80].

The multi-case study approach should allow the research to distinguish common organisational factors affecting quality and performance from factors that are highly context-specific. As such, this research can help to distinguish between general strategies for quality and performance improvement (strategies that appear to have been successful in more than one setting), and future initiatives that will need to be highly tailored to specific organisational contexts.

Discussion

Current stage of research

At the time of submission of this protocol, the project has been underway for 12 months. This time was used to finalise both the case study sites for Stream Three and the secondary outcomes for Stream Two, for which a comprehensive dictionary of data definitions has been completed (Additional file 3). We believe that this is an appropriate juncture to publish the protocol and data dictionary, now that the sites and outcomes to be measured have been determined. During this time the Stream One surveys have been completed, as have the first round of interviews in Stream Three. The analysis of the data from these streams will be undertaken in 2012, in parallel with the quantitative data collection and analysis for Stream Two.

There are other important related questions this project is not able to explore. One such question is whether patient experiences of ED treatment has changed as a consequence of implementation of the target. Although we have attempted to address areas of timeliness of care previously identified as being of concern to patients in our clinical outcomes, we recognise that this is only a surrogate for the patient's perspective in assessing the quality of healthcare. Another question not addressed by the current project is the effect of implementation of the target on the experiences of ED staff [76]. We are currently exploring how such questions can be better captured in the context of our research.

Ethical approval

Ethical approval for this study was granted by the New Zealand Multi-Region Ethics Committee (MEC 10/06/060) on 1st July 2010.

Authors' information

The Shorter Stays in Emergency Departments National Research Project team is a collaboration of researchers from the School of Population Health (Faculty of Medical and Health Sciences, University of Auckland) and the Emergency Department Research Team at the Auckland District Health Board. The lead investigators are Peter Jones and Linda Chalmers. The investigators are Shanthi Ameratunga, Toni Ashton, Peter Carswell, Elana Curtis, Papaarangi Reid, Joanna Stewart, Tim Tenbensel and Susan Wells. The research fellows are Alana Harper and James LeFevre, the data manager is Denish Kumar and the research administrator is Jayshree Ramesh-Sukha

References

  1. Health Targets: Shorter stays in Emergency Departments. 2009, Ministry of Health, New Zealand, [updated 2009; cited 2009 24/08]; Available from: http://www.moh.govt.nz/moh.nsf/indexmh/healthtargets-targets-emergencydepartments

    Google Scholar 

  2. Recommendations to Improve Quality and the Measurement of Quality in New Zealand Emergency Departments. 2009, [Working Party Recommendations]: Ministry of Health, New Zealand, [updated 2009 24/08/09; cited]; Available from: http://www.moh.govt.nz/moh.nsf/pagesmh/8783/$File/quality-ed-jan09.pdf

  3. Sprivulis PC, Da Silva J-A, Jacobs IG, Frazer ARL, Jelinek GA: The association between hospital overcrowding and mortality among patients admitted via Western Australian emergency departments. Medical Journal of Australia. 2006, 184 (5): 208.

    PubMed  Google Scholar 

  4. Richardson DB: Increase in patient mortality at 10 days associated with emergency department overcrowding. Medical Journal of Australia. 2006, 184 (5): 213-216.

    PubMed  Google Scholar 

  5. Bernstein SL, Aronsky D, Duseja R, Epstein S, Handel D, Hwang U, et al: The Effect of Emergency Department Crowding on Clinically Oriented Outcomes. 2009, Journal [serial on the Internet]. Date; (1): Available from: http://dx.doi.org/10.1111/j.1553-2712.2008.00295.x

    Google Scholar 

  6. Government of Western Australia: Four Hour Rule Program. 2008, [updated 2008 27/08/2009; cited]; Available from: http://www.health.wa.gov.au/fourhourrule/home/

    Google Scholar 

  7. Alberti G, Hancock C, Jackson B, et al: The NHS Plan: a plan for investment a plan for reform. Cm 4818-I. 2000, London: Department of Health, 1-147.

    Google Scholar 

  8. Alberti G: Transforming Emergency Care in England. Report No. 264639. 2004, London: Department of Health

    Google Scholar 

  9. Hauck K, Street A: Do targets matter? A comparison of English and Welsh National Health priorities. Health Econ. 2007, 16 (3): 275-290. 10.1002/hec.1153.

    Article  PubMed  Google Scholar 

  10. Gubb J: Have targets done more harm than good in the English NHS? Yes. BMJ. 2009, 338 (jan16_2): a3130-.

    Article  PubMed  Google Scholar 

  11. Miro O, Sanchez M, Espinosa G, Coll-Vinent B, Bragulat E, Milla J: Analysis of patient flow in the emergency department and the effect of an extensive reorganisation. Emerg Med J. 2003, 20 (2): 143-148. 10.1136/emj.20.2.143.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Brailsford SC, Lattimer VA, Tarnaras P, Turnbull JC: Emergency and On-Demand Health Care: Modelling a Large Complex System. J Oper Res Soc. 2004, 55 (1): 34-42. 10.1057/palgrave.jors.2601667.

    Article  Google Scholar 

  13. Lane DC, Husemann E: System dynamics mapping of acute patient flows. J Oper Res Soc. 2008, 59: 213-224. 10.1057/palgrave.jors.2602498.

    Article  Google Scholar 

  14. Bevan G, Hood C: What's measured is what matters: targets and gaming in the English public health care system. Public Administration. 2006, 84 (3): 517-538. 10.1111/j.1467-9299.2006.00600.x.

    Article  Google Scholar 

  15. Mayhew L, Smith D: Using queuing theory to analyse the government's 4-H completion time target in accident and emergency departments. Health Care Manag Sci. 2008, 11 (1): 11-21. 10.1007/s10729-007-9033-8.

    Article  CAS  PubMed  Google Scholar 

  16. Sibly E, Wiskin CM, Holder RL, Cooke MW: Short stay emergency admissions to a West Midlands NHS Trust: a longitudinal descriptive study, 2002-2005. Emerg Med J. 2007, 24 (8): 553-557. 10.1136/emj.2006.043901.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Kelman S, Friedman JN: Performance improvement and performance dysfunction: an empirical examination of distortionary impacts of the emergency room wait-time target in the English National Health Service. Journal of Public Administration Research and Theory. 2009, 19 (4): 917-946. 10.1093/jopart/mun028.

    Article  Google Scholar 

  18. van Ryn M, Saha S: Exploring Unconscious Bias in Disparities Research and Medical Education. JAMA. 2011, 306 (9): 995-996. 10.1001/jama.2011.1275.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Writing Committee M, Masoudi FA, Bonow RO, Brindis RG, Cannon CP, DeBuhr J, et al: ACC/AHA 2008 statement on performance measurement and reperfusion therapy: a report of the ACC/AHA task force on performance measures (Work Group to Address the Challenges of Performance Measurement and Reperfusion Therapy). Circulation. 2008, 118 (24): 2649-2661.

    Article  Google Scholar 

  20. Mills AM, Shofer FS, Chen EH, Hollander JE, Pines JM, Mills AM, et al: The association between emergency department crowding and analgesia administration in acute abdominal pain patients. Acad Emerg Med. 2009, 16 (7): 603-608. 10.1111/j.1553-2712.2009.00441.x.

    Article  PubMed  Google Scholar 

  21. Schull MJ, Vermeulen M, Slaughter G, Morrison L, Daly P, Schull MJ, et al: Emergency department crowding and thrombolysis delays in acute myocardial infarction.[Erratum appears in Ann Emerg Med. 2005 Jan;45(1):84]. Ann Emerg Med. 2004, 44 (6): 577-585. 10.1016/j.annemergmed.2004.05.004. [Research Support, Non-U.S. Gov't]

    Article  PubMed  Google Scholar 

  22. Mitchell R, Kelly AM, Kerr D, Mitchell R, Kelly A-M, Kerr D: Does emergency department workload adversely influence timely analgesia?. Emergency Medicine Australasia. [Comparative Study]. 2009, 21 (1): 52-58.

    Article  Google Scholar 

  23. Chu K, Brown A: Association between access block and time to parenteral opioid analgesia in renal colic: A pilot study. 2009, Journal [serial on the Internet]. Date; (1): Available from: http://dx.doi.org/10.1111/j.1742-6723.2008.01146.x

    Google Scholar 

  24. Pines J, Hollander J, Localio R, Metlay J: The association between emergency department crowding and hospital performance on antibiotic timing for pneumonia and percutaneous intervention for myocardial infarction. Acad Emerg Med. 2006, 13 (8): 873-10.1111/j.1553-2712.2006.tb01741.x.

    Article  PubMed  Google Scholar 

  25. Jones P, Schimanski K: The four hour target to reduce emergency department 'waiting time': a systematic review of clinical outcomes. Emerg Med Australas. 2010, 22 (5): 391-398. 10.1111/j.1742-6723.2010.01330.x.

    Article  PubMed  Google Scholar 

  26. Fullerton R, Wempe W: Lean manufacturing, non-financial performance measures, and financial performance. International Journal of Operations & Production Management. 2009, 29 (3): 214-10.1108/01443570910938970.

    Article  Google Scholar 

  27. Carlos FG, Mahmoud MY, João VL: The dimensionality and utilization of performance measures in a manufacturing operational context. Cross Cultural Management. 2007, 14 (4): 286-10.1108/13527600710830322.

    Article  Google Scholar 

  28. Ittner C, Larcker D, Meyer M: Subjectivity and the weighting of performance measures: evidence from a balanced scorecard. Account Rev. 2003, 78 (3): 725-10.2308/accr.2003.78.3.725.

    Article  Google Scholar 

  29. Boyne GA, Chen AA: Performance targets and public service improvement. Journal of Public Administration Research and Theory. 2007, 17 (3): 455-477.

    Article  Google Scholar 

  30. Degeling P, Maxwell S, Kennedy J, Coyle B: Medicine, management, and modernisation: a "danse macabre"?. Br Med J. 2003, 326 (7390): 649-10.1136/bmj.326.7390.649.

    Article  Google Scholar 

  31. Glouberman S, Mintzberg S: Managing the care of health and the cure of disease- part I: differentiation. Health Care Manage Rev. 2001, 26 (1): 56.

    Article  CAS  PubMed  Google Scholar 

  32. Mason S, Locer T, Careter A, Walters S, Stride C, Casson J: What are the organisational factors that influence waiting times in Emergency Departments. 2006, Sheffield: National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO), Contract No.: Document Number

    Google Scholar 

  33. Wanless D, Appleby J, Harrison ADP: Our Future Health Secured? A review of NHS funding and performance. 2007, London: Kings Fund

    Google Scholar 

  34. Robinson P: Data briefing. Four-hour target fuels admissions. Health Serv J. 2007, 117 (6078): 23.

    PubMed  Google Scholar 

  35. Robson B, Harris RA: Hauora: Māori Standards of Health IV. A Study of the years 2000-2005. 2007, Wellington, [updated 2007; cited 12/12/11]; Available from: http://www.hauora.maori.nz/hauora/#download

    Google Scholar 

  36. Blakely T, McLeod M: Will the financial crisis get under the skin and affect our health? Learning from the past to predict the future. N Z Med J [Review]. 2009, 122 (1307): 76-83.

    Google Scholar 

  37. MOH: A Portrait of Health: Key Results of the 2006/07 New Zealand Health Survey. 2008, Wellington, [updated 2008; cited 12/12/11]; Available from: http://www.health.govt.nz/publications/portrait-health-key-results-2006-07-new-zealand-health-survey

    Google Scholar 

  38. Smith L: Decolonizing methodologies: research and indigenous peoples. 1999, Dunedin: University of Otago Press

    Google Scholar 

  39. Rangahau: Principles of Kaupapa Māori. 2011, Wellington, [updated 2011; cited 2011 12th December]; Available from: http://www.rangahau.co.nz/research-idea/27/

    Google Scholar 

  40. Te Ropu Rangahau Hauora a Eru Pomare: Mana Whakamarama--equal explanatory power: maori and non Maori sample size in national health survey. 2002, Wellington: Ministry of Health and University of Otago

    Google Scholar 

  41. Valencia RR: The evolution of deficit thinking: educational thought and practice. 1997, Washington DC: The Palmer Press

    Google Scholar 

  42. Te Kupenga Hauora Māori. Tömaiora research Protocols 2011. [updated 2011; cited 12/12/11]; Available from: http://www.fmhs.auckland.ac.nz/faculty/tomaiora/default.aspx

  43. Lane DC, Husemann E: System dynamics mapping of acute patient flows. J Oper Res Soc. 2008, 52 (2): 213-224.

    Article  Google Scholar 

  44. Lane DC, Monefeldt C, Rosenhead JV: Looking in the wrong place for healthcare improvements: a system dynamics study of an accident and emergency department. Journal of the Operational Research Society. 2000, 51 (5): 518-531.

    Article  Google Scholar 

  45. Sterman J: Business dynamics: system thinking and modeling for a complex world. 2000, Boston: McGraw-Hill

    Google Scholar 

  46. McGarvey B, Hannon B: Dynamic modeling for business management. 2004, New York: Springer

    Book  Google Scholar 

  47. Krochmal P, Riley TA: Increased health care costs associated with ED overcrowding. Am J Emerg Med. 1994, 12 (3): 265-266. 10.1016/0735-6757(94)90135-X.

    Article  CAS  PubMed  Google Scholar 

  48. Freeman J, Croft S, Cross S, Yap C, Mason S: The impact of the 4 h target on patient care and outcomes in the Emergency Department: an analysis of hospital incidence data. Emerg Med J. 2010, 27 (12): 921-927. 10.1136/emj.2009.085431.

    Article  PubMed  Google Scholar 

  49. Policy on Quality Management in Emergency Medicine. 2002, [College Policy]: Australasian College for Emergency Medicine, [updated 2002 24/08/09; cited]; Available from: http://www.acem.org.au/media/policies_and_guidelines/quality.pdf

  50. Richardson DB: Increase in patient mortality at 10 days associated with emergency department overcrowding. Medical Journal of Australia. 2006, 184 (5): 213.

    PubMed  Google Scholar 

  51. Wood H, Fontain P, Harvey D, Catford G, Hills A, Holland B: 2009, Investigation into Mid Staffordshire NHS Foundation Trust: Healthcare Commission, Contract No.: Document Number

  52. ACEM/ACEN/CENA Draft Joint Policy Emergency Department Pain Management DP58 [College Policy]: 2008, Australasian College for Emergency Medicine, [updated 2008 24/08/09; cited]; Available from: https://www.acem.org.au/media/member_docs/DP58_NICS_joint_FACEM_CENA_ACEN_ED_Pain_Policy.pdf

  53. British Orthopaedic Association Standards for Trauma: Hip Fracture in the Older Person. 2007, [updated 2007; cited 7/10/09]; Available from: http://www.boa.ac.uk/Publications/Pages/BOASTs.aspx

    Google Scholar 

  54. Pines JM, Hollander JE, Localio AR, Metlay JP: The association between emergency department crowding and hospital performance on antibiotic timing for pneumonia and percutaneous intervention for myocardial infarction. Acad Emerg Med. 2006, 13 (8): 873-878. 10.1111/j.1553-2712.2006.tb01741.x.

    Article  PubMed  Google Scholar 

  55. Guttmann A, Razzaq A, Lindsay P, Zagorski B, Anderson GM: Development of measures of the quality of emergency department care for children using a structured panel process. Pediatrics. [Research Support, Non-U.S. Gov't]. 2006, 118 (1): 114-123.

    Google Scholar 

  56. Arnaud A, Vervloet D, Dugue P, Orehek J, Charpin J: Treatment of acute asthma: effect of intravenous corticosteroids and beta 2-adrenergic agonists. Lung. 1979, 156 (1): 43-48. 10.1007/BF02713990.

    Article  CAS  PubMed  Google Scholar 

  57. Willis C, Gabbe B, Cameron P: Measuring quality in trauma care. Injury. 2007, 38 (5): 527-537. 10.1016/j.injury.2006.06.018.

    Article  PubMed  Google Scholar 

  58. Stelfox HT, Straus SE, Nathens A, Bobranska-Artiuch B: Evidence for quality indicators to evaluate adult trauma care: a systematic review. Crit Care Med [Research Support, Non-U.S. Gov't Review]. 2011, 39 (4): 846-859.

    Google Scholar 

  59. Omundsen M, Dennett E: Delay to appendicectomy and associated morbidity: a retrospective review. ANZ J Surg. 2006, 76 (3): 153-155. 10.1111/j.1445-2197.2006.03673.x.

    Article  PubMed  Google Scholar 

  60. Ditillo MF, Dziura JD, Rabinovici R: Is it safe to delay appendectomy in adults with acute appendicitis?. Ann Surg. 2006, 244 (5): 656-660. 10.1097/01.sla.0000231726.53487.dd.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW: Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007, 297 (8): 831-841. 10.1001/jama.297.8.831.

    Article  CAS  PubMed  Google Scholar 

  62. Jansen JO, Grant IC: Communication with general practitioners after accident and emergency attendance: computer generated letters are often deficient. Emerg Med J. 2003, 20 (3): 256-257. 10.1136/emj.20.3.256.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Solutions to emergency department (ED) overcrowding: a literature review. 2009, Wellington: Ministry of Health

  64. Forero R, Hillman K, McCarthy S, et al: Access block and Emergency Department Overcrowding. Emerg Med Australas. 2010, 22: 119-135. 10.1111/j.1742-6723.2010.01270.x.

    Article  PubMed  Google Scholar 

  65. Richardson SK, Ardagh M, Gee P: Emergency department overcrowding: the Emergency Department Cardiac Analogy Model (EDCAM). Accid Emerg Nurs. 2005, 13 (1): 18-23. 10.1016/j.aaen.2004.10.010.

    Article  PubMed  Google Scholar 

  66. Malcolm L: Trends in hospital bed utilisation in New Zealand 1989 to 2006: more or less beds in the future?. N Z Med J. 2007, 120 (1264): 16-27.

    Google Scholar 

  67. Barton P: A kind of ritual Pakeha Tikanga-Maori Experiences of Hospitalisation. 2008, Albany: Massey University

    Google Scholar 

  68. Davis P, Lay-Yee R, Dyall L, Briant R, et al: Quality of hospital care for Maori patients in New Zealand: retrospective cross-sectional assessment. Lancet. 2006, 367 (9526): 1920-10.1016/S0140-6736(06)68847-8.

    Article  PubMed  Google Scholar 

  69. Locker TE, Mason SM: Are these emergency department performance data real?. Emerg Med J. 2006, 23 (7): 558-559. 10.1136/emj.2005.032748.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Locker TE, Mason SM: Analysis of the distribution of time that patients spend in emergency departments. Br Med J. 2005, 330 (7501): 1188-10.1136/bmj.38440.588449.AE.

    Article  Google Scholar 

  71. Wolff AM, Bourke J: Detecting and reducing adverse events in an Australian rural base hospital emergency department using medical record screening and review. Emerg Med J. 2002, 19 (1): 35-40. 10.1136/emj.19.1.35.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Kuan W, Mahdevan M: Emergency Unscheduled Returns: Can we do better?. Singapore Med J. 2009, 50 (11): 1068-1071.

    CAS  PubMed  Google Scholar 

  73. Ministry of Health, Ethnicity Data Protocols for the Health and Disability Sector: 2004, Wellington, [updated 2004; cited 2011 12th December]; Available from: http://www.health.govt.nz/publication/ethnicity-data-protocols-health-and-disability-sector

  74. Ellis C, Devlin G, Matsis P, Elliott J, Williams M, Gamble G: Acute Coronary Syndrome patients in New Zealand receive less invasive management when admitted to hospitals without invasive facilities. N Z Med J [Comparative Study Research Support, Non-U.S. Gov't]. 2004, 117 (1197): U954.

    Google Scholar 

  75. Sofaer S: Qualitative Methods: What Are They and Why Use Them?. Health Serv Res. 1999, 34 (5): 1101.

    CAS  PubMed  PubMed Central  Google Scholar 

  76. Mortimore A, Cooper S: The "4-hour target": emergency nurses' views. Emerg Med J. 2007, 24 (6): 402-404. 10.1136/emj.2006.044933.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Campbell SM, McDonald R, Lester H: The experience of pay for performance in English family practice: a qualitative study. Ann Fam Med. 2008, 6 (3): 228(7)

    Google Scholar 

  78. McDonald R, Harrison S, Checkland K, Campbell SM, Roland M: Impact of financial incentives on clinical autonomy and internal motivation in primary care: ethnographic study. BMJ. 2007, 334 (7608): 1357--10.1136/bmj.39238.890810.BE.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Lamont SS: "See and Treat": spreading like wildfire? A qualitative study into factors affecting its introduction and spread. Emerg Med J. 2005, 22 (8): 548-552. 10.1136/emj.2004.016303.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Mannion R, Davies HTO, Marshall MN: Cultural characteristics of "high" and "low" performing hospitals. J Heal Organ Manag. 2005, 19 (6): 431-439. 10.1108/14777260510629689.

    Article  CAS  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The Shorter Stays in Emergency Departments National Research Project is funded by a competitive three year grant of NZ$1,138,171.00 from the Health Research Council of New Zealand (HRC 10-588), which commenced on 1st October 2010. The funding body has no role in the design of the study, data collection, analysis or interpretation, or writing of the manuscript. The authors would like to acknowledge the considerable contributions to the project of by Jayshree Ramesh-Sukha (Project Administrator) and Denish Kumar (Data Manager) without whom this project would not be possible. We would also like to thank James leFevre (Research Fellow) who reviewed and edited the final manuscript prior to submission.

Author information

Authors and Affiliations

  1. Adult Emergency Department, Auckland City Hospital, Park Road, Grafton, Auckland, New Zealand

    Peter Jones & Alana Harper

  2. Department of Epidemiology and Biostatistics, School of Population Health, Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand

    Peter Jones, Susan Wells, Shanthi Ameratunga & Joanna Stewart

  3. Health Systems, School of Population Health, Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand

    Linda Chalmers, Peter Carswell, Toni Ashton & Tim Tenbensel

  4. Te Kupenga Hauora Māori, School of Population Health, Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand

    Elana Curtis & Papaarangi Reid

Authors
  1. Peter Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linda Chalmers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susan Wells
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shanthi Ameratunga
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Carswell
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Toni Ashton
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Elana Curtis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Papaarangi Reid
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Joanna Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Alana Harper
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Tim Tenbensel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Jones.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

PC, PJ, AH and TA developed Stream One of the research. PJ, SW, SA, JS and AH developed Stream Two, PJ and AH reviewed the literature to determine initial candidate markers of care, AH drafted the data dictionary. JS conducted the sample size calculation and will undertake statistical analysis. LC, TT and PC designed Stream Three. PR and EC were included at all stages of research design for all streams. TT has overall responsibility for Stream Four. PJ and AH drafted this manuscript, which was reviewed and revised by the other authors. All authors read and approved the final manuscript.

Electronic supplementary material

12913_2012_1949_MOESM1_ESM.DOC

Additional file 1: Survey of Initiatives Made and Resources Used to Help Meet the Shorter Stays in Emergency Departments Target (Stream One). (DOC 1020 KB)

Additional file 2: Interview Schedule for Qualitative Interviews (Stream Three). (DOC 470 KB)

Additional file 3: Data Dictionary for Quantitative Outcomes (Stream Two). (PDF 3 MB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Jones, P., Chalmers, L., Wells, S. et al. Implementing performance improvement in New Zealand emergency departments: the six hour time target policy national research project protocol. BMC Health Serv Res 12, 45 (2012). https://doi.org/10.1186/1472-6963-12-45

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1472-6963-12-45

Keywords

  • Health Policy
  • Emergency Medicine
  • Health Services Research
  • Quality of Health Care
  • Models
  • Theoretical
  • Inequalities

BMC Health Services Research

ISSN: 1472-6963

Contact us