- Study protocol
- Open Access
- Open Peer Review
A multistage mixed methods study protocol to evaluate the implementation and impact of a reconfiguration of acute medicine in Ireland’s hospitals
BMC Health Services Research volume 19, Article number: 766 (2019)
To address deficits in the delivery of acute services in Ireland, the National Acute Medicine Programme (NAMP) was established in 2010 to optimise the management of acutely ill medical patients in the hospital setting, and to ensure their supported discharge to primary and community-based care. NAMP aims to reduce inappropriate hospital admissions, reduce length of hospital stay and ensure patients receive timely treatment in the most appropriate setting. It does so primarily via the development of Acute Medical Assessment Units (AMAUs) for the rapid assessment and management of medical patients presenting to hospitals, as well as streamlining the care of those admitted for further care. This study will examine the impact of this programme on patient care and identify the factors influencing its implementation and operation.
We will use a multistage mixed methods evaluation with an explanatory sequential design. Firstly, we will develop a logic model to describe the programme’s outcomes, its components and the mechanisms of change by which it expects to achieve these outcomes. Then we will assess implementation by measuring utilisation of the Units and comparing the organisational functions implemented to that recommended by the NAMP model of care. Using comparative case study research, we will identify the factors which have influenced the programme’s implementation and its operation using the Consolidated Framework for Implementation Research to guide data collection and analysis. This will be followed by an estimation of the impact of the programme on reducing overnight emergency admissions for potentially avoidable medical conditions, and reducing length of hospital stay of acute medical patients. Lastly, data from each stage will be integrated to examine how the programme’s outcomes can be explained by the level of implementation.
This formative evaluation will enable us to examine whether the NAMP is improving patient care and importantly draw conclusions on how it is doing so. It will identify the factors that contribute to how well the programme is being implemented in the real-world. Lessons learnt will be instrumental in sustaining this programme as well as planning, implementing, and assessing other transformative programmes, especially in the acute care setting.
Ireland, as with other jurisdictions [1, 2], has seen a significant reduction in its acute beds with a 13% reduction in in-patient beds between 2007 and 2012 , and has a large unmet demand for long term care beds [3, 4]. This situation, along with continued growth in demand for emergency services, is resulting in patients waiting longer in overcrowded Emergency Departments (EDs) [5,6,7,8,9], and often receiving suboptimal care on trolleys and wards which are not fit for purpose [10, 11]. In view of this increased demand and reduced capacity, hospitals are finding innovative ways to make better use of existing bed stock by implementing interventions to reduce avoidable admissions, reduce variations in length of stay and improve the safe discharge of patients . The development of the discipline of Acute Medicine and the introduction of Acute Medical Units (AMUs) is seen as one such approach to manage the rates of increase . An AMU is defined as ‘... a dedicated facility within a hospital that acts as the focus for acute medical care for patients who have presented as medical emergencies to hospital or who have developed an acute medical illness while in hospital’ . These Units are also known in other jurisdictions as Acute Medical Assessment Units (AMAUs), Medical Assessment Units (MAUs), Acute Assessment Units (AAUs), Medical Assessment and Planning Units (MAPUs), and Admission and Planning Units (APUs). While there is wide variation in how these Units are designed and operated, it is recommended that they are co-located on the same floor with other acute and emergency services, and are staffed by acute medicine physicians or specialist consultants with an interest in acute medicine. It is expected that the presence of a senior decision maker expedites the clinical decision making process and improves patient care by facilitating timely review of each patient as they arrive in the Unit . This model of acute care delivery has been adopted in the UK, Australia and New Zealand [15, 16], and more recently the Netherlands . The majority of medical patients presenting to hospitals as emergencies in the UK are now assessed and treated in AMUs, either directly, or after triage in an Emergency Department , and these Units are considered essential for improving the quality of care for patients presenting to hospitals with complex medical conditions [19, 20].
The Irish National Acute Medicine Programme
The National Acute Medicine Programme (NAMP) was introduced in Ireland in 2010 to provide a framework for the delivery of acute medical services and to address deficits in the care of acutely ill medical patients presenting as emergencies to Irish hospitals . Central to the programme was the development of AMUs in all major hospitals, and similar functioning, but smaller AMAUs in smaller hospitals. [A note on terminology used in this study: a fully functioning AMU consists of an AMAU with an associated short stay ward (SSW) for patients whose length of hospital stay is not expected to be greater than 48 h. For consistency, we will refer to Units in the Irish setting as AMAUs, and identify those with an SSW]. As with the UK model, the purpose of these Units is to facilitate the streaming of medical patients either directly from GPs or from ED at triage, into a designated assessment area where they will be rapidly assessed and diagnosed by a senior decision maker (a consultant physician or a registrar/specialist registrar) within a 1 h target and the decision made within a 6 h target to discharge home, admit to an adjacent short stay unit (up to a 48 h stay), or admit to an in-patient ward [21, 22].
To assist with the implementation of this service reconfiguration, the National Acute Medicine Programme ‘categorised’ Irish hospitals into 4 generic hospital models, from the smaller Model 1 community/district hospitals to the largest Model 4 hospitals. The type of AMAU at each hospital was determined by the hospital’s model [see Additional file 1]. The Programme recognised that Units should be designed firstly around function, such as identifying and clarifying their role in the hospital’s acute services and specifying the patient groups to be assessed there, rather than form (e.g., physical layout and structure) and sites were given the flexibility to adapt the Units to suit local needs and resources . This approach has been highlighted in Australia as being of significant importance in the performance of AMUs . In addition to the establishment of these Units, the National Acute Medicine Programme identified four medical patient pathways - from ambulatory care through to care for complex patients requiring longer hospital stays - and recommended specific practice changes in each pathway [24, 25] [see Additional file 2].
While hospitals were not mandated to adopt this new framework for acute medical care, they were actively encouraged to do so. In 2010, when the NAMP model of care was published, there were eight acute public hospitals with an AMAU. Implementation started over the course of 2012 and 2013, with the last Unit opening in 2014. Currently there are 30 hospitals with an AMAU, representing over 88% of acute public hospitals. Seven out of nine model 4 hospitals, 16 of 17 model 3 hospitals and seven of eight model 2 hospitals have an operating AMAU .
Understanding successful implementation of AMUs
Effectiveness of AMUs in improving patient outcomes
There is mixed evidence on the effectiveness of AMUs in improving patient care. Two recent reviews have expanded upon the initial systematic review conducted by Scott et al. in 2009  and conclude that hospital length of stay, in-hospital mortality and 28-day readmission rates are reduced when AMUs are introduced into hospitals. However, the included studies were of moderate quality; the majority presented aggregate results (unadjusted for potential confounders), and relied on historical controls and ignored secular trends [17, 27]. A more recent systematic review by NICE assessed whether admission or assessment through an AMU (compared with direct admission to a general medical ward) increased hospital discharges, improved patient outcomes and hospital resource usage, and found that there is mixed evidence for the benefit of admission through an AMU . With stricter inclusion and exclusion criteria, their review was limited to just three observational studies [29,30,31], which they classed as very low quality. Recognising the continuing growth in the area of Acute Medicine and the fact that over 90% of hospitals in the UK now have an AMU, the NICE committee felt that ongoing assessment of AMUs was crucial, especially in terms of adherence to standards and quality indicators, and called for higher quality research on the impact of AMUs, including measuring improvements in patient flow and reduced length of hospital stay .
Components of AMUs related to better outcomes
The heterogeneity of the AMU models studied in these effectiveness reviews, in terms of Unit organisation, consultant work patterns, ward round frequency, policies on length of stay, and admission criteria, and the fact that most studies have examined a single site, makes it hard to deduce which elements of the AMU are associated with better patient outcomes [17, 27]. The Society of Acute Medicine in the UK called for research to describe what features of an AMU contribute most to improved patients outcomes . In response to this, Reid et al., conducted a second systematic review - this time to examine the evidence base on how best to deliver care in AMUs. They found limited evidence and a significant knowledge gap on the topic. The one component with consistent evidence of improved patient outcomes is the presence of a consultant for a sustained period . This has been associated with a reduction of potentially avoidable admissions to hospital , reductions in mortality and 28-day readmission rates , and reduced length of hospital stay . Hence, consultant presence is deemed a core component of AMUs worldwide [13, 15, 20, 33, 34] and the Royal College of Physicians in the UK have published recommendations on how to provide this consultant cover . Vaughan et al., synthesised the literature on the benefits of a multidisciplinary team (MDT) in the acute medical setting on patient experience and clinical outcomes . They found that there is a consistent, albeit methodologically flawed, body of evidence that supports MDT working in this setting. They highlight that the recent shift toward individualised care plans for patients, and the introduction of care bundles with specific interventions, necessitates a MDT approach to care. Whilst these care bundles and comprehensive care plans have not been extensively studied in the AMU setting, the literature supports the contentions that they are highly adaptable and promote MDT working, while certain components appear highly suitable for transfer into the AMU context .
Determinants of successful implementation of AMUs
There is a significant gap in the acute medicine literature concerning the factors influencing the implementation of these AMUs. To date, there are no published studies which have qualitatively examined the barriers and enablers to the establishment and embedding of these Units. The London Quality Standards programme which aimed to improve the quality of acute and emergency care, set out the minimum quality of care that patients with medical illnesses should expect when admitted to hospital. An evaluation of its implementation identified many barriers and enablers to adherence to standards in acute care , and it is likely that many of these will be of relevance to this study, given the similarity of programme objectives.
As other jurisdictions consider the expansion of AMUs , evaluating the recent, large scale, country-wide, implementation of Units into Irish hospitals provides an excellent opportunity to highlight the factors (contextual and others) which can facilitate or impede the implementation and impact of these Units on patient care.
Approach to evaluation and conceptual frameworks
We will use a mixed methods approach (a multistage evaluation with an embedded explanatory sequential design) to examine whether the programme is achieving its desired outcomes, and how these outcomes are affected by the context within which the programme is operating .
The UK Medical Research Council (MRC) guidance on process evaluation of complex interventions will serve as the overarching framework [39, 40]. This framework recognizes that to inform policy and practice, we need to understand not only whether interventions work but how they were implemented, their causal mechanisms, and how effects differ from one context to another . Programmes are frequently deemed to be ineffective, simply because they have not been implemented as planned [41, 42]. Therefore, evaluating how well a programme has been implemented is essential to understanding and interpreting an impact evaluation.
Damshroder’s ‘Consolidated Framework for Implementation Research’ (CFIR)  and Proctor’s ‘Conceptual Model of Implementation research’  will be used to understand the determinants of implementation and how they influence outcomes. The CFIR provides a comprehensive taxonomy of constructs that are likely to influence the implementation of complex programmes . When using the CFIR in post-implementation evaluation studies, a focus on outcomes is essential and the meaningful use of the CFIR in this regard involves linking CFIR constructs (i.e., the determinants of implementation) to outcomes (both implementation & programme outcomes) [43, 45]. A recent systematic review by Kirk et al., categorising the empirical use of the framework, found a dearth of studies linking determinants of successful implementation to such outcomes . Proctor et al., provide a model for distinguishing between implementation outcomes (e.g., adoption, reach and fidelity) and programme outcomes (e.g., service level outcomes - efficiency, effectiveness; patient level outcomes - satisfaction, quality-of-life) and highlights that a programme will not be effective if it is not implemented well . We will use these frameworks to examine the hypothesized relationships depicted in Fig. 1.
This study aims to evaluate the impact of Ireland’s National Acute Medicine Programme and identify the factors influencing its implementation and operation.
Elicit the programme’s theories, and ‘mechanisms of change’ necessary to achieve the desired outcomes
Assess how the programme has been implemented across hospitals by measuring utilisation of the Units and documenting which organisational functions (i.e., structures, resources and processes) have been put in place to support the programme
Identify the factors (contextual and others) which have influenced the implementation of the programme and its outcomes
Determine whether the programme is achieving its outcomes and measure how well variation across sites can be ‘attributed’ to the level of implementation.
This multistage mixed methods study uses an explanatory sequential design whereby qualitative research will be undertaken to explain quantitative findings (see Table 1 and Fig. 2). In Stage 1, documentary analysis and data from expert interviews with the national programme team will be used to develop the programme’s logic model, specifying the underlying programme theory. In Stage 2, implementation effectiveness will be examined by using routine hospital administrative data to assess utilisation of the Units, and by conducting surveys to assess the organisational functions (i.e., structures, resources and processes) put in place at each site to support the Units. In Stage 3, comparative case study work will be conducted at eight sites to explore in detail the factors that have influenced the programme’s implementation and its ability to achieve desired outcomes. In Stage 4, routine administrative hospital data will be analysed again, this time to examine the impact of the programme in reducing length of stay of medical patients. In Stage 5, data from Stages1 to 4 will be integrated to examine how variation in programme outcomes across sites is explained by the level of implementation, and components implemented.
Stage 1: Theory conceptualization
As suggested by the UK Medical Research Council guidance on process evaluation, unearthing the programme’s theory and depicting same in a logic model is a crucial first step in evaluating a complex intervention . A logic model can be used to present both process and impact theory and is a replica of what the programme is intended to be which can then be analysed [40, 41]. It can be used for identifying the programme’s functions, activities and outputs to assess fidelity, and to understand how the programme interacts with the organisation’s structures and functions.
Data collection and analysis
Following the guidance of Rossi et al. , a stepwise approach to eliciting the programme theory was taken, and this stage is completed. Rossi advises that to describe the theory embodied in an existing programme’s structure and operation, it is necessary that the evaluator work with stakeholders to draw out the theory that is represented in their actions and assumptions. Therefore, a logic model outlining the programme’s theory was developed by a combination of documentary review, key informant interviews, and in-person meetings with the NAMP team. Documents were reviewed to identify the underlying programme theory, the core components of the programme, the expected outcomes and the mechanisms as to how the programme expects to achieve these . These included the national plan ‘Report of the National Acute Medicine Programme (2010), ‘standards’ and ‘guidance’ for AMUs in other jurisdictions [13, 15, 48] and published literature on their operation  and impact on patient care [17, 27, 28]. Key informant interviews were conducted with the NAMP team - physicians, nurses and allied health professionals with expertise in acute medicine, and programme managers - (past and present NAMP members, n = 6) and an initial group meeting (current members, n = 6) held to understand programme processes and how they could be influenced by the system into which they were introduced. A first draft of the model was developed in the format recommended by the Kellogg Foundation with emphasis not only on the programme’s outcomes and components but the mechanisms by which it expects to achieve these outcomes  and revised through face-to-face discussions with the national team [see Fig. 3].
Stage 2. Assessing programme implementation
Studies in the UK and Australasia have shown considerable variation across hospitals in terms of compliance with recommendations on how care should be delivered in AMUs [50,51,52]. We are interested in examining whether the NAMP model of care has been implemented as designed. In our study, programme implementation will be assessed in terms of (i) ‘service utilisation’ (programme reach), defined by Rossi as ‘...the extent to which the intended targets actually receive programme services, and (ii) the ‘organisational functions implemented’ again, defined by Rossi as ‘whether the programme’s actual activities and arrangements sufficiently approximate the intended ones’ .
Data collection and analysis
Measuring the programme’s organisational functions focuses on how well the programme is organising its efforts and using its resources to accomplish the essential programme tasks . The logic model will be used to develop a survey to collect data on the structure of the Unit (e,g, location, bed capacity, opening hours), resources (e.g., priority access to diagnostics, medical and nursing staff and workforce patterns), processes and procedures (e.g., mode of access to the Unit, referral pathway from ED/GP, patient profiles to be seen, escalation policy, return policy,) and changes in acute care throughout the hospital (e.g., improved patient streaming, integrated discharge planning, use of a common screening tool). Surveys will be completed at each site by the AMAU lead physician or Clinical Nurse Manager (CNM).
Service utilisation (programme reach) will be measured by examining the proportion of ‘emergency medical patients’ streamed through each Unit for 2017 and the case-mix and characteristics of these patients.
Stage 3. Identifying factors that influence programme implementation and outcomes
To identify the factors which have influenced the implementation of the programme and its outcomes, we will conduct comparative case study work, using the approach by Yin which is suited to the complex nature of health services research, and allows for in-depth data gathering on organisational processes and programme impact [53, 54]. The purpose of this stage of the evaluation is to understand the factors that are influencing the level of utilisation of the Units in terms of the proportion of acute medical in-patients that are streamed through the AMAU, but also the factors that are influencing the programme’s ability to achieve its desired outcomes. Experience with the intervention, including participants’ perception of it and its compatibility with the hospital system will be explored in detail during this stage.
Selecting the sites for comparative case study research
Cases will be purposively sampled based on the level of utilisation identified during Stage 2, with four ‘high’ and four ‘low’ implementation sites selected. Sampling cases at either end of the implementation spectrum will allow us identify the factors that contribute to or hinder ‘successful’ implementation. This approach has been taken by Damshroder & Lowery in their study assessing implementation determinants for their propensity to distinguish between sites with high versus low implementation effectiveness .
Data collection at sites
We will conduct semi-structured interviews with health professionals (AMAU lead physician, clinical nurse manager for the AMAU, Assistant Director of Nursing for patient flow) to elicit information on the determinants of implementation [see Additional file 3 for list of CFIR constructs].
Analysing and interpreting the case studies
Data will be managed using NVivo 12. Qualitative data (interviews and documents) will be analysed using the Framework method which comprises five stages (familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation) [55, 56]. The CFIR framework will be applied as pre-defined deductive codes, however open coding will also be used to identify factors that do not fit within the definitions of CFIR constructs . A case memo will be created for each of the eight sites, and constructs rated to reflect the magnitude of their influence on implementation, using the approach recommended by the authors of the CFIR framework . A matrix will then be created listing the ratings for each construct for each site, and cross case comparison made between high and low implementation sites to identify patterns in ratings of the constructs that distinguish between high and low implementation effectiveness—i.e., constructs that were qualitatively correlated with implementation effectiveness .
Stage 4. Evaluating programme impact
Outcomes were identified during the development of the logic model, which involved an examination of how AMUs are evaluated elsewhere. Programme effectiveness will be assessed by examining changes over time in (i) rates of potentially avoidable admissions (ii) lengths of hospital stay of medical patients (iii) lengths of hospital stay of potentially avoidable medical conditions.
Data collection and analysis
Programme outcome measures will be derived from Hospital In-Patient Enquiry (HIPE) which is an administrative database of all public hospital admissions in Ireland, including episodes of care in the AMAU. Programme impact will be estimated by comparing monthly data from 2009 to 2017, using interrupted time series regression (ITS) and ARIMA (autoregressive integrated moving average), accounting for secular and seasonal trends, and using the proportion of patients treated in the Units as a time varying covariate. Models will be run for individual AMAUs and for all hospitals combined. Several sensitivity analyses will be conducted including using length of stay truncated at 30 days, to account for the deficiencies in community services which can skew the average LOS.
Stage 5. Integration of programme outcomes and programme implementation
To explain the variation in implementation and programme outcomes between sites, we will then construct a joint display table presenting the data for each site - the constructs that were identified as influential and the outcomes achieved - and examine patterns and inconsistencies across and between cases [58, 59] In this manner, the constructs which influenced implementation (Stage 3), the level of implementation (Stage 2) and the programme outcomes achieved (Stage 5), will be presented for each case in line with our conceptual framework from Fig. 1.
This protocol outlines a mixed methods study to evaluate whether the reconfiguration of acute medical care in hospitals, is effective in everyday practice. The study will examine the variation in implementation and effectiveness of Acute Medical Units from a national perspective, and be the first to comprehensively assess the factors that contribute to how well these Units are implemented and how well they perform. This work is timely as other jurisdictions consider the wide-scale introduction of Acute Medical Units . It addresses the call for high quality research (including qualitative studies) to describe which features of Acute Medicine contribute most to its success . The work of Reid et al., highlights the lack of research into the active ingredients of the AMU that contribute to its success and the clear gap in knowledge of how best to deliver care in the AMU [18, 27, 52, 60]. By examining variation in service and patient level outcomes in parallel to the organisational functions (e.g., structures, resources and processes of care), this study will assess the association between implementation of the AMU and outcomes achieved. The comparative case study will identify which components and processes contribute to improved outcomes and importantly, help decipher the factors that have influenced the successful establishment and operation of these Units. The results of this study will inform further refinement of the national programme and contribute to the design of more effective AMAUs.
Our study has some limitations. The fragmented health IT infrastructure in Irish hospitals, and the lack of a Unique Health Identifier, means we are unable to examine the trajectory of care received by patients streamed through the Units, and the impact on outcomes such as 30-day mortality, health services utilisation and quality of life. For this reason, we are examining the efficiency, effectiveness and timeliness of care  which are seen as ‘proximal’ outcomes upstream on the pathway to improved health outcomes . We will examine changes in potentially avoidable admissions and lengths of hospital stay; the most common outcomes examined in previous studies evaluating the effectiveness of Acute Medical Units. We are also limited in our ability to examine indicators of performance. For example, we are unable to track - for all hospitals participating in the programme- the patient journey from ED through to the AMAU and the length of time spent on this pathway, which is an important indicator of the timeliness of patient care. Work is underway to address these IT shortcomings with the introduction of an Acute Floor Information System (AFIS), which will facilitate tracking of the patient journey and enhance the collection and reporting of these key performance indicators. A second limitation is the inherent risk of confounding that presents in observational studies of this nature. We have tried to minimise these risks by the use of robust statistical techniques such as interrupted time series analysis  at the individual hospital level, and the triangulation of various data sources to elicit a greater understanding of how the programme is resulting in improved outcomes.
This study has many strengths; most notably its explanatory sequential design which strengthens the validity of our findings on what influences implementation and how implementation leads to better outcomes. According to Creswell, combining statistical trends (quantitative date) with personal experiences (qualitative data), provides a better understanding of the research problem than either form of data alone . Additionally, because complex interventions such as NAMP, tend to be highly context-specific in their effects, generalising the results of effect estimation for policy and practice requires more nuanced analyses of why these effects occur .
Second, programme and implementation theory will be used throughout; from the creation of the logic model which provides a blueprint of the programme to be analysed , to the use of the Consolidated Framework for Implementation Research  to guide data collection, measurement, coding, analysis and reporting of the findings of the comparative site work.
Third, we will use robust statistical methods to evaluate the performance and impact of these Units. Recent reviews of the literature on the effectiveness of these Units have highlighted the shortcomings in the research to date with many studies reporting effect estimates that have not taken into consideration potential biases (such as selection bias), confounding and underlying secular and seasonal trends [17, 27, 28]. We endeavour to minimise the influence of these on the effect estimates by using interrupted time series analysis and applying ARIMA modelling to estimate programme impact, adjusting for autocorrelation and seasonality [64,65,66].
Finally, the access to and collaboration with the national programme is a key strength of the study, as it facilities the co-development of the programme theory from the outset.
We expect the findings of this evaluation to be of interest to a wide audience given the growing need to demonstrate effectiveness of complex interventions. Findings on the mechanisms and contexts that optimise the implementation of this complex multi-faceted intervention will be useful to those developing and implementing other change programmes, especially given the growing realisation that failure to deliver effective services is largely attributable to the lack of knowledge on how best to implement and sustain these changes. This is a formative evaluation and the National team tasked with implementing and overseeing the programme are keenly interested in knowing what the determinants of a successful AMU are and how this information can be used to support sites struggling to reach full potential. Additionally, those in jurisdictions where the discipline of acute medicine is better developed and AMUs are well established, as well those in countries contemplating expansion of their AMUs, will be interested in the challenges that implementers face, and how context ‘interacts’ with and ‘shapes’ the programme being implemented.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author.
Acute Medical Assessment Unit
Acute Medical Unit
Consolidated Framework for Implementation Research
Health Service Executive
National Acute Medicine Programmme
Short Stay Ward
Ewbank, L., Thompson, J., McKenna, H., NHS hospital bed numbers: past, present, future., The King’s Fund, Editor. 2017, The King's Fund: https://www.kingsfund.org.uk/publications/nhs-hospital-bed-numbers.
OECD, editor. Health care activities: Hospital beds, in Health at a glance, OECD, Editor: OECD; 2015. http://www.oecd-ilibrary.org/sites/health_glance-2015-en/06/03/index.html?itemId=/content/chapter/health_glance-2015-32-en&mimeType=text/html. Accessed 12 Dec 2018.
Wren M-A, Keegan C, Walsh B, Bergin A, Eighan J, et al. Projections of demand for healthcare in Ireland, 2015–2030. First report from the hippocrates model, The Economic and Social Research Institute (ESRI), Editor. Dublin: ESRI; 2017.
Smyth B, et al. Planning for Health: Trends and Priorities to Inform Health Service Planning 2017. Report from the Health Service Executive. Dublin: Health Service Executive; 2017.
Pallin DJ, et al. Population aging and emergency departments: visits will not increase, Lengths-Of-Stay And Hospitalizations Will. Health Affairs. 2013;32(7):1306–12.
Ham C. Emergency department pressures need to be tackled through integrated urgent and emergency care. BMJ. 2015;350:h322.
O'Cathain A, et al. A system-wide approach to explaining variation in potentially avoidable emergency admissions: national ecological study. BMJ Qual Saf. 2014;23(1):47–55.
Pinkney J, et al. How can frontline expertise and new models of care best contribute to safely reducing avoidable acute admissions? A mixed-methods study of four acute hospitals. Health Serv Deliv Res. 2016;4(3).
Karakusevic S. Understanding patient flow in hospitals, Nuffield Trust, Editor. Nuffield Trust: London; 2016.
Carter EJ, Pouch SM, Larson EL. The relationship between emergency department crowding and patient outcomes: a systematic review. J Nurs Scholarsh. 2014;46(2):106–15.
Hoot NR, Aronsky D. Systematic Review of Emergency Department Crowding: Causes, Effects, and Solutions. Ann Emerg Med. 2008;52(2):126–136.e1.
Mason S, et al. Innovations to reduce demand and crowding in emergency care; a review study. Scand J Trauma Resusc Emerg Med. 2014;22:55.
Royal College of Physicians, Acute medical care. The right person, in the right setting - first time. Report of the Acute Medicine Task Force. London: Royal College of Physicians; 2007.
McNeill G, et al. What is the effect of a consultant presence in an acute medical unit? Clin Med (Lond). 2009;9(3):214–8.
ACI Acute Care Taskforce Medical Assessment Unit Working Group. NSW Medical Assessment Unit Model of care. Sydney: NSW Agency for Clinical Innovation; 2014.
Internal Medicine Society of Australia and New Zealand, Standards for Medical Assessment and Planning Units in Public and Private Hospitals. 2006.
van Galen LS, et al. Acute medical units: the way to go? A literature review. Eur J Intern Med. 2017;39:24–31.
Reid LEM, et al. How is it best to deliver care in acute medical units? A systematic review. QJM. 2017. https://doi.org/10.1093/qjmed/hcx161.
Schipper EM. Acute medical units, more capacity without increasing resources. Eur J Intern Med. 2017;39:e13.
Holland M, et al. Acute medical units, definitely the way to go. Eur J Intern Med. 2017;39:e10–1.
Royal College of Physicians of Ireland, et al. Report of the National Acute Medicine Programme. Dublin: National Acute Medicine Programme; 2010.
O'Reilly O, et al. National Acute Medicine Programme--improving the care of all medical patients in Ireland. J Hosp Med. 2015;10(12):794–8.
Jenkins P, Barton L, McNeil G. Contrasts in acute medicine: a comparison of the British and Australian systems for managing emergency medical patients. MJA. 2010;193(4):227–8.
Royal College of Physicians of Ireland and Health Service Executive, National Acute Medicine Programme. 2012.
Royal College of Physicians of Ireland and Health Service Executive, National Acute Medicine Programme. Key elements of the programme to deliver the best patient outcomes. 2012.
Scott I, Vaughan L, Bell D. Effectiveness of acute medical units in hospitals: a systematic review. Int J Qual Health Care. 2009;21(6):397–407.
Reid LE, et al. The effectiveness and variation of acute medical units: a systematic review. Int J Qual Health Care. 2016;28(4):433–46.
National Institute for Health and Care Excellence. Chapter 24 Assessment through acute medical units. In: Emergency and acute medical care in over 16s: service delivery and organisation. DRAFT NICE guideline for consultation; 2017.
Li JY, et al. Outcomes of establishing an acute assessment unit in the general medical service of a tertiary teaching hospital. Med J Aust. 2010;192(7):384–7.
Rooney T, et al. Impact of an acute medical admission unit on hospital mortality: a 5-year prospective study. Qjm. 2008;101(6):457–65.
Coary R, et al. Does admission via an acute medical unit influence hospital mortality? 12 years’ experience in a large Dublin hospital. Acute Med. 2014;13(4):152–8.
Bell D, et al. Consultant input in acute medical admissions and patient outcomes in hospitals in England: a multivariate analysis. PLoS One. 2013;8(4):e61476.
McGovern E. Acute medical assessment units: a literature review. Health Services Executive: Dublin; 2013.
Royal College of Physicians: Designing Services. Acute internal medicine services: Acute medical unit. 2017; Available from: http://www.rcpmedicalcare.org.uk/designing-services/specialties/acute-internal-medicine/services-delivered/acute-medical-unit/. Cited 2017 12/7/2017.
Royal College of Physicians, Delivering a 12-hour, 7-day consultant presence on the acute medical unit. Appendix 1: Example calculation of numbers of patient contacts per day., in Acute care toolkit 4. 2012.
Vaughan L, Mitchell A. Multi-disciplinary teams for better patient experience and clinical outcomes: the context of providing a multi-professional approach, in RCPE UK Consensus Conference on Acute medicine. London: Journal of the Royal College of Physicians of Edinburgh; 2013. p. 23–9.
Vaughan LM, Machaqueiro S, Gaskins M, Imison C. The London Quality Standards: A case study in changing clinical care. London: Research report, N. Trust, Editor; 2017.
Bamberger M, Rao V, Woolcock M. In: Tashakkori A, Teddlie C, editors. Editors Using mixed-methods in monitoring and evaluation: Experiences from international development, in Sage Handbook of Mixed- Methods in Social and Behavioral Research. Thousand Oaks: Sage Publications Inc; 2010. p. 613–42.
Moore GF, et al. Process evaluation of complex interventions. UK Medical Research Council (MRC) guidance, MRC population health science research network, editor. London: UK Medical Research Council; 2015.
Moore GF, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.
Rossi P, Lipsey M, Freeman H. Evaluation: a systematic approach. 7th ed. ed. Thousand Oaks: SAGE Publications, Inc; 2004.
Dobson D, Cook TJ. Avoiding type III error in program evaluation: results from a field experiment. Eval Porgram Plan. 1980;3(4):269–76.
Damschroder LJ, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
Proctor E, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76.
Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013;8:51.
Kirk MA, et al. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. 2016;11(1):72.
McHugh S, et al. Evaluating the implementation of a national clinical programme for diabetes to standardise and improve services: a realist evaluation protocol. Implement Sci. 2016;11(1):107.
NHS West Midlands Quality Review Service, Quality Standards for Acute Medical Units (AMUs). Version 2. 2012.
W.K Kellogg Foundation. Logic Model Development Guide: Using Logic Models to Bring Together Planning, Evaluation, and Action. Michigan: W. K Kellogg Foundation; 2004.
Ward D, et al. Acute medical care. The right person, in the right setting--first time: how does practice match the report recommendations? Clin Med (Lond). 2009;9(6):553–6.
McNeill GBS, et al. Optimizing care for acute medical patients: the Australasian medical assessment unit survey. Intern Med J. 2011;41(1a):19–26.
Reid L, Rae R. Structures in AMUs - Qualitative and Quantitative, in Edinburgh International Conference of Medicine. Past, Present & Future. Edinburgh: Health Service Executive; 2016.
Yin RK. Enhancing the quality of case studies in health services research. Health Serv Res. 1999;34(5 Pt 2):1209–24.
Fulop N, et al. Implementing changes to hospital services: factors influencing the process and ‘results’ of reconfiguration. Health Policy. 2012;104:128–35.
Gale NK, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117.
Crowe S, et al. The case study approach. BMC Med Res Methodol. 2011;11(1):100.
Desveaux L, et al. Improving the appropriateness of antipsychotic prescribing in nursing homes: a mixed-methods process evaluation of an academic detailing intervention. Implement Sci. 2017;12(1):71.
O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341:c4587.
Creswell JW. A concise introduction to mixed methods research; 2015.
Reid L, Lone NI, Morrison ZJ, Weir CJ, Jones MC. An Examination of Acute Medical Units in Scottish Hospitals, in IMSANZ 17. Wellngton: Health Service Executive; 2017.
Glasgow RE, Brownson RC, Kessler RE. Thinking about health-related outcomes: what do we need evidence about? Clin Transl Sci. 2013;6(4):286–91.
Wagner A, et al. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27:299–309.
Gillies C, Freemantle N, Grieve R, Sekhon J, Forder J. Advancing quantitative methods for the evaluation of complex interventions. In Raine R, Fitzpatrick R, Barratt H, Bevan G, black N, Boaden R, et al. challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016;4(16):37–54.
Gilmour S, et al. Using intervention time series analyses to assess the effects of imperfectly identifiable natural events: a general method and example. BMC Med Res Methodol. 2006;6:16.
Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348–55.
Nelson BK. Time series analysis using autoregressive integrated moving average (ARIMA) models. Acad Emerg Med. 1998;5(7):739–44.
We would like to acknowledge the valuable assistance of the NAMP team in drawing up the logic model and their patience in answering the many questions the primary researcher had in relation to understanding the programme’s theories, its mechanisms of change, and the programme’s components, as well as facilitating access to the sites. The team however, have not had input into the design or conduct of the evaluation. Additionally, we are very grateful to the Health Intelligence Unit of the HSE who developed the NQAIS Clinical tool, and were instrumental in the identification of measures to assess programme impact.
The Irish Health Services Executive (HSE), commissioned this evaluation which was peer reviewed and awarded funding (Project ref. 205377/14065). However, they had no role in the study design, data collection, analysis, interpretation of results or writing the manuscript.
Ethics approval and consent to participate
Ethical approval was granted by the Health Policy and Management and Centre for Global Health (HPM/CGH) Research Ethics Committee, Trinity College Dublin, Ireland in June 2017 (25/2017/01). Each participant in the study will be asked for written informed consent, and their data will be anonymised for analysis and reporting. Anonymity will be assured at each case study site and all participants will be given a unique ID number. Any potentially identifiable information will be removed prior to reporting and publishing the findings.
Consent for publication
EH, the primary author, receives an educational grant from the HSE. However, this agency does not have input into the design of the study or the analysis, and interpretation of data, or the writing of this manuscript. SMH, JB, LV and CN have no competing interests to declare.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Hurley, E., McHugh, S., Browne, J. et al. A multistage mixed methods study protocol to evaluate the implementation and impact of a reconfiguration of acute medicine in Ireland’s hospitals. BMC Health Serv Res 19, 766 (2019) doi:10.1186/s12913-019-4629-5
- Acute medicine
- Acute medical unit
- System reconfiguration
- Programme implementation
- Mixed methods