Mathematical modelling for health systems research: a systematic review of system dynamics and agent-based models

Abstract

Background

Mathematical modelling has been a vital research tool for exploring complex systems, most recently to aid understanding of health system functioning and optimisation. System dynamics models (SDM) and agent-based models (ABM) are two popular complementary methods, used to simulate macro- and micro-level health system behaviour. This systematic review aims to collate, compare and summarise the application of both methods in this field and to identify common healthcare settings and problems that have been modelled using SDM and ABM.

Methods

We searched MEDLINE, EMBASE, Cochrane Library, MathSciNet, ACM Digital Library, HMIC, Econlit and Global Health databases to identify literature for this review. We described papers meeting the inclusion criteria using descriptive statistics and narrative synthesis, and made comparisons between the identified SDM and ABM literature.

Results

We identified 28 papers using SDM methods and 11 papers using ABM methods, one of which used hybrid SDM-ABM to simulate health system behaviour. The majority of SDM, ABM and hybrid modelling papers simulated health systems based in high income countries. Emergency and acute care, and elderly care and long-term care services were the most frequently simulated health system settings, modelling the impact of health policies and interventions such as those targeting stretched and under resourced healthcare services, patient length of stay in healthcare facilities and undesirable patient outcomes.

Conclusions

Future work should now turn to modelling health systems in low- and middle-income countries to aid our understanding of health system functioning in these settings and allow stakeholders and researchers to assess the impact of policies or interventions before implementation. Hybrid modelling of health systems is still relatively novel but with increasing software developments and a growing demand to account for both complex system feedback and heterogeneous behaviour exhibited by those who access or deliver healthcare, we expect a boost in their use to model health systems.

Introduction

Health systems are complex adaptive systems [1]. As such, they are characterised by extraordinary complexity in relationships among highly heterogeneous groups of stakeholders and the processes they create [2]. Systems phenomena of massive interdependencies, self-organising and emergent behaviour, non-linearity, time lags, feedback loops, path dependence and tipping points make health system behaviour difficult and sometimes impossible to predict or manage [3]. Conventional reductionist approaches using epidemiological and implementation research methods are inadequate for tackling the problems health systems pose [4]. It is increasingly recognised that health systems and policy research need a special set of approaches, methods and tools that derive from systems thinking perspectives [5]. Health systems encompass a many tiered system providing services to local, district and national populations, from community health centres to tertiary hospitals. Attempting to evaluate the performance of such a multi-faceted organisation presents a daunting task. Mathematical modelling, capable of simulating the behaviour of complex systems, is therefore a vital research tool to aid our understanding of health system functioning and optimisation.

System dynamics model (SDM)

System dynamics models (SDM) and agent-based models (ABM) are the two most popular mathematical modelling methods for evaluating complex systems; while SDM are used to study macro-level system behaviour such as the movement of resources or quantities in a system over time, ABM capture micro-level system behaviour, such as human decision-making and heterogeneous interactions between humans.

While use of SDM began in business management [6, 7] it now has wide spread application from engineering to economics, from environmental science to waste and recycling research [8,9,10,11,12,13]. A SDM simulates the movement of entities in a system, using differential equations to model over time changes to system state variables. A stock and flow diagram can be used to provide a visual representation of a SDM, describing the relationships between system variables using stocks, rates and influencing factors. The diagram can be interpreted as mimicking the flow of water in and out of a bath tub [7]; the rates control how much ‘water’ (some quantifiable entity, resource) can leave or enter a ‘bath tub’ (a stock, system variable) which changes over time depending on what constraints or conditions (e.g. environmental or operational) are placed on the system. Often before the formulation of a stock and flow diagram, a causal loop diagram is constructed which can be thought of as a ‘mental model’ of the system [14], representing key dynamic hypotheses.

Agent-based model (ABM)

Unlike SDM, ABM is a ground-up representation of a system, simulating the changing states of individual ‘agents’ in a system rather than the broad entities or aggregate behaviour modelled in SDM. Aggregate system behaviour can however be inferred from ABM. Use of ABM to model system behaviour has been trans-disciplinary, with application in economics to ecology, from social sciences to engineering [15,16,17,18,19]. There can be multiple types of agent modelled, each assigned their own characteristics and pattern of behaviour [20, 21]. Agents can learn from their own experiences, make decisions and perform actions based on set rules (e.g. heuristics), informed by their interactions with other agents, their own assigned attributes or based on their interaction with the modelled environment [22]. The interactions between agents can result in three levels of communication between agents; one-to-one communication between agents, one-to-many communication between agents and one-to-location communication where an agent can influence other agents contained in a particular location [22].

Why use SDM and ABM to model health systems?

ABM and SDM, with their ability to simulate micro- and macro-level behaviour, are complementary instruments for examining the mechanisms in complex systems and are being recognised as crucial tools for exploratory analysis. Their use in mapping health systems, for example, has steadily risen over the last three decades. ABM is well-suited to explore systems with dynamic patient or health worker activity, a limitation of other differential equation or event-based simulation tools [23,24,25]. Unlike discrete-event simulation (DES) for example, which simulates a queue of events and agent attributes over time [26], the agents modelled in ABM are decision makers rather than passive individuals. Closer to the true system modelled, ABM can also incorporate ongoing learning from events whereby patients can be influenced by their interactions with other patients or health workers and by their own personal experience with the health system [21]. SDM has also been identified as a useful tool for simulating feedback and activity across the care continuum [27,28,29,30] and is highly adept at capturing changes to the system over time [31]. This is not possible with certain ‘snapshot in time’ modelling approaches such as DES [32]. SDM is best implemented where the aim of the simulation is to examine aggregate flows, trends and sub-system behaviour as opposed to intricate individual flows of activity which are more suited to ABM or DES [33].

There are also models that can accommodate two or more types of simulation, known as hybrid models. Hybrid models produce results closer to true system behaviour by drawing on the strengths of one or more modelling methods while reducing the limitations associated with using a single simulation type [27]. The activity captured in such models emulates the individual variability of patients and health professionals while retaining the complex, aggregate behaviour exhibited in health systems.

Health scientists and policy makers alike have recognised the potential of using SDM and ABM to model all aspects of health systems in support of decision making from emergency department (ED) optimisation [34] to policies that support prevention or health promotion [35]. Before implementing or evaluating costly health policy interventions or health service re-structuring in the real world, modelling provides a relatively risk-free and low budget method of examining the likely impact of potential health system policy changes. They allow the simulation of ‘what if’ scenarios to optimise an intervention [36]. They can help identify sensitive parameters in the system that can impede the success of initiatives and point to possible spill-over effects of these initiatives to other departments, health workers or patients. Perhaps most important of all, these modelling methods allow researchers to produce simulations, results and a graphical-user interface in relation to alternative policy options that are communicable to stakeholders in the health system [37], those responsible for implementing system-wide initiatives and changes.

Study aim and objectives

Given the increasing amount of literature in this field, the main aim of the study was to examine and describe the use of SDM and ABM to model health systems. The specific objectives were as follows: (1) Determine the geographical, and healthcare settings in which these methods have been used (2) Identify the purpose of the research, particularly the health policies or interventions tested (3) Evaluate the limitations of these methods and study validation, and (4) Compare the use of SDM and ABM in health system research.

Although microsimulation, DES and Markov models have been widely used in disease health modelling and health economic evaluation, our aim in this study was to review the literature on mathematical methods which are used to model complex dynamic systems, SDM and ABM. These models represent two tenants of modelling: macroscopic (top-level) and microscopic (individual-level) approaches. Although microsimulation and DES are individual-based models like ABM, individuals in ABM are “active agents” i.e. decision-makers rather than “passive agents” which are the norm in microsimulation and DES models. Unlike Markov models which are essentially one-dimensional, unidirectional and linear, SDM are multi-dimensional, nonlinear with feedback mechanisms. We have therefore focussed our review on SDM and ABM because they are better suited to characterise the complexity of health systems. This study reviews the literature on the use of SDM and ABM in modelling health systems, and identifies and compares the key characteristics of both modelling approaches in unwrapping the complexity of health systems. In identifying and summarising this literature, this review will shed light on the types of health system research questions that these methods can be used to explore, and what they add to more traditional methods of health system research. By providing an over overview of how these models can be used within health system research, this paper is also expected to encourage wider use and uptake of these methods by health system researchers and policy makers.

Methods

The review was conducted in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [38].

Search strategy and information sources

The literature on ABM and SDM of health systems has not been confined to a single research discipline, making it necessary to widen the systematic review to capture peer-reviewed articles found in mathematical, computing, medicine and health databases. Accordingly, we searched MEDLINE, EMBASE, Cochrane Library, MathSciNet, ACM Digital Library, HMIC, Econlit and Global Health databases for literature. The search of health system literature was narrowed to identify articles that were concerned with modelling facility-based healthcare, services and related healthcare financing agreements which had been excluded or were not the focus of previous reviews [34, 35, 39,40,41]. The search criteria used for MEDLINE was as follows, with full search terms for each database and search terms used to locate SDM and ABM literature found in Additional file 1:

(health system* OR health care OR healthcare OR health service* OR health polic* OR health facil* OR primary care OR secondary care OR tertiary care OR hospital*).ab,ti. AND (agent-based OR agent based).ab,ti. AND (model*).ab,ti.

In addition, the reference list of papers retained in the final stage of the screening process, and systematic reviews identified in the search, were reviewed for relevant literature.

Data extraction and synthesis

The screening process for the review is given in Fig. 1 (adapted from [38]). All search results were uploaded to Mendeley reference software where duplicate entries were removed. The remaining records were screened using their titles and abstracts, removing entries based on eligibility criteria given in Table 1. Post-abstract review, the full text of remaining articles was screened. Papers retained in final stage of screening were scrutinised, with data imported to Excel based on the following categories; publication date, geographical and healthcare setting modelled, purpose of research in addition to any policies or interventions tested, rationale for modelling method and software platform, validation and limitations of model. The results were synthesised using descriptive statistics and analysis of paper content that were used to answer the objectives.

Fig. 1

a Flow-chart for systematic review of SDMs and b ABMs of health systems (Database research discipline is identified by colour; mathematical and computing (red), medicine (blue) and health (green) databases). Adapted from PRISMA [38]

Table 1 Eligibility criteria for review

The studies were first described by three characteristics: publication date, geographical setting, and what aspect of the health system was modelled and why. These characteristics were chosen for the following reasons. Publication date (Fig. 2) allows us to examine the quantity of SDM and ABM studies over time. Geographical settings (Fig. 2, top) allows us to see which health systems have been studied, as health systems in LMIC are very different from those in developed countries. Studies are classified as modelling health systems in high, upper middle, lower middle and low income countries as classified by The World Bank based on economy, July 2018 [42]. Finally, we examined which aspects of the health system have been modelled and the types of research/policy questions that the models were designed to address, to shed light on the range of potential applications of these models, and also potential gaps in their application to date.

Fig. 2

Number of articles in the final review by year of publication and economic classification

The analysis of paper content was split into three sections; SDM use in health system research (including hybrid SDM-DES), ABM use in health system research (including hybrid ABM-DES) and hybrid SDM-ABM use in health system research. The quality of selected studies will not be presented as our aim was to compare and summarise the application of SDM and ABM in modelling health systems rather than a quality appraisal of studies.

Results

Study selection

The search initially yielded 535 citations for ABM and 996 citations for SDM of facility-based healthcare and services (see Fig. 1). Post-full text screening 11 ABM and 28 SDM papers were retained for analysis, six of which utilised hybrid modelling methods. Three of the hybrid modelling papers integrated SDM with DES [43,44,45], two integrated ABM with DES [24, 46] and one integrated SDM with ABM [47]. A summary table of selected papers is given in Table 2.

Table 2 Summary of studies included at full paper review (SDM) and studies included at full paper review (ABM)

Descriptive statistics

Publication date

The first SDM paper to model health systems was published in 1998 [56] whilst the first publication [66] utilising ABM came almost a decade later (Fig. 2). We found an increasing trend in publications for both modelling approaches, with 90.9% (10/11) and 71.4% (20/28) of all ABM and SDM articles, respectively, having been published in the last decade. The first hybrid modelling article was published in 2010 [43], using SDM and DES to model the impact of an intervention to aid access to social care services for elderly patients in Hampshire, England.

Geographical setting

The proportion of papers that modelled health systems in high, upper middle, lower middle and low income countries is presented in Fig. 2. Eighteen (18/28) papers that employed SDM simulated health systems in high income countries including England [33, 36, 43, 45, 50, 54, 56, 57] and Canada [28, 51, 62]. Four SDM papers simulated upper middle income country health systems, including Turkey [52, 59] and China [64], with a nominal number of papers (5/28) focussing on lower middle or low income countries (West Bank and Gaza [48, 55], Indonesia [37], Afghanistan [30] and Uganda [60]). Almost all ABM papers (9/11) modelled a high income country health system, including the US [20, 23, 25] and Austria [65]. Two (2/11) ABM papers described an upper-middle income based health system (Brazil [22, 67]). All six articles that implemented a hybrid SDM or ABM simulated health systems based in high income countries, including Germany [44] and Poland [47].

Healthcare setting and purpose of research

The healthcare settings modelled in the SDM, ABM and hybrid simulation papers are presented in Fig. 3. Healthcare settings modelled using SDM included systems that were concerned with delivering emergency or acute care (11/28) [28, 31, 36, 45, 47, 50, 56,57,58, 61, 62], elderly or long-term care services (LTC)(12/28) [28, 31, 36, 43,44,45, 49,50,51, 54, 61, 62] and hospital waste management (4/28) [37, 48, 52, 55]. Twenty of the SDM papers selected in this review assessed the impact of health policy or interventions on the modelled system. Common policy targets included finding robust methods to relieve stretched healthcare services, ward occupancy and patient length of stay [28, 31, 36, 43, 49, 50, 54, 58, 62], reducing the time to patient admission [33, 53, 61], targeting undesirable patient health outcomes [47, 58, 60, 63], optimising performance-based incentive health system policies [30, 59] and reducing the total cost of care [33, 54, 61]. The remaining eight papers explored factors leading to undesirable emergency care system behaviour [56, 57], simulating hospital waste management systems and predicting future waste generation [37, 48, 55], estimating future demand for cardiac care [44], exploring the impact of patient admission on health professionals stress level in an integrated care system [45], and variation in physician decision-making [32].

Fig. 3

The health system sector locations modelled in the SDM, ABM and hybrid modelling literature. Long-term care (LTC); Accountable care organisation (ACO); Maternal, newborn and child health (MNCH)

ABM papers modelled systems focussed on delivering emergency or acute care (4/11) [21, 22, 47, 67] and accountable care organisations (ACO) or health insurance reimbursement schemes (3/11) [23, 25, 65]. Nine of the ABM papers assessed the impact of health policy or interventions on the modelled system. Common policy targets included decreasing the time agents spent performing tasks, waiting for a service or residing in parts of the system [20, 22, 24, 67], reducing undesirable patient outcomes [23, 25, 47, 67], reducing the number of patients who left a health facility without being seen by a physician [22, 67] and optimising resource utility (beds and healthcare staff) [46, 66, 67]. The remaining two papers described simulation tools capable of comparing health insurance reimbursement schemes [65] and assessing risk, allocation of resources and identifying weaknesses in emergency care services [21].

Papers that utilised hybrid simulation, combining the strengths of two modelling approaches to capture detailed individual variability, agent-decision making and patient flow, modelled systems focussed on delivering elderly care or LTC services [43,44,45] and emergency or acute care [45, 47]. Four of the hybrid simulation papers assessed the impact of policy or intervention on the modelled system. Policy targets included improving access to social support and care services [43], reducing undesirable patient outcomes [47], decreasing patient waiting time to be seen by a physician [24] and improving patient flow through the system by optimising resource allocation [46]. The remaining two papers used hybrid simulation to estimate the future demand for health care from patients with cardiac disease [44] and model patient flow through an integrated care system to estimate impact of patient admission on health care professionals wellbeing [45].

SDM use in health systems research (including hybrid SDM-DES)

Rationale for using model

Gaining a holistic system perspective to facilitate the investigation of delays and bottlenecks in health facility processes, exploring counter-intuitive behaviour and monitoring inter-connected processes between sub-systems was cited frequently as reasons for using SDM to model health systems [28, 36, 37, 48, 56]. SDM was also described as a useful tool for predicting future health system behaviour and demand for care services, essential for health resource and capacity planning [48, 60]. Configuration of the model was not limited by data availability [28, 52, 64] and could integrate data from various sources when required [51].

SDM was described as a tool for health policy exploration and optimising system interventions [33, 36, 51, 54, 58, 64], useful for establishing clinical and financial ramifications on multiple groups (such as patients and health care providers) [63], identifying policy resistance or unintended system consequences [59, 61] and quantifying the impact of change to the health system before real world implementation [62]. The modelling platform also provided health professionals, stakeholders and decision makers with an accessible visual learning environment that enabled engagement with experts necessary for model conception and validation [48, 50, 55, 57]. The model interface could be utilised by decision makers to develop and test alternative policies in a ‘real-world’ framework that strengthened their understanding of system-wide policy impact [31, 49, 58, 61].

SDM-DES hybrid models enabled retention of deterministic and stochastic system variability and preservation of unique and valuable features of both methods [44], capable of describing the flow of entities through a system and rapid insight without the need for large data collection [43], while simulating individual variability and detailed interactions that influence system behaviour [43]. SDM-DES offered dual model functionality [44] vital for simulating human-centric activity [45], reducing the practical limitations that come with using either SDM or DES to model health systems such as attempting to use SDM to model elements which have non-aggregated values (e.g. patient arrival time) [45] which is better suited for DES.

Healthcare setting

Sixteen papers that utilised SDM modelled systems that were concerned with the delivery of emergency or acute care, or elderly care or LTC services.

Ten of the reviewed papers primarily modelled sectors of the health system that delivered emergency or acute care^1,2. Brailsford et al. [50], Lane et al. [56], Lane et al. [57] and Lattimer et al. [36] simulated the delivery of emergency care in English cities, specifically in Nottingham and London. Brailsford et al. [50] and Lattimer et al. [36] created models that replicated the entire emergency care system for the city of Nottingham, from primary care (i.e. General Practice surgeries) to secondary care (i.e. hospital admissions wards), to aid understanding of how emergency care was delivered and how the system would need to adapt to increasing demand. Lane et al. [56] and Lane et al. [57] modelled the behaviour of an ED in an inner-London teaching hospital, exploring the knock on effects of ED performance to hospital ward occupancy and elective admissions. Esensoy et al. [28] and Wong et al. [62] both modelled emergency care in Canada, Esensoy et al. [28] focussing on six sectors of the Ontario health system that cared for stroke patients while Wong et al. [62] simulated the impact of delayed transfer of General Internal Medicine patients on ED occupancy. Rashwan et al. [31], Walker et al. [61] and Mahmoudian-Dehkordi et al. [58] modelled patient flow through a generic emergency care facility with six possible discharge locations in Ireland, a sub-acute extended care hospital with patient flow from feeder facilities in Australia and an intensive care unit, ED and general wards in a generic facility.

Five of the SDM papers primarily simulated the behaviour of LTC facilities or care services for elderly patients³. Ansah et al. [49] modelled the demand and supply of general LTC services in Singapore with specific focus on the need for LTC and acute health care professionals. Desai et al. [54] developed a SDM that investigated future demand of care services for older people in Hampshire, England which simulated patient flow through adult social care services offering 13 different care packages. In modelling complex care service demand, Cepoiu-Martin et al. [51] explored patient flow within the Alberta continuing care system in Canada which offered supportive living and LTC services for patients with dementia. Brailsford et al. [43] used a hybrid SDM-DES model to investigate how local authorities could improve access to services and support for older people, in particular the long term impact of a new contact centre for patients. The SDM replicated the whole system for long term care, simulating the future demography and demand for care services and the nested DES model simulated the operational issues and staffing of the call centre in anticipation of growing demand for services. Zulkepli et al. [45] also used SDM-DES to model the behaviour of an integrated care system in the UK, modelling patient flow (DES) and intangible variables (SDM) related to health professionals such as motivation and stress levels.

Policy impact evaluation/testing

Twenty papers that utilised SDM tested the impact of policy or interventions on key health system performance or service indicators. The intended target of these policies ranged from relieving strained and under resourced healthcare services, decreasing healthcare costs to reducing patient mortality rates.

Ansah et al. [49], Brailsford et al. [50] and Desai et al. [54] aimed to reduce occupancy in acute or emergency care departments through policies that targeted elderly utilisation of these services. While demand for LTC services is expected to exponentially increase in Singapore, focus has been placed on expanding the acute care sector. Ansah et al. [49] simulated various LTC service expansion policies (static ‘current’ policy, slow adjustment, quick adjustment, proactive adjustment) and identified that proactive expansion of LTC services stemmed the number of acute care visits by elderly patients over time and required only a modest increase in the number of health professionals when compared with other policies. In Brailsford et al. [50] simulation of the entire emergency care system for Nottingham, England, policy testing indicated that while the emergency care system is operating near full capacity, yearly total occupancy of hospital beds could be reduced by re-directing emergency admissions from patients over 60 years of age (who make up around half of all admissions) to more appropriate services, such as those offered by community care facilities. To explore challenges that accompany providing care for an ageing population subject to budget restraints, Desai et al. [54] simulated the delivery and demand for social care services in Hampshire over a projected 5 year period. In offering care packages to only critical need clients and encouraging extra care services at home rather than offering residential care, the number of patients accessing acute care services reduced over the observed period.

Desai et al. [54], in addition to Taylor et al. [33] and Walker et al. [61], also examined policies that could reduce the total cost of care. Increasing the proportion of hired unqualified care workers (over qualified care workers who are employed at a higher cost rate) resulted in savings which could be fed back into care funding, although Desai et al. [54] remarked on the legal and practical limitations to this policy. Taylor et al. [33] examined the impact of shifting cardiac catheterization services from tertiary to secondary level hospitals for low risk investigations and explored how improvements could be made to services. Significant and stable improvements in service, including reduced waiting list and overall cost of service, were achieved with the implementation of strict (appropriate referral) guidelines for admitting patients. Walker et al. [61] modelled patient flow from feeder hospitals to a single sub-acute extended care facility in Victoria, Australia, to assess the impact of local rules used by the medical registrar for admission. The local admission policy which prioritised admissions from patients under the care of private doctors pushed the total cost of care over the facility budget by 6% whereas employing no prioritisation rule reduced the total cost of care to 3% under budget.

Semwanga et al. [60], Mahmoudian-Dehkordi et al. [58] and Worni et al. [63] evaluated the impact of health policy on undesirable patient outcomes (mortality and post-treatment complication rates). Semwanga et al. [60] tested the effectiveness of policies designed to promote maternal and neonatal care in Uganda, established from the literature. Policies that enabled service uptake, such as community health education, free delivery kits and motorcycle coupons were significant in reducing neonatal death over the simulated period. Mahmoudian-Dehkordi et al. [58] explored the intended and unintended consequences of intensive care unit resource and bed management policies on system performance indicators, including patient mortality. During a simulated crisis scenario, prioritising intensive care unit patient admission to general wards over emergency admissions was found to be the most effective policy in reducing total hospital mortality. Worni et al. [63] estimated the impact of a policy to reduce venous thromboembolism rates post-total knee arthroplasty surgery and identified unintentional consequences of the strategy. The policy prevented the reimbursement of patient care fees in the event that a patient was not taking the recommended prophylaxis medication and consequently develops venous thromboembolism. Simulation results indicated a positive 3-fold decrease in venous thromboembolism rates but an unintended 6-fold increase in the number of patients who develop bleeding complications as a result of compulsory prophylaxis treatment.

Validation (including sensitivity analysis)

Statistically-based models are usually used in quantitative data rich environments where model parameters are estimated through maximum likelihood or least-squares estimation methods. Bayesian methods can also be used to compare alternative statistical model structures. SDMs and ABMs on the other hand are not fitted to data observations in the traditional statistical sense. The data are used to inform model development. Both quantitative data and qualitative data (e.g. from interviews) can be used to inform the structure of the model and the parameters of the model. Furthermore, model structure and parameter values can also be elicited from expert opinion. This means that the nature of validation of ABMs and SDMs requires more scrutiny than that of other types of models.

With increasing complexity of such models, and to strengthen confidence in their use particularly for decision support, models are often subjected to sensitivity analysis and validation tests. Twenty-two papers that utilised SDM undertook model validation, the majority having performed behavioural validity tests (see Additional file 2 for details of validation methods for each model). Key model output such as bed occupancy [36, 50], department length of stay [62] and number of department discharges [31] were compared with real system performance data from hospitals [32, 33, 36, 48, 50, 54, 58, 59, 61, 62], local councils [54], nationally reported figs [31, 64]. as well being reviewed by experts [57, 60] as realistic. Others performed more structure orientated validity tests. Model conception [28, 60], development [30, 36, 50, 53, 54, 57, 62] and formulation [54, 56, 59] were validated by a variety of experts including health professionals [47, 53, 54, 57, 59, 62], community groups [56] and leaders [60], steering committees [36], hospital and care representatives [50, 56, 59], patient groups [60] and healthcare policy makers [60]. Further tests for structural validity included checking model behaviour when subjected to extreme conditions or extreme values of parameters [30, 31, 52, 57, 59, 60, 64], model dimensional consistency [31, 52, 57, 59, 60], model boundary adequacy [31] and mass balance [54] and integration error checks [31, 52]. Sensitivity analysis was performed to assess how sensitive model output was to changes in key parameters [49, 51, 57, 60, 64], to test the impact of parameters that had been based on expert opinion on model output [28] and varying key system parameters to test the robustness and effectiveness of policies [28, 30, 52, 53, 58] (on the assumption of imperfect policy implementation [28]).

Limitations of research

Most of the model limitations reported were concerned with missing parameters, feedback or inability to simulate all possible future health system innovations. Mielczarek et al. [44], Cepoiu-Martin et al. [51], Ansah et al. [49] and Rashwan et al. [31] did not take into account how future improvements in technology or service delivery may have impacted results, such as the possibility of new treatment improving patient health outcomes [51] and how this could impact the future utilisation of acute care services [49]. Walker et al. [61] and Alonge et al. [30] described how the models may not simulate all possible actions or interactions that occurred in the real system, such as all proactive actions taken by hospital managers to achieve budget targets [61] or all unintended consequences of a policy on the system [30]. De Andrade et al. [53] and Rashwan et al. [31] discussed the reality of model boundaries, that SDMs cannot encapsulate all health sub-sector behaviour and spill-over effects. Although these have been listed here as limitations, not accounting for possible future improvements in healthcare service or not simulating all possible actions in the modelled system did not prevent authors from fulfilling study objectives. When developing a SDM, it is not possible to account for all possible spill-over effects to other healthcare departments and this should not be attempted; model boundaries are set to only include variables and feedback that are pertinent to exploring the defined problem.

Simplification of model parameters was another common limitation. Wong et al. [62] stated that this would result in some model behaviour not holding in the real system, such as using weekly hospital admission and discharge averages in place of hourly rates due to the hospital recording aggregated data. This aggregation of model parameters may not have reflected real system complexity; Eleyan et al. [55] did not differentiate between service level and type of hospital when modelling health care waste production (described as future work) and Worni et al. [63] refrained from stratifying post-surgery complications by severity, potentially combining lethal and less harmful complications within the same stock (although this did not detract from the study conclusion that the rate of complications would increase as a result of the tested policy).

Data availability, lack of costing analysis and short time horizons were also considered credible limitations. Models that had been calibrated with real data were at risk of using datasets that contained measurement errors or incomplete datasets lacking information required to inform model structure or feedback [32]. Routine facility data required for model conception and formulation was unavailable which restricted the replication of facility behaviour in the model [36] and restricted validation of model behaviour [59], although it should be noted that this is only one method among many for SDM validation and the author was able to use other sources of data for this purpose. Lack of costing or cost effectiveness analysis when testing policies [60], particularly policies that required significant investment or capacity expansion [58], limited discussion on their feasibility in the real system. Models that simulated events over short time scales did not evaluate long term patient outcomes [33] or the long term effects of facility policies on certain groups of patient [57].

ABM use in health system research (including hybrid ABM-DES)

Rationale for using model

The model’s ability to closely replicate human behaviour that exists in the real system was frequently cited [20,21,22, 25, 66], providing a deeper understanding of multiple agent decision-making [23, 67], agent networks [25] and interactions [21, 22]. The modelling method was described as providing a flexible framework capable of conveying intricate system structures [20], where simulations captured agent capacity for learning and adaptive behaviour [20, 25] and could incorporate stochastic processes that mimicked agent transition between states [25]. ABM took advantage of key individual level agent data [25] and integrated information from various sources including demographic, epidemiological and health service data [65]. The visualisation of systems and interface available with ABM software packages facilitated stakeholder understanding of how tested policies could impact financial and patient health outcomes [23], particularly those experts in the health industry with minimal modelling experience [67].

Integrating DES and ABM within a single model ensured an intelligent and flexible approach for simulating complex systems, such as the outpatient clinic described in Kittipittayakorn et al. [24]. The hybrid model captured both orthopaedic patient flow and agent decision-making that enabled identification of health care bottlenecks and optimum resource allocation [24].

Healthcare setting

Seven papers that utilised ABM modelled systems that were either concerned with delivering emergency or acute care², ACOs or health insurance reimbursement schemes.

Liu et al. [21] and Yousefi et al. [22] modelled behaviour in EDs in Spanish and Brazilian tertiary hospitals. Liu et al. [21] simulated the behaviour of eleven key agents in the ED including patients, admission staff, doctors, triage nurses and auxiliary staff. Patients were admitted to the ED and triaged before tests were requested and a diagnosis issued. Over time, agent states changed based on their interaction with other agents such as when a doctor decided upon a course of action for a patient (sending the patient home, to another ward, or continue with diagnosis and treatment). For further details of agent type and model rules for each paper, see Additional file 3.

Yousefi et al. [22] modelled the activities of patients, doctors, nurses and receptionists in a ED. Agents could communicate with each other, to a group of other agents or could send a message to an area of the ED where other agents reside. They made decisions based on these interactions and the information available to them at the time. The main focus of the simulation was on patients who left the ED without being seen by a physician; patients decided whether to leave the ED based on a ‘tolerance’ time extracted from the literature, which changed based on their interaction with other agents. In an additional paper, Yousefi et al. [67] simulated decision-making by patients, doctors, nurses and lab technicians within a generic ED informed from the literature. Group decision-making was employed, whereby facility staff could interact with each other and reach a common solution for improving the efficacy of the department such as re-allocating staff where needed. Yousefi et al. [67], Yousefi et al. [22] and Liu et al. [21] each used a finite state machine (a computational model which describes an entity that can be in one of a finite number of states) to model interactions between agents and their states.

Liu et al. [25] and Alibrahim et al. [23] modelled the behaviour of patients, health providers and payers using series of conditional probabilities, where health providers had participated in an ACO in the United States. Liu et al. [25] presented a model where health providers within an ACO network worked together to reduce congestive heart failure patient healthcare costs and were consequently rewarded a portion of the savings from the payer agent (hypothetically, the Centers for Medicare and Medicaid Services). Patients were Medicare beneficiaries over the age of 65 who developed diabetes, hypertension and/or congestive heart failure and sought care within the network of health providers formed of three hospitals and 15 primary care physician clinics. Alibrahim et al. [23] adapted Liu et al. [25] ACO network model to allow patients to bypass their nearest medical provider in favour of an alternative provider. The decision for a patient to bypass their nearest health centre was influenced by patient characteristics, provider characteristics and the geographical distance between health providers. Providers were also given a choice on whether to participate in an ACO network, where they would then need to implement a comprehensive congestive heart failure disease management programme.

Einzinger et al. [65] created a tool that could be used to compare different health insurance reimbursement schemes in the Austrian health sector. The ABM utilised anonymous routine data from practically all persons with health insurance in Austria, pertaining to medical services accessed in the outpatient sector. In the simulation, patients developed a chronic medical issue (such as coronary heart disease) that required medical care and led to the patient conducting a search of medical providers through the health market. The patient then accessed care at their chosen provider where the reimbursement system, notified of the event via a generic interface, reimbursed the medical provider for patients care.

Policy impact evaluation/testing

Nine papers tested the impact of policy on key health system performance or service indicators. The intended target of these policies ranged from decreasing patient length of stay, to reducing the number of patients who leave without being seen by a physician to reducing patient mortality and hospitalisation rates.

Huynh et al. [20], Yousefi et al. [22], Yousefi et al. [67] and Kittipittayakorn et al. [24] tested policies to reduce the time agents spent performing tasks, waiting for a service or residing in parts of the system. Huynh et al. [20] modelled the medication administration workflow for registered nurses at an anonymous medical centre in the United States and simulated changes to the workflow to improve medication administration safety. Two policies were tested; establishing a rigid order for tasks to be performed and for registered nurses to perform tasks in the most frequently observed order (observed in a real medical centre) to see if this improved the average amount of time spent on tasks. Yousefi et al. [67] modelled the effects of group decision-making in ED compared with the standard approach for resource allocation (where a single supervisor allocates resources) to assess which policy resulted in improved ED performance. Turning ‘on’ group decision-making and starting the simulation with a higher number of triage staff and receptionists resulted in the largest reduction of average patient length of stay and number of patients who left without being seen. This last performance indicator was the subject of an additional paper [22], with focus on patient-to-patient interactions and how this impacted their decision to leave the ED before being seen by a physician. Four policies adapted from case studies were simulated to reduce the number of patients leaving the ED without being seen and average patient length of stay. The policy of fast-tracking patients who were not acutely unwell during triage performed well as opposed to baseline, where acutely ill patients were always given priority. Kittipittayakorn et al. [24] used ABM-DES to identify optimal scheduling for appointments in an orthopaedic outpatient clinic, with average patient waiting time falling by 32% under the tested policy.

Liu et al. [25], Alibrahim et al. [23] and Yousefi et al. [67] tested the impact of health policy on undesirable patient outcomes (patient mortality and hospitalisation rates). Liu et al. [25] modelled health care providers who operated within an ACO network and outside of the network and compared patient outcomes. Providers who operated within the ACO network worked together to reduce congestive heart failure patient healthcare costs and were then rewarded with a portion of the savings. As part of their membership, providers implemented evidence-based interventions for patients, including comprehensive discharge planning with post-discharge follow-up; this intervention was identified in the literature as key to reducing congestive heart failure patient hospitalisation and mortality, leading to a reduction in patient care fees without compromising the quality of care. The ACO network performed well, with a 10% reduction observed in hospitalisation compared with the standard care network. In another study [23] six scenarios were simulated with combinations of patient bypass capability (turned “on” or “off”) and provider participation in the ACO network (no ACO present, optional participation in ACO or compulsory participation in ACO). Provider participation in the ACO, in agreement with Liu et al. [25], led to reduced mortality and congestive heart failure patient hospitalisation, with patient bypass capability marginally increasing provider ACO participation. Yousefi et al. [67] also modelled the impact of group decision-making in ED on the number of patient deaths and number of wrong discharges i.e. patients sent to the wrong sector for care after triage and are then discharged before receiving correct treatment.

Validation (including sensitivity analysis)

Nine of the 11 papers that utilised ABM undertook model validation, consisting almost exclusively of behavioural validity tests. Model output, such as patient length of stay and mortality rates, was reviewed by health professionals [46, 66] and compared with data extracted from pilot studies [20], health facilities (historical) [22, 24, 46, 65, 66], national health surveys [65] and relevant literature [23, 25]. Papers presented the results of tests to determine the equivalence of variance [20] and difference in mean [20, 24] between model output and real data. Structural validity tests included extreme condition testing [23, 46] and engaging health care experts to ensure the accuracy of model framework [22, 47]. Sensitivity analysis was performed to determine how variations or uncertainty in key parameters (particularly where they had not been derived from historical or care data [65]) affected model outcomes [23, 25].

Limitations of research

The majority of model limitations reported were concerned the use or availability of real system or case data. Huynh et al. [20], Yousefi et al. [67] and Liu et al. [25] formulated their models using data that was obtainable, such as limited sample data extracted from a pilot study [20], national average trends [25] and data from previous studies [67]. Yousefi et al. [22] case study dataset did not contain key system feedback, such as the tolerance time of patients waiting to be seen by a physician in the ED, although authors were able to extract this data from a comparable study identified in the literature.

Missing model feedback or parameters, strict model boundaries and simplification of system elements were also considered limitations. Huynh et al. [20], Hutzschenreuter et al. [66] and Einzinger et al. [65] did not model all the realistic complexities of their system, such as all possible interruptions to tasks that occur in patient care units [20], patient satisfaction of admission processes [66] (which will be addressed in future work), how treatment influences the course of disease or that morbid patients are at higher risk of developing co-morbidity than healthier patients, which would affect the service needs and consumption needs of the patient [65]. To improve the accuracy of the model, Huynh et al. stated that further research is taking place to obtain real, clinical data (as opposed to clinical simulation lab results) to assess the impact of interruptions on workflow. Liu et al.’s [21] model boundary did not include other hospital units that may have been affected by ED behaviour and they identify this as future work, for example to include hospital wards that are affected by ED behaviour. Alibrahim et al. [23] and Einzinger et al. [65] made simplifications to the health providers and networks that were modelled, such as assuming equal geographical distances and identical care services between health providers in observed networks [23], limiting the number of factors that influenced a patients decision to bypass their nearest health provider [65] and not simulating changes to health provider behaviour based on service utilisation or reimbursement scheme in place [23]. Alibrahim et al. [23] noted that although the model was constrained by such assumptions, the focus of future work would be to improve the capability of the model to accurately study the impact of patient choice on economic, health and health provider outcomes.

SDM-ABM use in health system research

A single paper used hybrid SDM-ABM to model health system behaviour. Djanatliev et al. [47] developed a tool that could be used to assess the impact of new health technology on performance indicators such as patient health and projected cost of care. A modelling method that could reproduce detailed, high granularity system elements in addition to abstract, aggregate health system variables was sought and a hybrid SDM-ABM was selected. The tool nested an agent-based human decision-making module (regarding healthcare choices) within a system dynamics environment, simulating macro-level behaviour such as health care financing and population dynamics. A case study was presented to show the potential impact of Mobile Stroke Units (MSU) on patient morbidity in Berlin, where stroke diagnosis and therapy could be initiated quickly as opposed to standard care. The model structure was deemed credible after evaluation by experts, including doctors and health economists.

Comparison of SDM and ABM papers

The similarities and differences among the SDM and ABM body of literature are described in this section and shown in Table 3. A high proportion of papers across both modelling methods simulated systems that were concerned with emergency or acute care. A high number of SDM papers (11/28) simulated patient flow and pathways through emergency care [28, 31, 36, 45, 47, 50, 56,57,58, 61, 62] with a subset evaluating the impact of policies that relieved pressure on at capacity ED’s [28, 36, 50, 58, 62]. ABM papers simulated micro-level behaviour associated with emergency care, such as health professional and patient behaviour in EDs and what impact agent interactions have on actions taken over time [21, 22, 47, 67]. ACOs and health insurance reimbursement schemes, a common modelled healthcare setting among the ABM papers [23, 25, 65] was the focus of a single SDM paper [63] while health care waste management, a popular healthcare setting for SDM application [37, 48, 52, 55] was entirely absent among the selected ABM literature. SDM and ABM were both used to test the impact of policy on undesirable patient outcomes, including patient mortality [23, 25, 58, 60, 67] and hospitalisation rates [23, 25]. Interventions for reducing patient waiting time for services [24, 33, 53, 61, 67] and patient length of stay [22, 31, 67] were also tested using these methods, while policy exploration to reduce the total cost of care was more frequent among SDM studies [33, 54, 61].

Table 3 Comparison of content between SDM, ABM and hybrid models of health systems literature

SDM and ABM software platforms provide accessible, user-friendly visualisations of systems that enable engagement with health experts necessary for model validation [48, 50, 55, 57] and facilitate stakeholder understanding of how alternative policies can impact health system performance under a range conditions [31, 49, 58, 61]. The ability to integrate information and data from various sources was also cited as rationale for using SDM and ABM [51]. Reasons for using SDM to model health systems, as opposed to other methods, included gaining a whole-system perspective crucial for investigating undesirable or counter-intuitive system behaviour across sub-systems [28, 36, 37, 48, 56] and identifying unintended consequences or policy resistance with tested health policies [59, 61]. The ability to replicate human behaviour [20,21,22, 25, 66] and capacity for learning and adaptive behaviour [20, 25] was frequently cited as rationale for using ABM to simulate health systems.

Validation of SDMs and ABMs consisted mostly of behavioural validity tests where model output was reviewed by experts and compared to real system performance data or to relevant literature. Structural validity tests were uncommon among ABM papers while expert consultation on model development [30, 36, 50, 53, 54, 57, 62, 63], extreme condition [30, 31, 52, 57, 59, 60, 64] and dimensional consistency tests [31, 52, 57, 59, 60] were frequently reported in the SDM literature. The inability to simulate all actions or interactions that occur in the real system [20, 30, 61, 65, 66] and simplification of model parameters [23, 55, 62, 63, 65] were described as limitations in both SDM and ABM papers. Data availability for model conception and formulation [20, 22, 25, 32, 36, 67] and the impact of model boundaries (restricting exploration of interconnected sub-system behaviour [21, 31, 53]) were also cited limitations common to both sets of literature. Lack of costing analysis [58, 60], short time horizons [33, 57] and an inability to model future improvements in technology or service delivery [31, 44, 49, 51] were additionally cited among the SDM papers.

Discussion

Statement of principal findings

Our review has confirmed that there is a growing body of research demonstrating the use of SDM and ABM to model health care systems to inform policy in a range of settings. While the application of SDM has been more widespread (with 28 papers identified) there are also a growing number of ABM being used (11), just over half of which used hybrid simulation. A single paper used hybrid SDM-ABM to model health system behaviour. To our knowledge this is the first review to identify and compare the application of both SDM and ABM to model health systems. The first ABM article identified in this review was published almost a decade after the first SDM paper; this reflects to a certain extent the increasing availability of SDM and ABM dedicated software tools with the developments in ABM software lagging behind their SDM modelling counterparts.

Emergency and acute care, and elderly care and LTC services were the most frequently simulated health system setting. Both sets of services are facing exponential increases in demand with constraints on resources, presenting complex issues ideal for evaluation through simulation. Models were used to explore the impact and potential spill over effects of alternative policy options, prior to implementation, on patient outcomes, service use and efficiency under various structural and financial constraints.

Strengths and weaknesses of the study

To ensure key papers were identified, eight databases across four research areas were screened for relevant literature. Unlike other reviews in the field [39, 40], there was no restriction placed on publication date. The framework for this review was built to provide a general overview of the SDM and ABM of healthcare literature, capturing papers excluded in other published reviews as a result of strict inclusion criteria. These include reviews that have focussed specifically on compiling examples of modelled health policy application in the literature [35] or have searched for papers with a particular health system setting, such as those that solely simulate the behaviour of emergency departments [34]. One particularly comprehensive review of the literature had excluded papers that simulated hospital systems, which we have explicitly included as part of our search framework [39].

The papers presented in this review, with selection restricted by search criteria, provide a broad picture of the current health system modelling landscape. The focus of this review was to identify models of facility-based healthcare, purposely excluding literature where the primary focus is on modelling disease progression, disease transmission or physiological disorders which can be found in other reviews such as Chang et al. [39] and Long et al. [41]. The data sources or details of how data was used to conceptualise and formulate models are not presented in this paper; this could on its own be the focus of another study and we hope to publish these results as future work. This information would be useful for researchers who want to gain an understanding of the type and format of data used to model health systems and best practice for developing and validating such models.

Literature that was not reported in English was excluded from the review which may have resulted in a small proportion of relevant papers being missed. Papers that described DES models, the other popular modelling method for simulating health system processes, were not included in this review (unless DES methods are presented as part of a hybrid model integrated with SDM or ABM) but have been compiled elsewhere [68,69,70]. Finally, the quality of the papers was not assessed.

Implications for future research

A nominal number of SDM papers (9/28), an even lower proportion of ABM papers (2/11) and none of the hybrid methods papers simulated health systems based in low- or middle-income countries (LMICs). The lower number of counterpart models in LMICs can be attributed to a lack of capacity in modelling methods and perhaps the perceived scarcity of suitable data; however, the rich quantitative and qualitative primary data collected in these countries for other types of evaluation could be used to develop such models. Building capacity for using these modelling methods in LMICs should be a priority and generating knowledge of how and which secondary data to use in these settings for this purpose. In this review, we observed that it is feasible to use SDM to model low-income country health systems, including those in Uganda [60] and Afghanistan [30]. The need to increase the use of these methods within LMICs is paramount; even in cases where there is an absence of sufficient data, models can be formulated for LMICs and used to inform on key data requirements through sensitivity analysis, considering the resource and healthcare delivery constraints experienced by facilities in these settings. This research is vital for our understanding of health system functioning in LMICs, and given the greater resource constraints, to allow stakeholders and researchers to assess the likely impact of policies or interventions before their costly implementation, and to shed light on optimised programme design.

Health system professionals can learn greatly from using modelling tools, such as ABM, SDM and hybrid models, developed originally in non-health disciplines to understand complex dynamic systems. Understanding the complexity of health systems therefore require collaboration between health scientists and scientists from other disciplines such as engineering, mathematics and computer science. Discussion and application of hybrid models is not a new phenomenon in other fields but their utilisation in exploring health systems is still novel; the earliest article documenting their use in this review was published in 2010 [43]. Five of the six hybrid modelling papers [43,44,45,46,47] were published as conference proceedings (the exception Kittipittayakorn et al. [24]), demonstrating the need to include conference articles in systematic reviews of the literature in order to capture new and evolving applications of modelling for health systems research.

The configuration and extent to which two distinct types of models are combined has been described in the literature [71,72,73,74,75]. The hybrid modelling papers selected in this review follow what is described as ‘hierarchical’ or ‘process environment’ model structures, the former where two distinct models pass information to each other and the latter where one model simulates system processes within the environment of another model [72]. Truly ‘integrated’ models, considered the ‘holy grail’ [43] of hybrid simulation, where elements of the system are simulated by both methods of modelling with no clear distinction, were not identified in this review and in the wider literature remain an elusive target. In a recent review of hybrid modelling in operational research only four papers were identified to have implemented truly integrated hybrid simulation and all used bespoke software, unrestricted by the current hybrid modelling environments [76].

Of the six hybrid modelling papers, only Djanatliev et al. [47] presented a model capable of both ABM and SDM simulation. The crucial macro- and micro- level activity captured in such models represent feedback in the wider, complex system while retaining the variable behaviour exhibited by those who access or deliver healthcare. With increasing software innovation and growing demand for multi-method modelling in not only in healthcare research but in the wider research community, we need to increase their application to modelling health systems and progress towards the ‘holy grail’ of hybrid modelling.

Conclusions

We identified 28 papers using SDM methods and 11 papers using ABM methods to model health system behaviour, six of which implemented hybrid model structures with only a single paper using SDM-ABM. Emergency and acute care, and elderly care and LTC services were the most frequently simulated health system settings, modelling the impact of health policies and interventions targeting at-capacity healthcare services, patient length of stay in healthcare facilities and undesirable patient outcomes. A high proportion of articles modelled health systems in high income countries; future work should now turn to modelling healthcare settings in LMIC to support policy makers and health system researchers alike. The utilisation of hybrid models in healthcare is still relatively new but with an increasing demand to develop models that can simulate the macro- and micro-level activity exhibited by health systems, we will see an increase in their use in the future.