Skip to main content

Table 3 Comparison of content between SDM, ABM and hybrid models of health systems literature

From: Mathematical modelling for health systems research: a systematic review of system dynamics and agent-based models

 

SDM papers

ABM papers

Hybrid papers

Purpose of research

Testing policies or interventions:

• to relieve at-capacity healthcare services, reduce ward occupancy and patient length of stay [28, 31, 36, 43, 49, 50, 54, 58, 62].

• to reduce time to patient admission and treatment [33, 53, 61]

• to reduce delayed discharges [31]

• to increase the uptake of healthcare services and level of healthcare provision [60]

• to target undesirable patient health outcomes (morbidity, mortality, post-treatment complications) [47, 58, 60, 63].

• to optimise performance-based incentive policies against health professional productivity, quality of care and volume of services [30, 59].

• to reduce the total cost of care [33, 47, 58, 60, 61, 63].

• to reduce deficit of health professionals [51]

• to reduce generation of incineration-only health care waste [52]

• to increase the number of patients who currently do not seek medical care [64]

Other:

• explore factors leading to undesirable emergency care system behaviour [56, 57]

• simulating hospital waste management systems and predicting future waste generation [37, 48, 55].

• estimating future demand for cardiac care [44].

• exploring the impact of patient admission on health professionals stress level in an integrated care system (IC) [45].

• exploring variation in physician decision-making [32].

Testing policies or interventions:

• to decrease the time agents spent performing tasks, waiting for a service or residing in parts of the system [20, 22, 24, 67].

• to reduce undesirable patient outcomes (mortality and hospitalisation) [23, 25, 47, 67].

• to reduce the number of patients who left a health facility without being seen by a physician [22, 67].

• to reduce number of patients who are wrongly discharged [67]

• to optimise utility of resources (staff, beds) [46, 66, 67].

• on bypass rate of patients accessing care at alternative facilities [23]

• to reduce total cost of care [25]

Other:

• Create tools capable of comparing health insurance reimbursement schemes [65].

• Assessing risk, allocation of resources and identifying weaknesses in emergency care services [21].

Testing policies or interventions:

SDM-DES

• to improve access to social support and care services [43].

ABM-DES

• to decrease patient waiting time to be seen by a physician [24].

• to improve patient flow and length of stay through the system by optimising resource allocation [46].

SDM-ABM

• to reduce undesirable patient outcomes (morbidity) [47].

Other:

SDM-DES

• Estimate the future demand for health care from patients with cardiac disease [44].

• Model patient flow through an integrated care system to estimate impact of patient admission on health care professional’s wellbeing [45].

Healthcare setting modelled

• Cardiology care [33, 53]

• Elderly care or LTC services [28, 31, 36, 49,50,51, 54, 61, 62]

• Emergency or acute care [28, 31, 36, 50, 56,57,58, 61, 62]

• Hospital waste management [37, 48, 52, 55]

• ACO or health insurance schemes [63]

• MNCH [32, 60]

• Orthopaedic care [63]

• Cardiology care [66]

• Emergency or acute care [21, 22, 67]

• ACO or health insurance schemes [23, 25, 65]

SDM-DES

• Cardiology care [44]

• Elderly care or LTC services [43,44,45]

• Emergency or acute care [45]

ABM-DES

• MNCH [46]

• Orthopaedic care [24]

SDM-ABM

• Emergency or acute care [47]

Rationale for using model

• Gain holistic perspective of system to investigate delays and bottlenecks in health facility processes, exploring counter-intuitive behaviour and monitoring interconnected processes between sub-systems over time [28, 30, 31, 36, 37, 48, 56, 58].

• Useful tool for predicting future health system behaviour and demand for care services, essential for health resource and capacity planning [48, 60].

• Configuration of model was not limited by data availability [28, 52, 64] and could integrate data from various sources when required [51].

• Used as a tool for health policy exploration and optimising health system interventions [33, 36, 51, 54, 57, 58, 64].

• Useful for establishing clinical and financial ramifications on multiple groups (such as patients and health care providers) [63].

• Identifying and simulating feedback, policy resistance or unintended system consequences [59, 61].

• Quantifying the impact of change to the health system before real world implementation [62].

• Visual learning environment enabled engagement with stakeholders necessary for model conception and validation [48, 50, 55, 57].

• Utilised by decision makers to develop and test alternative policies in a ‘real-world’ framework [31, 49, 58, 61].

• Suitable for quantitative analyses [53].

• Fast running simulation [54].

• Ability to closely replicate human behaviour that exists in the real system [20,21,22, 25, 66].

• Provides deeper understanding of multiple agent decision-making [23, 67], agent networks [25] and interactions [21, 22].

• Provides flexible framework capable of conveying intricate system structures [20], where simulations captured agent capacity for learning and adaptive behaviour [20, 25].

• Could incorporate stochastic processes that mimicked agent transition between states [25].

• Took advantage of key individual level agent data [25] and integrated information from various sources [65].

• Simulation allows patients to have multiple medical problems at the same time [65].

• Model can be made generalisable to other settings [65].

• Visualization of system facilitated stakeholder understanding of tested policy impact [23], particularly those in the health industry with minimal modelling experience [67].

SDM-DES

• Enabled retention of deterministic and stochastic system variability and preservation of unique and valuable features of both methods [44].

• Capable of simulating flow of entities through system and provides rapid insight without need for large data collection [43].

• Can simulate individual variability and detailed interactions that influence system behaviour [43].

• Offered dual model functionality [44] vital for simulating human-centric activity [45], reducing the practical limitations that come with using a single simulation method to model health systems [45].

ABM-DES

• Captured both patient flow through system and agent decision-making that enabled identification of health care bottlenecks and optimum resource allocation [24].

SDM-ABM

• Could reproduce detailed, high granularity system elements in addition to abstract, aggregate health system variables [47].

Methods of validation

Behavioural validity tests:

• Model output reviewed by experts [57, 60].

• Model output compared with historical data and relevant literature [31,32,33, 36, 48, 50, 54, 58, 59, 61, 62, 64].

Structural validity tests:

• Model conception [28, 60], development [30, 36, 50, 53, 54, 57, 62] and formulation [54, 56, 59] validated by experts.

• Extreme condition or value testing [30, 31, 52, 57, 59, 60, 64].

• Dimensional consistency checks [31, 52, 57, 59, 60].

• Model boundary accuracy checks [31].

• Mass balance checks [54].

• Integration error checks [31, 52].

Sensitivity analysis

• to assess how sensitive model output was to changes in key parameters [49, 51, 57, 60, 64].

• to test the impact of parameters that had been based on expert opinion on model output [28].

• to test the robustness and effectiveness of policies [28, 30, 52, 53, 58, 63] (on the assumption of imperfect policy implementation [28]).

Behavioural validity tests:

• Model output reviewed by experts [46, 66].

• Model output compared with historical data and relevant literature [20, 22,23,24,25, 46, 65, 66].

• F-test [20] and T-test [20, 24] (equivalence of variance and difference in mean tests).

Structural validity tests:

• Extreme condition or value testing [23, 46].

• Model framework reviewed by experts [22, 47].

Sensitivity analysis:

• to determine how variations or uncertainty in key parameters (particularly where they had not been derived from historical or care data [65]) affected model outcomes [23, 25].

Behavioural validity tests:

ABM-DES

• Model output reviewed by experts [46].

• Model output compared with historical data [24, 46].

• T-test (difference in mean tests) [24].

Structural validity tests:

ABM-DES

• Extreme condition or value testing [46].

SDM-ABM

• Model framework reviewed by experts [47].

Sensitivity analysis:

SDM-DES

• To assess how sensitive model output was to changes in key parameters [44].

Study limitations

• Did not consider how future improvements in technology or service delivery may impact results [31, 44, 49, 51].

• May not have simulated all possible actions or interactions that occurred in real system [30, 61].

• Model cannot encapsulate all health sub-sector behaviour and spill-over effects [31, 53].

• Simplification of real system in model [55, 62, 63].

• Lack of facility data required for model conception, formulation and validation [32, 36, 59].

• Lack of costing or cost effectiveness analysis [58, 60].

• Simulation was over a short time scale and did not evaluate long term patient outcomes [33, 57].

• Assumptions made in model development may not be generalisable to other settings [36, 63].

• Discussion with stakeholders that contributed to model development was not performed systematically [51].

• Quantifying model uncertainty was limited [64].

• Model parameterised with best information available, sometimes missing key data [20, 22, 25, 67].

• Did not model all real system complexity, simplifications made to agents and their attributes [20, 23, 65, 66].

• Did not consider all hospital units affected by possible spill-over effects [21].

SDM-DES

• Did not consider how future improvements in technology may impact results [44].

• Did not model all real system complexity, stable number of patients with disease per age group [44].

• Lack of technology support led to simplifications in configuration of model (how information was passed between two distinct models) [45].

ABM-DES

• Need more case studies to externally validate model [24].

Software platform

• iThink or STELLA (same software) [33, 36, 37, 48, 50, 54, 55, 57, 60, 61].

• MATLAB and Simulink [30].

• Vensim [28, 32, 52, 53, 62,63,64].

• Did not state [31, 49, 51, 56, 58, 59].

• AnyLogic [23, 25, 65].

• Java [66].

• Netlogo [20,21,22, 67].

SDM-DES

• Vensim and Simul8 [43, 45].

• Does not state [44].

ABM-DES

• AnyLogic [24, 46]

SDM-ABM

• AnyLogic [47].