Skip to main content
Download PDF

Cost-effectiveness analysis of Smart Triage, a data-driven pediatric sepsis triage platform in Eastern Uganda

BMC Health Services Research volume 23, Article number: 932 (2023) Cite this article

Abstract

Background

Sepsis, characterized by organ dysfunction due to presumed or proven infection, has a case-fatality over 20% in severe cases in low-and-middle income countries. Early diagnosis and treatment have proven benefits, prompting our implementation of Smart Triage at Jinja Regional Referral Hospital in Uganda, a program that expedites treatment through a data-driven triage platform. We conducted a cost-effectiveness analysis of Smart Triage to explore its impact on patients and inform multicenter scale up.

Methods

The parent clinical trial for Smart Triage was pre-post in design, using the proportion of children receiving sepsis treatment within one hour as the primary outcome, a measure linked to mortality benefit in existing literature. We used a decision-analytic model with Monte Carlo simulation to calculate the cost per year-of-life-lost (YLL) averted of Smart Triage from societal, government, and patient perspectives. Healthcare utilization and lost work for seven days post-discharge were translated into costs and productivity losses via secondary linkage data.

Results

In 2021 United States dollars, Smart Triage requires an annuitized program cost of only $0.05 per child, but results in $15.32 saved per YLL averted. At a willingness-to-pay threshold of only $3 per YLL averted, well below published cost-effectiveness threshold estimates for Uganda, Smart Triage approaches 100% probability of cost-effectiveness over the baseline manual triage system. This cost-effectiveness was observed from societal, government, and patient perspectives. The cost-effectiveness observed was driven by a reduction in admission that, while explainable by an improved triage mechanism, may also be partially attributable to changes in healthcare utilization influenced by the coronavirus pandemic. However, Smart Triage remains cost-effective in sensitivity analyses introducing a penalty factor of up to 50% in the reduction in admission.

Conclusion

Smart Triage’s ability to both save costs and avert YLLs indicates that patients benefit both economically and clinically, while its high probability of cost-effectiveness strongly supports multicenter scale up. Areas for further research include the incorporation of years lived with disability when sepsis disability weights in low-resource settings become available and analyzing budget impact during multicenter scale up.

Trial registration

NCT04304235 (registered on 11/03/2020, clinicaltrials.gov).

Peer Review reports

Background

Sepsis is a syndrome characterized by an inflammatory state triggered by a presumed or proven infection that results in organ dysfunction and/or death [1]. In severe cases in low-and-middle income countries (LMICs), disease-specific mortality and case fatality ratios can exceed 20% [2, 3]. In Uganda, the burden of sepsis weighs heavily on the healthcare system and pediatric population, with a neonatal prevalence of 24.4% [4]. International sepsis treatment guidelines recommend delivering a bundle of care within one hour (defined in this paper as a “timely sepsis bundle”), based on evidence that earlier treatment reduces mortality and improves patient outcomes [5]. This bundle consists of antimicrobials alongside fluid and oxygen as clinically indicated [5].

In high income countries, evidence-based triage strategies aimed at expediting sepsis treatment [6] involve laboratory investigations often inaccessible in LMIC settings, where delays in treatment leading to a high case fatality ratio are commonplace [7]. LMIC-specific guidelines such as the Emergency Triage Assessment and Treatment (ETAT) [8] and others [9, 10], show good predictive power for mortality and improve clinical outcomes when successfully implemented [11,12,13]. However, the complexity of these guidelines face challenges in implementation, particularly in resource-poor environments where patient throughput and new-staff turnover is high [14, 15]. Infrastructural barriers and medication shortages further delay sepsis treatment even if a sick child has been effectively identified as having sepsis [15].

The Smart Triage program, which was implemented at the pediatric outpatient department at Jinja Regional Referral Hospital (JRRH) from 2020–2021, targeted these barriers in several ways [16]. First, we developed and implemented a model that used clinical variables for risk stratification based on the child’s predicted need for admission. This allowed sicker children to be prioritized during assessment by the existing JRRH clinical team [17]. This model was housed in a mobile application which required minimal training (compared to existing triage guidelines) for use by new team members. Secondly, a Bluetooth patient and treatment tracking system enhanced patient flow and organization by conveying patient status and risk stratification to the clinical team through an electronic dashboard in an automated manner. Thirdly, outcome data from this patient tracking system, such as time taken to deliver a sepsis bundle of care, was used to drive local quality improvement cycles. This combination of quality improvement cycles with a data-driven approach has been shown to effectively improve patient outcomes in the LMIC setting [18].

In high-income settings, economic evaluations of quality improvement initiatives aimed at delivering early sepsis care have demonstrated reduced cost and increased benefit over standard of care [19,20,21]. However, such analyses have not been comprehensively conducted for triage programs in LMICs. LMIC analyses that do exist may report partial costs only [14], or report intermediate outcomes such as quality improvement scores with limited clinical interpretability [22]. In addition, these analyses focus on the perspective of the healthcare system [13, 22, 23], but do not consider out-of-pocket costs and productivity losses. In LMICs such as Uganda that are yet to implement universal health coverage (UHC), where patients may incur out-of-pocket costs near or exceeding that of monthly earnings [24], inclusion of the societal perspective is important to explore the economic impact of healthcare interventions on patients. Lastly, no LMIC analysis has utilized economic modelling techniques to incorporate healthcare utilization costs and link intermediate outcomes to clinical endpoints. To address these gaps in the literature, we conducted a cost-effectiveness of the Smart Triage program using a decision analytic model from a societal perspective, using secondary linkage data to link healthcare utilization to costs, lost work to productibility losses, and process outcomes to clinical endpoints. Results of this analysis may be used both to explore the impact of Smart Triage on patients and to inform scale up of Smart Triage to other Ugandan centers.

Methods

Objectives

  1. 1.

    Determine the incremental cost effectiveness ratio (ICER) for years-of-life lost (YLL) averted by the Smart Triage program compared to the baseline triage infrastructure at JRRH, using the pre-intervention phase at JRRH as the control.

  2. 2.

    Determine the probability of cost-effectiveness of the Smart Triage program under different willingness-to-pay thresholds, compared to the baseline triage infrastructure at JRRH.

  3. 3.

    Investigate the impact of out-of-pocket costs on cost-effectiveness by secondary analyses separating the government and patient perspectives.

Description of primary clinical trial

The Smart Triage program, and corresponding clinical trial, were implemented in the pediatric outpatient department at JRRH in Uganda, which serves children up to age 18 years presenting at various degrees of illness severity. A concurrent control site where Smart Triage would not be implemented was originally planned, but coronavirus disease (COVID) pandemic-induced delays in site initiation precluded the use of this site as a control for our analysis. Therefore, the clinical trial, and data used for this analysis, is of a pre-post design. The comparator to Smart Triage was a manual triage system used in the pre-implementation phase at JRRH that is based on ETAT, a set up common in many comparable LMIC centers [25]. The pre-intervention phase stretched from April 2020 to December 2020, while the post-intervention phase stretched from April 2021 to December 2021. January through March 2021 corresponded to the bulk of program inception/development, and outcome data from these months were not included in our analysis.

All presenting patients, except for those with scheduled appointments or procedures, were eligible for enrollment. Primary outcome was defined as the difference, between the pre- and post-implementation phases, in the proportion of children who received a timely sepsis bundle (defined as within one hour of arrival at the outpatient department). This choice of outcome is based on benchmarks established by international guidelines [1], and improving this outcome is associated with decreased mortality [26]. Enrolled children were followed up via telephone to their caregiver for in-hospital mortality (if admitted), 7-day post-discharge mortality, and 7-day post-discharge healthcare utilization rates. We optimized our follow-up rate through rigorous checks for phone number accuracy, repeat calls to unreached caregivers through a standardized process, and use of shared phone numbers, such as that of a neighbor, should a caregiver not have a personal number. This scheme has achieved a greater than 95% follow-up rate in previous studies by our group in the same region [27]. Further details on the Smart Triage program, clinical trial protocol, sample size calculations, and telephone follow-up have been published elsewhere [16] with appropriate clinical trial registration on clinicaltrials.gov (NCT04304235, 11/03/2020). Ethics approval was obtained from Makerere University School of Public Health (MUSPH) Higher Degrees, Research and Ethics Committee (protocol number 743). MUSPH provided approval for the parent study and this economic evaluation to be conducted at Jinja Regional Referral Hospital in Jinja, Uganda; all study procedures were performed in accordance with the Declaration of Helsinki.

Economic model structure and assumptions

We conducted our analysis using a decision analytic model from a societal perspective, using secondary data from the literature to link health resource utilization (HRU) rates to costs and process outcomes to clinical endpoints [26, 28]. Sub-analyses were done from the government perspective to inform policy formation, and from the patient perspective to isolate economic impact on patients. The model (Fig. 1) mirrors a patient’s clinical pathway from the time of presentation at JRRH to the post-discharge period up to seven days. The structure of this model were validated in consultation with local clinicians at JRRH, and the post-discharge components mirror that of a recently conducted cost-effectiveness analysis of a post-discharge follow-up program for sepsis in Uganda [29]. Lastly, Smart Triage was implemented with heavy advocacy and leadership by local stakeholders for sustainability beyond the trial end date. Therefore, we assumed a program duration of five years for annuitization of program costs and calculation of total patients impacted.

Fig. 1
figure 1

Decision analytic model for patient care pathway before and after Smart Triage implementation. USD = United States dollars, HOC = Health opportunity cost, WHO = World Health Organization, GDP = Gross domestic product

Full size image

Key assumptions of our model include the sustainability of Smart Triage beyond study end date, the generalizability of secondary linkage data used in the model to the Smart Triage population, and the adequacy of our seven-day duration of patient follow-up. The rationale behind these assumptions, and further details on our model, are previously published [30]. Our analysis was designed and conducted according to the 2013 version of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement [31], the most recent version at the time of analysis design, and is also in compliance with the revised 2022 version [32].

Model inputs

As the Smart Triage trial was powered for the pre-post difference in proportion of children receiving a timely sepsis bundle and not mortality difference, the estimated improvement in the latter for any observed improvement in the former was calculated through the literature odds ratio for mortality improvement if given a timely sepsis bundle vs. not [26]. YLLs were then calculated via the product of: 1) mortality rate, 2) total number of patients, 3) difference between the average life expectancy in Uganda [33] and the average age of death in the Smart Triage trial. The pre-post difference in YLL represents the YLL-averted.

In terms of HRU costs, unit costs of care center visits with and without admission were obtained from the Decade of Vaccine Economics project [28, 34]. This source provided aggregate Ugandan HRU costing data from 2018 in pediatric pneumonia, a leading cause of sepsis in LMICs [35]. Costs were segregated by government vs. patient perspectives and outpatient visits vs. inpatient stays. Government costs included facility and medical costs attributable to a clinical visit not directly borne by the patient. For patient costs, both direct costs (e.g. hospital fees) and indirect costs excluding productivity losses (e.g. transportation) were included. Over 45 healthcare centers across Uganda were surveyed. The types and frequency of HRU observed in the Smart Triage trial were then multiplied with the relevant unit cost to obtain costing input for our analysis. For example, if a patient was admitted to JRRH for five days, the unit cost per diem for admission from the government perspective would be multiplied by five to obtain the corresponding cost on the system for admission. The same calculation would be repeated from the patient perspective to obtain out-of-pocket costs. To track productivity losses, we measured caregivers’ lost days of work directly from our trial, then multiplied by the average daily wage in Uganda from the International Labor Organization [36] to obtain lost wages. We counted as caregivers: parents, relatives, and non-family caregivers (e.g. neighbor).

Lastly, program costs were tabulated directly from our trial; costs solely attributable to research were excluded. Program costs were counted from a public payer (i.e., the government’s) perspective, as patients were not charged out-of-pocket fees for Smart Triage due to the reusable nature of program components. All costs were inflated to 2021, the implementation year of Smart Triage, via the gross domestic product (GDP) deflator of the country in which they incurred, then converted to United States dollar (USD) using 2021 average exchange rates. 2021 was chosen as that was the implementation year of Smart Triage.

Perspectives of analysis

Our analysis from the government perspective includes costs of care not borne out-of-pocket by patients (e.g. healthcare worker salary, hospital utilities, Smart Triage program costs). Our analysis from the patient perspective includes out-of-pocket costs directly related to healthcare (e.g. hospital fees), costs indirectly related to having a hospitalized child (e.g. transportation), and productivity losses (lost wages). Our analysis from the societal perspective combines these two perspectives to create a wholistic picture of Smart Triage’s cost-effectiveness.

Statistical analyses

Bivariate analyses were performed using the chi squared test for discrete outcomes and student t test for continuous outcomes. Fischer exact test was used in place of chi-squared test when the number of events were less than five. The incremental cost-effectiveness ratio (ICER) is represented by: (post–pre difference in cost)/(post–pre difference in YLL). The ICER was calculated by multiplying out the decision analytic model after filling in probabilities for each branch point, and summing the corresponding total costs and YLL for each terminal node [37]. Monte Carlo simulations with 10,000 iterations were used to simulate uncertainty of the ICER. Each of the 10,000 simulated ICERs were then plotted on a cost-effectiveness plane, with the vertical axis as costs and horizontal axis as YLL-averted. On this plane, simulations in the northeast quadrant represent increased clinical efficacy at an added cost, in the northwest quadrant reduced clinical efficacy at an added cost (should not adopt intervention), in the southwest quadrant reduced clinical efficacy but with cost savings, and in the southeast quadrant increased clinical efficacy with cost savings (economic dominance – adopt intervention). The proportion of simulations falling under each of these scenarios were calculated.

Next, the probability of cost-effectiveness under varying thresholds for willingness to pay per YLL averted were plotted on a cost-effectiveness acceptability curve (CEAC). The CEAC was generated by modelling increasing monetary amounts a payer would be willing to pay to avert one YLL; these amounts correspond to lines with increasing slopes on the cost-effectiveness plane. The proportion of simulated ICERs that fall below each line on the cost-effectiveness plane corresponds to the probability of cost-effectiveness at that willingness to pay threshold [37]. Therefore, plotting “proportion of ICERs below the line” on the vertical axis and “slope of line” on the horizontal axis creates the CEAC.

For the above analyses, annuitization and discounting rates of 3% per annum were used for costs and YLL, in congruence with a systematic review of other LMIC economic evaluations [38], assuming a five year program duration. All analyses were performed using R [39]. Additional details regarding statistical analyses and the Monte Carlo simulation may be found in our published protocol [30].

Definition of cost-effectiveness

As mentioned above, the CEAC plots the probability of cost-effectiveness of Smart Triage over the baseline triage infrastructure at increasing willingness to pay thresholds. A cost-effectiveness threshold of a country is its chosen willingness to pay threshold to adopt an intervention if, at or below that threshold, an intervention has a high probability of cost-effectiveness over the comparator. However, most LMICs, including Uganda, do not have an formally established cost-effectiveness threshold. Therefore, for our analysis, we selected a threshold based on projected health opportunity costs (HOC) of Uganda (threshold = $174.12 in 2021 USD). As many cost-effectiveness analyses have historically used multiples of GDP per capita ($883.89 in 2021 USD for Uganda), as recommended by the World Health Organization (WHO), we have also stated our results in the context of this standard [40]. See “Discussion – strengths of analysis” section for our rationale for adopting the HOC-based threshold, along with limitations for using either threshold.

Sensitivity analyses

We performed one-way deterministic sensitivity analyses using different annuitization rates (0%, 1%, 5%), and by removing the assumption that Smart Triage would continue past the study end date. The upper limit of 5% was chosen in congruence with variations in economic evaluation guidelines in global health [41], stemming from the variations in economic growth rates of different LMICs [42]. In a scenario-based sensitivity analysis, we also excluded months where patient load at JRRH outpatient department was lower than 50% of the annual median, to account for months where patient load abnormally slowed down due to COVID-pandemic induced travel restrictions. The rationale for this sensitivity analysis was that the primary outcome of Smart Triage, receiving a timely sepsis bundle, is a process outcome that may be affected by decreases in patient load from reasons such as travel restrictions. For this sensitivity analysis, the months excluded were September 2020 from the pre-intervention phase; and January, June, July, August 2021 from the post-intervention phase; corresponding to months where patient load reduced to lower than 50% of the annual median. For each one-way sensitivity analysis, we also applied a probabilistic sensitivity analysis using Monte Carlo simulation to account for parameter uncertainties within our model, as described previously.

Since the cost-effectiveness of Smart Triage was driven mainly by a reduction in admissions rather than an improvement in receiving a timely sepsis bundle, we performed a post-hoc sensitivity analysis whereby the absolute reduction in admission was further reduced in magnitude by 25%, 50%, and 75%. The rationale for this sensitivity analysis is that a portion of the reduction in admission rate may be driven by other factors independent from the efficacy of Smart Triage, including changing HRU behaviors throughout the COVID pandemic.

Results

Demographics and conduct of Smart Triage trial

The sex and age distribution of children were similar between the pre- and post-intervention phases, with the vast majority (over 85%) being children under 5 years. Malnutrition rates and HIV positivity were evenly distributed between phases. The rate of HIV positivity was low, but more than a third of patients have never been tested. Among admitted patients, greater than 90% were admitted for suspected infection. Diagnoses for non-admitted patients were not gathered, due to constraints in trial human resources. Approximately 10% of patients received a triage category of “emergency”, while the remaining 90% were split between “priority” and “non-urgent.” Of note, 41 patients were missing triage priority data due to technical difficulties. Data recovery was unfortunately not possible despite the best efforts of the research team. However, triage priority was not an input variable to the economic model and therefore does not affect our analysis and key findings. Lastly, greater than 90% of patients resided outside of the Jinja Central municipality, thereby necessitating some form of transportation to JRRH. We achieved a high rate of follow-up, with only 1.8% and 1.4% of patients lost to follow-up in the two phases respectively (Table 1).

Table 1 Demographics of patients enrolled into the Smart Triage trial at Jinja Regional Referral Hospital
Full size table

Clinical outcomes

The parent clinical trial did not find statistically significant differences between the pre and post-implementation phases in the proportion of children receiving a timely sepsis bundle, in-hospital mortality, or 7-day post-discharge mortality. However, for total mortality, there was a statistically significant improvement in the post-intervention phase (absolute risk reduction = -0.6% [95% CI -1.2 to -0.05, p = 0.016]). In the pre-implementation phase, 0.7% (95% CI 0.4 to 1.3) of children received a timely sepsis bundle, compared with 0.7% (95% CI 0.4 to 1.2) of children in the post-implementation phase, (p > 0.99). Among admitted patients, we found in-hospital mortality rates of 2.9% (95% 1.5 to 5.5) and 1.1% (95% CI 0.22 to 3.4) during the pre- and post-implementation phases respectively (p = 0.13). Finally, follow-up completed 7 days after return home showed a 7-day mortality rate of 0.2% (95% CI 0.0 to 0.6) and 0.1% (95% CI 0.0 to 0.4) in the pre- and post-implementation phases respectively (p = 0.798) (Table 2). The average age of death during the Smart Triage trial, used to calculate YLL averted in the Monte Carlo simulation, was 2.0 years (95% CI 0.7 to 3.0).

Table 2 Clinical outcomes in the pre-implementation and post-implementation phases of Smart Triage
Full size table

Program costs

The cost of implementing the Smart Triage platform, annuitized over a five-year duration, was $1369.82 per year in 2021 USD. With an average of 28,000 children presenting at JRRH’s outpatient department per year from 2020 to 2022, the average program cost per child is $0.05. Program costs were comprised of infrastructural costs (e.g. electronic dashboard indicating patient triage status), shared consumables (e.g. reusable radiofrequency patient tracking identification bands), or wages (e.g. one-time technologist fee for set-up). Details are available in Additional file 1.

Health resource utilization

There was a clinically and statistically significant reduction in the admission rate at JRRH between the pre-intervention and post-intervention phases (-8.4% [95% CI -11.1 to -5.7], p < 0.001). There were also reductions in seeking care (-2.8% [95% CI -4.5 to -1.1], p = 0.001) and readmissions at any health facilities (-1.2% [95% CI -2.1 to -0.3], p = 0.008) after leaving JRRH. The mean length of stay for primary admission and readmission for both phases were 4 days (Table 3).

Table 3 Health resource utilization probabilities in the pre-implementation and post-implementation phases
Full size table

For each pattern of HRU (admit vs. non-admit at JRRH and readmission vs. seek care again without readmission vs. no revisit at all after leaving JRRH), average days of work missed are shown in Table 4, along with corresponding losses in productivity. As expected, HRU patterns that involve admission and/or readmission results in greater missed work, and therefore higher productivity losses.

Table 4 Average days of paid employment missed by caregivers, stratified by health resource utilization pattern
Full size table

Cost effectiveness of Smart Triage – societal perspective (base-case analysis)

From the societal perspective, Smart Triage resulted in an incremental cost of -$1321.86 per 1000 children (95% CI -4735.48 to 444.73) and an incremental YLL averted of 86.3 years per 1000 children (95% CI 37.7 to 152.3), corresponding to an average ICER of -$15.32 per YLL averted ($15.32 saved per YLL averted). On the cost-effectiveness plane, 91.9% of the Monte Carlo simulations fall within the southeast quadrant, corresponding to both cost-savings and improved clinical outcome, while the remaining 8.1% of simulations fall within the northeast quadrant, corresponding to improved clinical outcome at an added cost (Fig. 2, Additional file 2). On the CEAC, 95% probability of cost-effectiveness was reached at approximately $3 USD, well before both the HOC-based threshold of $174.12 USD and WHO threshold for an “extremely cost-effective intervention” of 1 × GDP per capita ($883.89 USD) (Fig. 3).

Fig. 2
figure 2

Cost effectiveness of Smart Triage relative to the baseline triage infrastructure at JRRH, societal perspective. USD = United States dollars, YLL averted = years of life lost averted, JRRH = Jinja Regional Referral Hospital

Full size image
Fig. 3
figure 3

Cost effectiveness acceptability curve of Smart Triage relative to the baseline triage infrastructure at JRRH. USD = United States dollars, HOC = Health opportunity cost, WHO = World Health Organization, GDP = Gross domestic product

Full size image

Government and patient perspectives

From the government perspective, Smart Triage resulted in an incremental cost of $5.34 per 1000 children (95% CI -442.37 to 554.87) and an incremental YLL averted of 86.3 years per 1000 children (95% CI 37.2 to 153.1), corresponding to an average ICER of $0.06 per YLL averted. 52.7% of the Monte Carlo simulations result in both cost-savings and improved clinical outcome, while the remaining 47.3% result in improved clinical outcome at an added cost (Additional file 2). The ICER was less cost-effective compared to base case (Fig. 4), but 95% probability of cost-effectiveness was nonetheless reached well before both HOC-based and WHO thresholds (Fig. 3).

Fig. 4
figure 4

Tornado diagram showing variations in 95% confidence interval for ICER for various one-way sensitivity analyses. ICER = Incremental cost effectiveness ratio, YLL averted = Years of life lost averted, USD = United States dollars

Full size image

From the patient perspective, Smart Triage resulted in an incremental cost of -$1319.33 per 1000 children (95% CI -4604.14 to 319.00) and a YLL averted of 86.4 years per 1000 children (37.0 to 151.8), corresponding to an average ICER of -$15.27 per YLL averted ($15.27 saved per YLL averted). 93.9% of the Monte Carlo simulations result in both cost-savings and improved clinical outcome, while the remaining 6.1% result in improved clinical outcome at an added cost (Additional file 2). The ICER distribution was comparable to base case (Fig. 4), and 95% probability of cost-effectiveness was reached well before both HOC-based and WHO thresholds (Fig. 3).

Sensitivity analyses

The cost-effectiveness of Smart Triage was preserved in sensitivity analyses assuming annuitization rates of 0%, 1%, or 5%. Namely, 89.0%, 92.0%, and 91.9% respectively of the Monte Carlo simulations resulted in both cost-savings and improved clinical outcomes, while the remaining 11.0%, 8.0%, and 8.1% respectively resulted in improved clinical outcomes at an added cost (Additional file 3). ICERs showed similar distributions to the base analysis (Fig. 4). For all three scenarios, the probability of Smart Triage being cost-effective reached 95% by $4 USD, well under the HOC-based and WHO thresholds.

If we remove the assumption that Smart Triage would be sustained after study end date for a program duration of five years, cost-effectiveness was still maintained. In this scenario, only costs incurred/saved and YLL averted during the study timeframe were counted, and program costs were not annuitized. 89.2% of the Monte Carlo simulations resulted in both cost-savings and improved clinical outcomes, while 10.8% resulted in improved clinical outcomes with an added cost (Additional file 3). The ICER showed a similar distribution to the base analysis (Fig. 4). The probability of Smart Triage being cost-effective reaches 95% at $4 USD, well under the HOC-based and WHO thresholds.

Removal of months where patient attendance at the JRRH outpatient department dropped to 50% of the annual median showed an increase in cost savings and a corresponding shift towards added cost-effectiveness of the ICER compared to the base analysis. However, the confidence interval for cost and ICER were much wider than that of the base analysis (Fig. 4).

Our post-hoc analysis of reducing the absolute reduction in admissions by a penalty factor of 25%, 50%, and 75% of the observed value showed that a high probability of cost-effectiveness was attained below both HOC-based and WHO thresholds with a penalty factor of up to 50% (Fig. 5).

Fig. 5
figure 5

Cost effectiveness acceptability curve of Smart Triage, applying different penalty factors for admission rate reduction. USD = United States dollars, HOC = Health opportunity cost, WHO = World Health Organization, GDP = Gross domestic product

Full size image

Discussion

Interpretation and impact of analysis

Our analysis shows Smart Triage achieves greater than 95% cost-effectiveness under a willingness to pay threshold calculated based on Uganda’s HOC (see “Discussion—strengths of analysis” section for further discussion). Furthermore, this cost-effectiveness is achieved well under the WHO threshold for defining an intervention as “extremely cost-effective” [43], which is one GDP per capita of the country of implementation ($858.10 in 2021 USD for Uganda) and is much higher than the HOC-based threshold of $174.12. For our base case analysis from the societal perspective, Smart Triage results in $15.32 USD saved per YLL averted (-$1321.86 [95%CI -4735.48 to 444.73] for 86.3 YLL averted [95%CI 37.7 to 152.3] per 1000 children), with a program cost of $0.05 USD per child. This small initial investment is only a minute fraction of the existing average hospital spending of $7 to $12 per child in Uganda for pneumonia, one of the most common causes of sepsis in LMICs [34]. Cost-effectiveness was not sensitive to changes in annuitization rate up to 5%. In addition, even if Smart Triage were not sustainable past the study end date, with incurred costs concentrated into the study period and no further YLL averted past the study period, cost-effectiveness persists.

We conducted our base case analysis from the societal perspective, with a sub-analysis from the patient perspective, to capture the economic impact of Smart Triage on patients, not just on the healthcare system like previous analyses [13, 22]. The patient perspective is important as a significant portion of healthcare costs are borne out of pocket in Uganda, which does not yet have UHC. In a Ugandan cross-sectional survey of pediatric sepsis patients, the hospital fee and medications alone cost $15 USD, increasing to $59 USD in private not-for-profit facilities [24]. In context, the average monthly wage in Uganda is $57 USD [36]. Additional costs not directly related to healthcare, such as transportation, meals while hospitalized, and hired childcare for siblings further increase the already heavy financial burden of having a hospitalized child [24, 44]. In this context, our analysis reassuringly indicates a high probability of economic dominance of Smart Triage from both the societal and patient perspectives (91.9% and 93.9%), suggesting that Smart Triage results in both cost-savings for patients and benefits in YLLs-averted, benefiting patients both economically and clinically.

From the government perspective, not only did the probability of cost-effectiveness approach 100% well before the HOC-based threshold (Fig. 3), but half of the Monte Carlo simulations were economically dominant over the pre-existing standard (Additional file 2), suggesting both cost savings and YLLs averted. A cross-sectional survey of Ugandan policy stakeholders showed that 95% felt that cost-effectiveness was an important factor in policy formulation, but unfortunately, a lack of personnel trained in economic evaluation coupled with limitations in information technology and data availability often prevent the use of cost-effectiveness analyses in drafting policy [45]. To this end, a favorable cost-effectiveness result for Smart Triage at JRRH provides strong economic rationale for further testing in additional centers to explore the adoption of Smart Triage into policy.

To assist in scaling up Smart Triage, our low program cost per child may be used to project initial investment cost at subsequent sites based on patient capacity, while scale up costs not encountered in our single center study may be recorded during scale up and used to model the budget impact of a wider adoption of Smart Triage in Uganda [46]. Additionally, implementation research on other successfully adopted health programs in sub-Saharan Africa show that a well-delineated scalable unit is paramount to successful expansion from a single test site to subsequent sites [47]. To this end, Smart Triage is designed to be implemented in an entire outpatient department, representing a clearly defined scalable unit. To further facilitate cost-effective scale up, we have minimized program costs by using non-proprietary and locally-acquired equipment where possible. Also, the training of local Smart Triage “champions” who would subsequently train new staff facilitated local familiarly with the platform that was sustained by existing local leadership rather than costlier extended support from high income country partners. These techniques to minimize capital cost and maximize local ownership are proven strategies towards sustainability in other newly scaled healthcare interventions in sub-Saharan Africa [27, 48, 49], and will be used during Smart Triage’s scale up.

Our analysis provides an evidence-based model and input parameters that could advance health economics research in the areas of pediatric sepsis care in Uganda and elsewhere. Our study findings also present policy implications and further research needs for the system-wide strengthening of pediatric sepsis care in Uganda. To this end,we are working with our partners at Walimu and the United Catholic Medical Bureau to expand the Smart Triage platform to four additional hospitals in Uganda. At these sites, we will conduct external validation of the model and solidify the platform’s scalability for further dissemination [50].

Strengths of analysis

Our use of economic modelling techniques enabled the calculation of costs from healthcare utilization rates and linkage of intermediate outcomes (timely sepsis bundle) to clinical outcomes (YLL-averted). This technique results in a more comprehensive cost-effectiveness estimate to inform policy and allows for an estimate involving clinical endpoints such as YLL-averted despite our parent trial being powered for timely sepsis bundle, a process outcome. A previous economic evaluation of an ETAT-based pediatric triage system in Kenya, the ETAT + , also showed cost-effectiveness. However, this analysis utilized only program costs and the primary outcome of the parent cluster randomized trial [22]. As this primary outcome was a multi-domain quality improvement score, the ICER was difficult to interpret clinically. Another trial examining the ETAT + in Rwanda reported partial costs, but did not conduct a full cost-effectiveness analysis [14]. One analysis utilizing decision analytic modelling showed cost-effectiveness for the integrated management of neonatal childhood illness program in India, a program with some parallels to ETAT [51]. However, the interventions included encapsulated the entire inpatient and post-discharge timeline, with only a small fraction focusing on triage. To our knowledge, our use of economic modelling to evaluate a pediatric sepsis triage program is novel in the LMIC setting.

Another strength of our analysis is our novel use of an HOC-based willingness to pay threshold to define cost-effectiveness. Historically, cost-effectiveness analyses have used multiples of GDP per capita as defined by the WHO. While easy to understand, use of GDP per capita does not take into account an intervention’s cost-effectiveness relative to other existing interventions in a country [40, 52]. However, as all healthcare budgets are finite, funding a new intervention will by definition result in not funding other proposed interventions and/or cutting back on existing interventions. The HOC of a given country is defined as the current impact, per currency unit spent on existing healthcare interventions, on the disability adjusted life years (DALYs) averted in that country. Therefore, basing a cost-effectiveness threshold on HOC assesses novel interventions in the context of existing programs and allows for optimum allocation of resources in resource limited settings. The exact methodology to determine this threshold uses instrumental variables to account for hidden confounders and reverse causation between currency spent and DALYs averted. These methods are aptly described in the reference publication [53]. The HOC-based threshold for Uganda is $174.12 in 2021 USD.

Lastly, our analysis includes post-discharge follow-up data in healthcare utilization, clinical outcome, and productivity loss. These data add comprehensiveness to our analysis, in contrast to existing literature that largely focuses on the inpatient period [12, 14, 22]. To this end, our high rate of follow-up also minimizes missing data. Our inclusion of productivity losses and indirect costs of healthcare (e.g. transportation and purchased meals while in hospital) even after discharge is important, for as mentioned, the economic burden of illness on families is not limited to the direct costs of healthcare such as hospital fees and medication costs [24, 34].

Limitations of analysis

An important limitation of this analysis is that we did not achieve an improvement in the primary target outcome – proportion of children receiving a timely sepsis bundle. Instead, the cost effectiveness observed was driven by other factors, in particular a reduction in admission rate, that may have been confounded by factors such as COVID-related shifts in HRU behavior. The Smart Triage trial was originally designed as a difference in differences study, with a control site able to account for changes unrelated to Smart Triage implementation at JRRH [16]. Unfortunately, delays induced by the pandemic prevented trial initiation at the control site until well after Smart Triage implementation at JRRH, thereby precluding the usability of the control site for this analysis. Reassuringly, in corresponding with local clinicians, no major changes to the triage infrastructure apart from Smart Triage were implemented within the duration of our study. Symptom screening of patients for diversion to COVID designated centers (of which JRRH was not) was implemented in the initial months of the pandemic and encapsulated both phases of the study, thereby less likely to introduce bias. Of course, while these observations are reassuring, the cost-effectiveness of Smart Triage from this analysis requires confirmation in additional sites in a “new normal” era.

Despite potential confounding, a reduction in admission due to Smart Triage is entirely plausible. For example, bringing sicker patients to the front of the assessment queue may have given these patients priority for consideration of hospital admission, a limited resource in LMIC settings. Simultaneously, a low-risk classification by Smart Triage’s data-driven algorithm may have added confidence for clinicians to treat patients with mild to moderate symptoms as outpatients, whereas prior to Smart Triage, a conservative recommendation of admission may have been offered to these patients. While the observed reduction in admissions that drove cost-effectiveness is likely multi-factorial in etiology, our post-hoc sensitivity analysis reassuringly showed that the cost-effectiveness of Smart Triage under the HOC-based threshold is preserved when a penalty factor of up to 50% is applied to the reduction in admission rate.

An additional limitation to our analysis lies in our inability to model years lived with disability due to the lack of long-term quality-of-life data and corresponding disability weights in sepsis survivors in the LMIC context. Extrapolation from HIC data would be inaccurate given the vast differences in social support systems and standards of care between HICs and LMICs. However, our model may be rerun to incorporate such data when available, to calculate incremental cost per DALY averted of Smart Triage. This re-analysis would be informative, as the HOC-based threshold is based on DALYs averted and not YLLs averted. Mathematically speaking, as DALYs are calculated by summing YLLs and years lived with disability, some YLLs averted by Smart Triage may be offset by years lived with disability. This offset would potentially lower the denominator for ICER, increasing the ICER and diluting cost-effectiveness. Nonetheless, our observation of cost-effectiveness well before the HOC-based threshold provides ample room to maintain cost-effectiveness even if years lived with disability were incorporated, despite this potential dilution.

A third limitation lies in our inability to count opportunity costs unrelated to missed wages, such as the death of livestock or spoilage of unattended crops that may have been used to generate income [24]. The variable nature of the source of these losses makes them difficult to count comprehensively, and a review of existing LMIC costing studies show a large variation in the breadth of opportunity costs captured [44]. However, the severity of these opportunity costs is likely linked to higher HRU rates. Therefore, if Smart Triage can indeed reduce HRU such as need for admission, these opportunity costs would likely be reduced.

Finally, scaling up of interventions must consider costs of scale up as well as budget impact [46], which are not captured in a cost-effectiveness analysis at a single site. Nonetheless, the high cost-effectiveness of Smart Triage at JRRH provides ample buffer for unanticipated scale up costs, while the low program cost is reassuring from a budget perspective. A subsequent budget impact analysis, informed by scale up costs measured during a multicenter trial, may be conducted to confirm Smart Triage’s adoptability as an intervention for pediatric sepsis triage at the regional and national levels.

Conclusion

Smart Triage reached a high probability of cost-effectiveness as defined by an HOC-based cost-effectiveness threshold, which assesses proposed interventions based on the country-specific per dollar impact on healthcare outcomes. This cost-effectiveness provides strong rationale for multicenter scale up. Also, Smart Triage benefits patients both clinically and economically, in terms of YLLs averted and cost savings respectively. Our analysis represents the first application of economic modelling methods to evaluate the cost-effectiveness of a pediatric sepsis triage intervention, providing a more comprehensive analysis than those based on only program costs and surrogate outcomes. Smart Triage’s cost-effectiveness was primarily driven by a reduction in admissions, which may be confounded by changes in healthcare utilization induced by factors such as the COVID pandemic. Reassuringly, sensitivity analyses using a penalty factor of up to 50% for reduction in admissions show persistent cost-effectiveness. Areas for future research include confirmation of our results in a post-pandemic “new normal” era, calculation of cost per DALY averted using long-term disability data when available, and modelling of budget impact based on scale up costs measured through a larger multicenter trial. To this end, we are actively pursuing expansion into additional sites in collaboration with local agencies.

Availability of data and materials

The datasets analysed during the current study are available in the Pediatric Sepsis Colab repository, hosted on Borealis, https://borealisdata.ca/dataverse/smart_triage. The corresponding author (EL) may be contacted for data.

Abbreviations

ARR:

Absolute risk reduction

CEAC:

Cost effectiveness acceptability curve

CHEERS:

Consolidated Health Economic Evaluation Reporting Standards

COVID:

Coronavirus disease

DALY:

Disability adjusted life year

ETAT:

Emergency Triage Assessment and Treatment

GDP:

Gross domestic product

HOC:

Health opportunity cost

HRU:

Health resource utilization

ICER:

Incremental cost effectiveness ratio

JRRH:

Jinja Regional Referral hospital

LMIC:

Lower middle income country

NS:

Not significant

RR:

Relative risk

UHC:

Universal health care

USD:

United States dollar

WHO:

World Health Organization

YLL:

Years of life lost

References

  1. Weiss SL, Peters MJ, Alhazzani W, Agus MS, Flori HR, Inwald DP, et al. Surviving sepsis campaign international guidelines for the management of septic shock and sepsis-associated organ dysfunction in children. Intensive Care Med. 2020;46(1):10–67.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Tan B, Wong JJM, Sultana R, Koh JCJW, Jit M, Mok YH, et al. Global case-fatality rates in pediatric severe sepsis and septic shock: A systematic review and meta-analysis. JAMA Pediatrics. 2019;173(4):352–62.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200–11.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Abate BB, Kasie AM, Reta MA, Kassaw MW. Neonatal sepsis and its associated factors in East Africa: a systematic review and meta-analysis. Int J Public Health. 2020;65:1623–33.

    Article  PubMed  Google Scholar 

  5. Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 2017;43(3):304–77.

    Article  PubMed  Google Scholar 

  6. Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated Pediatric Risk of Mortality score. Crit Care Med. 1996;24(5):743–52.

    Article  CAS  PubMed  Google Scholar 

  7. Papali A, McCurdy MT, Calvello EJ. A “three delays” model for severe sepsis in resource-limited countries. J Crit Care. 2015;30(4):861.e9-.e14.

    Article  PubMed  Google Scholar 

  8. World Health Organization. Emergency triage assessment and treatment (ETAT). Geneva: World Health Organization; 2005. Available from: https://www.who.int/maternal_child_adolescent/documents/9241546875/en/. [Updated 2016 March 17; Cited 2021 May 01].

    Google Scholar 

  9. World Health Organization. Handbook IMCI: Integrated management of childhood illness. Geneva: World Health Organization; 2005. Available from: https://apps.who.int/iris/handle/10665/42939. [Updated 2012 June 16; Cited 2021 May 1].

    Google Scholar 

  10. Cheema B, Stephen C, Westwood A. Paediatric triage in South Africa. S Afr J Child Health. 2013;7(2):43–5.

    Article  Google Scholar 

  11. Hansoti B, Jenson A, Keefe D, De Ramirez SS, Anest T, Twomey M, et al. Reliability and validity of pediatric triage tools evaluated in low resource settings: a systematic review. BMC Pediatr. 2017;17(1):37.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Molyneux E, Ahmad S, Robertson A. Improved triage and emergency care for children reduces inpatient mortality in a resource-constrained setting. Bull World Health Organ. 2006;84:314–9.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Robison JA, Ahmad ZP, Nosek CA, Durand C, Namathanga A, Milazi R, et al. Decreased pediatric hospital mortality after an intervention to improve emergency care in Lilongwe, Malawi. Pediatrics. 2012;130(3):e676–82.

    Article  PubMed  Google Scholar 

  14. Hategeka C, Mwai L, Tuyisenge L. Implementing the Emergency Triage, Assessment and Treatment plus admission care (ETAT+) clinical practice guidelines to improve quality of hospital care in Rwandan district hospitals: healthcare workers’ perspectives on relevance and challenges. BMC Health Serv Res. 2017;17(1):256.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Mupara LU, Lubbe JC. Implementation of the Integrated Management of Childhood Illnesses strategy: challenges and recommendations in Botswana. Glob Health Action. 2016;9(1):29417.

    Article  PubMed  Google Scholar 

  16. Mawji A, Li E, Komugisha C, Akech S, Dunsmuir D, Wiens MO, et al. Smart triage: triage and management of sepsis in children using the point-of-care Pediatric Rapid Sepsis Trigger (PRST) tool. BMC Health Serv Res. 2020;20(1):1–13.

    Article  Google Scholar 

  17. Mawji A. Smart Triage Jinja Model Vancouver, vol. 1. Canada: Borealis; 2021.  Available from: https://doi.org/10.5683/SP2/10BME4.

  18. Wagenaar BH, Hirschhorn LR, Henley C, Gremu A, Sindano N, Chilengi R. Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia. BMC Health Serv Res. 2017;17(3):830.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Noritomi DT, Ranzani OT, Monteiro MB, Ferreira EM, Santos SR, Leibel F, et al. Implementation of a multifaceted sepsis education program in an emerging country setting: clinical outcomes and cost-effectiveness in a long-term follow-up study. Intensive Care Med. 2014;40(2):182–91.

    Article  PubMed  Google Scholar 

  20. Afshar M, Arain E, Ye C, Gilbert E, Xie M, Lee J, et al. Patient outcomes and cost-effectiveness of a sepsis care quality improvement program in a health system. Crit Care Med. 2019;47(10):1371–9.

    Article  PubMed  Google Scholar 

  21. Khowaja AR, Willms AJ, Krause C, Carriere S, Ridout B, Kennedy C, et al. The Return on Investment of a Province-Wide Quality Improvement Initiative for Reducing In-Hospital Sepsis Rates and Mortality in British Columbia, Canada. Crit Care Med. 2022;50(4):e340.

    Article  PubMed  Google Scholar 

  22. Barasa EW, Ayieko P, Cleary S, English M. A multifaceted intervention to improve the quality of care of children in district hospitals in Kenya: a cost-effectiveness analysis. PLoS Med. 2012;9(6):e1001238.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Manzi A, Mugunga JC, Iyer HS, Magge H, Nkikabahizi F, Hirschhorn LR. Economic evaluation of a mentorship and enhanced supervision program to improve quality of integrated management of childhood illness care in rural Rwanda. PloS One. 2018;13(3):e0194187.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Krepiakevich A, Khowaja A, Kabajaasi O, Nemetchek B, Ansermino JM, Kissoon N, et al. Out of pocket costs and time/productivity losses for pediatric sepsis in Uganda: a mixed-methods study. BMC Health Serv Res. 2021;21(1):1–9.

    Article  Google Scholar 

  25. Kang K, Chandler H, Espinosa V, Kissoon N. Systems for paediatric sepsis: a global survey. West Indian Med J. 2014;63(7):703.

    CAS  PubMed  Google Scholar 

  26. Evans IV, Phillips GS, Alpern ER, Angus DC, Friedrich ME, Kissoon N, et al. Association between the New York sepsis care mandate and in-hospital mortality for pediatric sepsis. JAMA. 2018;320(4):358–67.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Wiens M, Kumbakumba E, Larson C, Ansermino J, Singer J, Kissoon N, et al. Postdischarge mortality in children with acute infectious diseases: derivation of postdischarge mortality prediction models. BMJ Open. 2015;5(11):e009449.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. de Broucker G, Ekirapa Kiracho E, Ssebagereka A, Constenla D, Aloysius M, Patenaude B. Cost of pneumonia in children under 5 in Uganda (2017-18), vol. 2. Cambridge: Harvard Dataverse; 2020. Available from: https://doi.org/10.7910/DVN/4CF954. [Updated 2020 February 04; Cited 2020 October 13].

  29. Khowaja A, Wiens M, Kissoon N, Ansermino M. Economic evaluation of the Smart Discharges program in Uganda, vol. 1. Scholars Portal Dataverse; 2020. Available from: https://doi.org/10.5683/SP2/GXXG6X. [Updated 2020 December 9; Cited 2021 March 15].

  30. Li EC, Grays S, Tagoola A, Komugisha C, Nabweteme AM, Ansermino JM, et al. Cost-effectiveness analysis protocol of the Smart Triage program: A point-of-care digital triage platform for pediatric sepsis in Eastern Uganda. PLoS One. 2021;16(11):e0260044.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. Int J Technol Assess Health Care. 2013;29(2):117–22.

    Article  PubMed  Google Scholar 

  32. Husereau D, Drummond M, Augustovski F, de Bekker-Grob E, Briggs AH, Carswell C, et al. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. Int J Technol Assess Health Care. 2022;38(1):e13.

    Article  PubMed  Google Scholar 

  33. World Health Organization. Global Health Observatory data repository: life expectancy and healthy life expectancy. World Health Organization; 2020. Available from: https://apps.who.int/gho/data/node.main.688. [Cited 2022 March].

  34. Ekirapa-Kiracho E, De Broucker G, Ssebagereka A, Mutebi A, Apolot RR, Patenaude B, Constenla D. The economic burden of pneumonia in children under five in Uganda. Vaccine X. 2021;8:100095.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Institute for Health Metrics and Evaluation. GBD Compare Data Visualization. Seatlle: IHME, University of Washington; 2016. Available from: https://vizhub.healthdata.org/gbd-compare/.

    Google Scholar 

  36. International Labour Organization. Ilostat–ILO Database Of Labour Statistics. Geneva: International Labour Organization; 2017. Available from: https://ilostat.ilo.org/data/country-profiles/. [Updated 2020; cited 2022 March].

    Google Scholar 

  37. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford university press; 2015.

  38. Santatiwongchai B, Chantarastapornchit V, Wilkinson T, Thiboonboon K, Rattanavipapong W, Walker DG, et al. Methodological variation in economic evaluations conducted in low-and middle-income countries: information for reference case development. PLoS One. 2015;10(5):e0123853.

    Article  PubMed  PubMed Central  Google Scholar 

  39. R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2019. Available from: https://www.R-project.org/.

    Google Scholar 

  40. Cameron D, Ubels J, Norström F. On what basis are medical cost-effectiveness thresholds set? Clashing opinions and an absence of data: a systematic review. Glob Health Action. 2018;11(1):1447828.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Daccache C, Rizk R, Dahham J, Evers SM, Hiligsmann M, Karam R. Economic evaluation guidelines in low-and middle-income countries: a systematic review. Int J Technol Assess Health Care. 2022;38(1):e1.

    Article  Google Scholar 

  42. Haacker M, Hallett TB, Atun R. On discount rates for economic evaluations in global health. Health Policy Plan. 2020;35(1):107–14.

    PubMed  Google Scholar 

  43. Tan-Torres Edejer T, Acharya A, Adam T, Baltussen R, Evans DB, Hutubessy R, et al. Making choices in health: WHO guide to cost-effectiveness analysis. Geneva: World Health Organization; 2003. Available from: https://www.who.int/choice/book/en/. [Updated 2017 October 16; Cited 2020 November 21].

    Google Scholar 

  44. McIntyre D, Thiede M, Dahlgren G, Whitehead M. What are the economic consequences for households of illness and of paying for health care in low-and middle-income country contexts? Soc Sci Med. 2006;62(4):858–65.

    Article  PubMed  Google Scholar 

  45. Musuuza JS, Singer ME, Mandalakas AM, Katamba A. Key actors’ perspectives on cost-effectiveness analysis in Uganda: a cross-sectional survey. BMC Health Serv Res. 2014;14:539.

    PubMed  PubMed Central  Google Scholar 

  46. Donaldson C, Currie G, Mitton C. Cost effectiveness analysis in health care: contraindications. BMJ. 2002;325(7369):891–4.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Barker PM, Reid A, Schall MW. A framework for scaling up health interventions: lessons from large-scale improvement initiatives in Africa. Implement Sci. 2015;11(1):1–11.

    Article  Google Scholar 

  48. Bennett S, Mahmood SS, Edward A, Tetui M, Ekirapa-Kiracho E. Strengthening scaling up through learning from implementation: comparing experiences from Afghanistan, Bangladesh and Uganda. Health Res Policy Syst. 2017;15(2):5–16.

    Google Scholar 

  49. English M, Wamae A, Nyamai R, Bevins B, Irimu G. Implementing locally appropriate guidelines and training to improve care of serious illness in Kenyan hospitals: a story of scaling-up (and down and left and right). Arch Dis Child. 2011;96(3):285–90.

    Article  PubMed  Google Scholar 

  50. BC Children’s Hospital. Smart Triage. Vancouver: BC Children’s Hospital; 2022. Available from: https://guides.library.uq.edu.au/referencing/vancouver/webpages. [Updated 2022; Cited 2023 Jul 28].

    Google Scholar 

  51. Prinja S, Bahuguna P, Mohan P, Mazumder S, Taneja S, Bhandari N, et al. Cost effectiveness of implementing integrated management of neonatal and childhood illnesses program in district Faridabad, India. PloS One. 2016;11(1):e0145043.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Marseille E, Larson B, Kazi DS, Kahn JG, Rosen S. Thresholds for the cost–effectiveness of interventions: alternative approaches. Bull World Health Organ. 2014;93:118–24.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Ochalek J, Lomas J, Claxton K. Estimating health opportunity costs in low-income and middle-income countries: a novel approach and evidence from cross-country data. BMJ Glob Health. 2018;3(6):e000964.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We acknowledge the contributions of Ms. Alishah Mawji as clinical trial coordinator, Mr. Dustin Dunsmuir as lead developer, and Ms. Sela Greys as assistant analyst. We also acknowledge the support of the University of British Columbia in Vancouver, Canada and the Jinja Regional Referral Hospital in Uganda.

Funding

This research was funded by the Wellcome Trust [Grant number 215695/B/19/Z]. For the purposes of open access, the author has applied a CC BY public copyright license to any Author Accepted Manuscript version arising from this submission. The funding body had no role in study design, data collection/analysis/interpretation, or manuscript writing.

Author information

Authors and Affiliations

  1. Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, BC, Canada

    Edmond C. K. Li, Yashodani Pillay & J. Mark Ansermino

  2. Department of Anesthesiology, Royal Columbian Hospital, Vancouver, BC, Canada

    Edmond C. K. Li

  3. Jinja Regional Referral Hospital, Jinja, Uganda

    Abner Tagoola

  4. World Alliance for Lung and Intensive Care Medicine in Uganda, Kololo, Kampala, Uganda

    Clare Komugisha & Annette Mary Nabweteme

  5. Center for International Child Health, British Columbia Children’s Hospital, Vancouver, BC, Canada

    Yashodani Pillay & J. Mark Ansermino

  6. Faculty of Applied Health Sciences, Brock University, St. Catharines, ON, Canada

    Asif R. Khowaja

Authors
  1. Edmond C. K. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abner Tagoola
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clare Komugisha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Annette Mary Nabweteme
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yashodani Pillay
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. J. Mark Ansermino
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Asif R. Khowaja
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EL led model design, analysis, and manuscript writing. AT, CK, and AN spearheaded the Smart Triage clinical trial’s local design and implementation, while YP did the same as a Canadian collaborator. JA was principal investigator for the Smart Triage clinical trial. AK was lead health economist for this economic evaluation. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Edmond C. K. Li.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from Makerere University School of Public Health (MUSPH) Higher Degrees, Research and Ethics Committee (protocol number 743). MUSPH provided approval for the parent study and this economic evaluation to be conducted at Jinja Regional Referral Hospital in Jinja, Uganda; all study procedures were performed in accordance with the Declaration of Helsinki.

Our analysis was designed and conducted according to the 2013 version of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, the most recent version at the time of analysis design, and is also in compliance with the revised 2022 version.

Written informed consent for the parent clinical trial, as well as follow-up analyses including this economic evaluation, was obtained from all caregivers/participants prior to enrollment. Caregivers (parent/legal guardian) were approached for consent for children under 16.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary table 1.

Program costs of Smart Triage in 2021 United States Dollars. Costs marked with * were originally incurred in 2021 Ugandan Shillings and were converted into USD by the average conversion rate in 2021 as per the World Bank (3587.075 Ugandan Shillings to 1 USD). Abbreviations: USD = United States Dollar, JRRH = Jinja Regional Referral Hospital.

Additional file 2: Supplementary table 2.

Detailed Monte Carlo simulation results from societal, government, and patient perspectives. Costs in 2021 USD. Abbreviations: YLL averted = years of life lost averted, ICER = incremental cost effectiveness ratio, USD = United States dollars.

Additional file 3:

Supplementary table 3. Details of Monte Carlo simulations for sensitivity analyses. Abbreviations: USD = United States dollars, YLL averted = years of life lost averted, ICER = incremental cost-effectiveness ratio.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, E.C.K., Tagoola, A., Komugisha, C. et al. Cost-effectiveness analysis of Smart Triage, a data-driven pediatric sepsis triage platform in Eastern Uganda. BMC Health Serv Res 23, 932 (2023). https://doi.org/10.1186/s12913-023-09977-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-023-09977-5

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us