 Research
 Open access
 Published:
Value of information analysis for pandemic response: intensive care unit preparedness at the onset of COVID19
BMC Health Services Research volume 23, Article number: 485 (2023)
Abstract
Background
During the early stages of the COVID19 pandemic, there was considerable uncertainty surrounding epidemiological and clinical aspects of SARSCoV2. Governments around the world, starting from varying levels of pandemic preparedness, needed to make decisions about how to respond to SARSCoV2 with only limited information about transmission rates, disease severity and the likely effectiveness of public health interventions. In the face of such uncertainties, formal approaches to quantifying the value of information can help decision makers to prioritise research efforts.
Methods
In this study we use Value of Information (VoI) analysis to quantify the likely benefit associated with reducing three key uncertainties present in the early stages of the COVID19 pandemic: the basic reproduction number (\(R_0\)), case severity (CS), and the relative infectiousness of children compared to adults (CI). The specific decision problem we consider is the optimal level of investment in intensive care unit (ICU) beds. Our analysis incorporates mathematical models of disease transmission and clinical pathways in order to estimate ICU demand and disease outcomes across a range of scenarios.
Results
We found that VoI analysis enabled us to estimate the relative benefit of resolving different uncertainties about epidemiological and clinical aspects of SARSCoV2. Given the initial beliefs of an expert, obtaining more information about case severity had the highest parameter value of information, followed by the basic reproduction number \(R_0\). Resolving uncertainty about the relative infectiousness of children did not affect the decision about the number of ICU beds to be purchased for any COVID19 outbreak scenarios defined by these three parameters.
Conclusion
For the scenarios where the value of information was high enough to justify monitoring, if CS and \(R_0\) are known, management actions will not change when we learn about child infectiousness. VoI is an important tool for understanding the importance of each disease factor during outbreak preparedness and can help to prioritise the allocation of resources for relevant information.
Background
When confronted with an emerging infectious disease, public health managers face the challenge of allocating finite health resources to maximise outbreak preparedness [1, 2]. However, many important characteristics of emerging infectious diseases are unknown in the early stages of an outbreak, which complicates the allocation of funds when it is most urgently needed. Faced with this challenge, public health managers can decide to allocate available resources under uncertainty to manage an outbreak, or invest some of these resources to reduce uncertainty by collecting additional information about the disease outbreak. Under time and resource constraints, strategically collecting the most valuable information is even more critical [3] as obtaining new information can be time consuming and come with both direct and opportunity costs [4, 5]. In the case of an emerging infectious disease, time and resources invested in collecting information could instead be invested directly into outbreak preparedness or interventions. An example of such a decision, relevant in the early stages of the COVID19 pandemic, was the acquisition of Intensive Care Unit (ICU) beds. While it was clear that increased capacity of health care systems would be needed, yet determining the optimal level of investment was not straightforward. Value of Information (VoI) analysis is a decision tool that helps managers make transparent and accountable decisions when faced with this dilemma of either acting under uncertainty or gathering more information about a given situation before acting [6].
VoI analysis was first developed by Raiffa & Schlaifer [6] in 1961 for application in investment risk analysis. VoI has been applied to ecological and environmental conservation problemsolving and analysis [3, 4, 7,8,9,10]. In the field of epidemiology and public health economics, VoI has been applied for the prioritization of information gathering during vaccination and other nonpharmaceutical interventions, and for the assessment of health technology adoption [11,12,13]. Bradbury et al [12] provides a compelling example of how VoI analysis can inform decision making. In that study, VoI was applied to the analysis of an emergency vaccination campaign for FootandMouth Disease (FMD) in livestock. There was substantial uncertainty around vaccine efficacy, the time lag between vaccination and conferral of immunity on the animals against the virus, and the daily capacity to deliver vaccines. With some defined uncertainties (expanded into 27 scenarios), management actions and objectives, Bradbury et al [12] found that resolving these uncertainties would lead to an average savings of 55 million pounds among other benefits in reducing the number of infected animals as well as the duration of the epidemic. The implication of this finding is that they could identify the most valuable uncertainty to resolve.
Faced with the COVID19 pandemic, many questions were raised about the capacity of healthcare systems around the world to cope with emerging disease outbreaks [14, 15]. During the early outbreak of COVID19 in Wuhan, 75% of deaths resulted from lack of access to mechanical ventilation [16]. Similar experiences in Italy and Singapore showed the importance of ICU preparedness for respiratory disease outbreaks [17, 18]. Increasing the number of ICU beds and other critical care preparations will increase the survival rate of severe COVID19 cases. Noting that acquiring more ICU beds alone is not sufficient and appropriate management practices are essential to optimum utilisation of hospital resources [17, 19, 20].
Here, we explore how managers can use VoI to decide whether to invest in gathering further information about COVID19 disease dynamics before purchasing ICU beds or to purchase ICU beds directly without collecting further information. We study how three uncertain parameters relate to ICU demand: the basic reproduction number (\(R_0\)), case severity (CS) and the relative infectiousness of children compared to adults (CI). We aim to discover which parameter is the most valuable to enable good decisions around ICU preparedness. We use a dynamic transmission model and a clinical pathways model [21] to quantify potential ICU demand, and then apply the VoI analysis.
Method
VoI analysis estimates the value to be gained from resolving uncertainty [3, 6]. In this section, we present an application of VoI to the problem of deciding the appropriate level of investment in ICU bed capacity prior to a novel infectious disease outbreak, given uncertainty about local transmission and severity characteristics of the disease. We first define the problem context used as our case study. We present the management objectives in terms of preparing for a COVID19 outbreak in a resourceconstrained setting. We define the management actions that can be chosen to meet the objective(s). We then introduce two VoI methods that assess how resolving transmission and severity uncertainties affects the choice of management actions. We also describe the models used to generate the input data for our analysis.
Problem context and case study
The World Health Organization Regional Office for the West Pacific (WPRO) monitors the public health performance of about 1.9 billion people across 37 countries in the West Pacific region. Across the WPRO countries, the healthcare system is generally under stress and requires strengthening especially for novel outbreaks [22]. ICU capacity varies significantly across countries in the WPRO region, with a capacity range in some as low as just a few (if any) beds to enable care of severe cases of COVID19 [14]. Others do have ICU beds but their numbers may be insufficient during a large outbreak. Information such as the characteristics of the population and nature of the outbreak are important features that influence the required number of ICU beds in these countries. Unfortunately, this information is often missing. Here, we use VoI to provide insight into which information is most valuable to prepare for a COVID19 outbreak in the WPRO region. We considered a typical urban settlement in the AsiaPacific Islands with an approximate population of 450,000 with 9 ICU beds for COVID19 (Table 1). This choice corresponds to a relatively high populated settlement within the AsiaPacific islands. The model inputs for this population are presented in the accompanying Supplementary information (Section 1, Table S1).
Objective and actions
The first step in formulating VoI analysis is defining the management objective and the actions that will lead to achieving these objectives. This approach is necessary because a piece of new information is only valuable if it will lead to a change in outcomes.
The broad objective is to minimise the health impact of COVID19 through the optimal provisioning of ICU beds for both COVID19 and other existing diseases within a population. We translate this broad objective into: minimising the deficit or excesses in newly acquired ICU beds for COVID19 outbreak preparation. The outcome measure is the sum of ICU bed shortfall over the duration of the outbreak. Because increasing the number of ICU beds for COVID19 also comes at a real cost and an opportunity cost, our objective includes minimising surplus allocation of ICU beds to COVID19 that could result in an adverse effect on other existing diseases in the given population. Hence, we intend to minimise the absolute difference between any newly acquired ICU beds before an outbreak plus existing ICU beds, and the actual number of ICU beds utilised to contain an outbreak.
We define the actions available to a health manager as various increase options in ICU beds capacity from 0 to 200. We consider only the first five discrete actions available to a decisionmaker (using increments of 50 beds):

\(a_0\): Do nothing.

\(a_1\): Increase capacity by 50 ICU beds.

\(a_2\): Increase capacity by 100 ICU beds

\(a_3\): Increase capacity by 150 ICU beds

\(a_4\): Increase capacity by 200 ICU beds
The choice of maximum expansion of up to 200 ICU beds is a reflection of the plausible range for a Lowandmedium Income Country (LMIC), based on discussion with health system researchers within the region under study. Further, the idea of ICU bed is not limited to the equipment itself but the personnel and logistics required to operate the ICU beds. Therefore, we limit the actions to the maximum plausible irrespective of actual demand.
There are a range of methods that assist in VoI analysis [3, 6]. The expected value of perfect information (EVPI) quantifies how beneficial it is to resolve all uncertainty before making a decision. The expected value of partial information (EVPXI) estimates the improvement in decision outcomes if information is resolved about one (or more) parameter. Both approaches calculate expected values and therefore assume a risk neutral situation [23].
Expected value of perfect information
We define a COVID19 outbreak scenario, s, as a combination of values of three epidemiological parameters: case severity CS, basic reproduction number \(R_0\), and child infectiousness CI. These parameters are uncertain and correlate with early factors used in determining the number of patients that may need access to the ICU of a hospital during a COVID19 outbreak [24].
Given an uncertain scenario \(s \in S\), and a set of alternative actions \(a \in A\), the EVPI is the difference between the expected outcomes under certainty (\(EV_{certainty}\)) and uncertainty (\(EV_{uncertainty}\)) [3, 6].
Because our outcomes are measured as a shortfall (which is a form of health cost instead of health benefit), we use absolute values for the calculation of EVPI to avoid negative values during computation. Under uncertainty, it is assumed that decisionmakers will implement the action that minimises the expected cost outcomes under all possible scenarios. Given a number of scenarios, N, in a set, S:
with V(a, s) the cost function that represents the outcome of implementing action a in scenario s, and \(p_s\) is the probability of scenario s occurring.
Under certainty, it is assumed that decisionmakers will first invest in resolving uncertainty prior to making a decision a that minimises the cost of an outbreak scenario.
We define our cost function V(a, s) such that it incorporates the effect of other diseases requiring ICU beds. For each scenario s, and corresponding action a, the cost function V(a, s) measures the level of shortfall or excess in ICU beds in preparedness for a COVID19 outbreak:
Where U(a, s) is the utility of an action a in scenario s, and \(\gamma\) is a discount factor that takes value between 0 and 1 to represent the impact on other diseases when newlyacquired ICU beds (B(a)) are allocated to COVID19 to the detriment of other nonCOVID severe illnesses. In countries where bed allocation to COVID19 does not cause problems for other existing conditions in the society \(\gamma\) should be set to 0. Countries that cannot afford to have any extra allocation to COVID19 would assign \(\gamma = 1\).
We define the utility of an action a, in a given scenario s, as
Where X(s) is the shortfall in ICU beds required to address an outbreak under scenario s and B(a) is the number of ICU beds acquired while implementing an action, a. The utility U(a, s) is optimised when the acquired ICU beds B(a) exactly matches a possible shortfall in ICU beds X(s) during an outbreak scenario s.
In accordance with our objective, our cost function V(a, s) penalises a shortage of ICU beds for COVID19 (U(a, s)) but also recognises that excess allocation of ICU beds to COVID19 to the detriment of other diseases that also require ICU beds (\(\gamma U(a,s)\)) is not an optimal decision.
Expected Value of Partial Perfect Information (EVPXI)
EVPI determines the value of resolving all uncertainty in a decision process. However, EVPI does not recommend which uncertain scenario to resolve first [3]. This can be useful in cases where EVPI is high enough and health managers need to determine which of the uncertain scenarios is most important to resolve. The calculation of EVPXI helps guide such a decision. EVPXI calculates the improvement in expected outcomes when resolving uncertainty about a scenario and is expressed as:
Where \(EV_{certainty(y)}\) corresponds to the expected value when uncertainty about scenario y being true or false is resolved. Because we are using a cost function rather than a payoff, we use an absolute value to ensure that a positive value is returned as EVPXI. When computing the expected value of the hypothesis y, we have one of two outcomes: either y is true (with probability \(p_{y = true}\)) or it is not (with probability \(p_{y = false}\)). Formally:
where the first term calculates the expected value in the case where scenario y is true with probability \(p_{y = true}\), and the second term calculates the expected values in the case where y is false [3]. If y is false, then calculating the expected outcomes requires determining \(p_{sy}=false\), i.e. the renormalised probability distribution computed for the remaining uncertain scenarios. For example, let \(p_s = \{0.2, 0.5,0.3\}\) be our probabilities in three scenarios: \(s_1\), \(s_2\) and \(s_3\). When we resolve \(s_1\) to be false, then our probabilities in \(s_2\) or \(s_3\) change to \(p_{sy = false}=\{0.625,0.375\}\).
The value of resolving individual parameters
The scenario modelling is useful to estimate the expected outcomes of a management decision. However, from a research perspective, it might be easier to quantify the effect of resolving uncertainty about only one parameter at a time, instead of combined values represented by a scenario. For example, health managers could study only how case severity CS being Low, Medium or High affects ICU requirements. To find out which parameter uncertainty is worth reducing, we can reapply the EVPI Eq. (1) . However, we first need to redefine X(s) in the case where s is now explicitly differentiating the contribution of each parameter, i.e. \(s = \{s_{R_0},s_{CS},s_{CI}\}\). The value of resolving an individual parameter becomes the marginal expected outcomes over the set of unknown parameters and can be expressed as:
For example, if we are only interested in studying subscenarios in which \(s_{CS} \in \{Low, Medium, High\}\), we can compute the expected outcomes for \(X_{CS=Low}\), \(X_{CS=Medium}\) and \(X_{CS=High}\) assuming a uniform distribution over the values of the parameters \(R_0\) and CI. Specifically, to compute the cost for the subscenario where CS is High, we take an average of the model outputs of nine scenarios where \(X_{CS} = High\):
In other words, the term X(s) in Eq. (4) is replaced with the average value \(X(S_{CS}=High)\). Repeat the calculations for \(X(S_{CS}= Medium)\) and \(X(S_{CS}=Low)\). Hence, for resolving the case severity parameter uncertainty, we are concerned with only three broad scenarios where \(CS = (LowMediumHigh)\) . With these scenarios, we compute an EVPI for CS. We then repeat this process for \(R_0\) and CI. The parameter with highest EVPI will need to be resolved first.
Infectious disease modelling and simulation
We briefly describe the model employed to compute the shortfall of ICU beds X(s) for each of the 27 outbreak scenarios defined by three parameters, each stratified into three value space. We use a combined transmission model (TM) and clinical pathway model (CPM) to estimate the cost of a COVID19 outbreak in terms of the shortfall in ICU beds and explored how this cost changes for different combinations of disease parameters \(R_0\), CS, and CI. We build on previously published SusceptibleExposedInfectiousRecovered (SEIR) and clinical pathway models [21] developed to inform intervention decisions for both Influenza and COVID19 outbreaks [25] to compute X(s) (Fig. 1).
The SEIR model compartmentalises a population into susceptible, exposed, infected or recovered segments. It uses differential equations to represent how the number of people in each compartment changes over time based on some model parameters including \(R_0\) [26] and relative infectiousness of children (CI). The clinical pathway model, on the other hand, applies to individuals who become infected and also present themselves for medical attention. Patient presentation at different levels of severity is represented by the CS level parameter. The model’s population is agestructured such that for each scenario, the proportion of individuals in an age group with severe infection differs by age group. For example, the age ranges \(019\) in (\(CS_{Low}\)) will have 0.062% of its population having a severe infection. The proportion for the age range 60  69 years for \(CS_{Low}\) is 15.4% but for \(CS_{Medium}\), the proportion increases to 66% for this age range 6069 years. Details of these proportions are provided in Section 1 of the Supplementary information.
We selected parameter combinations as follows. We first stratified each parameter range into three categories: Low, Medium or High (Table 2) based on observations made during the early stages of the COVID19 pandemic (see Section 1 in Supplementary information). To ensure efficient coverage of parameter space, we selected specific parameter values from each of the nine possible combinations of these categories using Latin Hypercube Sampling (LHS)[27]. Hence, when simulating a parameter value from the Low, Medium or High category, we first divided the category into 200 subintervals of equal probability and then selected a value uniformly at random from each of those subintervals, yielding 200 parameter values within a category. This approach helped us to integrate uncertainty within the decision process.
In Table 2, the value of CS determines the proportion of cases that become severe within the population. Only severe cases get hospitalized and only a fraction of the hospitalized cases end up in the ICU unit. Hence, the higher the value of CS, the higher the proportion of people that are likely to require an ICU bed.
A summary of the models applied to this study is shown in Fig. 1.
We define \(R_0\) as:
where \(\lambda\) is the probability of getting infected per contact between an infectious and a susceptible individual, \(\kappa\) is the contact rate between infectious and susceptible individuals, and d is the duration of infectiousness of infected individuals before recovery or death.
The shortfall in ICU beds (after the available ICU beds are exhausted) required per outbreak scenario, X(s), was estimated through simulations. During the TM and CPM simulations, the values of \(R_0\), CS and CI were sampled 1000 times within their uniform range of values. However, during the VoI simulations, the expert beliefs in Table 3 were used to scale the output of the models (X(s)) for each of the 27 scenarios presented in Table 4.
Sensitivity analysis on prior information
In the previous sections, we have assumed that each scenario was assigned a prior probability of occurrence \(p_s\). This prior information influences the result of value of information analysis. In the absence of prior knowledge, we assumed equal chance over the values of the unknown parameters (\(R_0, CS, CI\)) being Low, Medium or High.
To study the influence of these probabilities \(p_s\) on the value of information, we also defined unequal chances over the parameter categories. We considered priors where we had a “Strong belief” that all the parameters are either Low, Medium or High. Table 3 provides the probability distribution over each of the parameter categories for these three priors. For example, for the “belief in low outbreak”, we will assume that the scenario corresponding to \(s_{R_0}=s_{CS}=s_{CI}=Low\) will have the highest probability \(p(s_{R_0}=Low) = p(s_{CS}=Low) = p(s_{CI}=Low) = 0.5\).
Assuming independence, the joint probability, \(p(R_0,CS, CI)\) that an epidemic belongs to one of the 27 outbreak scenarios is the combinations of Low, Medium and High parameter values:
As an example, a belief that a future outbreak will have Low \(R_0\), High Severity and Low Child Infectiousness has the probability:
Hence, our sensitivity analysis represents the 27 hypothetical scenarios of a novel disease outbreak based on all possible combinations of the 3 uncertain parameters under consideration, each grouped into high, medium or low values.
Results
Cost of an outbreak from simulations
The first result we provide is the shortfall in ICU beds, X(s), for each scenario \(s_1\) to \(s_{27}\) obtained through simulations of the transmission and clinical pathway models (Table 4). The X(s) values reported in this table are the \(95^{th}\) percentile of the maximum ICU bed shortfall for all the simulations per scenario. The choice of \(95^{th}\) percentile guarantees that 95% of ICU requirement per outbreak scenario is met.
The scenarios where \(X(s) = 0\) imply that existing ICU beds are enough to take care of any severe outbreak. It could also mean that the outbreak is not severe enough to cause critical illness. So, there is no shortfall in ICU beds that requires additional purchase or assignment to COVID19. When case severity is Low, \(X(s)=0\) irrespective of the value of \(R_0\) and child infectiousness. X(s) is also zero when case severity is Medium but \(R_0\) is Low; child infectiousness will not matter. In the other extreme, when case severity is High and \(R_0\) is also High, \(X > 1000\) ICU beds irrespective of the value of child infectiousness. A decision tree is provided in Fig. S3 of supplementary information to show the combinations of parameters and how large the X(s) can be in terms of excess ICU bed demand.
Expected value of perfect information
The expected value of perfect information (EVPI) depends on the prior information on the most likely scenario (beliefs) and the impact of other diseases (denoted by \(\gamma\)). Given that \(\gamma\) will be countrydependent, we vary the value of \(\gamma\) to interpret our results.
Under No prior knowledge (Fig. 2  blue curves), EVPI varied from 0 to 65 ICU beds depending on how we accounted for the impact of other disease (\(\gamma =0\) to \(\gamma =1\)).
Recall that \(\gamma\) is a discount factor that represents the impact on other diseases when newlyacquired ICU beds (B(a)) are allocated to COVID19 to the detriment of other nonCOVID severe illnesses
When \(\gamma = 0\) and EVPI equals zero or is very small, it means that there is no need to gather more information before acting. When \(\gamma =0.8\), there is a significant consequence of acting without collecting further information. In such case, if the cost of collecting further information before acting is less than the financial or health cost of 65 ICU beds, the management should invest in resolving the parameter (\(R_0\), CS and CI) uncertainties before deciding which action (number of ICU beds to purchase for COVID19) to take.
For a given prior belief in either low, medium, or severe outbreak (Fig. 2), the EVPI varies for a fixed \(\gamma\). At \(\gamma = 0.8\), belief in severe outbreak has the highest EVPI. This means that there is more value in collecting further information about other severe nonCOVID disease before determining the number of ICU beds to acquire for COVID19. Information gathering is less important, if there is a belief in medium outbreak and even less important when there is a belief in low outbreak. However, the EVPI of 33 ICU beds for belief in low outbreak is substantial and thus further information to resolve parameter uncertainties could be sought if the cost of such information is lower than the financial or health cost of 33 ICU beds. When there is a maximum concern about nonCOVID severe illness (\(\gamma = 1)\), then belief in severe outbreak has the highest EVPI of 91 ICU beds. This means that further information will be valuable to confirm this belief before choosing the best available management action.
The results show that the value of information increases as the excess allocation discounting factor, \(\gamma\), increases until a peak is reached. This trend is consistent across all scenarios. Only belief in severe outbreak increases EVPI continuously with \(\gamma\) until \(\gamma = 1\). Hence, the value of learning about the severity of COVID19 is affected by our belief that other diseases will also be severe. Conversely, if one believes that excess allocation of ICU beds to COVID19 will increase the severity of other preexisting diseases, then there is greater value in learning if COVID19 will actually be severe in order to ensure optimum allocation of ICU beds without excesses.
We note that when \(\gamma =0\), the \(EVPI = 0\). If the impact of other nonCOVID conditions do not matter in a country, the best action is to purchase or allocate the maximum available ICU bed to COVID19. There is no value in seeking further information about the nature of an outbreak.
Expected value of partial information and per parameter EVPI
The decision to gather more information depends on the value of the EVPI being ‘high enough’. The threshold for a high enough EVPI depends on the context and on the risk preference of health managers. If the EVPI is deemed high, collecting more information is desirable. In this case, we may want to determine which parameter is most important to collect information about. EVPXI per scenario and parameterlevel EVPI help us to prioritise information collection at the scenario and parameter levels.
Figure 2 shows that the prior belief in severe outbreak has large EVPI across \(\gamma > 0\). We, therefore, use this prior of belief in severe outbreak to study which scenario and parameters we should be resolving first when there are inadequate resources to resolve all other scenarios and parameters uncertainty.
Table 5 presents the EVPXI at \(\gamma ={1}\) and assuming belief in severe outbreak. Scenario 17 has the highest value of information and therefore should be resolved first. It also shows a number of scenarios where EVPXI equals zero. These zerovalue scenarios belong mostly to where the parameters take Low values. That means that there is no value in gathering information on these scenarios before acting. A second cluster is seen where EVPXI are less than 4 ICU beds. We consider these scenarios to have low value of information. For these two groups of scenarios, management could continue without resolving uncertainty for these scenarios. However, for the third cluster the EVPXI range from 5 to 10 ICU beds, it is reasonable to assess that there will be a significant impact of acting without gathering further information about the state of the parameters that define the scenarios in this cluster. The third cluster contains the scenario where no parameter assumes Low value.
Scenario 17 represents the case where \(R_0\), case severity and child infectiousness are Medium, High and Medium, respectively (see Tables 4 and 5). There is high value in resolving this scenario first, which means that resolving if this scenario is occurring or not would have the highest gain in management outcomes in comparison with other scenarios. However, due to the dependence in these computation methods, after each high value scenario is resolved to be actually true or false, the most uncertain scenario could change. We cannot determine if scenario 18 (second highest EVPXI) should be resolved next until we first resolve scenario 17. The EVPXI will need to be recomputed before the next scenario with the highest expected value of information is determined.
If there are inadequate resources to collect more information about all three parameters that constitute each scenario, then EVPXI cannot tell us which parameter we should focus our research on. To solve this problem, we conducted EVPI on separate parameters. Figure 3 shows the value of resolving the uncertainty surrounding each of the three parameters without having the knowledge of the other two parameters.
Figure 3 shows that case severity is the parameter that mostly drives the value of information for optimum ICU bed acquisition. The excesses and shortages are determined by the case severity of COVID or nonCOVID diseases in a country (\(\gamma\)). This is explained in Fig. 4 using EVPXI for case severity (Low, Medium and High).
From Fig. 4, the highest research priority is resolving if the case severity of disease outbreak is high or not. With no sufficient resources to conduct research about all the severity levels, a binary test on High Severity will reduce the uncertainty surrounding the optimum allocation of ICU beds between COVID and nonCOVID patients. The EVPXI will then be computed again for low case severity (CS(Low)) or medium case severity (CS(Medium)) to determine which hypothesis will be resolved next. Similar analysis can be performed for \(R_0\). The outcome of similar analysis for \(R_0\) is shown in Fig. 5.
Finally, an equivalent analysis for Child Infectiousness shows zero value for all its possible states (Low, Medium, or High). Therefore, if case severity and \(R_0\) are known, management action will not change when we obtain new knowledge about child infectiousness. Thus, there is no need to spend any research funds on studying child infectiousness before deciding on optimal ICU bed allocation.
Discussion
We have investigated how VoI analysis can help guide COVID19 outbreak preparedness when considering the requirement for ICU beds in the early stages of an outbreak.
Our results suggest that reducing uncertainty about case severity and \(R_0\) were most valuable because increased knowledge of these factors would have the greatest impact on a decision about how many ICU beds were required. In contrast, reducing uncertainty about child infectiousness was less likely to affect a decision about ICU bed requirements. Hence, in the context of this case study, VoI analysis could enable a manager to make more informed decisions about where they allocate resources in the early stages of an outbreak. Spending resources on studying child infectiousness before deciding on how many ICU beds should be allocated to COVID19 is unnecessary, as it will not affect the decision outcomes. Spending resources on resolving severity and transmissibility, on the other hand, could affect a decision outcome about investment in ICU capacity. Hence, this study illustrates how mathematical modeling and VoI analysis can provide evidence to enable health managers to decide how resources should be allocated, leading to better decisions about outbreak preparedness.
Note that our analysis was conditioned on the specific question that was being addressed about ICU capacity, and does not mean that the relative infectiousness of children is never a valid factor to consider in preparing for an outbreak. If the question concerns other forms of preparedness, such as providing general ward beds and health workers, VoI analysis may well reveal that child infectiousness is an important factor to be resolved.
To put this work into practical context and in line with existing literature, we consider financial and health costs associated with public health decisions during a COVID19 outbreak. In financial context, an ICU bed unit costs $37,500 (range: $25,000  $50,000) [28] to be purchased and installed. So, taking belief in severe outbreak and \(\gamma = 0.7\) as an example, we have EVPI \(=\) 69 ICU beds which would cost $2.59 million. However, human health and life are not directly measured in dollars. The chosen health cost of information is the number of deaths and other COVID19 morbidity that will result from a shortfall of 69 ICU beds for patients that may require them. Hence, considering health cost, a unit shortage in ICU beds could increase the mortality rate of the initial strain of COVID19 by 0.0034% of a given population [14, 29] in a nonvaccinated population. Based on our study objective, having perfect information about an outbreak helps in optimal allocation of ICU beds to avert this loss, which can be significant as this projection means that 34 people in every one million could die. This benefit of optimal allocation, irrespective of the outbreak scenario, is equivalent to the dollar cost or health cost of (not having) 69 ICU beds when they are required for either COVID or nonCOVID severe cases.
From the cost analysis above, we can see that VoI analysis helps to explain why the choices made by the manager are the best given the information available at the time. This justification is necessary in order to account for possible losses or gains in the long run. For this study, VoI could help to explain that $2.59 million needs to be spent on purchasing ICU beds in order to increase the chance of saving 34 people per 1 million of the population from dying from a severe COVID19 infection. For the population size considered in this study (about half a million) and for the \(\gamma = 0.7\), only half of the dollar cost ($1.29 million) should be spent.
VoI analysis is an integral part of the adaptive management process in fields such as ecology [3, 9] and public health management [12] . Adaptive management is a structured and iterative process that incorporates uncertainty for robust decisionmaking. A new iteration is required when new information becomes available through a monitoring and feedback process. As more information about an outbreak becomes available, VoI analysis helps public health managers to refine their management actions to optimise health outcomes. When VoI is high, the expected benefit of gathering further information is justified  triggering an adaptive management process to gather the new evidence that can be used to review management actions [3]. The required information can be obtained by carrying out further research in the literature or setting up experiments to obtain the current values of the parameters. This new information may result in a different choice of action that will lead to a different outcome than the action chosen before new information became available.
In the early outbreak of an infectious disease when not much is known about the disease, mathematical modeling helps one to study the dynamics of the disease. Mathematical models enables the study of whatif or hypothetical scenarios to understand certain characteristics of the disease such as duration, peak time, transmissibility, severity and the impact of various possible interventions. Mathematical modeling, therefore, provides the expert information when there is no existing information from experts in the outbreak. Also, mathematical models reduce the burden of assembling numerous experts in order to provide a distribution for various parameters instead of relying on a single data point. In our case study, we can see how we utilised models to generate the expert input used in our VoI analysis. Beyond this input generation, mathematical models have become an invaluable tool in predicting, assessing and controlling potential outbreaks [21, 30].
In our case study, we chose to use the shortfall of ICU beds as a proxy for the costs associated with a severe COVID19 outbreak. An alternative would be to quantify the economic costs associated with not having sufficient ICU beds, for example, using Qualityadjusted life years (QALY) or Disabilityadjusted life years (DALY). Note also that while the case study used in this paper has focused solely on preparedness in terms of ICU bed capacity, there are many management decisions required for pandemic preparedness and response. Other preparedness includes stocking prophylactic drugs, recruitment of more health workers and quarantine services, among others [29, 31]. However, the general approach proposed in this paper can be applied to a broad range of decisions involved in the pandemic response.
A potential limitation of our work is that in our exemplar scenario, we focused on three uncertain parameters: severity, child infectiousness and \(R_0\). It is possible that model parameters for which we assumed fixed values, such as the latent period, may also affect model outcomes. While the values we used for these fixed parameters were based on the best available evidence at the time, the impact of uncertainty in these parameters remains to be explored. However, the focus of our work is on an example of how to utilise VoI for preparedness for a novel respiratory disease outbreak, with COVID used as a case study.
Availability of data and materials
An Excel sheet for oneoff computation of VoI as used in this work is available at https://github.com/KingPeter2014/VOIProject/blob/main/VoICUDataMultivariate2.xlsx. The code for advanced and extended analysis can be found at https://github.com/KingPeter2014/VoIAnalysis.git.
Abbreviations
 COVID:

Corona Virus Diseases
 VoI:

Value of Information
 EVPI:

Expected Value of Perfect Information
 EVPXI:

Expected Value of Partial Perfect Information
 ICU:

Intensive Care Unit
 SEIR:

SusceptibleExposedInfectiousRecovered
 LHS:

Latin Hypercube Sampling
References
Figueroa CA, Harrison R, Chauhan A. Priorities and challenges for health leadership and workforce management globally: a rapid review. BMC Health Serv Res. 2019;19(239). https://doi.org/10.1186/s1291301940807.
Reich MR, Javadi D, Ghaffar A. Introduction to the special issue on “effective leadership for health systems’’. Health Syst Reform. 2016;2(3):171–5.
Nicol S, Ward K, Stratford D, Joehnk KD, Chades I. Making the best use of experts’ estimates to prioritise monitoring and management actions: A freshwater case study. J Environ Manag. 2018;215:294–304. https://doi.org/10.1016/j.jenvman.2018.03.068.
Marlow NJ, Thomas ND, Williams AAE, Macmahon AB, Lawson J, Hitchen Y, et al. Cats (Felis catus) are more abundant and are the dominant predator of woylies (Bettongia penicillata) after sustained fox (Vulpes vulpes) control. Aust J Zool. 2015;63(1):18. https://doi.org/10.1071/ZO14024.
Martin TG, Camaclang AE, Possingham PPH, Maguire LA, Chadès I. Timing of Protection of Critical Habitat Matters: Timely critical habitat protection. Conserv Lett. 2016;10(3):308–16.
Raiffa H, Schlaifer R. Applied Statistical Decision Theory. Cambridge: Division of Research, Graduate School of Business Administration, Havard University; 1961.
Runge MC, Converse SJ, Lyons JE. Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program. Biol Conserv. 2011;144(4):1214–23. https://doi.org/10.1016/j.biocon.2010.12.020.
Yokota F, Thompson KM. Value of Information Analysis in Environmental Health Risk Management Decisions: Past, Present, and Future. Risk Anal. 2004;24(3):635–50. https://doi.org/10.1111/j.02724332.2004.00464.x.
Canessa S, GuilleraArroita G, LahozMonfort JJ, Southwell DM, Armstrong DP, Chadès I, et al. When do we need more data? A primer on calculating the value of information for applied ecologists. Methods Ecol Evol Br Ecol Soc. 2015;6:1219–28. https://doi.org/10.1111/2041210X.12423.
Holling CS. Adaptive Environmental Assessment and Management. NewYork: John Wiley & Sons Ltd; 1978. http://pure.iiasa.ac.at/id/eprint/823/1/XB78103.pdf. Accessed 21 Oct 2021.
Wilson ECF. A Practical Guide to Value of Information Analysis. PharmacoEconomics. 2015;33:105–21. https://doi.org/10.1007/s402730140219x.
Bradbury NV, Probert W, Shea K, Runge M, Fonnesbeck C, Keeling M. Quantifying the Value of Perfect Information in Emergency Vaccination Campaigns. PLoS Comput Biol. 2017;13(2):1005318. https://doi.org/10.1371/journal.pcbi.1005318.
McKone TE, Winterfeldt DV. Defining the Decision Framework and the Value of Exposure Information in Military Deployments. In: McKone TE, Huey BM, Downing E, Duffy LM, editors. Strategies to Protect the Health of Deployed U.S. Forces: Detecting, Characterizing, and Documenting Exposures. USA: The National Academy of Sciences; 2000. p. 147 – 160. https://www.ncbi.nlm.nih.gov/books/NBK225344/. Accessed 19 July 2021.
Shereen MA, Khan S, Kazmi A, Bashir N, Siddique R. COVID19 infection: Origin, transmission, and characteristics of human coronaviruses. J Adv Res. 2020;24:91–8. https://doi.org/10.1016/j.jare.2020.03.005.
World Health Organization. WHO Coronavirus Disease (COVID19) Dashboard. 2020. https://covid19.who.int/. Accessed 19 July 2020.
Xie J, Tong Z, X G, Du B, Qiu H, Slutsky AS. Critical care crisis and some recommendations during the COVID19 epidemic in China. Intensive Care Med. 2020. https://doi.org/10.1007/s00134020059797.
Goh KJ, Wong J, Tien JCC, Ng SY, Duu Wen S, Phua GC, et al. Preparing your intensive care unit for the COVID19 pandemic: practical considerations and strategies. Crit Care J. 2020;24(215). https://doi.org/10.1186/s13054020029164.
Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the COVID19 outbreak in Lombardy, Italy: early experience and forecast during an emergency response. JAMA. 2020;323(16):1545–6. https://doi.org/10.1001/jama.2020.4031.
Christian MD, Devereaux AV, Dichter JR, Rubinson L, Kissoon N. Introduction and executive summary: care of the critically ill and injured during pandemics and disasters: CHEST consensus statement. Chest. 2014;146:8–34.
Rubinson L, Nuzzo JB, Talmor DS, O’Toole T, Kramer BR, Inglesby TV. Augmentation of hospital critical care capacity after bioterrorist attacks or epidemics: recommendations of the Working Group on Emergency Mass Critical Care. Crit Care Med. 2005;33:2393–403.
Moss R, McCaw JM, Cheng AC, Hurt AC, McVernon J. Reducing Disease Burden in an Influenza Pandemic by Targeted Delivery of Neuraminidase Inhibitors: Mathematical Models in the Australian Context. BMC Infect Dis. 2016;16(1):552. https://doi.org/10.1186/s1287901618667.
WHO West Office. Western Pacific Regional Strategy for Health Systems Based on the Values of Primary Health Care. 2010. https://apps.who.int/iris/bitstream/handle/10665/207483/9789290615019_eng.pdf. Accessed 21 July 2021.
Zaleskiewicz T. Beyond Risk Seeking and Risk Aversion: Personality and the Dual Nature of Economic Risk taking. Eur J Personal. 2001;15:105–22. https://doi.org/10.1002/per.46.
COVID19 G. Early indicators of intensive care unit bed requirement during the COVID19 epidemic: A retrospective study in IledeFrance region, France. PLoS ONE. 2020;15(11):0241406. https://doi.org/10.1371/journal.pone.0241406.
Moss R, Wood J, Shearer F, Black A, Glass K, Cheng A, et al. Coronavirus Disease Model to Inform TransmissionReducing Measures and Health System Preparedness. Australia Emerg Infect Dis. 2020;26(12):2844–53. https://doi.org/10.3201/eid2612.202530.
Carcione J, Juan S, Claudio B, Jing B. A Simulation of a COVID19 Epidemic Based on a Deterministic SEIR Model. J Front Public Health. 2020. https://doi.org/10.3389/fpubh.2020.00230.
Imam RL. Latin Hypercube Sampling. In: Melnick EL, Everitt BS, editors. Encyclopedia of Quantitative Risk Analysis and Assessment. 2008. https://doi.org/10.1002/9780470061596.risk0299.
Medtronic. HPCResorces 2021. 2020. https://hcpresources.medtronic.com/blog/highacuityventilatorcostguide. Accessed 14 May 2021.
Gandjour A. How Many Intensive Care Beds are Justifiable for Hospital Pandemic Preparedness? A Costeffectiveness Analysis for COVID19 in Germany. Appl Health Econ Health Policy. 2021;19:181–90. https://doi.org/10.1007/s40258020006322.
Siettos CI, Russos L. Mathematical modeling of infectious disease dynamics. Landes Biosci. 2013;4(4):295–306. https://doi.org/10.4161/viru.24041.
Shea K, Bjørnstad ON, Krzywinski M, et al. Uncertainty and the management of epidemics. Nat Methods. 2020;17:867–8. https://doi.org/10.1038/s4159202009434.
Acknowledgements
SPARK/SPECTRUM RESEARCH CONSORTIA.
Funding
NHMRC Centre of Research Excellence under the scheme Supporting Participatory Evidence generation to Control Transmissible diseases in our Region Using Modelling (SPECTRUM). Grant ID: APP1170960. This work was also supported by the Department of Foreign Affairs and Trade Australia, ASEAN Pacific Infectious Disease Detection and Response Program 2019 through the project: Strengthening Preparedness in the AsiaPacific Region through Knowledge (SPARK).
Author information
Authors and Affiliations
Contributions
PE, NG, and IC designed the VoI experiments while all authors participated in designing the disease transmission models and experiments. PE created the VoI models while PE, CMB and PTC created the transmission models. CMB and PTC collected data for the transmission models. PE ran all the simulations. PE, NG and IC performed the analysis and interpreted the results. PE and IC wrote the manuscript. All authors have read and approved the manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Appendix
Appendix
This paper contains a Supplementary information on the parameters and methods used for the simulation and clinical pathway models.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Eze, P.U., Geard, N., Baker, C.M. et al. Value of information analysis for pandemic response: intensive care unit preparedness at the onset of COVID19. BMC Health Serv Res 23, 485 (2023). https://doi.org/10.1186/s12913023094794
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12913023094794