Essential information: Uncertainty and optimal control of Ebola outbreaks

Edited by Alan Hastings, University of California, Davis, CA, and approved April 20, 2017 (received for review October 21, 2016)
May 15, 2017
114 (22) 5659-5664

Significance

The 2014 Ebola outbreak illustrates the complexities of decision making in the face of explosive epidemics; management interventions must be enacted, despite imperfect or missing information. The wide range in projected caseload generated attention as a source of uncertainty, but debate did not address whether uncertainty affected choice of action. By reevaluating 37 published models, we show that most models concur that reducing funeral transmission and reducing community transmission are robust and effective management actions to minimize projected caseload. Although models disagreed about absolute caseload, this measure has little relevance for evaluating candidate interventions. Our study highlights the importance of projecting the impact of interventions and is applicable to management of other epidemic outbreaks where rapid decision making is critical.

Abstract

Early resolution of uncertainty during an epidemic outbreak can lead to rapid and efficient decision making, provided that the uncertainty affects prioritization of actions. The wide range in caseload projections for the 2014 Ebola outbreak caused great concern and debate about the utility of models. By coding and running 37 published Ebola models with five candidate interventions, we found that, despite this large variation in caseload projection, the ranking of management options was relatively consistent. Reducing funeral transmission and reducing community transmission were generally ranked as the two best options. Value of information (VoI) analyses show that caseloads could be reduced by 11% by resolving all model-specific uncertainties, with information about model structure accounting for 82% of this reduction and uncertainty about caseload only accounting for 12%. Our study shows that the uncertainty that is of most interest epidemiologically may not be the same as the uncertainty that is most relevant for management. If the goal is to improve management outcomes, then the focus of study should be to identify and resolve those uncertainties that most hinder the choice of an optimal intervention. Our study further shows that simplifying multiple alternative models into a smaller number of relevant groups (here, with shared structure) could streamline the decision-making process and may allow for a better integration of epidemiological modeling and decision making for policy.
The devastating 2014 Ebola outbreak in West Africa is the largest ever recorded (1, 2). It resulted in 28,646 cases and 11,323 deaths by March 27, 2016 (WHO report; apps.who.int/ebola/ebola-situation-reports) and engendered an outpouring of concern for those affected. A large number of epidemiological models were developed and published (24). To date, we have identified 55 published Ebola models. Most of these models (50 of 55) projected caseloads as the preferred way to predict epidemic trajectory. However, caseload projections varied widely between models, drawing a great deal of attention and causing intense debate (5, 6).
Caseload projection is critical for predicting the size of an epidemic and planning management efforts, and it can vary from model to model for several reasons, such as differences in model structure, parameterization, and other assumptions. Despite model-specific variations in caseload projections, a critical question for decision making is whether different models lead to different management recommendations or different rankings of alternative management actions. If all models agree on the optimal management, then differences in projections are not a critical concern for decision making. Otherwise, if models disagree with respect to the ranking of management recommendations, then the optimal intervention is model-specific, which means that policymakers face the question of which model(s) to rely on to make management decisions; a closer examination of the source of the disagreement is then warranted.
Here, with the objective of minimizing the Ebola caseload, we explored the management recommendations of a large set of published Ebola models. We considered 37 published compartmental Ebola models that varied widely in model structure, parameterization, or both (Table S1). Among them, the SEIR compartment model is the most commonly adopted model framework, where individuals progress through susceptible (S) to exposed (E), infectious (I), and then removed (R) compartments through either recovery or death. Because hospital settings and funerals have been identified as critical transmission sources and targets for intervention, some models explicitly include a hospital (H) or funeral (F) compartment. Based on model structure, we classified 37 models into four categories: models with both H and F explicitly represented (referred to as SEIHFR), models with only H explicitly represented (SEIHR), models with only F explicitly represented (SEIFR), and models with neither H nor F compartments (SEIR). To ensure consistency, we recoded all of the models within the same stochastic environment by simulating the epidemic birth and death processes using the Gillespie algorithm with a tau-leaping approximation (3, 7). We then identified five management actions that are broadly applied to control Ebola: reducing community transmission, reducing hospital transmission, reducing funeral transmission, increasing hospitalization, and reducing case fatality ratio (2, 3, 8, 9). We projected the caseload under five management interventions and identified the optimum management for each model. We then used value of information (VoI) (10, 11) analyses to quantify the potential improvement in caseload outcomes achieved by resolving model-specific uncertainty and identify the key uncertainties to resolve to achieve that improvement.
Table S1.
The 37 published compartmental Ebola models used for this analysis
Model* no.Model typeOutbreak countryOutbreak yearR0Ref.Caseload projected in original publication?
1SEIHFRGuinea20141.467Yes
2SEIHFRDRC19764.7113Yes
3SEIHFRWest Africa2015NA24Yes
4SEIHFRWest Africa20141.8025Yes
5SEIHFRDRC19952.709Yes
6SEIHFRUganda20002.709Yes
7SEIHFRLiberia20142.223Yes
8SEIHFRSierra Leone20141.783Yes
9SEIHRWest Africa20142.0026No
10SEIHRNigeria20141.50–2.0027Yes
11SEIHRLiberia20141.9028Yes
12SEIHRSierra Leone20141.3628Yes
13SEIHRWest Africa20151.9429Yes
14SEIFRWest Africa20151.6024Yes
15SEIFRGuinea20151.7924Yes
16SEIFRLiberia20151.8124Yes
17SEIFRSierra Leone20151.3224Yes
18SEIFRGuinea20141.1030Yes
19SEIFRLiberia20141.5330Yes
20SEIFRSierra Leone20141.3330Yes
21SEIRGuinea20141.5131Yes
22SEIRLiberia20141.5931Yes
23SEIRSierra Leone20142.5331Yes
24SEIRGuinea20141.1532Yes
25SEIRSierra Leone20141.2432Yes
26SEIRDRC19951.8333Yes
27SEIRUganda20001.3433Yes
28SEIRWest Africa20142.1025Yes
29SEIRDRC19951.3834Yes
30SEIRDRC19951.3634Yes
31SEIRLiberia2014NA35Yes
32SEIRSierra Leone2014NA35Yes
33SEIRDRC19953.6536No
34SEIRUganda20001.7936No
35SEIRGuinea20141.3037Yes
36SEIRLiberia20141.6937Yes
37SEIRSierra Leone20140.9937Yes
DRC, Democratic Republic of the Congo.
*
Model number is as in Dataset S1; parameter values are given in Dataset S1. Links to code are given in SI Text: Parameters.R, Functions.R, and Running models.R.
NA means that R0 was not reported.
The Ebola outbreak highlighted the challenges and opportunities that arise when multiple modeling groups contribute models and projections to help inform decision making. The VoI framework allows us to study the robustness of conclusions in the face of multiple alternative models and discover important model sensitivities with respect to the ranking of interventions. By studying an ensemble of models, we can identify actions that are robust in the face of uncertainty as well as sources of uncertainty that warrant immediate study.

Results

In our simulations, starting with 10 infectious cases in a population of 10,000 individuals, the mean projected caseload was 5,615 ± 2,705 (SD), ranging from 184 to 9,887 cases (Fig. 1). Despite this difference in caseload projections, the majority of the models suggested similar management recommendations (Fig. 1).
Fig. 1.
Unconstrained caseload projections (Upper) and ranks of five management actions (Lower) under 37 published compartmental Ebola models with SEIHFR (representing susceptible, exposed, infectious, hospitalized, funeral, and removed compartments), SEIHR, SEIFR, or SEIR structures. For each model, five management actions were ranked from the worst (with highest caseload projection) as shown in light red to the best (with lowest caseload projection) as shown in dark red. Simulated population size is 10,000 people, and the effectiveness is 30% for each management action.
We define the effect of an intervention as the percentage change compared with a no intervention baseline. The effect of an intervention on underlying rates (e.g., transmission and hospitalization) is unlikely to be known a priori; therefore, we first projected caseload assuming that each intervention resulted in a 30% change in affected parameters. Given a 30% change for each intervention, the majority (22 of 37) of the models recommended reducing funeral transmission as the optimal action, and 29 of 37 or 36 of 37 models ranked it as among the top 2 or 3, respectively. Reducing community transmission was optimal for 10 of 37 models, and 33 of 37 models ranked it in the top 2. Reducing the case fatality ratio was optimal for 3 of 37 models, and reducing hospital transmission was optimal for 2 of 37 models, whereas increasing hospitalization was not optimal in any model (Fig. 1). The optimal management recommendation was closely associated with model structure; for example, all SEIR and the majority (four of five) of SEIHR models recommended reducing funeral transmission as the optimal action, whereas the majority (six of eight) of SEIHFR models recommended reducing community transmission (Fig. 1).
The final epidemic size (i.e., the total caseload) is linked to the basic reproductive ratio, R0, in SEIR-like compartmental models, such as those analyzed here (12). To provide a deeper mathematical understanding of our individual model results, we therefore conducted elasticity (proportional sensitivity) analyses of R0 to the parameters associated with five interventions in the full SEIHFR models (in which all parameters could be explicitly perturbed). This analysis revealed that the ranking of the elasticities was the same as that of the associated interventions, in which reducing community transmission ranked as the first (6/8 models recommended it as optimal) and reducing funeral transmission ranked as the second (Fig. 1 and Table S2); if we only have a single model, alternative interventions can be ranked by their effect on R0.
Table S2.
The elasticity of R0 to parameters associated with the interventions for the full SEIHFR models
InterventionsParameters affectedModel 1Model 2Model 3Model 4Model 5Model 6Model 7Model 8AverageRank
Community transmissionβIc1, βIc20.860.150.680.460.190.970.760.740.601
Funeral transmissionβFc, βFh0.140.150.220.310.660.030.190.190.242
Case fatality ratioδ1, δ2, δH1, δH20.45−0.220.670.300.64−0.010.15−0.250.223
Hospital transmissionβIh1, βIh20.00*0.710.110.230.160.000.050.070.174
Hospitalizationh1, h2−0.080.68−0.50−0.060.04−0.71−0.08−0.04−0.105
Average gives the mean elasticity across eight SEIHFR models. Rank indicates the rank of the absolute value of the average elasticity. Model number and description of the parameters are as given in Dataset S1.
*
The hospital transmission parameter is not included in this model; thus, the calculated elasticity is zero. The elasticity rank of this intervention is unchanged if this model is left out of the analysis.
To illustrate the sensitivity of the optimal intervention to the intervention effect size, we also compared caseload projections under a particular intervention over a gradient of changes ranging from 10 to 100% in increments of 10% with projections under the rest of the four interventions with the baseline change of 30%. These analyses highlight that whether an intervention is ranked as optimal depends on its effect size. Reducing funeral transmission is optimal in 22 of 37 models if the effect is over 30% and optimal in 31 of 37 models if interventions associated with burials lead to an 80% reduction. Reducing community transmission is rarely optimal when the effect size is less than 30%, but it is recommended as the best intervention in 32 of 37 models if it can be reduced by 50%. Interventions aimed at reducing the case fatality ratio must achieve a reduction of 60% to be optimal in 26 of 37 of the models (Fig. 2 and Fig. S1). Notably, increasing hospitalization and reducing hospital transmission were rarely optimal interventions, even at 100% effectiveness.
Fig. 2.
The number of models for which a particular intervention is recommended as optimal for the interventions of (A) reducing funeral transmission, (B) reducing community transmission, (C) reducing case fatality ratio, (D) reducing hospital transmission, and (E) increasing hospitalization. Evaluation was based on comparisons of caseload projections under each specific management action over a gradient of changes ranging from 10 to 100% against all of the other management actions with a baseline intensity of 30%.
Fig. S1.
The management effect required for each of five management actions to become the optimal action under 37 published compartmental Ebola models. All 37 models are presented within the row for each action reordered by rank for that action. Evaluation was based on comparisons of caseload projections under each specific management action over a gradient of changes ranging from 10 to 100% against all of the other management actions with a baseline intensity of 30% change. The number of models for which a particular intervention is recommended as optimal generally increased with the intervention efficacy. Considering the intervention of reducing funeral transmission as an example, if the efficacy of an intervention to reduce funeral transmission is below 10%, no models recommend that intervention as optimal. When the efficacy increases to 20%, 4/37 models recommend it as optima; when the efficacy increases to 30%, 22/37 models recommend it as optimal. The number of models recommending an intervention to reduce funeral transmission as optimal is positively correlated with the efficacy of the intervention.
We calculated the expected value of perfect information (EVPI) (10, 11), which quantifies the maximum achievable improvement in management that could be obtained by identifying a single model as “best” before the implementation of specific decisions (Materials and Methods has a formal definition). The EVPI analysis showed that the expected improvement in management outcomes caused by resolving all model-specific uncertainties is an 11% reduction in caseload. We further conducted an analysis of the expected value of partial information (EVPXI) (10), which quantifies the expected improvement in management performance by resolving a subset of uncertainties. In particular, we quantified the relative contribution of uncertainty about model structure (SEIR, SEIHR, SEIFR, or SEIHFR) and caseload projection (models that projected low, low intermediate, high intermediate, or high case burden) to expected management outcomes. The EVPXI analysis illustrates that targeting the uncertainties in model structure could improve management (i.e., reduction in caseload) by 9% (82% of total EVPI), but targeting uncertainties in caseload projection could only achieve 1% improvement of management (12% of total EVPI). Thus, the two most important take home messages from our analyses are, although differences in caseload projections from the various models dominated much of the public discussion, (i) that intervention rankings are not affected by this issue and (ii) that resolving uncertainty in model structure is important for identifying optimal response strategies.

Discussion

Identifying the uncertainties that affect the choice of management intervention is critical for focusing scientific inquiry on questions that will improve the management of an epidemic. Conditional on a single model, a conventional approach is to evaluate the sensitivity of outcomes and management recommendations to parametric uncertainty. However, it is increasingly common that there are multiple independent models that can contribute to the evaluation of candidate interventions and policy development (2, 3, 8, 9). Thus, we present a framework for integrating model output to identify actions that are robust to the parametric, structural, and other uncertainties reflected in an ensemble of models. Our study showed that, despite large differences in caseload projections, management recommendations are broadly consistent across 37 published Ebola models; reducing funeral transmission and reducing community transmission are generally ranked as the top two best management options, whereas hospital-associated actions are rarely the best. Focusing on individual SEIHFR models, the same rank order was found for the proportional sensitivity of R0 to the parameters associated with each of five interventions, in which reducing community transmission ranked as the first and reducing funeral transmission ranked as the second. This result aligns with classical theory (12): if we only have a single model, interventions that most affect the basic reproductive ratio are the best. Both funeral transmission (2, 9) and community transmission (9, 13) were identified as critical transmission sources and, therefore, targets for intervention against Ebola in previous studies. Despite the broad consensus among model recommendations, our EVPI analysis showed that resolving model uncertainties could improve management by 11%. Considering the 2014 Ebola outbreak, which had a caseload of 28,646 and a case fatality ratio of over 50%, this improvement would represent a reduction of 3,266 cases and 1,633 deaths averted. By conducting model–class-specific analyses of EVPXI, we found that the VoI for model structure was far higher than for caseload projections. Thus, the ranking of interventions was not strongly correlated with caseload projections, although expected caseload does provide information on how much effort will be required to halt the epidemic. The ranking of interventions differed more between than within model structures. This result could be a reflection of the inherent differences in the dynamics of different model structures or caused by differences between explicit and implicit representations of interventions within the same model structure. When the target compartments for specific interventions were not explicitly represented in the originally published models, then implementing interventions involved more subtle decisions. Although we tried to achieve the best standardization, our choices (detailed in SI Text) may hamper fair comparisons. An obvious solution is to focus on models that consistently and explicitly represent both compartments and interventions (implying an important role for the integration of operations research and epidemic modeling to ensure that modeled interventions are realistic and reflect real world constraints). However, we garnered important insights by studying all 37 models rather than restricting our analysis to 8 parameterized SEIHFR models.
Our study chose a 30% change to illustrate the management ranking based on caseload; we did not specifically consider the operational cost or constraints inherent in achieving that level of effect with each intervention. In practice, the same percentage change in one intervention might be harder or more expensive to achieve than another. Therefore, it is also important to consider operational and economic constraints (14, 15). Our analysis showed that some interventions, like reducing funeral transmission, can be ranked highly even if they only achieve low effectiveness, but others, such as reducing community transmission and reducing the case fatality ratio, are only ranked high if highly effective. In contrast, interventions targeting hospitalization rate and hospital transmission were rarely optimal, even at 100% effectiveness. Thus, although additional operational or behavioral information may be necessary to evaluate the potential effect of these interventions (e.g., because of interactions with social and cultural processes) (16), the framework that we have presented can be used to estimate minimal levels of effectiveness necessary to consider different classes of interventions. It also needs to be noted that, for purposes of illustration, we evaluated each management action separately in this study. However, management interventions are not mutually exclusive, and combined interventions should be studied. The VoI decision-theoretic approach can easily extend to this situation.
In this study, we focused on minimizing cases as our objective, because most of the published models projected caseload. However, recommended interventions may differ when considering different objectives, such as minimizing mortality or epidemic duration (17). In the case of the 2014 Ebola outbreak, intervention recommendations for the objective of minimizing deaths were the same as for minimizing caseload, because the case fatality ratio was quite similar for different compartments in the published models.
Overall, our study shows that, although differences in Ebola caseload projections were a subject of concern, identification and resolution of uncertainties that hinder management (here, model structure) are more relevant to the selection of optimal response strategies. Our work adds to a growing body of literature on uncertainty in disease dynamics from the standpoint of the decision maker and emphasizes that the uncertainty that is of interest epidemiologically (here, final epidemic size) may not be the same as the uncertainty that is most relevant to management (ranking of candidate interventions).
Our work used VoI to quantify the expected benefit of resolving uncertainty in a decision-making context conditional on model-specific projections. For the published Ebola models, we showed that achieving scientific resolution on model structure achieves 82% of the expected benefit of reducing all uncertainty considered. Scientific understanding (legitimate scientific differences of opinion/data) is frequently described in multiple competing models. Previous methods have assessed potential interventions conditional on individual models or used ensemble prediction (18). Instead, we propose the use of VoI to identify which scientific differences of opinion lead to different management recommendations; research should then prioritize learning about these operationally relevant uncertainties. Thus, our analysis is explicitly not focused on selecting a best model (as is commonly done). Rather, we use all models to identify actions that are robust and uncertainties that are important across the suite of candidate models. For Ebola, simplifying a large set of models into a smaller number of relevant classes (at the level of model structure) may allow for better integration of epidemiological modeling and decision making for policy. In other outbreak situations, resolution of other sources of uncertainty may be more important. More generally, our study shows how VoI analysis may provide “rules of thumb” to guide the decision-making process for other epidemics, such as Zika or avian influenza, where significant uncertainty about individual models remains but timely decision making is required.

Materials and Methods

Literature Survey of Published Ebola Models.

We conducted a literature survey for any published Ebola models on the Institute for Scientific Information Web of Knowledge and through the Google search engine using the search terms Ebola and model. We identified a total of 55 mathematical models in 35 publications and 1 online final report. The majority (37 of 55) of the models were compartmental models, whereas the rest (18 of 55) adopted one or more of a variety of modeling approaches, such as branching process models (19) and spatial models (20). Our study focused on 37 compartment models (a list is given in Table S1); this model type was the most widely used modeling approach for Ebola epidemic projection and management evaluation (2, 3, 9). The consistent framework of compartmental models allows for a general comparison of projected epidemic dynamics among models. Additionally, the widely proposed interventions in practice are either explicitly or implicitly applicable to these models, and therefore, the effectiveness of different interventions can be compared within and among models.

Compartment Models.

In an Ebola compartment model, individuals in a population are classified into different states as represented by different model compartments based on their health status. All individuals remain in the susceptible (S) compartment until they contract the virus through contact with infectious individuals; they then enter the exposed (E) compartment, where they are infected but are not yet infectious. Exposed individuals move to the infectious (I) compartment after a certain latent period, at which point they start to show symptoms and become infectious to other individuals. Infectious individuals will either remain in the community or be hospitalized; both are removed (R) from the chain of transmission through either recovery or death. Deceased individuals may infect others until they are buried.
Compartment models may incorporate different subsets of compartments to explore different transmission mechanisms or evaluate the effectiveness of different interventions. Hospital settings and transmission during funeral practices have been identified as critical transmission sources of Ebola. A number of published Ebola compartment models have addressed these transmission mechanisms by explicitly including a hospitalized (H) compartment and a funeral (F) compartment (3, 8, 9, 13). Based on the model structure, we classified 37 models into four categories: models with both H and F explicitly represented (8 models; referred to as SEIHFR), models with only H explicitly represented (5 models; referred to as SEIHR), models with only F explicitly represented (7 models; referred to as SEIFR), and models with neither H nor F compartments (i.e., these mechanisms of transmission were implicitly incorporated in overall transmission; 17 models; referred to as SEIR models). To ensure consistency, we recoded all of the 37 models within the same stochastic environment (epidemic birth and death processes simulated using the Gillespie algorithm with a tau-leaping approximation; details are given below) (3, 7) using R 3.2.1 (21). A figure of the global model, within which all of the 37 models can be represented as submodels, is presented in Fig. S2. Parameters for 37 models are listed in Dataset S1. Additionally, to ensure correct representation of the published models, we recalculated the basic reproductive number (R0) (Dataset S1) using the next generation framework (22). Links to code to run all of the models are available in SI Text: Parameters.R, Functions.R, and Running models.R.
Fig. S2.
Flow diagram of the global model to represent each of 37 cited models as a submodel. Compartments in the global model are described as below. A full description of model parameters is in Dataset S1. Links to code to run all of the models are available in SI Text: Parameters.R, Functions.R, and Running models.R. Ec, exposed (infected but not yet infectious) individuals in the community; Ecd, exposed (infected and detectable but not yet infectious) individuals in the community; Ehw, exposed healthcare workers in hospital; Erh, exposed healthcare workers in the community; Ev, hospital-visiting exposed individuals; Fc, infectious (dead) individuals in the funeral compartment from the community; Fh, patients who died in the hospital entering the funeral compartment; Ic1, infectious individuals in the early infectious compartment in the community; Ic2, infectious individuals in the late infectious compartment in the community; Iccc1, infectious individuals at the early infectious stage in Ebola community care centers; Iccc2, infectious individuals at the late infectious stage in Ebola community care centers; Iccc2, Ebola virus-negative patients with Ebola-like symptoms in Ebola community care centers; Ih1, infectious individuals at the early infectious stage in the hospital; Ih2, infectious individuals at the late infectious stage in the hospital; Ih2, Ebola virus-negative patients with Ebola-like symptoms in the hospital; Ihw1, infectious healthcare workers in hospital; Irh1, infectious healthcare workers in the community; Iv1, hospital-visiting infectious individuals at the early infectious stage; R, individuals removed from the chain of transmission through either recovery or burial; Rccc, recovered individuals in Ebola community care centers; Rh, recovered individuals in the hospital; Sc, susceptible individuals in the community; Shw, susceptible healthcare workers in the hospital; Srh, susceptible healthcare workers in the community; Sv, hospital-visiting susceptible individuals.

Management Actions.

By surveying the literature, we selected five interventions that were broadly applied to control Ebola outbreaks: reducing community transmission, increasing hospitalization, reducing hospital transmission, reducing case fatality ratio, and reducing funeral transmission (2, 3, 8). Reducing community transmission (i.e., transmission in the community) is a general intervention that is achieved in a variety of ways, such as by providing household sanitation kits, improving contact tracing, improving self-quarantine of sick individuals in the community, reducing individual mobility and border crossing, and increasing community awareness through educational campaigns (2, 3, 8). Hospitalization increase can be realized by improving contact tracing and intensifying campaigns to identify and isolate patients, building more Ebola Treatment Centers, increasing the number of beds, and increasing necessary supplies and public support (3, 9). A reduction in hospital transmission can be achieved by encouraging the use of personal protective equipment for healthcare personnel treating infected cases and reducing hospital visits (8). Hydration of infected individuals has proved to be an important way to reduce mortality of Ebola cases, and various other new pharmaceutical approaches are being explored for future outbreaks. Funeral transmission (i.e., transmission at funerals) reduction can be achieved through improved funeral practices to increase safe burial by reducing risky behavior (2, 8).

Intervention Implementation.

Interventions can be modeled by changing the parameters thought to be influenced by the corresponding management actions. A reduction in transmission in the community, in hospitals, or at funerals can be modeled by reducing the transmission coefficients associated with these classes. Increasing hospitalization can be simulated by increasing the hospitalization ratio, and reducing mortality can be modeled by decreasing the case fatality ratio. Each intervention was assessed in each model in terms of the objective to minimize the Ebola caseload; for each model, we projected caseload under five interventions and then ranked the interventions from the best (lowest caseload) to the worst (highest caseload). All five interventions can be explicitly implemented in the full SEIHFR models; for the other models with some compartments unspecified, the corresponding interventions need to be simulated implicitly. For example, reducing hospital transmission can only be explicitly applied to the SEIHFR and SEIHR models, in which the H compartment is explicitly represented, but cannot be applied in the SEIFR or SEIR models, where the H compartment is unspecified. To be able to evaluate management options and conduct VoI analysis across all of the published models, including those that do not explicitly represent all infectious compartments, we calculated the implicit effect of an intervention via the average proportional contribution of the target transmission to the overall transmission based on the full SEIHFR models. For example, a proportional reduction of ΔH in hospital transmission can be explicitly simulated by multiplying the coefficient of hospital transmission by 1 − ΔH in the SEIHFR and SEIHR models. However, for the SEIFR or SEIR models where the H compartment is unspecified, an implicit simulation method needs to be applied. If PH is the proportional contribution of hospital transmission to the overall transmission, then a reduction of ΔH in hospital transmission can be implicitly simulated via multiplying the coefficient of overall transmission by the factor 1 − ΔHPH. A detailed description of the implicit simulation of the other interventions is provided in SI Text. The parameters used for the implicit management simulations were all based on the mean of the parameters across all of the published SEIHFR full models as shown in Table S3.
Table S3.
Parameters that were affected by five interventions, including reducing community transmission, reducing hospital transmission, reducing funeral transmission, reducing case fatality ratio, and increasing proportion of hospitalized cases
ParameterBiological meaningParameters affected by the interventionsMean value of parameters in full models
com.tra.hos.tra.fun.traFatalityHospitalization
βCTransmission coefficient in the communityYes    0.24
βHTransmission coefficient in the hospital Yes   0.10
βFTransmission coefficient at funerals  Yes  0.53
θProportion of hospitalized cases    Yes0.67
δCCase fatality ratio in the community   Yes 0.63
δHCase fatality ratio in the hospital   Yes 0.55
WCWeights of βC   YesYes0.45
WHWeights of βH   YesYes0.25
WFWeights of βF   YesYes0.29
PCContribution of βC to βCHFYesYesYesYesYes0.38
PHContribution of βH to βCHFYesYesYesYesYes0.09
PFContribution of βF to βCHFYesYesYesYesYes0.53
WCH_CWeights of βC in βCH    Yes0.64
WCH_HWeights of βH in βCH    Yes0.36
PCH_CContribution of βC to βCHYesYes  Yes0.81
PCH_HContribution of βH to βCHYesYes  Yes0.19
DCContribution of δC to δCH   YesYes0.51
DHContribution of δH to δCH   YesYes0.49
WCF_CWeights of βC in βCF   YesYes0.61
WCF_FWeights of βF in βCF   YesYes0.39
PCF_CContribution of βC to βCFYes YesYesYes0.24
PCF_FContribution of βF to βCFYes YesYesYes0.53
com.tra., Reducing community transmission; fatality, reducing case fatality ratio; fun.tra., reducing funeral transmission; hospitalization, increasing proportion of hospitalized cases; hos.tra., reducing hospital transmission.
The effect of an intervention, which we define as the percentage change compared with baseline, may not be known a priori. For illustration, we first projected caseload considering a 30% change for each of five interventions. Based on the caseload projection, we evaluated five interventions as well as the outcomes expected without any intervention for each of 37 models. We ranked them from the best (lowest caseload) to the worst (highest caseload). To illustrate how to identify at what effect size a particular intervention shifts to be best, we also compared caseload projections under a particular intervention over a gradient of changes ranging from 10 to 100% (with an interval of 10%) with the projections under the rest of the four interventions with the baseline change of 30%. For example, to assess the intervention of reducing community transmission, we did caseload projections by reducing community transmission from 10 to 100% for all 37 models, compared each projection under each level of change against the other four interventions with a change of 30%, and then ranked the intervention.
We implemented stochastic simulations for all models using Gillespie’s algorithm (7) with a tau-leaping approximation (3) to capture the random nature of epidemic birth and death processes (23). We performed 100 stochastic simulations for each management intervention–model combination, with 10 initial infectious individuals in a population of 10,000 individuals. When the parameters in the original publication were time-dependent, we fixed baseline parameters at values used at the start of the epidemic and/or set them to the no intervention baseline. To be able to conduct this broad comparison across all models, hospitalization capacity was not modeled, because only a few models considered this factor. Links to code to assess all interventions in all of the models in this study are given in SI Text: Parameters.R, Functions.R, and Running models.R. Additionally, to examine how sensitive R0 is to parameters associated with five interventions, we conducted an elasticity (proportional sensitivity) analysis (Table S2). We calculated R0 using the next generation framework and estimated derivatives numerically by the method of difference. We limited this analysis to the full SEIHFR models, in which all associated parameters were explicitly represented, thus allowing a perturbation of the full set of associated parameters. We then compared the rank of the elasticity of the parameters with the rank of the associated interventions.

VoI Analysis.

We calculated EVPI, which quantifies the maximum achievable improvement in management that could be obtained by resolving uncertainties before the implementation of specific decisions (10, 11). It is quantified as
EVPI=j=1npj(optaCa,j)optaj=1npjCa,j,
[1]
where n is the total number of models, Ca,j represents management performance (i.e., caseload in this study) associated with taking intervention a under model j, pj is the weight associated with model j (i.e., the belief that model j is the true model; subject to the constraint that the pj sum to one), and opta indicates the optimum over all interventions (10, 11). In this initial analysis, we weighed the models equally.
EVPI describes the benefit of resolving all sources of uncertainty. In practice, because of limited time and resources, it may not be possible to collect all of the required information. In this case, it is more realistic to prioritize a subset of uncertainties to resolve to maximize management improvement. Analysis of the EVPXI quantifies how much management performance could be improved by resolving a subset of uncertainties and therefore provides a useful tool to identify which subset of uncertainties should be given priority if time and resources are limited (10). EVPXI is calculated as
EVPXI=i=1qoptajsipjCa,joptaj=1npjCa,j,
[2]
where n models are grouped into i = 1 … q mutually exclusive and exhaustive sets, set si has ni models in it, pj is the weight associated with model j, and Ca,j is the management performance under model j and intervention a. Therefore, EVPXI quantifies the improvement in management performance by resolving the uncertainty associated with si (10). We conducted EVPXI analyses for four subsets of models with different types of structures (SEIHFR, SEIHR, SEIFR, and SEIR) and also, four subsets of models with different ranges of caseload projections (<2,500, 2,500–5,000, 5,000–7,500, and >7,500 cases) to evaluate the improvement in management by resolving uncertainties in model structure and the range of caseload projections.

SI Text

Implicit Simulation of Interventions.

Our goal here is to provide a comprehensive EVPI analysis of all 37 compartmental models published for Ebola outbreaks. Because not all models have all compartments relevant to all proposed interventions, we were faced with some parameterization choices. Here, we outline our method for intervention simulation by defining a set of weights based on the mean of the parameters of all of the published full SEIHFR models to implicitly represent the contribution of unspecified compartments to the overall transmission in other models.
In a full SEIHFR model, there are three transmission compartments, which are community (C), hospital (H), and funeral (F). The relative contribution of each transmission compartment to the overall transmission coefficients (βCHF) depends on the absolute magnitude of their transmission coefficients (βC, βH, and βF) and their weights (WC, WH, and WF), which reflect the fraction of infectious individuals in each compartment. Therefore,
βCHF=WCβC+WHβH+WFβF,
[S1]
and the weights that may be time varying are
WC=11+θ+(1θ)δC+θδH,
[S2]
WH=θ1+θ+(1θ)δC+θδH,
[S3]
and
WF=(1θ)δC+θδH1+θ+(1θ)δC+θδH,
[S4]
where θ is the proportion of infectious cases that are hospitalized. Parameters δC and δH are case fatality ratios in the community and hospital, respectively.
The proportional contribution of each transmission compartment to the overall transmission βCHF can be written as
PC=WCβCβCHF,
[S5]
PH=WHβHβCHF,
[S6]
and
PF=WFβFβCHF.
[S7]
Correspondingly, we write βC, βH, and βF as
βC=PCβCHFWC,
[S8]
βH=PHβCHFWH,
[S9]
and
βF=PFβCHFWF.
[S10]
In SEIR models, transmission in heterogeneous compartments, such as H and F, is implicit, and an overall transmission coefficient βCHF is used to implicitly reflect the combined contributions of βC, βH, and βF. The choice to reduce ΔC in community transmission can be reflected in βCHF by reducing the contribution of βC based on the mean of the full SEIHFR models. The new effective transmission β′CHF is
βCHF=(1PC)βCHF+(1ΔC)PCβCHF.
[S11]
Similarly, the management option of reducing hospital transmission by ΔH can be achieved by recalculating β′CHF as
βCHF=(1PH)βCHF+(1ΔH)PHβCHF,
[S12]
and the management option of reducing funeral transmission by ΔF can be quantified by recalculating β′CHF as
βCHF=(1PF)βCHF+(1ΔF)PFβCHF.
[S13]
Because the H compartment is unspecified in an SEIR model, we represent the management option of increasing the proportion of hospitalized cases θ by Δh in terms of a change in βCHF, as a function of θ, and thus, as a change in the weight of hospital transmission WH. Increasing the proportion of hospitalized cases will also consequently lead to a shift in the number of people who would recover/die in the community who instead recover/die in the hospital. Given that the case fatality ratio might differ between the community and hospital settings (8, 24), changing the proportion of hospitalized cases also consequently changes the weight of funeral transmission (WF) and the weight of community transmission (WC), both of which depend on the proportion of hospitalized cases and case fatality ratio as given in Eqs. S2 and S4. We included this to address the possibility that differences in case fatality ratio were important. However, the existing differences in case fatality ratio in this study are small. The new weights of W′C, W′H, and W′F under the intervention of increasing the proportion of hospitalized cases can be calculated by replacing θ with (1 + Δh)θ based on Eqs. S2S4. Thus, β′CHF can be calculated based on Eqs. S1 and S8S10 according to
βCHF=WCβC+WHβH+WFβF
=WCPCβCHFWC+WHPHβCHFWH+WFPFβCHFWF.
[S14]
Following the same logic as for hospitalization interventions, the management option of reducing the case fatality ratio in the community (δC) and in hospitals (δH) both by ΔD in the SEIR model is done by a change in βCHF occurring as a function of δC and δH and changes in WC, WH, and WF. The new weights of W′C, W′H, and W′F under the intervention of reducing the case fatality ratio can be calculated by replacing δC and δH with (1 − ΔD)δC and (1 − ΔD)δH based on Eqs. S2S4. Thus, β′CHF can be calculated based on Eqs. S1 and S8S10 according to
βCHF=WCPCβCHFWC+WHPHβCHFWH+WFPFβCHFWF.
[S15]
In an SEIFR model, the funeral transmission coefficient βF is represented explicitly via the compartment F, whereas the compartments C and H are unspecified, and the transmission coefficients βC and βH are implicitly represented by a combined transmission coefficient βCH. The weights of βC (written as WCH_C) and βH (written as WCH_H) can be calculated based on the full SEIHFR models as
WCH_C=WCWC+WH=11+θ
[S16]
and
WCH_H=WHWC+WH=θ1+θ.
[S17]
The overall transmission βCH can be written as
βCH=WCH_CβC+WCH_HβH.
[S18]
Therefore, the proportional contributions of community and hospital transmission are
PCH_C=WCH_CβCβCH
[S19]
and
PCH_H=WCH_HβHβCH,
[S20]
and βC and βH can be written as
βC=PCH_CβCHWCH_C
[S21]
and
βH=PCH_HβCHWCH_H.
[S22]
The intervention of a reduction of ΔC in community transmission can be achieved via a change in βCH by reducing the contribution of βC based on the full SEIHFR models, for which β′CH is
βCH=(1PCH_C)βCH+(1ΔC)PCH_CβCH.
[S23]
Similarly, the management option of reducing hospital transmission by ΔH involves a change in βCH by reducing the contribution of βH. Thus, β′CH is given by
βCH=(1PCH_H)βCH+(1ΔH)PCH_HβCH.
[S24]
The intervention of reducing funeral transmission by ΔF is direct for the SEIFR model, with βF as
βF=(1ΔF)βF.
[S25]
Because the H compartment in an SEIFR model is unspecified, the intervention of increasing the hospitalization proportion θ by Δh involves a change in βCH, as a function of the proportion of θ, through a change in W′CH_C and WCH_H. W′CH_C and WCH_H are calculated based on Eqs. S16 and S17 by replacing θ with (1 + Δh)θ. β′CH can be calculated based on Eqs. S18S22 according to
βCH=WCH_CβC+WCH_HβH
=WCH_CPCH_CβCHWCH_C+WCH_HPCH_HβCHWCH_H.
[S26]
Because the case fatality ratio in hospitals may differ from that in the community, changing hospitalization may result in a change in the overall case fatality ratio. Given that the H compartment is unspecified in an SEIFR model, δC and δH are implicitly represented by an overall case fatality ratio δCH. Based on the full SEIHFR models, δCH can be written as
δCH=(1θ)δC+θδH,
[S27]
where the contributions of δC and δH to δCH are
DC=(1θ)δCδCH
[S28]
and
DH=θδHδCH.
[S29]
Accordingly, δC and δH can be written as
δC=DCδCH1θ
[S30]
and
δH=DHδCHθ.
[S31]
The new case fatality ratio δ′CH under the management option of increasing hospitalization is calculated as
δCH=[1(1+Δh)θ]δC+(1+Δh)θδH
=[1(1+Δh)θ]DCδCH1θ+(1+Δh)θDHδCHθ.
[S32]
The intervention of reducing case fatality ratio in the community (δC) and in the hospital (δH) both by ΔD can be directly modeled by multiply δCH by (1 − ΔD).
In an SEIHR model, whereas the hospital transmission coefficient βH is represented explicitly via the compartment H, the transmission compartments of C and F are unspecified, and the transmission coefficients βC and βF are implicitly represented by the compound transmission coefficient βCF. The weights of βC (WCF_C) and βF (WCF_F) are calculated based on the full SEIHFR models as
WCF_C=WCWC+WF=11+(1θ)δC+θδH
[S33]
and
WCF_F=WFWC+WF=(1θ)δC+θδH1+(1θ)δC+θδH.
[S34]
The transmission coefficient βCF is
βCF=WCF_CβC+WCF_FβF.
[S35]
Therefore, the proportional contributions of βC and βF to the transmission βCF can be written as
PCF_C=WCF_CβCβCF
[S36]
and
PCF_F=WCF_FβFβCF.
[S37]
Correspondingly, βC and βF can be written as
βC=PCF_CβCFWCF_C
[S38]
and
βF=PCF_FβCFWCF_F.
[S39]
The intervention of a reduction of ΔC in community transmission can be captured by a change in βCF by reducing the contribution of βC. Therefore, we calculate β′CF as
βCF=(1PCF_C)βCF+(1ΔC)PCF_CβCF.
[S40]
The management option of reducing funeral transmission by ΔF is captured through a change in βCF by reducing the contribution of βF. Therefore, β′CF is calculated as
βCF=(1PCF_F)βCF+(1ΔF)PCF_FβCF.
[S41]
The intervention of reducing hospital transmission by ΔH is direct in an SEIHR model, and the β′H is
βH'=(1ΔH)βH.
[S42]
The direct effect of the intervention of increasing the hospitalization proportion by ΔH is explicitly modeled by replacing θ with (1 + Δh)θ in the SEIHR model. The change in the case fatality ratio cannot be directly represented in the SEIHR model given that the funeral compartment is unspecified. It can only be reflected through altering βCH by changing the weights in community and funeral transmissions. The WCF_C and WCF_F can be calculated based on Eqs. S33 and S34; β′CF can be calculated based on Eqs. S35S39 according to
βCF=WCF_CβCF+WCF_FβCF
=WCF_CPCF_CβCFWCF_C+WCF_FPCF_FβCFWCF_F.
[S43]
Following the same logic, the intervention of reducing case fatality ratio in the community (δC) and hospital (δH) both by ΔD is achieved implicitly through a change in βCH by altering the weights of community (W′CF_C) and funeral (W′CF_F) transmission via replacing δC and δH with (1 − ΔD)δC and (1 − ΔD)δH based on Eqs. S33 and S34. The β′CH can be calculated based on Eqs. S35S39 and S43.

The R Code to Run the Analyses Described Here Is Provided in the Following Links.

Acknowledgments

We thank Amalie McKee, Chris Baker, and Brian Lambert for help and comments. We acknowledge funding from National Science Foundation Ebola Research and Policy for Infectious Disease Dynamics (RAPIDD) Grant DMS-1514704, NIH Ecology and Evolution of Infectious Diseases (EEID) Award 1 R01 GM105247-01, and Biotechnology and Biological Sciences Research Council (BBSRC) Grant BB/K010972/4. Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the US Government.

Supporting Information

Supporting Information (PDF)
Dataset_S01 (XLSX)

References

1
MEJ Woolhouse, A Rambaut, P Kellam, Lessons from Ebola: Improving infectious disease surveillance to inform outbreak management. Sci Transl Med 7, 307rv5 (2015).
2
A Pandey, et al., Strategies for containing Ebola in West Africa. Science 346, 991–995 (2014).
3
CM Rivers, ET Lofgren, M Marathe, S Eubank, BL Lewis, Modeling the impact of interventions on an epidemic of Ebola in Sierra Leone and Liberia. PLoS Curr, November 6, 2014).
4
J-P Chretien, S Riley, DB George, Mathematical modeling of the West Africa Ebola epidemic. eLife 4, e09186 (2015).
5
D Butler, Models overestimate Ebola cases. Nature 515, 18 (2014).
6
C Rivers, Ebola: Models do more than forecast. Nature 515, 492 (2014).
7
DT Gillespie, Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys 115, 1716–1733 (2001).
8
FB Agusto, MI Teboh-Ewungkem, AB Gumel, Mathematical assessment of the effect of traditional beliefs and customs on the transmission dynamics of the 2014 Ebola outbreaks. BMC Med 13, 96 (2015).
9
J Legrand, RF Grais, PY Boelle, AJ Valleron, A Flahault, Understanding the dynamics of Ebola epidemics. Epidemiol Infect 135, 610–621 (2007).
10
MC Runge, SJ Converse, JE Lyons, Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program. Biol Conserv 144, 1214–1223 (2011).
11
K Shea, MJ Tildesley, MC Runge, CJ Fonnesbeck, MJ Ferrari, Adaptive management and the value of information: Learning via intervention in epidemiology. PLoS Biol 12, e1001970 (2014).
12
RM Anderson, RM May Infectious Diseases of Humans: Dynamics and Control (Oxford Univ Press, Oxford, 1991).
13
A Camacho, et al., Potential for large outbreaks of Ebola virus disease. Epidemics 9, 70–78 (2014).
14
P Klepac, R Laxminarayan, BT Grenfell, Synthesizing epidemiological and economic optima for control of immunizing infections. Proc Natl Acad Sci USA 108, 14366–14370 (2011).
15
P Klepac, ON Bjørnstad, CJE Metcalf, BT Grenfell, Optimizing reactive responses to outbreaks of immunizing infections: Balancing case management and vaccination. PLoS One 7, e41428 (2012).
16
S Funk, GM Knight, VAA Jansen, Ebola: The power of behaviour change. Nature 515, 492 (2014).
17
WJ Probert, et al., Decision-making for foot-and-mouth disease control: Objectives matter. Epidemics 15, 10–19 (2016).
18
T Lindström, M Tildesley, C Webb, A Bayesian ensemble approach for epidemiological projections. PLOS Comput Biol 11, e1004187 (2015).
19
JM Drake, et al., Transmission models of historical Ebola outbreaks. Emerg Infect Dis 21, 1447–1450 (2015).
20
JP D’Silva, MC Eisenberg, Modeling spatial invasion of Ebola in West Africa. arXiv:150708367. (2015).
21
; R Development Core Team, The R Project for Statistical Computing. Available at www.r-project.org/. Accessed May 1, 2017. (2015).
22
O Diekmann, JAP Heesterbeek, JAJ Metz, On the definition and the computation of the basic reproduction ratio R0 in models for infectious diseases in heterogeneous populations. J Math Biol 28, 365–382 (1990).
23
NTJ Bailey The Mathematical Theory of Epidemics (Hafner Publ Co, New York, 1957).
24
MC Eisenberg, et al., Forecasting and Uncertainty in Modeling the 2014-2015 Ebola Epidemic in West Africa. arXiv:150105555. (2015).
25
MFC Gomes, et al., Assessing the international spreading risk associated with the 2014 west african ebola outbreak. PLoS Curr, September 2, 1014).
26
D Chowell, C Castillo-Chavez, S Krishna, X Qiu, KS Anderson, Modelling the effect of early detection of Ebola. Lancet Infect Dis 15, 148–149 (2015).
27
FO Fasina, et al., Transmission dynamics and control of Ebola virus disease outbreak in Nigeria, July to September 2014. Euro Surveill 19, 20920 (2014).
28
A Khan, M Naveed, M Dur-E-Ahmad, M Imran, Estimating the basic reproductive ratio for the Ebola outbreak in Liberia and Sierra Leone. Infect Dis Poverty 4, 13 (2015).
29
AJ Kucharski, et al., Evaluation of the benefits and risks of introducing Ebola community care centers, Sierra Leone. Emerg Infect Dis 21, 393–399 (2015).
30
JS Weitz, J Dushoff, Modeling post-death transmission of Ebola: Challenges for inference and opportunities for control. Sci Rep 5, 8751 (2015).
31
CL Althaus, Estimating the reproduction number of Ebola virus (EBOV) during the 2014 outbreak in West Africa. PLoS Curr 6, 1–5 (2014).
32
S Bashar, M Percy, R Singhai, Predicting the 2014 Ebola Outbreak in West Africa Using Network Analysis. Final Report:1–10. Available at snap.stanford.edu/class/cs224w-2014/projects2014/cs224w-45-final.pdf. Accessed August 4, 2016. (2014).
33
G Chowell, NW Hengartner, C Castillo-Chavez, PW Fenimore, JM Hyman, The basic reproductive number of Ebola and the effects of public health measures: The cases of Congo and Uganda. J Theor Biol 229, 119–126 (2004).
34
PE Lekone, BF Finkenstädt, Statistical inference in a stochastic epidemic SEIR model with control intervention: Ebola as a case study. Biometrics 62, 1170–1177 (2006).
35
MI Meltzer, et al., Estimating the future number of cases in the Ebola epidemic–Liberia and Sierra Leone, 2014-2015. MMWR Suppl 63, 1–14 (2014).
36
MJ Ferrari, ON Bjørnstad, AP Dobson, Estimation and inference of R0 of an infectious pathogen by a removal method. Math Biosci 198, 14–26 (2005).
37
J Shaman, W Yang, S Kandula, Inference and forecast of the current West African Ebola outbreak in Guinea, Sierra Leone and Liberia. PLoS Curr, October 31, 2014).

Information & Authors

Information

Published in

Go to Proceedings of the National Academy of Sciences
Proceedings of the National Academy of Sciences
Vol. 114 | No. 22
May 30, 2017
PubMed: 28507121

Classifications

Submission history

Published online: May 15, 2017
Published in issue: May 30, 2017

Keywords

  1. value of information
  2. VoI
  3. epidemiological outbreak management
  4. decision making

Acknowledgments

We thank Amalie McKee, Chris Baker, and Brian Lambert for help and comments. We acknowledge funding from National Science Foundation Ebola Research and Policy for Infectious Disease Dynamics (RAPIDD) Grant DMS-1514704, NIH Ecology and Evolution of Infectious Diseases (EEID) Award 1 R01 GM105247-01, and Biotechnology and Biological Sciences Research Council (BBSRC) Grant BB/K010972/4. Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the US Government.

Notes

This article is a PNAS Direct Submission.

Authors

Affiliations

Department of Biology, The Pennsylvania State University, University Park, PA 16802;
Center for Infectious Disease Dynamics, The Pennsylvania State University, University Park, PA 16802;
Ottar N. Bjørnstad
Department of Biology, The Pennsylvania State University, University Park, PA 16802;
Center for Infectious Disease Dynamics, The Pennsylvania State University, University Park, PA 16802;
Matthew J. Ferrari
Department of Biology, The Pennsylvania State University, University Park, PA 16802;
Center for Infectious Disease Dynamics, The Pennsylvania State University, University Park, PA 16802;
Riley Mummah
Department of Biology, The Pennsylvania State University, University Park, PA 16802;
Center for Infectious Disease Dynamics, The Pennsylvania State University, University Park, PA 16802;
Patuxent Wildlife Research Center, US Geological Survey, Laurel, MD 20708;
Christopher J. Fonnesbeck
Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, TN 37203;
Michael J. Tildesley
Zeeman Institute for Systems Biology & Infectious Disease Epidemiology Research, School of Life Sciences and Mathematics Institute, University of Warwick, Coventry CV4 7AL, United Kingdom
William J. M. Probert
Zeeman Institute for Systems Biology & Infectious Disease Epidemiology Research, School of Life Sciences and Mathematics Institute, University of Warwick, Coventry CV4 7AL, United Kingdom
Katriona Shea1 [email protected]
Department of Biology, The Pennsylvania State University, University Park, PA 16802;
Center for Infectious Disease Dynamics, The Pennsylvania State University, University Park, PA 16802;

Notes

1
To whom correspondence may be addressed. Email: [email protected] or [email protected].
Author contributions: S.-L.L., O.N.B., M.J.F., R.M., M.C.R., C.J.F., M.J.T., W.J.M.P., and K.S. designed research; S.-L.L., O.N.B., M.J.F., and K.S. performed research; S.-L.L., O.N.B., M.J.F., and K.S. analyzed data; and S.-L.L., O.N.B., M.J.F., R.M., M.C.R., C.J.F., M.J.T., W.J.M.P., and K.S. wrote the paper.

Competing Interests

The authors declare no conflict of interest.

Metrics & Citations

Metrics

Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.


Citation statements




Altmetrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited by

    Loading...

    View Options

    View options

    PDF format

    Download this article as a PDF file

    DOWNLOAD PDF

    Get Access

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Personal login Institutional Login

    Recommend to a librarian

    Recommend PNAS to a Librarian

    Purchase options

    Purchase this article to access the full text.

    Single Article Purchase

    Essential information: Uncertainty and optimal control of Ebola outbreaks
    Proceedings of the National Academy of Sciences
    • Vol. 114
    • No. 22
    • pp. 5551-E4523

    Media

    Figures

    Tables

    Other

    Share

    Share

    Share article link

    Share on social media