Content uploaded by Joseph R Bennett
Author content
All content in this area was uploaded by Joseph R Bennett on Feb 19, 2018
Content may be subject to copyright.
1
When to monitor and when to act: value of information theory for multiple
management units and limited budgets
Joseph R. Bennett
1*
, Sean L. Maxwell
2
, Amanda E. Martin
1
, Iadine Chadès
3,4
, Lenore Fahrig
1
,
Benjamin Gilbert
5
1
Department of Biology, Carleton University, 1125 Colonel By Drive, Ottawa Ontario, K1S
5B6, Canada
2
School of Geography, Planning and Environmental Management, University of Queensland,
Brisbane, QLD 4072, Australia
3
CSIRO, Brisbane, QLD 4001, Australia
4
ARC Centre of Excellence for Environmental Decisions, University of Queensland, Brisbane,
QLD 4072, Australia
5
Department of Ecology & Evolutionary Biology, University of Toronto, Toronto, Ontario,
Canada M5S 3B2
*Author for correspondence: joseph.bennett@carleton.ca; ph: +1-613-520-2600 x3124
Running Title: Multi-unit value of information theory
Word count:
Summary – 267
Main paper – 6164
Acknowledgements – 17
References - 1374
Tables and Figure legends – 584
Number of tables and figures: 1 figure, 7 tables
Number of references: 55
Page 1 of 56 Journal of Applied Ecology
2
Abstract 1
1. The question of when to monitor and when to act is fundamental to applied ecology, and 2
notoriously difficult to answer. Value of information (VOI) theory holds great promise to 3
help answer this question for many management problems. However, VOI theory in 4
applied ecology has only been demonstrated in single-decision problems, and has lacked 5
explicit links between monitoring and management costs. 6
2. Here, we present an extension of VOI theory for solving multi-unit decisions of whether 7
to monitor before managing, while explicitly accounting for monitoring costs. Our 8
formulation helps to choose the optimal monitoring/management strategy among groups 9
of management units (e.g. species, habitat patches), and can be used to examine the 10
benefits of partial and repeat monitoring. 11
3. To demonstrate our approach, we use case simulated studies of single-species protection 12
that must choose among potential habitat areas, and classification and management of 13
multiple species threatened with extinction. We provide spreadsheets and code to 14
illustrate the calculations and facilitate application. Our case studies demonstrate the 15
utility of predicting the number of units with a given outcome for problems with 16
probabilities of discrete states, and the efficiency of having a flexible approach to manage 17
according to monitoring outcomes. 18
4. Synthesis and applications. The decision to act or gather more information can have 19
serious consequences for management. No decision, including the decision to monitor, is 20
risk-free. Our multi-unit expansion of Value of Information (VOI) theory can reduce the 21
risk in monitoring/acting decisions for many applied ecology problems. While our 22
approach cannot account for the potential value of discovering previously unknown 23
threats or ecological processes via monitoring programs, it can provide quantitative 24
Page 2 of 56Journal of Applied Ecology
3
guidance on whether to monitor before acting, and which monitoring/management 25
actions are most likely to meet management objectives. 26
Keywords: Value of Information, VOI, decision theory, monitoring, optimization, multiple 27
management units, threatened species, habitat protection, management strategy 28
Page 3 of 56 Journal of Applied Ecology
4
Introduction 29
The question of how much information is needed to inform management is central to applied 30
ecology. Information from monitoring is often vital to effective decision making. Unfortunately, 31
many management decisions are made based on inadequate information (Sutherland et al. 2004), 32
which can lead to inefficient or counter-productive choices (Cook et al. 2010). Monitoring, 33
particularly if it is conducted over long time periods, has also led to the discovery of important 34
environmental stressors (e.g. Wintle et al. 2010 Lindenmayer et al. 2012). 35
However, monitoring costs money and time, both of which can be in short supply. Monitoring 36
instead of managing can also be a way of avoiding difficult but necessary decisions (Nichols and 37
Williams 2006). Indeed, there are cases of threatened species being monitored continuously until 38
they are extinct (Martin et al. 2012; Lindenmayer et al. 2013). Information gathering, beyond 39
what is necessary to make an effective decision, risks dissipating resources that could have been 40
used for management, and missing critical windows of opportunity (Chadès et al. 2008; 41
McDonald-Madden et al. 2010; Martin et al. 2012). 42
Value of information (VOI) theory provides an important tool to choose monitoring strategies. 43
By explicitly modelling the value gained by monitoring, VOI theory can be used to determine 44
whether additional information would be useful for a specific management question. The utility 45
of VOI theory has been demonstrated for invasive species management (Hauser et al. 2009; 46
Moore and Runge 2012), disease control (Shea et al. 2014), threatened species management 47
(Runge et al. 2011; Williams and Johnson 2015; Maxwell et al. 2015; Canessa et al. 2015), and 48
conservation reserve selection (Runting et al. 2013; Mazor et al. 2016). These applications of 49
Page 4 of 56Journal of Applied Ecology
5
VOI suggest that decisions involving a combination of monitoring and management should first 50
assess the value of monitoring strategies. 51
Despite the applicability of VOI to decision making, there are two key limitations of 52
conventional VOI used in applied ecology. The first limitation is that VOI theory has not been 53
fully developed for multi-unit management, which occurs when managing multiple species or 54
multiple habitat patches. In the decision theory literature, multi-unit problems have recently been 55
explored (Keisler 2004; Brickel and Zan 2009; Zan and Brickel 2013), but with either strict 56
assumptions of identical unit costs, or simulation to examine the influence of relaxed 57
assumptions, rather than generalized theory that is broadly applicable. In applied ecology, VOI 58
theory has only been presented for problems involving decisions for single management units or 59
single decisions applied to grouped sets of management units (e.g. all species or habitat patches 60
in a study), rather than individual decisions among management units (e.g. which species to 61
monitor versus manage, which habitats to monitor versus manage). This limitation makes VOI 62
difficult to implement for many real-world problems in applied ecology. 63
The second limitation is that when monitoring results and financial or time costs are not 64
explicitly related, VOI cannot directly answer the question of when to monitor and when to act. 65
For example, a VOI exercise may suggest value in monitoring, but if the monitoring is expensive 66
or the timeframe for successful management is limited, this value may be diminished or negated. 67
This limitation is closely related to the first limitation, because budgets are often insufficient to 68
manage all units and must be carefully allocated among them (e.g. Wilson et al. 2009; Joseph et 69
al. 2009). 70
Recently, several authors have attempted to overcome the challenge of explicitly considering 71
monitoring costs in VOI analysis for problems in applied ecology. Maxwell et al. (2015) 72
Page 5 of 56 Journal of Applied Ecology
6
calculated the financial value of perfect information for a conservation problem by estimating the 73
cost of optimal population management with current information and with uncertainties resolved. 74
Mazor et al. (2016) used systematic reserve selection software to infer that intensive monitoring 75
information provided better reserve selection outcomes than more extensive information. Shea et 76
al. (2014) used financial value of information to infer whether a monitoring strategy would be 77
worthwhile. Although these approaches provide important advances for quantifying the value of 78
monitoring, they did not explicitly incorporate monitoring costs, nor determine the trade-off 79
between monitoring and management costs. 80
Here, we present an extension of VOI theory for solving problems of monitoring versus action 81
across multiple management units, while explicitly considering the cost of monitoring. In doing 82
so, we link VOI theory with decision theory for optimal management. We also show that where 83
decisions for management units are based on probabilities of discrete states (e.g. “is species X 84
present in unit Y”), we can calculate the expected number of units with a given state and make 85
decisions to monitor or act more efficiently than using aggregated single-decision VOI 86
calculations. The theory presented is general, and we show how it can be used to understand risk 87
in decision making when there is uncertainty in data. Using these calculations, we can explore a 88
wide range of questions, including trade-offs between monitoring effort and cost, and the 89
benefits of partial or repeat monitoring. We demonstrate potential applications using two simple, 90
simulated case studies of threatened species conservation. 91
Materials and methods 92
Formulation 93
Page 6 of 56Journal of Applied Ecology
7
VOI analysis addresses the question of whether information to reduce uncertainty about a 94
problem is worth gathering. Across multiple management units, it can be used to address whether 95
additional information would be useful for more efficient management, given limited budgets 96
(Keisler 2004; Zan and Brickel 2009). To calculate VOI across multiple management units that 97
explicitly accounts for costs, we require estimates of the following (Table 1): 1) cost of 98
management actions; 2) prior probabilities for states of the units (these can be non-informative 99
priors); 3) expected values of our management actions (i.e., the estimated benefit of actions, 100
given possible states of the management units); 4) monitoring accuracy; and 5) monitoring cost. 101
Monitoring cost can be financial, and thus restrict management options, or can be incurred via 102
impacts on management goals, such as when delays due to monitoring reduce management 103
efficacy. 104
Estimates of these parameters can contain considerable uncertainty. However, such estimates are 105
commonly used in setting conservation priorities (e.g., Ball et al. 2009; McCarthy et al. 2010; 106
Joseph et al. 2009; Bennett et al. 2014), and indeed are implicit in every conservation decision 107
regarding monitoring versus acting. With estimates of these parameters, we can relate monitoring 108
and management costs, and use these to calculate trade-offs between monitoring first versus 109
acting on current information. As shown below, we can also explore plausible ranges for 110
parameter estimates, to determine the ranges of conditions in which we would decide to monitor 111
before acting. 112
Below, we briefly describe conventional single-decision VOI theory, using modified terminology 113
of Canessa et al. (2015), outlined in Table 1. We then present a VOI formulation for decisions 114
across multiple management units that accounts for both monitoring and management costs. We 115
Page 7 of 56 Journal of Applied Ecology
8
provide a full mathematical formulation in Appendix S1, and step-by-step walkthroughs for the 116
case studies in Appendices S2-S4. 117
VOI for a single management unit 118
According to VOI theory, one can quantify the expected value of a given action on a scale 119
compatible with the management objectives, e.g. the 50-year probability of extinction, or 120
population size. The value of the action depends on the true state of the management unit(s), e.g. 121
whether the species is present or absent in a given habitat patch. However, there is uncertainty 122
about which state the management unit is in. 123
The expected value of an action a
i
under uncertainty can be calculated as follows: 124
1. 125
This is the sum of all possible values for the action a
i
for all states s of the management unit, 126
with each value weighted by its respective probability of the state s being true. 127
The expected value of the best management action under uncertainty can be calculated as 128
follows: 129
. 2a. 130
This is the maximum expected value from Eq. 1 among all potential management actions. The 131
same equation can be formulated as a decision problem by introducing a binary decision variable 132
x
i
identifying whether action a
i
is implemented: 133
2b. 134
Page 8 of 56Journal of Applied Ecology
9
The standard formulation used herein assumes a single management action can be taken for a 135
unit
, and represents the expected value of the management action one would 136
logically take with current information. 137
Although absolute certainty rarely exists in environmental decision-making, the expected value 138
of the best management action under certainty, which is calculated as follows, is nonetheless 139
useful for comparing with the expected value of sampling or monitoring information: 140
3a. 141
This is the sum of all the best management actions for all the possible states of the management 142
unit, weighted by the probabilities of each state being true. This differs from the expected value 143
of the best management action under uncertainty in that it sums across all possible states (instead 144
of taking the single best management action with uncertainty). Thus, it is always equal to or 145
greater than the expected value of the best management action under uncertainty. 146
The expected value under certainty can also be formulated as a decision problem: 147
3b. 148
where the decision variable
identifies which action
to implement for each possible state s. 149
The difference between the expected value of the best management action under certainty and 150
uncertainty is termed the expected value of perfect information (EVPI), and is calculated as 151
follows: 152
4. 153
Page 9 of 56 Journal of Applied Ecology
10
Monitoring will typically improve upon current information, and increase the expected value of 154
the best management action. Specifically, monitoring will change our belief about the probability 155
of each state s being true. In VOI, probabilities for each state s being true are estimated for each 156
possible monitoring result using Bayes Theorem, i.e. probability(state s | result y) = 157
probability(result y | state s) × prior probability (state s) / probability(result y) (Raiffa 1968). 158
The expected value of the best management action when information from monitoring (y) is 159
incorporated is: 160
5a. 161
The expected value after monitoring information can also be formulated as a decision problem: 162
5b. 163
where
identifies the action a
i
to implement for each possible monitoring result y. This is the 164
expected value of the best management action for each monitoring result y, weighted by 165
probabilities of obtaining the monitoring results. 166
The expected value of monitoring information (frequently termed the expected value of sampling 167
information, EVSI, in the VOI literature), is the difference between the expected value of the 168
best management action after monitoring and the expected value of the best management action 169
under uncertainty (i.e., before monitoring): 170
6. 171
VOI across multiple management units with a limited budget 172
There are two key differences in calculation between VOI across multiple management units and 173
conventional VOI. The first is that VOI across multiple management units allocates decisions 174
Page 10 of 56Journal of Applied Ecology
11
among individual units, providing the flexibility to maximize our best overall values among units 175
by ranking and summing expected values for management decisions within a budget, as opposed 176
to choosing among general strategies presented as a single decision problem (Brickel and Zan 177
2009; Zan and Brickel 2013). This flexibility has important implications. For example, if we are 178
deciding whether to protect a single habitat patch, our optimal decision may be to protect even 179
after monitoring does not confirm presence of our species of interest (because there is a 180
diminished, but non-zero posterior probability it is present). But if we are deciding which of a set 181
of habitat patches to protect, we may only protect patches where we actually found the species. 182
The second difference from conventional VOI is that for problems involving probabilities of 183
discrete states, prior probabilities can be used to estimate the number (or fraction) of 184
management units with a given state, using the linearity of expectation property of random 185
variables, i.e. . This allows us to allocate management or monitoring 186
decisions among units to maximize the summed expected values among units, within a given 187
budget. If our budget was unlimited, we would calculate expected value of the best decisions 188
given current, perfect or monitoring information for all management actions and our expected 189
values would simply be the summed results of conventional VOI equations across all 190
management units. In the far more likely scenario of a limited budget, we can calculate the 191
expected value of decisions across management units and choose the optimal set of decisions that 192
maximizes the summed expected value, subject to our budget. 193
The expected value of the optimal group of management actions under uncertainty is as follows: 194
7. 195
Page 11 of 56 Journal of Applied Ecology
12
This is the maximum (subject to the budget B) of the summed expected values of management 196
actions among units under uncertainty (Eq. 1), where A
j
is the suite of available actions for unit j, 197
and
is a decision variable which identifies which action
to implement for a given unit j (see 198
Appendix S1 for details). 199
Equation 7 can be formulated as a knapsack problem, whereby a decision-maker with limited 200
capacity must choose among management options to maximize value. Knapsack problems have a 201
long history in optimization research, and many algorithms to solve them have been proposed 202
(see Martello and Toth 1990 and Kellerer et al. 2004 for detailed reviews). In applied ecology, 203
examples of the knapsack problem include setting priorities for managing species (e.g. Joseph et 204
al. 2009; Bennett et al. 2014), and managing threats to biodiversity (e.g. Carwardine et al. 2014). 205
Finding an exact solution to a knapsack problem can be challenging and require considerable 206
computer resources, especially when investments and returns are strongly correlated and the 207
number of potential actions and management units is large (Pisinger 2005). However, heuristics 208
can be used to find approximate solutions. One well-known approximation first proposed by 209
Dantzig (1957), is to rank potential actions among management units by the expected cost-210
effectiveness of management based on current knowledge (i.e., expected value of decision under 211
uncertainty / action cost), and choose units sequentially according to rank, with the goal of 212
managing the units with the greatest summed expected value. In conservation, this approach has 213
formed the basis of prioritization protocols for threatened species (e.g. Joseph et al. 2009; 214
Government of New South Wales 2013; Bennett et al. 2014), and an analogous technique has 215
been used for reserve selection (Moilanen 2007). Although this approach can be less efficient 216
than more complex approaches (Kellerer et al. 2004), particularly when costs of actions are large 217
compared to the overall budget, it is intuitive and easy to illustrate. We use it in our case studies, 218
Page 12 of 56Journal of Applied Ecology
13
but note that our methods are compatible with other techniques. We also note that individually 219
suboptimal actions may sometimes be optimal for multi-unit problems (see Appendix S1 for 220
details). In the simple case where the cost of management for all units is equal (e.g. if we are 221
trying to allocate equal-sized predator exclosures among habitats), costs can be normalized to 222
one, and the expected cost-effectiveness of management in a unit is simply the expected value of 223
the chosen management action under uncertainty for that unit. 224
Expected value of optimal group of management actions under certainty 225
Among multiple management units the expected value of the optimal group of management 226
actions under certainty can be calculated as follows: 227
8. 228
This is the maximum, subject to the budget B, of the summed probabilities of possible states s 229
for each unit, multiplied by the values of actions
among all units for each possible state. For 230
discrete probability distributions, we can use prior probabilities among all management units to 231
predict the expected number (or fraction) of units with a given state using the linearity of 232
expected values property for random variables. We can then calculate the expected values of 233
potential management decisions for units with the predicted states, and calculate the expected 234
value of the optimal group of management actions among units that can fit within our budget. 235
The expected value of perfect information is calculated with Eq. 4, using expected values of the 236
optimal group of management actions for perfect versus current information for multiple 237
management units. Again, when management costs are equal among units, the expected cost-238
effectiveness for a unit is simply the expected value of the chosen management action for a given 239
unit. 240
Page 13 of 56 Journal of Applied Ecology
14
The expected value of the optimal group of management actions after monitoring is calculated as 241
follows: 242
9. 243
This is the maximum, subject to the budget B, of the summed expected values of actions
244
among all units j after monitoring. Specifically, it is the maximum (subject to budget) summed 245
value among all units, of the probability of result y, multiplied the value of action
given state 246
s, weighted by the probability p|s of state s given result y. For probabilities of discrete states, 247
probabilities of obtaining a result y can be used to calculate the expected number or fraction of 248
units with each outcome, and monitoring accuracy can be incorporated into the expected value of 249
a management action given the monitoring outcome, as per Eq. 5a,b. As with Eq. 8, EV
monitoring
is 250
maximized by allocating the budget to management actions among the units that yield the 251
highest summed expected values. The expected value of monitoring information is calculated 252
using Eq. 6 above, but using results for multiple management units. 253
Explicitly considering cost of monitoring 254
To account for monitoring cost via the budget, we calculate the expected values using the net 255
budget after subtracting monitoring costs. It is also possible to account for the cost of monitoring 256
in a single-decision context, by removing any potential management actions that cannot be 257
afforded if monitoring costs reduce the management budget. In contrast, the cost of delays 258
caused by monitoring can be directly incorporated into the expected value of management 259
actions, as we illustrate in Case Study 2 below. 260
Case Study 1 – Habitat protection for threatened plant conservation 261
Page 14 of 56Journal of Applied Ecology
15
In this case study, a conservation agency is initiating a program to protect occurrences of a 262
threatened plant species on private land, by arranging 20-year stewardship agreements. Although 263
we use simulated parameters to simplify illustration of our method, we endeavoured to make 264
them as realistic as possible, using approximate costs of surveys, taxes and land agreements from 265
farmland in southern Ontario, Canada. 266
As part of stewardship agreements, the agency will pay landowners the equivalent of 20 years of 267
taxes in exchange for protecting one hectare parcels of private land. Appendix Table S1 provides 268
details of cost calculations. Briefly, the estimated survey cost is $500 per parcel, and the cost of 269
the 20-year stewardship program is $5,000 per parcel. The agency has a total budget of $40,000 270
for this program, which is sufficient to protect eight parcels if all funds are used for stewardship 271
agreements and none for surveys. 272
The agency is considering 20 parcels potentially containing the species (Table S1). For ten of the 273
parcels, the probability of occurrence is estimated as ~0.5, due to recorded occurrence in an 274
outdated survey. The remaining 10 parcels have estimated probabilities of occurrence of ~0.1, 275
based on habitat suitability only. The agency wants to know if it should survey the parcels before 276
deciding which ones to protect, or if it should arrange protection without monitoring. For 277
simplicity, we measure value of a management action (V(a,s)) as conserved occurrences of the 278
threatened species; V(a,s) = 1 if a parcel in which the species occurs is protected. No other 279
management action contributes to species persistence; thus the other management actions (not 280
protecting a parcel, or protecting a parcel where the species does not occur) have V(a,s) = 0. 281
We evaluate whether the agency should survey the parcels before deciding which ones to protect, 282
using both conventional VOI and VOI for multiple management units. We present detailed 283
calculations for a detection probability of 0.8 with no false positives, and parcels with an 284
Page 15 of 56 Journal of Applied Ecology
16
estimated prior probability of occurrence of 0.5. We provide worksheets with additional 285
calculations in Appendix S2, including calculations for 0.1 probability of occurrence, and 286
alternative scenarios including monitoring only a subset of parcels, different management costs 287
for some parcels, and repeat surveys. These spreadsheets are intended to further demonstrate the 288
flexibility of our approach, and to facilitate understanding of the calculations.
289
To illustrate the influence of variation in survey accuracy estimates, we calculate VOI across a 290
realistic range of survey detection probability from 0.05 to 0.95 (e.g. Chen et al. 2009; 2013). We 291
also explore the influence of variation in monitoring costs. While our estimate of $500 per 1-ha 292
area is compatible with current consultant rates for single surveys, other monitoring options (e.g. 293
free citizen science or more expensive, intensive programs) might be available. Thus, we 294
simulate monitoring program costs across a range from $0 to $1500 per parcel. 295
Case Study 2 – classification and management of species threatened with extinction 296
A conservation agency wishes to prioritize management of species based on extinction risk. 297
Here, we consider the ‘cost’ of monitoring as the probability that an endangered species will go 298
extinct during monitoring, and assume that the financial cost of monitoring is separate from the 299
management funding pool. 300
In this case study, there are three threat categories, ‘endangered, ‘threatened’, and ‘not 301
threatened’ based on estimated extinction risk. After initial population monitoring, 20 species 302
have been classified as ‘threatened’; however, there is considerable uncertainty as to whether the 303
classification is accurate. The agency believes there is a 50% probability that the classification is 304
accurate, a 25% probability that extinction risk has been underestimated such that the category 305
‘threatened’ is too low (i.e., a species classed as ‘threatened’ is really ‘endangered’), and a 25% 306
Page 16 of 56Journal of Applied Ecology
17
probability the category ‘threatened’ is too high (i.e., a species classed as ‘threatened’ is really 307
‘not threatened’). This means that we expect approximately 5 of these 20 species to be 308
endangered, 10 to be threatened, and 5 to be not threatened. 309
The agency has the budget to undertake long-term management for 17 of its 20 ‘threatened’ 310
species. It can either act now, or choose to undertake a second round of monitoring to better 311
understand their threat status. However, it estimates that during the time it takes to monitor, each 312
endangered species will have a 10% probability of going extinct. In this round of monitoring, 313
there is again a 50% probability that risk is correctly estimated for a given threat category, a 25% 314
probability that risk is underestimated for a given category, and a 25% probability that risk is 315
overestimated for a given category. However, for species assigned to the category ‘endangered’, 316
any overestimate of risk would still assign them to this category; thus, there is a 75% probability 317
of correctly assigning this category. Likewise, for species assigned to the category ‘not 318
threatened’, there is no lower category, so there is a 75% chance of correctly assigning these 319
species. 320
Because the agency wants to manage species with the greatest threat, its value structure for 321
management is as follows: for managing an endangered species, the value is 2; for managing a 322
threatened species, the value is 1; for managing a non-threatened species, the value is 0. The 323
agency wants to maximize the summed value among managed species. In Appendix S3, we 324
present an alternative value structure with negative (penalty) values for management of non-325
threatened species, and non-management of threatened and endangered species. 326
Results 327
Case Study 1 - Habitat protection: Conventional single-decision VOI 328
Page 17 of 56 Journal of Applied Ecology
18
The expected value of protecting a parcel with 0.5 prior probability of occurrence and current 329
information (Eq. 2) is the sum of the values for the action ‘protect’ when the species is present or 330
absent, weighted by the probability of each state, which is (1×0.5)+(0×0.5)=0.5 (Table 2). The 331
expected value of not protecting a parcel, similarly calculated, equals 0. Thus, the best 332
management action in this case is ‘protect’ and its expected value with current information is 0.5. 333
In our case, there is only a value for protecting an occurrence. Thus, the expected value of the 334
best management action under certainty, which is the sum of the values of the best management 335
actions for any state of the management unit, weighted by the probability of each state, is 336
0.5+0=0.5, and the expected value of perfect information (Eq. 4) is 0. 337
If the parcel is monitored and the species is found, the probability of occurrence is 1 because 338
there are no false positives, whereas if the species is not found, the updated probability of 339
occurrence is (probability not found | present) × (prior probability) / (probability not found) = 340
0.17 (Table 3). The probabilities of obtaining the results ‘found’ and ‘not found’, based on prior 341
probabilities, are 0.4 and 0.6 respectively (Table 3). The expected value of protection after the 342
species is ‘found’ in a parcel equals 1. However, even if the species is not found in a parcel, the 343
expected value of protecting that parcel is 0.17 due to the possibility of a non-detection error. 344
For monitored parcels with 0.5 prior probability of occurrence, the expected value of the decision 345
after monitoring is thus (1×0.4)+(0.17×0.6)=0.5, and the expected value of monitoring 346
information (Eq. 6) is 0. This result is logical, since the best decision for a single parcel only is to 347
‘protect’ with or without monitoring. 348
Habitat protection: VOI across multiple management units with a limited budget 349
Page 18 of 56Journal of Applied Ecology
19
Using current information, the expected value of protecting parcels with 0.5 probability is 0.5 350
(Eq. 2). The expected value of protecting parcels with 0.1 probability is 0.1. Protecting the 351
parcels with 0.5 probability is more cost-effective, and the best strategy is to protect as many of 352
these as possible. Our expected value of the optimal group of management actions using current 353
information is our predicted number of occurrences conserved: 0.5×8=4. With perfect 354
information, we would expect to find 0.5×10+0.1×10=6 parcels with occurrences on average. 355
The expected value of perfect information (EVPI; Eq. 4), given that we could afford to protect all 356
6 expected occurrences, is thus 6-4=2. 357
If we monitored before protecting, we would predict the outcomes outlined in Table 4, i.e. our 358
monitoring would find, on average, 4.8 occurrences and fail to find 1.2 occurrences. Our top-359
ranked management options would be to protect parcels where the focal species was found, 360
which would lead to 4.8 expected occurrences; Table 4). 361
If monitoring had no cost, we would have sufficient resources to protect eight parcels, including 362
the expected 4.8 parcels with found occurrences (our two best ranked outcomes), plus 3.2 of the 363
next most cost-effective parcels we can afford, which in this case are those with 0.5 prior 364
probability and no found occurrence. However, the cost of monitoring ($10,000) would sacrifice 365
protection of two parcels, leaving resources for protecting the predicted 4.8 found occurrences, 366
and an additional 1.2 parcels from the next-ranked outcome (Table 4). Thus, the expected value 367
of the optimal group of management actions would be 4.8+1.2×0.17=5 occurrences, and the 368
expected value of monitoring information, once the monitoring budget is considered, is 5–4=1. 369
In other words, we would expect to protect an additional parcel that currently houses the focal 370
species if we were to monitor, even though we would have to sacrifice protecting two parcels. 371
Thus, the agency would justifiably spend the $10,000 cost of the monitoring program. Note how 372
Page 19 of 56 Journal of Applied Ecology
20
the VOI across parcels with our limited budget leads to a different decision (monitor first) than if 373
we had extrapolated conventional VOI to any eight parcels. 374
Sensitivity to parameter variation 375
When we calculate VOI across a realistic range of survey detection probability from 0.05 to 0.95, 376
we can see that the expected value of monitoring information is negative when detectability falls 377
below 0.5 (Fig. 1a). Thus, in this case if the species cannot be detected with >50% probability 378
when present, it is better to act on existing information rather than monitor all parcels. When we 379
calculate VOI across a range of monitoring program costs from $0 to $1500 per parcel, we see 380
that VOI is negative for monitoring costs >$1000 per parcel (Fig. 1b). 381
Case Study 2 – Classification of threatened species: Single decision VOI 382
Using Eq. 1, the expected value of managing a species currently classified as ‘threatened’ with 383
current information is (probability it is endangered × value if endangered) + (probability it is 384
threatened × value if threatened) + (probability it is not threatened × value if not threatened), 385
which is 0.25×2+0.5×1+0.25×0=1. The expected value of not managing is zero. Since there is no 386
value for the decision to not manage, the expected value of perfect information is the same as the 387
expected value of current information. 388
The expected value of monitoring information calculations are shown in Tables 5 and 6. In this 389
case, we are accounting for the probability that our monitored species may be ‘endangered’, and 390
if so may go extinct during the course of monitoring (Table 6). We then subtract this from the 391
expected value of the decision with monitoring information, with the result being 0.95. Using Eq. 392
6, the expected value of monitoring information is 0.95-1=-0.05, which would lead to the 393
decision ‘do not monitor’. 394
Page 20 of 56Journal of Applied Ecology
21
VOI across multiple management units 395
The expected value of the optimal group of management actions under uncertainty is simply the 396
summed expected value of the best management actions with current information for the 17 397
species we can manage, 1×17=17. For the expected value of perfect information, we assume we 398
would manage up to 17 species with the greatest threat. We would manage the 5 expected 399
endangered and 10 expected threatened species for a total value of 5×2+10×1=20, and an 400
expected value of perfect information of 20-17=3. 401
When calculating the expected value of monitoring information, we must account for the 402
potential extinction of endangered species (10% probability per species) while monitoring. Based 403
on the prior probability of 0.1 that we will lose an endangered species during the course of 404
monitoring and the prior expectation of five endangered species, we would expect to lose 0.5 405
species on average. Taking this into account, our calculation of the expected value of the optimal 406
group of management actions after monitoring is presented in Table 7. The expected value of 407
monitoring information is 18.2-17=1.2. Since the expected value of monitoring information is 408
positive, the agency would be justified in monitoring before acting, despite the risk of extinction 409
while monitoring is taking place. This contrasts with the single-decision VOI that suggests 410
monitoring would not be recommended. 411
Discussion 412
Value of information (VOI) theory can help to guide monitoring, by explicitly estimating the 413
expected value of management actions before and after monitoring. However, two key 414
limitations of VOI in applied ecology have been its application within a single-decision 415
framework, rather than over multiple decisions across management units, and the lack of explicit 416
Page 21 of 56 Journal of Applied Ecology
22
consideration of monitoring costs. In the broader literature, VOI for multi-unit problems has 417
been explored, but with simplified parameters (e.g. perfect information; Keisler 2004), equal 418
costs among all units (Zan and Brickel 2009), or via simulation to explore non-equal costs or 419
non-identical probabilities (Brickel and Zan 2013). Our application is more general, and 420
explicitly links monitoring costs to non-monetary value measurements (e.g. number of species or 421
sites conserved). 422
In addition, we also demonstrate a crucial property of VOI across multiple management units for 423
probabilities of discrete states: prior probabilities can be summed among units, to estimate the 424
expected number of units corresponding to each potential result. This gives the flexibility and 425
realism of allowing different monitoring or management decisions among units (even with the 426
same prior probabilities), to maximize expected value. To our knowledge, this property of VOI 427
has not previously been recognized. Previous applications (Keisler 2004; Brickel and Zan 2009; 428
Soares et al. 2012; Zan and Brickel 2013) have calculated value of information based on summed 429
individual expected values among units. 430
This property has important implications. As we have shown in our case studies, calculating VOI 431
across multiple units can increase the expected value of monitoring information compared to a 432
single-decision case, because it can allow estimation of the expected number of units with each 433
possible monitoring result, and the choice of management among units to maximize total 434
expected values. We note, however, that this property is applicable to decisions involving 435
discrete states (e.g. presence/absence of a species, threat categories), while problems involving 436
continuous distributions (e.g. expected amount of a particular habitat among patches), and non-437
independence among units or value objectives require further exploration. For example, in cases 438
where sampled units from a single population lead to increased accumulated knowledge to 439
Page 22 of 56Journal of Applied Ecology
23
improve a model (e.g. Soares et al. 2012), extrapolation of probabilities across units will not be 440
applicable and simulation-based approaches may be necessary. 441
Multiple-unit decisions with limited budgets are very common in applied ecology. For example, 442
spatial resource management allocations often consider many management units (e.g. Wilson et 443
al. 2006; Leathwick et al. 2010), and threatened species conservation programs typically rank 444
species based on priorities (e.g. Joseph et al. 2009; IUCN 2017). Even the more quantitative 445
approaches to these issues are often based on limited data, because time and financial resources 446
are scarce. These limitations are frequently acknowledged, and caution is urged in implementing 447
recommendations where parameters are uncertain (e.g. Game and Grantham 2008; Moilanen et 448
al. 2014). 449
Various quantitative methods of addressing uncertainty in decision-making have been 450
implemented for conservation problems, including sensitivity analysis to examine the influence 451
of input parameters (e.g. Ardron et al. 2008), assigning greater weight to more certain data or 452
outcomes (Tulloch et al. 2013; Moilanen et al. 2014), and using either upper or lower bounds of 453
uncertainty estimates as the basis for decisions (Moilanen et al. 2014). 454
Multi-unit VOI analysis provides a distinct and complementary approach to addressing 455
uncertainty in multi-unit problems, through examining the potential utility of new information in 456
making more efficient decisions. By calculating the expected value of monitoring information 457
across management units and explicitly linking monitoring and management costs, agencies can 458
better partition their budgets to reducing uncertainty versus immediate action. 459
Exploring the influence of uncertainty in parameter estimates on VOI can also be highly 460
informative in targeting monitoring to reduce the most influential uncertainties, or determining 461
Page 23 of 56 Journal of Applied Ecology
24
the plausible ranges of parameters for which monitoring is worthwhile. For example, in Case 462
Study 1, the management agency could use information regarding detectability (e.g. is the 463
species easily recognizable or is it cryptic?), as well as the expected thoroughness of surveys 464
(Chen et al. 2009, 2013; McCarthy et al. 2013) to help determine whether to monitor first or 465
protect without monitoring. Likewise, if monitoring costs are uncertain but can be estimated 466
within a reasonable range, value of information could be calculated across this range. It is 467
important to note that the results of our artificial case study analyses are particular to the 468
parameters we chose. For example, thresholds for positive value of information we identified for 469
monitoring accuracy and cost in Case Study 1 cannot be applied to other studies, and sensitivity 470
should be evaluated on a case-by-case basis. More complex consideration of uncertainty across 471
multiple variables could assign probability distributions to these variables and use sensitivity 472
analysis to assign a probability that a given strategy (e.g. monitor first) is optimal. 473
Another potential limitation of VOI theory is its perceived complexity (Canessa et al. 2015). This 474
is a potential problem with decision-theoretic or evidence-based approaches in general 475
(Possingham et al. 2000; Pullin and Knight 2005). Managers have limited time and must have 476
expertise in several aspects of decision-making, so there is a tendency to use experience-based 477
information in decision-making (Pullin and Knight 2005; Cook et al. 2010). We hope that by 478
illustrating our methods using multiple formats (including customizable spreadsheets and code in 479
the Supporting Information that detail calculation steps), that we can diminish this barrier. 480
Both individual and multi-unit VOI theory are also limited in their ability to account for 481
uncertainties. Although the potential influence of uncertainties in VOI input parameters can be 482
examined using simulation across reasonable ranges of parameter estimates, VOI theory can only 483
resolve questions around so-called “known unknowns” (e.g. survey accuracy), and cannot 484
Page 24 of 56Journal of Applied Ecology
25
account for surprise results that often occur during monitoring programs (Doak et al. 2008; 485
Wintle et al. 2010). While decisions to monitor must be carefully considered in light of limited 486
resources (McDonald-Madden et al. 2010, Lindenmayer et al. 2013), the benefits of long-term 487
monitoring for finding new threats and solutions cannot be discounted, especially when 488
programs are sufficiently adaptable to incorporate new information into monitoring protocols 489
(Lindenmayer and Likens 2010). 490
A final consideration is that VOI theory generally calculates expected values of information, 491
based on probabilities. This uses the implicit assumption of risk neutrality. The actual states of a 492
system or results of monitoring may be different from those that are expected, and the actual 493
values of decisions can be affected accordingly. Such uncertainties may be uncomfortable for 494
conservation and resource management agencies, which appear to be generally risk averse 495
(Tulloch et al. 2015). Explorations of relationships between the degree of risk aversion and 496
single-decision VOI suggest non-linear relationships that are highly dependent on input 497
parameters (Hilton 1981; Willinger 1989; Eeckhoudt & Godfroid 2000). Intuitively, a strategy of 498
repeated monitoring can improve certainty before acting. However, given that monitoring can 499
have substantial financial costs, and can also lead to missed management opportunities, the 500
decision to monitor is not risk-free. The extension of VOI theory we have presented can provide 501
an objective framework for making justifiable decisions on when to monitor and when to act in 502
many management scenarios. 503
Authors’ Contributions 504
JRB conceived the ideas, designed the methodology and led the writing of the manuscript. SM 505
helped conceive the ideas and design the methodology and case studies. AEM and LF helped 506
Page 25 of 56 Journal of Applied Ecology
26
design the methodology and case studies. IC and BG helped with mathematical formulation and 507
designing the methodology; IC wrote the optimization formulation. All authors contributed 508
critically to the drafts and gave final approval for publication. 509
Acknowledgements 510
JRB, LF and BG are funded by the Natural Sciences and Engineering Research Council of 511
Canada (NSERC). The authors thank M. Runge and three anonymous reviewers for helpful 512
comments. 513
Data accessibility 514
Data for this paper are simulated. Code for generating and analysing data is available from the 515
Dryad Digital Repository. DOI: 10.5061/dryad.r464h5d (Bennett et al. 2018). Spreadsheets for 516
analysing data are also provided as Supporting Information. 517
References 518
Ardron, J.A., Possingham, H.P. & Klein, C.J., (2008) Marxan good practices handbook. Pacific 519
Marine Analysis and Research Association, Vancouver. 520
Ball, I., Possingham, H.P. & Watts, M. (2009) Marxan and Relatives: Software for Spatial 521
Conservation Prioritization. Spatial conservation prioritisation: Quantitative methods and 522
computational tools, pp. 185-195. Oxford University Press, Oxford. 523
Bennett, J.R., Maxwell, S.L., Martin, A.E., Chadès, I., Fahrig, L. & Gilbert, B. (2018). Data 524
from: When to monitor and when to act: value of information theory for multiple management 525
units and limited budgets. Dryad Digital Repository: DOI: 10.5061/dryad.r464h5d. 526
Page 26 of 56Journal of Applied Ecology
27
Bennett, J.R., Elliott, G., Mellish, B., Joseph, L.N., Tulloch, A.I., Probert, W.J., Di Fonzo, M.M., 527
Monks, J.M., Possingham, H.P. & Maloney, R. (2014) Balancing phylogenetic diversity and 528
species numbers in conservation prioritization, using a case study of threatened species in New 529
Zealand. Biological Conservation, 174, 47-54. 530
Bickel, J.E. & Zan, K. (2009) Resource Allocation and the Value of Information. In NSF CMMI 531
Research and Innovation Conference 2009. 532
Canessa, S., GuilleraArroita, G., LahozMonfort, J.J., Southwell, D.M., Armstrong, D.P., 533
Chadès, I., Lacy, R.C. & Converse, S.J. (2015) When do we need more data? A primer on 534
calculating the value of information for applied ecologists. Methods in Ecology and Evolution, 6, 535
1219-1228. 536
Carwardine J., Nicol S., van Leeuwen S., Walters, B., Firn J., Reeson A., Martin T.G. & Chadès 537
I. (2014) Priority threat management for Pilbara species of conservation significance, CSIRO 538
Ecosystems Sciences, Brisbane, Australia. 539
Chadès, I., McDonald-Madden, E., McCarthy, M.A., Wintle, B., Linkie, M. & Possingham, H.P. 540
(2008) When to stop managing or surveying cryptic threatened species. Proceedings of the 541
National Academy of Sciences, 105, 13936-13940. 542
Chen, G., Kéry, M., Zhang, J. & Ma, K. (2009) Factors affecting detection probability in plant 543
distribution studies. Journal of Ecology, 97, 1383-1389. 544
Chen, G., Kéry, M., Plattner, M., Ma, K. & Gardner, B. (2013) Imperfect detection is the rule 545
rather than the exception in plant distribution studies. Journal of Ecology, 101, 183-191. 546
Page 27 of 56 Journal of Applied Ecology
28
Cook, C. N., Hockings, M., & Carter, R. W. (2010) Conservation in the dark? The information 547
used to support management decisions. Frontiers in Ecology and the Environment, 8, 181-186. 548
Dantzig, G.B., 1957. Discrete-variable extremum problems. Operations Research, 5, 266-288. 549
Doak, D.F., Estes, J.A., Halpern, B.S., Jacob, U., Lindberg, D.R., Lovvorn, J., Monson, D.H., 550
Tinker, M.T., Williams, T.M., Wootton, J.T. & Carroll, I. (2008) Understanding and predicting 551
ecological dynamics: are major surprises inevitable. Ecology, 89, 952-961. 552
Eeckhoudt, L. and Godfroid, P., 2000. Risk aversion and the value of information. The Journal 553
of Economic Education, 31, 382-388. 554
Game, E.T. & Grantham, H.S. (2008) Marxan User Manual: For Marxan version 1.8.10. 555
University of Queensland, St. Lucia, Queensland, Australia, and Pacific Marine Analysis and 556
Research Association, Vancouver, British Columbia, Canada 557
Government of New South Wales (2013) Saving our Species Technical Report. Office of 558
Environment and Heritage, Sydney, Australia. 559
Hauser, C.E. & McCarthy, M.A. (2009) Streamlining ‘search and destroy’: costeffective 560
surveillance for invasive species management. Ecology Letters, 12, 683-692. 561
Hilton, R.W., 1981. The determinants of information value: Synthesizing some general results. 562
Management Science, 27, 57-64. 563
International Union for Conservation of Nature and Natural Resources (IUCN). (2017) IUCN 564
Red List. http://www.iucnredlist.org/ 565
Joseph, L.N., Maloney, R.F. & Possingham, H.P. (2009) Optimal allocation of resources among 566
threatened species: a project prioritization protocol. Conservation Biology, 23, 328-338. 567
Page 28 of 56Journal of Applied Ecology
29
Keisler, J., (2004) Value of information in portfolio decision analysis. Decision Analysis, 1, 177-568
189. 569
Kellerer, H., Pferschy, U. & Pisinger, D. (2004) Knapsack problems. Springer-Verlag, Berlin. 570
Keisler, J. (2004) Value of information in portfolio decision analysis. Decision Analysis, 1, 177-571
189. 572
Leathwick, J.R., Moilanen, A., Ferrier, S. & Julian, K., (2010) Complementarity-based 573
conservation prioritization using a community classification, and its application to riverine 574
ecosystems. Biological Conservation, 143, 984-991. 575
Lindenmayer & Likens (2010) Effective Ecological Monitoring. CSIRO Publishing, 576
Collingwood, Australia. 577
Lindenmayer, D.B., Likens, G.E., Andersen, A., Bowman, D., Bull, C.M., Burns, E., Dickman, 578
C.R., Hoffmann, A.A., Keith, D.A., Liddell, M.J. & Lowe, A.J. (2012) Value of longterm 579
ecological studies. Austral Ecology, 37, 745-757. 580
Lindenmayer, D.B., Piggott, M.P. & Wintle, B.A. (2013) Counting the books while the library 581
burns: why conservation monitoring programs need a plan for action. Frontiers in Ecology and 582
the Environment, 11, 549-555. 583
Martello, S. & Toth, P. (1990) Knapsack problems: algorithms and computer implementations. 584
Wiley. 585
Martin, T.G., Nally, S., Burbidge, A.A., Arnall, S., Garnett, S.T., Hayward, M.W., Lumsden, 586
L.F., Menkhorst, P., McDonaldMadden, E. & Possingham, H.P. (2012) Acting fast helps avoid 587
extinction. Conservation Letters, 5, 274-280. 588
Page 29 of 56 Journal of Applied Ecology
30
Maxwell, S.L., Rhodes, J.R., Runge, M.C., Possingham, H.P., Ng, C.F. & McDonaldMadden, 589
E. (2015) How much is new information worth? Evaluating the financial benefit of resolving 590
management uncertainty. Journal of Applied Ecology, 52, 12-20. 591
Mazor, T., Beger, M., McGowan, J., Possingham, H.P. & Kark, S. (2016) The value of migration 592
information for conservation prioritization of sea turtles in the Mediterranean. Global Ecology 593
and Biogeography, 25, 540–552. 594
McCarthy, M.A., Thompson, C.J., Hauser, C., Burgman, M.A., Possingham, H.P., Moir, M.L., 595
Tiensin, T. & Gilbert, M. (2010) Resource allocation for efficient environmental management. 596
Ecology Letters, 13, 1280-1289. 597
McCarthy, M.A., Moore, J.L., Morris, W.K., Parris, K.M., Garrard, G.E., Vesk, P.A., Rumpff, 598
L., Giljohann, K.M., Camac, J.S., Bau, S.S. & Friend, T. (2013) The influence of abundance on 599
detectability. Oikos, 122, 717-726. 600
McDonald-Madden, E., Baxter, P.W., Fuller, R.A., Martin, T.G., Game, E.T., Montambault, J. & 601
Possingham, H.P. (2010) Monitoring does not always count. Trends in Ecology & Evolution, 25, 602
547-550. 603
Moilanen, A., 2007. Landscape zonation, benefit functions and target-based planning: unifying 604
reserve selection strategies. Biological Conservation, 134, 571-579. 605
Moilanen, A., Pouzols, F.M., Meller, L., Veach, V., Arponen, A., Leppänen, J. & Kujala, H., 606
(2014) Zonation–Spatial Conservation Planning Methods and Software. Version 4. User Manual. 607
Helsinki, Finland: University of Helsinki. 608
Page 30 of 56Journal of Applied Ecology
31
Moore, J.L. & Runge, M.C. (2012) Combining Structured Decision Making and Valueof609
Information Analyses to Identify Robust Management Strategies. Conservation Biology, 26, 810-610
820. 611
Nichols, J.D. & Williams, B.K. (2006) Monitoring for conservation. Trends in Ecology & 612
Evolution, 21, 668-673. 613
Pisinger, D. (2005) Where are the hard knapsack problems? Computers & Operations Research, 614
32, 2271-2284. 615
Possingham, H., Ball, I. & Andelman, S. (2000) Mathematical methods for identifying 616
representative reserve networks. In Quantitative Methods for Conservation Biology (pp. 291-617
306). Springer New York. 618
Pullin, A.S. & Knight, T.M. (2005) Assessing Conservation Management's Evidence Base: a 619
Survey of ManagementPlan Compilers in the United Kingdom and Australia. Conservation 620
Biology, 19, 1989-1996. 621
Raiffa, H. (1968) Decision Analysis: Introductory Lecturers on Choices under Uncertainty. 622
Addison-Wesley, Reading, Ma. 623
Runge, M.C., Converse, S.J. & Lyons, J.E. (2011) Which uncertainty? Using expert elicitation 624
and expected value of information to design an adaptive program. Biological Conservation, 144, 625
1214-1223. 626
Runting, R.K., Wilson, K.A. & Rhodes, J.R. (2013) Does more mean less? The value of 627
information for conservation planning under sea level rise. Global Change Biology, 19, 352-363. 628
Page 31 of 56 Journal of Applied Ecology
32
Soares, M.O., Welton, N.J., Harrison, D.A., Peura, P., Hari, M.S., Harvey, S.E., Madan, J.J., 629
Ades, A.E., Palmer, S.J. & Rowan, K.M. (2012) An evaluation of the feasibility, cost and value 630
of information of a multicentre randomised controlled trial of intravenous immunoglobulin for 631
sepsis (severe sepsis and septic shock): incorporating a systematic review, meta-analysis and 632
value of information analysis. Health Technology Assessment, 16. 633
Shea, K., Tildesley, M.J., Runge, M.C., Fonnesbeck, C.J. & Ferrari, M.J. (2014) Adaptive 634
management and the value of information: learning via intervention in epidemiology. PLoS 635
Biology, 12, p.e1001970. 636
Sutherland, W.J., Pullin, A.S., Dolman, P.M., & Knight, T.M. (2004) The need for evidence-637
based conservation. Trends in Ecology & Evolution, 19, 305-308. 638
Tulloch, A.I., Maloney, R.F., Joseph, L.N., Bennett, J.R., Fonzo, M.M., Probert, W.J., O'Connor, 639
S.M., Densem, J.P. & Possingham, H.P. (2015) Effect of risk aversion on prioritizing 640
conservation projects. Conservation Biology, 29, 513-524. 641
Tulloch, V.J., Possingham, H.P., Jupiter, S.D., Roelfsema, C., Tulloch, A.I. & Klein, C.J., (2013) 642
Incorporating uncertainty associated with habitat data in marine reserve design. Biological 643
Conservation, 162, 41-51. 644
Williams, B.K. & Johnson, F.A. (2015) Value of information in natural resource management: 645
technical developments and application to pinkfooted geese. Ecology and Evolution, 5, 466-474. 646
Willinger, M., (1989) Risk aversion and the value of information. Journal of Risk and Insurance, 647
56, 104-112. 648
Page 32 of 56Journal of Applied Ecology
33
Wilson, K.A., McBride, M.F., Bode, M. & Possingham, H.P. (2006) Prioritizing global 649
conservation efforts. Nature, 440, 337-340. 650
Wilson, K.A., Carwardine, J. & Possingham, H.P. (2009) Setting conservation priorities. Annals 651
of the New York Academy of Sciences, 1162, 237-264. 652
Wintle, B.A., Runge, M.C., & Bekessy, S.A. (2010) Allocating monitoring effort in the face of 653
unknown unknowns. Ecology Letters, 13, 1325-1337. 654
Yokota, F. & Thompson, K.M. (2004) Value of information literature analysis: a review of 655
applications in health risk management. Medical Decision Making, 24, 287-298. 656
Zan, K. & Bickel, J.E. (2013) Components of portfolio value of information. Decision Analysis, 657
10, 171-185. 658
659
Page 33 of 56 Journal of Applied Ecology
34
660
661
Fig. 1. a) Expected value of monitoring information in Case Study 1, across a range of 662
detectability for the focal species; b) expected value of monitoring information in Case Study 1, 663
across a range of monitoring costs. 664
665
Page 34 of 56Journal of Applied Ecology
35
Table 1: Terms of Equations 666
Symbol Term
Expected value of a management action
s State of a management unit
A Set of all possible management actions for a given state s
a Individual management action
x Binary decision variable identifying whether an action is taken
P
s
Prior probability of a state s
V(a,s) Value of a management action for a state s
y Monitoring result
U Number of management units (e.g. habitats, species)
B Budget
c Cost of an individual management action
667
668
Page 35 of 56 Journal of Applied Ecology
36
Table 2: Prior probabilities and values for parcels with 0.5 probability of occurrence in Case 669
Study 1. 670
Present Absent Expected value
under uncertainty
Prior Probability 0.5 0.5
Values of Actions ‘Protect’ 1 0 1×0.5+0×0.5=0.5
‘Do not Protect’ 0 0 0×0.5+0×0.5=0
671
672
Page 36 of 56Journal of Applied Ecology
37
Table 3: Monitoring accuracy and updated beliefs for Case Study 1 parcels with 0.5 prior 673
probability of occurrence, assuming 0.8 probability of detection and no false positives. 674
Present Absent Probability of result
Expected accuracy
of monitoring
Found 0.8 0 0.5×0.8+0.5×0=0.4
Not found 0.2 1 0.5×0.2+0.5×1=0.6
Updated belief
after monitoring
Found 1 0
Not found 0.2×0.5/(0.2×0.5
+ 1×0.5)=0.17 1-0.17=0.83
675
676
Page 37 of 56 Journal of Applied Ecology
38
Table 4: Expected values and cost-effectiveness ranks of protection, across the four possible 677
survey results for parcels in Case Study 1. Since the cost of protecting each parcel is the same, 678
the cost of protecting each parcel is normalized to one and the expected cost-effectiveness of a 679
management action is the value of the action itself (expressed in number of protected 680
occurrences). 681
Prior
probability of
occurrence
Found in
survey?
Expected
value of
protecting,
given this
result
Expected
value of
decision to
protect
(single
parcel)
Ranking by
expected
cost-
effectiveness
(value of
protecting /
cost)
Expected
value among
parcels (value
of protecting
× number of
parcels with
this prior)
0.5 y 1.00 0.4 1 (tie) 4
0.1 y 1.00 0.08 1 (tie) 0.8
0.5 n 0.17 0.1 2 1
0.1 n 0.02 0.02 3 0.2
Predicted total
occurrences
found
4+0.8=4.8
Predicted total
occurrences
not found
1+0.2=1.2
682
683
Page 38 of 56Journal of Applied Ecology
39
Table 5: Case Study 2 probability of results after monitoring species currently classified as 684
‘threatened’. 685
Classification
Probability
one category
too low
Probability
correct
Probability
one category
too high
Probability of classification after
monitoring
Endangered 0 0.75 0.25 0.75×0.25+0.25×0.5=0.3125
Threatened 0.25 0.5 0.25 0.25×0.25+0.5×0.5+0.25×0.25=
0.375
Not
Threatened 0.25 0.75 0 0.25×0.5+0.75×0.25=0.3125
686
687
Page 39 of 56 Journal of Applied Ecology
40
Table 6: Expected value of monitoring information for single-decision in Case Study 2. Full calculation for updated belief of
688
‘endangered’ after monitoring result ‘endangered’ is shown.
689
Updated beliefs after monitoring result ‘endangered’
Endangered Threatened Not threatened Expected value if managed
(prob result ‘endangered’ | endangered) /
(prob result ‘endangered’)
=(0.75×0.25)/(0.75×0.25+0.25×0.5)=0.6
0.4 0 2×0.6+1×0.4=1.6
Updated beliefs after monitoring result ‘threatened’
Endangered Threatened Not threatened Expected value if managed
0.167 0.67 0.167 2×0.167+1×0.67+0×0.167=1
Updated beliefs after monitoring result ‘not threatened’
Endangered Threatened Not threatened Expected value if managed
0 0.4 0.6 1×0.4+0×0.6=0.4
Expected value of best management
action with monitoring if no extinction
risk during monitoring
0.3125×1.6+0.375×1+0.3125×0.4=1
Expected value of best management
action with monitoring including
extinction risk during monitoring
0.3125×1.6+0.375×1+0.3125×0.4-
(0.1×0.25×2)=0.95
690
Page 40 of 56Journal of Applied Ecology
41
Table 7: Expected value of actions after monitoring for Case Study 2.
691
692
Monitoring
result
Expected
number of
species with
this result
across 20
species
Expected
value of
managing one
species
Ranking by
expected cost-
effectiveness
(exp. value of
managing / cost)
Total expected
value
Total expected
value managed
Leftover budget after
each result category
managed
‘endangered’ 0.3125×20-
0.1×5=5.75
*
1.6 1 5.75×1.6=9.2 9.2 17-5.75=11.25
‘threatened’ 7.5 1 2 7.5×1=7.5 7.5 11.25-7.5=3.75
‘not
threatened
6.25 0.4 3 6.25×0.4=2.5 3.75×0.4=1.5 0
Total expected value 9.2+7.5+1.5=18.2
*This calculation accounts for the predicted rate of extinction during monitoring 693
694
Page 41 of 56 Journal of Applied Ecology
ValueofInformation
Detectability
0.2 0.4 0.6 0.8
-1.0 0.0 0.5 1.0 1.5
ValueofInformation
MonitoringCost($/parcel)
0 500 1000 1500
-2 -1 0 1
a) b)
Page 42 of 56Journal of Applied Ecology
Bennett et al., When to monitor and when to act: value of information theory
for multiple management units and limited budgets
Appendix S1 – Supplementary Formulation and Table S1
Optimization formulations for Bennett et al, multi-unit VOI.
1. Expected value under uncertainty
The expected value under uncertainty represents the expected value of a management action one would
logically implement with current information, i.e. an action a
i
in the set of possible actions that
maximizes
. Formally:
The calculation of the expected value under uncertainty assumes a single management action will be
implemented under uncertainty. The cost of implementing an action under uncertainty is defined by
).
2. Expected value under uncertainty and limited budget
When introducing a budget constraint, Equation (1) is now subject to the cost of the best action being
smaller or equal to an available budget B. We refer to this value as the expected value under uncertainty
and limited budget:
. Finding a solution to equation (1) under limited budget, means
finding the binary decision variables x
i
solution of:
Page 43 of 56 Journal of Applied Ecology
subject to
(c
1
) only one action is selected
from A, the suite of possible
actions.
(c
2
) the cost c of the selected
action is lower than our
budget.
(c
3
) the decision variables
identifies which action
to
implement.
(c
4
) priors
obey
probability laws
This problem can easily be solved by calculating and ranking the expected value for each action, and
selecting the highest value for which the cost of the associated action is below the budget. Because we
only select one action, the optimization is straight forward.
3. Expected value under uncertainty and limited budget for multiple management units
When accounting for multiple management units, we aim to calculate the expected value under
uncertainty. We assume:
- Managers are uncertain about which hypothesized state
is true in each management unit j.
- The uncertain hypotheses
and associated priors p(s) might not be same across all
management units j. The sum of the priors for a given management unit j equals to 1 i.e. We have
Page 44 of 56Journal of Applied Ecology
- The actions that managers could implement a
i
j
in each management unit might be different. Each
action has a cost, c(a
i
j
).
- For each management unit j, the value of implemented action a
i
j
for each hypothesized
is
V(a
i
j
,
s). This value representing the management outcomes is measured in the same unit across
all management units.
There are two ways of calculating the expected value under uncertainty and limited budget for multiple
management units.
First formulation
A first formulation consists in calculating the expected value under uncertainty for multiple management
units,
, by allowing to choose a group of actions that are individually optimal for their
respective management unit within our limited budget. Formally,
subject to:
(c
1
) the decision variables
identify
which group of management unit
maximizes the expected value under
uncertainty.
(c
2
) the cost of the selected actions is less
than budget B.
Where
is the expected value under uncertainty calculated for management unit j (Eq. 1) and
the cost of the optimal action under uncertainty calculated for management unit j. In
this formulation, if an optimal action for a management unit is too expensive to be selected, a less
Page 45 of 56 Journal of Applied Ecology
expensive suboptimal action would not be selected even though it could add some substantial
management outcomes if implemented.
To account for this case, we propose a second, optimal formulation.
Optimal formulation
The optimal formulation allows us to select suboptimal actions at the management unit scale that could be
optimal when considering a group of actions we would implement for multiple management units.
Formally:
subject to:
(c
1
) a maximum of one
action is selected per
management unit j, from the
suite A
j
of possible actions in
this unit.
(c
2
) the cost of the selected
actions is less than budget B.
(c
3
) the decision variables
identify which actions
to
implement.
(c
4
) priors
obey
probability laws.
Page 46 of 56Journal of Applied Ecology
This problem is also a knapsack problem, however its formulation allows us to select a suboptimal action
at the unit management level if the suboptimal action is part of the optimal solution at the level of all
management units.
4. Expected value under certainty
The expected value under certainty represents the expected value one would get if uncertainty was
resolved (with probability ) prior to implementing an action a
i
in the set of possible actions that
maximizes
. Formally:
The calculation of the expected value under certainty averages the maximum management outcomes we
would get if uncertainty was resolved prior to making a decision. For discrete distributions and multiple
units, the expected number of units with each state can be predicted, and the expected cost for these states
can often be known. However, for other problems, the cost of implementing an action under certainty can
therefore be estimated using an expectation:
5. Expected value under certainty and limited budget
Calculating the expected value under certainty under limited budget can give manager a revised estimate
of the value of perfect information, formally:
Page 47 of 56 Journal of Applied Ecology
subject to:
(c
1
) only one action is selected for
each possible state s.
(c
2
) the expected cost of the selected
actions is lower than our budget.
(c
3
) the decision variables
identifies which action
to
implement for each hypothesized s.
(c
4
) priors obey probability laws
Under a limited budget, calculating the expected value under certainty for a single management unit is a
knapsack optimization problem. The decision variables
identify the actions
we would
implement once the true state s is discovered (with probability ). While this formulation is not
particularly helpful when dealing with one management unit, it becomes useful when dealing with
multiple units. In the single management unit case, unaffordable actions should be removed from the set
of actions available as they will not be implemented. They should therefore not contribute to the expected
value of information under certainty, or used to calculate the expected value of perfect information
(EVPI). However, when dealing with multiple units, when affordable actions are bundled together their
expected cost should not exceed our budget, as they would, otherwise, artificially increase the value of
perfect information and therefore create the false impression that by reducing uncertainty we could
potentially have much bigger gains than with limited budgets.
Page 48 of 56Journal of Applied Ecology
6. Expected value under certainty and limited budget for multiple management units
Similar to the EV under uncertainty there are two possible formulations of the expected value under
certainty and limited budget for multiple management units.
First formulation
A first formulation consists in calculating the expected value under certainty for multiple management
units,
, by allowing us to choose actions that are individually optimal for their respective
management unit within our limited budget. Formally,
subject to:
(c
1
) the decision variables
identify
which group of management unit
maximizes the expected value under
certainty.
(c
2
) the expected cost is less than budget B.
Where
is the expected value under certainty calculated for management unit j (Eq. 1) and
the cost of the optimal action under certainty calculated for management unit j. In this
formulation, if an optimal action for a management unit is too expensive to be selected, a less expensive
suboptimal action would not be selected even though it could add some substantial management
outcomes if implemented.
Optimal formulation
The optimal formulation allows us to select suboptimal actions at the management unit scale that could be
optimal when considering a group of actions we would implement for multiple management units.
Page 49 of 56 Journal of Applied Ecology
subject to:
(c
1
) a maximum of one
action is selected per
possible state s.
(c
2
) the expected cost of
the selected actions is
less than budget B.
(c
3
) the decision
variable
identifies
which actions
to
implement.
(c
4
) priors
obey
probability laws.
This optimization problem is also a knapsack problem. In this formulation, actions that are too expensive
will not be selected under certainty.
7. Expected value of perfect information calculations
We can now easily derive the calculation of the expected value of perfect information using the
formulations under different assumptions (Table 1).
Page 50 of 56Journal of Applied Ecology
Table 1. Expected value of perfect information formulations under different assumptions
Expected value of perfect information Formulation
Single management unit
Single management unit with limited budget
Multiple management units with limited budget
using optimal actions at each unit.
Multiple management units with limited budget
using potentially suboptimal actions at each
unit.
8. Expected value of monitoring information
The expected value of imperfect monitoring (or sampling) information measures the expected
improvement of management outcomes from monitoring the system.
With
the expected value one would acquire when monitoring the system and receiving new
monitoring information Y on which hypothesis is the true hypothesis. Formally,
Where the probability of a hypothesized state s being true given the new data y is updated using Bayes
theorem:
Page 51 of 56 Journal of Applied Ecology
The calculation of
conveniently simplifies to:
We define the cost of the expected value of monitoring as
where represents the cost of monitoring.
9. Expected value of monitoring information under limited budget
Under a limited budget B, the
must account for the cost of managing and monitoring,
, we might therefore not be able to implement some of our initial management actions,
and their management outcomes should not be accounted for. Under a limited budget, finding
is equivalent to solving the following optimization problem:
subject to:
(c
1
) only one action is
selected for each
monitoring result
.
(c
2
) the sum of the
expected cost of the
selected actions and cost of
monitoring is lower than
Page 52 of 56Journal of Applied Ecology
our budget.
(c
3
) the decision variables
identifies which action
to implement for each
monitoring result y.
The expected value of monitoring information with a limited budget is therefore
10. Expected value of monitoring information under limited budget for multiple management
units
First formulation
The expected value of monitoring with a limited budget when dealing with multiple management units is
calculated as follows:
subject to:
(c
1
) only one action is
selected for each
monitoring result
.
(c
2
) the sum of the
expected cost is lower than
our budget.
(c
3
) the decision variables
identifies which unit
.
Page 53 of 56 Journal of Applied Ecology
Similar to the EVPI formulations for multiple units, the calculation of
assumes that we
would exclude the contribution of units that are too expensive.
The expected value of monitoring information with a limited budget for multiple management unit is
therefore
.
Optimal formulation
The formulation of
allows managers to account for the value of monitoring of suboptimal
actions so that the overall cost fits our budget.
subject to:
(c
1
) only one action is
selected for each
monitoring result (and
per management unit j).
(c
2
)
takes {0,1}, and
indicates whether
monitoring cost should
be accounting for at
management unit j.
(c
3
) the sum of the
expected cost of the
selected actions and
Page 54 of 56Journal of Applied Ecology
cost of monitoring is
lower than our budget.
(c
4
) the decision
variables
identifies
which action
to
implement for each
monitoring result y and
management unit j.
The expected value of monitoring information with a limited budget for multiple management unit that
allows suboptimal action to be selected is therefore:
.
This last equation concludes our formulation of the EVSI with a limited budget for multiple management
units.
Page 55 of 56 Journal of Applied Ecology
Table S1: Financial parameters for Case Study 1
Parameter Value
Consultant daily rate for
surveys
$1,000
Days to survey one parcel 0.5
Per parcel survey cost $1000 x 0.5 = $500
Survey cost for 20 parcels $500 x 20 = $10,000
Land value per ha $50,000
Agricultural land tax rate
per year
0.005
Annual tax $50,000 x 0.005 = $250
Cost per parcel of 20-year
agreement
$250 x 20 = $5,000
Agency budget $40,000
Page 56 of 56Journal of Applied Ecology
