Stable emergent heterogeneous agent distributions in noisy environments

Abstract

A mathematical model is introduced for the study of the behavior of a spatially distributed group of heterogenous agents which possess noisy assessments of the state of their immediate surroundings. We define general sensing and motion conditions on the agents that guarantee the emergence of a type of "ideal free distribution" (IFD) across the environment, and focus on how individual and environmental characteristics affect this distribution. In particular, we show the impact of the agents' maneuvering and sensing abilities for different classes of environments, and how spatial constraints of the environment affect the rate at which the distribution is achieved. Finally, we apply this model to a cooperative vehicle control problem and present simulation results that show the benefits of an IFD-based distributed decision-making strategy

Full text

Stable Emergent Heterogeneous Agent Distributions in Noisy Environments
Jorge Finke and Kevin M. Passino
Abstract—A mathematical model is introduced for the study
of the behavior of a spatially distributed group of heterogenous
agents which possess noisy assessments of the state of their
immediate surroundings. We define general sensing and motion
conditions on the agents that guarantee the emergence of a
type of “ideal free distribution” (IFD) across the environment,
and focus on how individual and environmental characteristics
affect this distribution. In particular, we show the impact of the
agents’ maneuvering and sensing abilities for different classes of
environments, and how spatial constraints of the environment
affect the rate at which the distribution is achieved. Finally,
we apply this model to a cooperative vehicle control problem
and present simulation results that show the benefits of an
IFD-based distributed decision-making strategy.
I. INTRODUCTION
The ideal free distribution concept from ecology charac-
terizes how animals optimally distribute themselves across
a finite number of habitats. The word “ideal” refers to the
assumption that animals have perfect sensing capabilities for
simultaneously determining habitat “suitability” (assumed to
be a correlate of Darwinian fitness) for all habitats. Moreover,
the “ideal” part of the IFD assumption supposes that each
animal will move to maximize its fitness. “Free” indicates
that animals can move at no cost and instantaneously to
any habitat regardless of their current location. If an animal
perceives one habitat as more suitable, it moves to this habitat
in order to increase its own fitness. This movement will,
however, reduce the new habitat’s suitability, both to itself
and other animals in that habitat. The IFD is an equilibrium
distribution where no animal can increase its fitness by
unilateral deviation from one habitat to another.
After the IFD notion was introduced in [1]-[2], different
models have been developed based on this concept (so
called IFD models), each trying to explain how different
groups behave as a whole in different environments. In
particular, many of these models try to relax the ideal and
free assumptions of the IFD by taking into account individual
and environmental characteristics, which are essential in
understanding the underlying dynamics of the entire group.
For instance, in [3] the authors discuss the concept of travel
cost and constraints in IFD models (e.g., they consider how
the cost of traveling between habitats might diminish the
expected benefits of moving to another habitat). Here, the
IFD model we introduce extends the one in [4], [5]. Like
This work was supported by the AFRL/VA and AFOSR Collaborative
Center of Control Science (Grant F33615-01-2-3154). We gratefully ac-
knowledge the help from Andy Sparks and Corey Schumacher at AFRL.
J. Finke and K. M. Passino are with the Department of Electrical
and Computer Engineering at The Ohio State University. Please address
correspondence to K. M. Passino, 2015 Neil Avenue, Columbus, Ohio
43210-1272, passino@ece.osu.edu.
in [4], [5] it is built on a graph, so that the graph topology
defines the interconnections between habitats (nodes) via a
set of arcs. By not requiring that every node has an arc to
every other node, the graph topology allows us to represent
removal of both the ideal and free restrictions to the original
IFD model. The author of [6] introduces the concept of
“interference” as the direct effect caused by the presence of
several competitors in the same habitat. Here, we consider a
general class of habitat suitability functions, which allows
us to model environments in which interference between
individuals may noticeably impact group behavior. Other
related studies take into account that animals may differ in
“competitive ability,” as in [7], [8]. Unlike in [1]-[5], and
[6] we consider an approach similar to [7], [8] in that we let
every individual have a certain “capacity,” which is assumed
to be a correlate of its competitive strength, its sensing ability
(e.g., an individual may have noisy sensors), its maneuvering
ability (e.g., its speed or turn radius), or other individual
characteristics that would affect the suitability of the habitat
it settles at. We allow individuals to differ in their capacity,
have different assessments about habitats, and study how
differences in the capacities among individuals affect the
optimal distribution.
The main contributions of this paper are as follows. In
Section II we develop a discrete agent model that captures
individual agents’ motion dynamics across the environment.
We establish a wide class of agent strategies (i.e., “prox-
imate” decision-making mechanisms) that will lead to an
emergent behavior of the group that is a “type of IFD”
(which later, for simplicity we will refer as an IFD). By
this, we mean one of many possible IFD realizations that
are in some sense close to an IFD that is achieved under
the original assumptions [1], [2]. Here we must consider
a wide class of distributions since the sensing noise and
discretization that quantify agent capacity both generally
make it impossible to achieve perfect suitability equalization
as is demanded by the original IFD concept. In Section III
we show how an “invariant set” of spatially distributed
discrete individuals can represent the IFD and use Lyapunov
stability analysis of this set to illustrate that there is a
wide class of resulting agent movement trajectories across
nodes that still achieves a desirable distribution. Finally,
in Section IV we use the problem of dynamic allocation
of vehicles during a cooperative surveillance mission as an
engineering application of the model and results.
II. A DISCRETE AGENT MODEL AND THE ENVIRONMENT
In theoretical ecology, a common approach in modeling
is to assume the existence of a large population in the

environment. Under such an assumption the total number of
individuals in any region or habitat of the environment can
be adequately represented by a continuous variable. Such an
approach was used in [1], [2] and [4], [5]. Here, we extend
the model in [4], [5] to allow for a finite number of discrete
agents. As in [4], [5] we assume that individuals (agents) may
move and distribute themselves over Navailable habitats
(nodes) and let H={1,...N}. Moreover, we define the
suitability of node i∈Has si(xi), where xirepresents the
state of node i. However, we do not require the existence
of a large number of individuals in the environment, and we
assume instead, that xiis described with a discrete variable.
This allows us to capture individual agent characteristics by
taking into account, for example, different agent capacities.
Hence, here we assume that xi∈R+= [0,∞), represents
the total agent capacity at node i, which results from multiple
discrete agents being present at that node. Let there be a
fixed number of agents in the environment. The capacity
of each agent stays constant, so that total agent capacity
C=PN
i=1 xiis fixed. Let εc∈R+be the minimum agent
capacity required to be present at any node i∈H(i.e.,
either so that all suitability functions are well defined at
any state, or as an additional constraint on the environment).
We assume that C > N εc. Note that the value of εcwill
depend on the lowest agent capacity of any agent, and the
minimum number of agents allowed at any node. In fact, we
assume that the total agent capacity in the environment can
be partitioned into discrete blocks. Each block represents a
particular agent, and its size is assumed to be a correlate
of its capacity (competitive capability). We assume that the
largest capacity of any agent in the environment is given by
x > 0, and the smallest capacity of any agent is given by x,
so that x≥x > 0. Moreover, assume the following:
•Node suitability changes relate to total node agent
capacity changes: We assume that for all si(xi),i∈H,
there exist constants ci,ci∈R,ci,ci>0, such that
−ci≤si(yi)−si(zi)
yi−zi
≤ −ci(1)
for any yi, zi∈[εc, C],yi6=zi. Thus, si(xi)is a
strictly monotonically decreasing function in its argu-
ment xi∈[εc, C], so that as the total agent capacity in
node iincreases, the suitability of the node decreases.
Moreover, we assume that limxi→∞ si(xi) = 0 for all
i∈H.
•Strictly positive suitability: We assume that the func-
tions si(xi)>0for all i∈H, and all xi∈[εc, C].
A. Environmental Constraints on Agent Sensing and Motion
The interconnection of nodes is described by a bidirec-
tional graph, (H, A), where A⊂H×H(i.e., a graph where
(i, j)∈Aimplies that (j, i)∈A). We assume that for
every i∈H, there must exist some j∈H,i6=j, such
that (i, j)∈Aand there exists a path between any two
nodes, in order to ensure that every node is connected to the
graph. If (i, j)∈A, this represents that an agent at node
ican sense its neighboring node jand can move from i
to j. According to the definition of (H, A), if an agent is
at iand can move to j(sense the suitability at j), agents
at jcan also move from jto i(sense the suitability at
i, respectively). We also assume that if (i, j)∈A, agents
at node iknow the total agent capacity at node j,xj, and
also xi. However, we do not assume that agents have perfect
sensor capabilities to measure its own or the suitability levels
of its neighboring nodes. In particular, for agents at node
i, where (i, j)∈A, “sensing node j” implies that agents
at node iknow sj(xj) + w, where wis “sensing noise”
that can change over time randomly, but −w≤w≤wfor
known constants w, w ≥0. Let si
j(xj) = sj(xj) + wdenote
the perception (i.e., the noisy measured value) by agents at
node iof the suitability level of node jwith total agent
capacity xj. In some cases one might want to assume that
wdepends on xi. For instance, si
j(xj) = sj(xj) + w(xi)
with w=w, and |w(x′
i)|>|w(x′′
i)| ≥ 0for x′′
i> x′
i
represents sensing conditions where a larger agent capacity
at node iresults in a better suitability perception of its
neighboring node j(e.g., due to better sensing capacities of
the individual agents, agreement strategies among different
agents at the same node that improve their individual sensing
abilities, or averaging strategies which compensate for the
error present in individual suitability assessments). Other
sensing conditions may require that si
j(xj) = sj(xj) + wij ,
where wij is the sensing noise present when agents at
node imeasure the suitability level of node j, in order
to represent that different habitats may be measured with
different accuracy. Here, we simply assume that if w(k)is
the sensing noise present in an agent’s perception at time
k, then it may be that w(k1)6=w(k2)for k16=k2, which
produces a general framework to represent that the sensing
capabilities of the agents may change over time (e.g., as
agents discover their surroundings, their ability to assess the
suitability levels of neighboring nodes may change).
Note that an agent’s perception about the suitability level
of a neighboring node may differ from its actual value by at
most max{w, w}. Also, note that given a node ℓ∈H, and
two neighboring nodes i, j such that (ℓ, i)∈Aand (ℓ, j)∈A
with si(xi)> sj(xj), if si(xi)−sj(xj)>2 max{w, w},
then the measured values of the suitability levels of nodes i
and jby agents at node ℓare such that, sℓ
i(xi)> sℓ
j(xj),
regardless of the sensing noise wpresent during the measure-
ments. In other words, if si(xi)−sj(xj)>2 max{w, w},
then the two sets of all possible measured values of the
suitability levels of the corresponding nodes iand j, given
si(xi)and sj(xj), do not overlap. Conversely, note that
these sets may only overlap if 0< si(xi)−sj(xj)≤
2 max{w, w}. Moreover, if (j, i)∈A, then |sj
i(xi)−
si(xi)| ≤ max{w, w}, and therefore |sj
i(xi)−sj(xj)| ≤
3 max{w, w}. Finally, since |sj
j(xj)−sj(xj)| ≤ max{w, w},
we obtain that |sj
i(xi)−sj
j(xj)| ≤ 4 max{w, w}, regardless
of the noise wpresent during the measurement. Let us define
W= 4 max{w, w}as the maximum difference between
the measured suitability value of a neighboring node and
the perception of the suitability level of the node where the

sensing agents are located, given that the actual suitability
levels of both nodes iand jare close enough (i.e., they do
not differ by more than 2 max{w, w}).
We use the distributed discrete event system model-
ing methodology from [9]. Let Rεc= [εc,∞)and
X=nx∈RN
εc:PN
i=1 xi=Co⊂RN
+be the sim-
plex over which the xidynamics evolve. Let x(k) =
[x1(k), x2(k),...,xN(k)]⊤∈ X be the state vector, where
xi(k)represents the total agent capacity at node iat time
index k≥0. Constraints on our model below will en-
sure that x(k)∈ X for all k≥0. Let I(x) = {i∈
H:xi> εc, x ∈ X } represent the set of nodes at
state x, such that each node i∈I(x)is occupied by a
certain number of agents which results in the total agent
capacity at node iexceeding the value of εc. Similarly,
let U(x) = H−I(x)represent the set of nodes at state
xwhose total agent capacity equals the minimum agent
capacity εc. The size of the set I(x)is denoted by NI. Let
M= maxi{si(xi)−si(xi+x) : for all xi∈[εc, C ]}for
all i∈H. In other words, Mis the maximum change in
suitability that could occur by having an agent of maximum
capacity leave any node. Figure 1 shows an example of a
system with N= 3 nodes and perfect sensing capabilities
so that w=w= 0. Note that a horizontal band of width
M > 0crossing at least one sicurve represents an IFD state
for some total agent capacity in the environment C.
15 10 15 20 25 30 35
Total agent capacity at each resource/node
Suitability functions for resources 1, 2 and 3
s (x )
2
x'3x'2x'1
M
2
IFD pattern xd
IFD realization x'
c
ε=7
s (x )
1 1
s (x )
3 3
s (0)
1
s (0)
2
s (0)
3
M
Fig. 1. Suitability functions si(xi)for three fully connected nodes with
x=x= 1,w=w= 0,εc= 7, and C= 36. Under perfect sensing
conditions the IFD distribution is reached when all agents are distributed
in such a way that at state xneighboring nodes isuch that i∈I(x)have
suitability levels that do not differ by more than M. After the IFD is reached
there is no movement of agents between nodes. For the example shown in
the plot, while agents distribute themselves over nodes 1 and 2, node 3
remains with the minimum agent capacity εcat the desired distribution.
Node i= 3 is called a truncated node. The suitability level s3(εc)is
too low to be chosen by any agent at other nodes. Note also that there
may exist different distributions of the total agent capacity that correspond
to neighboring suitability levels of nodes i∈I(x)differing by at most
M. Each such distribution is called an IFD realization. The light-colored
vertical bands represent all possible distributions of agent capacity for which
the IFD pattern is achieved. We denote the set of all IFD realizations by
Xdand will describe it mathematically in Section III. The dark-colored
vertical bars illustrates a particular distribution x′= [7,12,17]⊤, and its
resultant suitability levels that satisfied the IFD pattern (e.g., note that x′=
[7,11,18]⊤and x′= [7,10,19]⊤would also result in suitability levels
that satisfy the IFD pattern).
For a general graph topology, the best we can generally
hope to do with only local information and a distributed
decision-making strategy under perfect sensing capabilities is
to distribute agent capacities in such a way that the suitability
levels between any two connected nodes remain within M.
In particular, we can guarantee that |si(xi)−sj(xj)| ≤ Mfor
all (i, j)∈Asuch that i, j ∈I(x)at the desired distribution.
Note that the value of Mdepends on the particular shape
of all the suitability functions (i.e., the suitability function
of any node is bounded by Equation (1)), the total agent
capacity in the environment C, and the largest capacity of
any agent x. In particular, note that since Equation (1) applies
for all i∈Hand any yi, zi∈[εc, C], if we let yi=xiand
zi=xi+x, we can bound Mby xmini{ci} ≤ M≤
xmaxi{ci}. Similarly, m= mini{si(xi)−si(xi+x) :
for all xi∈[εc, C]}for all i∈H. Equation (1) guarantees
that M, m > 0.
B. Agent Sensing, Coordination, and Motion Requirements
Let Ebe a set of events and let ei,p(i)
α(i,k)represent the
event that one or more agents move from node i∈Hto
neighboring nodes ℓ∈p(i)at time k, where p(i) = {j:
(i, j)∈A}. Note that movement of agents from node i
to neighboring nodes decreases xisince node ireduces its
total agent capacity and consequently increases si(xi). Let
αℓ(i, k)denote the total agent capacity of the agents that
move from node i∈Hto node ℓ∈p(i)at time k. Let
the list α(i, k) = (αj(i, k), αj′(i, k),...,αj′′ (i, k)) such that
j < j′<··· < j′′ and j, j′,...,j′′ ∈p(i)and αj≥0for
all j∈p(i)represent the total agent capacity of the agents
that move to all neighboring nodes of node i; the size of the
list α(i, k)is |p(i)|and remains constant for all time k≥0
for all i∈H, since the topology of the graph (H, A)is
assumed to be time invariant (i.e., α(i, k)∈R|p(i)|
Cfor all
k, where RC= [0, C]). Let {ei,p(i)
α(i,k)}represent the set of all
possible combinations of how agents can move from node i
to its neighboring nodes for all k. Let the set of events be
described by E=Pnei,p(i)
α(i,k)o − {∅} (P(·)denotes the
power set). Notice that each event e(k)∈ E is defined as
aset, with each element of e(k)representing the transition
of possibly multiple agents among neighboring nodes in the
graph. Multiple elements in e(k)represent the simultaneous
movements of agents, i.e., migrations out of multiple nodes.
An event e(k)may only occur if it is in the set defined by
an “enable function,” g:X → P (E)−{∅}. State transitions
are defined by the operators fe:X → X ,where e∈ E. We
now specify gand fefor e(k)∈g(x(k)), which define the
agents’ sensing and motion:
•If for a node i∈H,si
j(xj)−si
i(xi)≤Mfor all (i, j)∈
A, then ei,p(i)
α(i,k)∈e(k)such that α(i, k) = (0,...,0)
is the only enabled event. Hence, agents at the most
suitable node that they know of do not move.
•If for node i∈H,si
j(xj)−si
i(xi)> M for some j
such that (i, j)∈A, then the only ei,p(i)
α(i,k)∈e(k), are

ones with α(i, k) = (αj(i, k) : j∈p(i)), such that:
(i)xi(k)−X
ℓ∈p(i)
αℓ(i, k)≥εc
(ii)si
i
xi(k)−X
ℓ∈p(i)
αℓ(i, k)

<max
j{si
j(xj(k)) : j∈p(i)} − W
(iii)If αj(i, k)>0for some j∈p(i),then
αj∗(i, k)≥xfor some
j∗∈ {j:si
j(xj(k)) ≥si
ℓ(xℓ(k)) for all ℓ∈p(i)}
(iv)αj(i, k) = 0 for any j∈p(i)such that
si
i(xi(k)) > si
j(xj(k)) and xj(k) = εc
Condition (i)guarantees that at any node there is at
least εcagent capacity. It is required so that conditions
(ii)and (iii)are well defined at all times. To interpret
conditions (ii)−(iv)it is useful to note that reducing
(increasing) the total agent capacity at a node always
increases (decreases, respectively) the suitability at that
node. The three conditions constrain how agents can
move based on their capacities and in terms of node
suitabilities. Note that agents may also move from
higher suitability nodes to lower suitability nodes as
long as all conditions are satisfied. Without condition
(ii), there could be a sustained migration oscillation
between nodes. Condition (iii)implies that at least one
agent must move to the neighboring node perceived
with the highest suitability. Without condition (iii)
some high suitability node could be ignored by the
agents and the IFD distribution might not be achievable.
Condition (ii)together with condition (iii)guarantees
that the highest suitability node is strictly monotonically
decreasing over time. Finally, without condition (iv)
some agents would still be free to move to nodes
with lower suitability levels, and the desired distribution
would not be maintained.
•If e(k)∈g(x(k)),ei,p(i)
α(i,k)∈e(k), then x(k+ 1) =
fe(k)(x(k)), where xi(k+ 1) equals xi(k)plus
X
{j:i∈p(j),ej,p(j)
α(j,k)∈e(k)}
αi(j, k)−X
{j:j∈p(i),ei,p(i)
α(i,k)∈e(k)}
αj(i, k)
Note that if x(0) ∈ X ,x(k)∈ X ,k≥0.
Let ENdenote the set of all infinite sequences of events
in E. Let Ev⊂ ENbe the set of valid event trajectories
for the model (i.e., ones that are physically possible). Event
e(k)∈g(x(k)) is composed of a set of what we will call
“partial events.” Define a partial event of type ito represent
the movement of α(i, k)agents from node i∈Hto its
neighbors p(i)so that conditions (i)−(iv)are satisfied at
time k. A partial event of type iwill be denoted by ei,p(i)and
the occurrence of ei,p(i)indicates that some agents located at
node i∈Hmove to other nodes. Partial events must occur
according to the “allowed” event trajectories. The allowed
event trajectories define the degree of asynchronicity of the
model at the node level. We define two possibilities for the
allowed event trajectories:
First, for allowed event trajectories Ei⊂Ev, assume that
each type of partial event occurs infinitely often on each
event trajectory E∈Ei. The assumption is met if at each
node all agents do not ever stop trying to move (e.g., if each
agent persistently tries to move to neighboring nodes). This
corresponds to assuming “total asynchronism” [10].
Second, for allowed event trajectories EB⊂Ev, assume
that there exists B > 0, such that for every event trajectory
E∈EB, in every substring e(k′),...,e(k′+ (B−1)) of
Ethere is the occurrence of every type of partial event (i.e.,
for every i∈H, the partial event ei,p(i)∈e(k), for some k,
k′≤k≤k′+B−1). This corresponds to assuming “partial
asynchronism” [10].
III. EMERGENT AGENT DISTRIBUTION
The set
Xd={x∈ X :for all i∈H, either |si(xi)−sj(xj)|
≤M+Wfor all j∈p(i)such that xj6=εc
and si(xi)> sj(xj)for all j∈p(i)such that
xj=εc,or xi=εc}(2)
is an invariant set that represents all possible distributions
of the total agent capacity Cat the IFD since for x∈ Xd,
|si(xi)−sj(xj)| ≤ M+Wfor all i, j ∈I(x)such that
(i, j)∈A, and si(xi) = si(εc)for all i∈U(x). It can
be shown that according to the definition of the enable
function gthere is no agent movement between nodes, so
that α(i, k) = (0,...,0) for all i∈Hwhen x(k)∈
Xd. Moreover, note that there exist many different agent
distributions that belong to Xd. Any agent distribution such
that the distribution of the total agent capacities x∈ Xdis an
IFD realization. Note that according to the definition of Xdit
is possible for unconnected nodes (i.e., ones such that (i, j)/∈
A) in the set I(x)to have suitabilities that differ by more
than Mwhen the distribution is achieved. This could happen
if two nodes i, j such that i, j ∈I(x)with high suitability
levels when x∈ Xdare separated by a node with minimum
agent capacity (e.g., in an environment represented by a line
topology of the graph (H, A)). However, any two nodes that
are linked according to the graph (H, A)(i.e., ones such that
(i, j)∈A)and belong to the set I(x)must have suitability
levels that differ at most by M+Wat the desired distribution.
Hence, depending on the graph’s connectivity, there could be
isolated “patches” of nodes where only nodes belonging to
the same patch have suitability levels that differ by at most
M+W(i.e., forming an environment of different patches).
Moreover, note that the formation of patches depends on the
total agent capacity in the environment, the initial distribution
x(0), and random agent migration between nodes.
Theorem 1 (Stability for a fully connected environment,
any total agent capacity): Given a fully connected graph
(H, A),εc>0, any population size with total agent capacity
C, and agent motion conditions (i)−(iv), the invariant set
Xdis asymptotically stable in the large with respect to Ei
and exponentially stable in the large with respect to EB.

Due to space constraints we do not include any proofs
here. For detailed information about the proofs of any of the
theorems the reader should contact the authors.
Note that asymptotic/exponential stability in the large
implies that for any initial distribution of agent capacity, the
invariant set will be achieved. This result provides general
sufficient conditions on when a distribution satisfying the
IFD pattern is achieved. However, the size of Xdis not
necessarily one, since there are many possible IFD real-
izations that may be achieved. Theorem 1 guarantees that
under the above stated sensing and motion conditions one
of them will be reached. Moreover, our analysis considers
all environments which can be modeled by a wide class of
suitability functions. It includes functions which have been
found to be useful in biology, like the one originally used to
introduce the IFD concept in [1], and the one in [8] which
introduced the interference model, among others.
Note also that Theorem 1 requires εc>0because if
εc= 0 at a truncated node i, then si(xi)equals infinity
for certain suitability functions (e.g., si(xi) = ai
xi). The
proof of Theorem 1 considers the dynamic emergence of
different patches when the environment is modeled by a
fully connected topology. Patches emerge as agents distribute
themselves over the nodes, and the total agent capacity is
small enough.
Theorem 2 (Stability for a not fully connected envi-
ronment, but sufficient total agent capacity): Given any
(H, A),εc≥0, and agent motion conditions (i)−(iv),
there exists a constant C > N εcsuch that if the total agent
capacity in the environment is at least C, then the invariant
set Xdis asymptotically stable in the large with respect to
Eiand exponentially stable in the large with respect to EB.
Theorem 2 considers a general interconnection topology,
which allows us to consider less restrictive agent sensing
and motion abilities. For this case we show that for a large
enough total agent capacity Cthere are no isolated patches
in the environment at the desired distribution. Theorem 2 is
an extension of the load balancing [10] theorems in [9], [11]
to the case when the “discrete virtual load” is a nonlinear
function of the state.
IV. APPLICATION: COOPERATIVE VEHICLE CONTROL
Suppose we wish to design a multi-vehicle guidance
strategy to enable a group of vehicles to perform surveillance
of some region where the goal is to make the proportion
of vehicles visiting a set of predefined areas match the
relative importance of monitoring each area. This vehicle
distribution goal must be achieved in spite of vehicle sensing,
communication, and motion constraints (the combination of
which requires a decentralized vehicle guidance strategy with
each vehicle making independent decisions). Assume that the
ith vehicle obeys a Dubin’s model with (constant) velocity
vand minimum turn radius T(i.e., vehicles will either
travel on the minimum turning radius or on straight lines).
Assume also that the region under surveillance can be divided
into Nequal-size square ℓ×ℓareas. These areas are the
nodes i∈H. The connectedness of the areas is modeled
by the topology of the graph (H, A). We assume that new
targets continually pop-up at points in the surveillance region
according to some stochastic process. We let Richaracterize
the (average) rate of appearance of pop-up targets in area
i, and assume it is constant but unknown to the vehicles.
We assume that pop-up target locations in area iare known
only to vehicles currently in iand that they stay exposed
until they are visited by some vehicle. When a vehicle
starts approaching a target, the target is considered to be
“attended,” and a vehicle may visit a new target only after
the target being approached has been reached. Once the target
is reached, the vehicle may perform various tasks and it is
then ignored for the rest of the mission.
The suitability level of an area is defined as the (average)
rate of appearance of unattended targets (i.e., targets which
have appeared but are not being or have not been attended
by any vehicle). Figure 2 shows two classes of suitability
functions for different intra-area vehicle coordination strate-
gies and target pop-up rates Ri. The left plot assumes that
vehicles located in the same area coordinate in order to
decide which targets within that area to attend (i.e., after a
target is reached, a vehicle approaches the closest target that
is not being approached by any other vehicle). The right plot
assumes that vehicles located over the same region do not
coordinate and they randomly approach any target located
within the area they are monitoring. Here, since our focus
is on the relative proportioning of area monitoring and not
intra-area coordination, we use the no intra-area coordination
approach in the remainder of the paper (conceptually similar
results to those below are obtained for specific intra-area
coordination).
0 2 4 6 8 10
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
Number of vehicles x
Suitability functions s (x ) for different R
0 2 4 6 8 10
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
R = 1/2
i
R = 1/2
i
R = 2/5
i
R = 1/3
i
R = 1/4
i
R = 1/4
i
R = 2/5
i
R = 1/3
i
ii i
iNumber of vehicles xi
Fig. 2. Suitability functions si(xi)for an area with ℓ= 2.5km and
vehicles at speed v= 15m/s, T= 100m with (left) and without (right)
intra-area coordination strategies to decide which targets within the area to
approach. Each data point represents 60 simulation runs with varying target
pop-up locations. The error bars are sample standard deviations from the
mean.
Assume that the vehicles only have noisy perceptions
about the suitability levels of the area they are monitoring
and its neighboring areas. To define the perception by any
vehicle about the suitability level of an area, we use a system
identification approach to determine a parameterized model
of the expected suitability function of that area, ˆsi(xi). In
particular, under the above assumptions and according to
Figure 2, for a fixed Tthe expected suitability functions

are of the form
ˆsi(xi) = ˆ
Ri−ˆr(v, ℓ)xi(3)
for all i∈H, where ˆ
Riis the expected target pop-up rate for
area i(targets/s), and ˆr(v, ℓ)is the expected rate of targets
being attended by each vehicle moving at speed vin an
area of size ℓ×ℓ(targets/s/vehicle). A vehicle’s perception
about the suitability level of an area will depend on how
the different parameters in Equation (3) are affected by its
limited sensing and maneuvering capabilities.
While maneuvering constraints on the vehicles (i.e., an
increasing minimum turn radius) may diminish the expected
rate ˆr(v, ℓ)for all vehicles in an area, the expected suitability
function shape stays the same as in Equation (3). Further-
more, note that in many applications, knowing the value of
ˆ
Riin Equation (3) usually requires that vehicles estimate
the number of targets that have appeared in that area in a
time window divided by the length of that window. Here we
assume that vehicles have good sensing capabilities and use a
large enough window in estimating the rate of appearance of
targets (e.g., so that vehicles monitoring area ican ultimately
obtain ˆ
Riand ˆ
Rjfor all j∈p(i)within 10% of Riand Rj,
respectively).
We define the perception by a vehicle located over area
iabout the suitability level of a neighboring area jas
si
j(xj) = ˆsj(xj)and this will be used in the movement rules
defined in Section II-B. As the mission progresses, vehicles
decide to move from one area to another only if the proposed
conditions (i)−(iv)are satisfied. Figure 3 shows two typical
different IFD realizations for 20 vehicles in a region divided
into four areas, and where a line topology is used. While the
plots illustrate that good vehicle surveillance distributions
are achieved, different IFD realizations can emerge due to
the discrete nature of vehicle capabilities (compare left and
right plots).
200 400 600 800
0
Time (s)
200 400 600 800
0
0.02
0.04
0.06
0.08
0.1
0.12
Time (s)
Suitabilities (targets/s)
Area 1
Area 2
Area 3
Area 4
Fig. 3. Two possible IFD realizations for vehicles deployed in an
environment divided into four areas connected by a line topology.
Next, using ideas from [10] we define two cooperative
sensing strategies to try to reduce the effects of the perception
noise won the mission performance. In particular, we assume
that every vehicle that is able to measure the suitability level
of an area, will cooperate with other vehicles by sharing with
them its own perception about that area. We first implement
a synchronous averaging strategy, where at any time k
all vehicles may exchange their current perceptions about
neighboring areas, and any vehicle evaluating conditions
(i)−(iv)uses the average value of all sensing vehicles in
order to define its current perception about an area. Note that
such an approach generally requires a fast and synchronized
communication network. Hence, we define an asynchronous
agreement algorithm, where those vehicles able to measure
the suitability of area itry to reach a common value
by exchanging their perceptions and combining them by
forming convex combinations. Figure 4 shows an example of
the typical different IFD realizations for these two strategies
and the no-cooperative sensing case (i.e., where vehicles just
use their own perception to evaluate conditions (i)−(iv)).
Note that the ultimate distribution has less variation when
cooperative sensing is used. We have also run Monte Carlo
simulations that show that when the ultimate distribution has
less variation vehicles require more time to achieve it.
200 400 600 800
0
0.02
0.04
0.06
0.08
0.1
0.12
Suitabilities (targets/s)
Time (s)
200 400 600 800
0
Time (s)
200 400 600 800
0
Time (s)
Area 1
Area 2
Area 3
Area 4
Fig. 4. Effects of implementing a synchronous and partially asynchronous
iterative methods to try to reduce the effects of the sensing noise won
the mission performance with 20 vehicles; no cooperative sensing (left),
agreement strategy (middle), averaging strategy (right).
REFERENCES
[1] S. D. Fretwell and H. L. Lucas, “On territorial behavior and other
factors influencing distribution in birds,” Acta Biotheoretica, vol. 19,
pp. 16–36, 1970.
[2] G. A. Parker, “The reproductive behaviour and the nature of sexual
selection in scatophaga stercoraria,” Evolution, vol. 28, pp. 93–108,
1974.
[3] M. Astrom, “Travel cost and the ideal free distribution,” OIKOS,
vol. 69, pp. 516–519, 1994.
[4] J. Finke and K. M. Passino, “Stable cooperative multiagent spatial
distributions,” in Proceedings of the IEEE Conference on Decision
and Control and the European Control Conference, (Seville, Spain),
December 2005.
[5] J. Finke and K. M. Passino, “Local agent sensing, coordination, and
motion requirements for stable emergent agent group distributions,”
submitted to the IEEE Transactions on Automatic Control, 2005.
[6] W. J. Sutherland, “Aggregation and the ideal free distribution,” Journal
of Animal Ecology, vol. 52, pp. 821–828, 1983.
[7] W. J. Sutherland, From Individual Behaviour to Population Ecology.
New York: Oxford University Press, 1996.
[8] G. A. Parker and W. J. Sutherland, “Ideal free distributions when
individuals differ in competitive ability: phenotype-limited ideal free
models,” Animal Behavior, vol. 34, pp. 1223–1242, 1986.
[9] K. M. Passino and K. L. Burgess, Stability Analysis of Discrete Event
Systems. John Wiley and Sons, Inc., NY, 1998.
[10] D. Bertsekas and J. Tsitsiklis, Parallel and Distributed Computation:
Numerical Methods. Belmont, Massachusetts: Athena Scientific, MA,
1997.
[11] K. L. Burgess and K. M. Passino, “Stability analysis of load balancing
systems,” Int. Journal of Control, vol. 61, pp. 357–393, February 1995.