Content uploaded by Zhongjing Ma
Author content
All content in this area was uploaded by Zhongjing Ma on May 19, 2016
Content may be subject to copyright.
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013 67
Decentralized Charging Control of Large Populations
of Plug-in Electric Vehicles
Zhongjing Ma, Duncan S. Callaway, Member, IEEE, and Ian A. Hiskens, Fellow, IEEE
Abstract—This paper develops a strategy to coordinate the
charging of autonomous plug-in electric vehicles (PEVs) using
concepts from non-cooperative games. The foundation of the
paper is a model that assumes PEVs are cost-minimizing and
weakly coupled via a common electricity price. At a Nash equi-
librium, each PEV reacts optimally with respect to a commonly
observed charging trajectory that is the average of all PEV
strategies. This average is given by the solution of a fixed point
probleminthelimitofinfinite population size. The ideal solution
minimizes electricity generation costs by scheduling PEV demand
to fill the overnight non-PEV demand “valley”. The paper’s
central theoretical result is a proof of the existence of a unique
Nash equilibrium that almost satisfies that ideal. This result is
accompanied by a decentralized computational algorithm and a
proof that the algorithm converges to the Nash equilibrium in
the infinite system limit. Several numerical examples are used
to illustrate the performance of the solution strategy for finite
populations. The examples demonstrate that convergence to the
Nash equilibrium occurs very quickly over a broad range of
parameters, and suggest this method could be useful in situations
where frequent communication with PEVs is not possible. The
method is useful in applications where fully centralized control is
not possible, but where optimal or near-optimal charging patterns
are essential to system operation.
Index Terms—Decentralized control, Nash equilibrium, non-co-
operative games, optimal charging control, plug-in electric vehicles
(PEVs), plug-in hybrid electric vehicles (PHEVs).
I. INTRODUCTION
VEHICLES that obtain some or all of their energy from
the electricity grid (including pure electric vehicles and
plug-in hybrids) may achieve significant market penetration
over the next few years. Such vehicles, which we generically
refer to as plug-in electric vehicles (PEVs), will reduce con-
sumption of exhaustible petroleum resources and may reduce
Manuscript received December 13, 2010; revised June 24, 2011; accepted
August 31, 2011. Manuscript received in final form October 25, 2011. Date of
publication November 15, 2011; date of current version December 14, 2012.
Recommended by Associate Editor S. Varigonda. This work was supported in
part by the Michigan Public Service Commission under Grant PSC-08-20, by
the National Science Foundation under EFRI-RESIN Grant 0835995, and by the
Excellent Young Scholars Research Fund of Beijing Institute of Technology.
Z. Ma is with the School of Automation and the Key Laboratory of Complex
System Intelligent Control and Decision, Beijing Institute of Technology, Min-
istry of Education, Beijing 100081, China (e-mail: mazhongjing@bit.edu.cn).
D. S. Callaway is with the Energy and Resources Group, University of Cali-
fornia, Berkeley, CA 94720 USA (e-mail: dcal@berkeley.edu).
I. A. Hiskens is with the Department of Electrical Engineering and Com-
puter Science, University of Michigan, Ann Arbor, MI 48109 USA (e-mail:
hiskens@umich.edu).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TCST.2011.2174059
Fig. 1. Typical (non-PEV) base demand in summer for the region managed by
the MISO. An ideal valley-filling profile is also shown.
pollutant emissions including greenhouse gases. As their pop-
ulation grows, however, electric power system operation will
become more challenging. For example, if a large number of
PEVs began charging around the time most people finish their
evening commute, a new demand peak could result, possibly
requiring substantial new generation capacity and ramping
capability [1].
In the United States, the average vehicle is driven about 28
miles per day, and the average one-way commute time is around
25 min [2]. This implies PEVs will spend a significant amount of
time parked and available to charge. On the other hand, charging
times for electric vehicles are likely to be 8 h or less, well under
the time that most PEVs will actually be available for charging.
This suggests that there could be useful flexibility with respect
to when and how fast vehicles charge. If vehicle charging can be
coordinated, it could be possible to construct aggregated charge
profiles that avoid detrimental power system impacts and mini-
mize system-wide costs.
A number of recent studies have explored the potential im-
pacts of high penetrations of PEVs on the power grid [3]–[6]. In
general, these studies assume that PEV charging patterns “fill
the valley” of night-time demand. For example, the overnight
dipinFi
g. 1 would be replaced by a total demand that remained
relatively constant during the charging period. However, these
studies do not address the issue of how to coordinate PEV
charging patterns. Possible coordination strategies can be
divided into the following two categories.
1) In centralized strategies, a central operator dictates
precisely when and at what rate every individual PEV
will charge. Decisions could be made on the basis of
system-level considerations only, or they could factor
vehicle-level preferences, for example desired charging
1063-6536/$26.00 © 2011 IEEE
68 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013
interval, final state of charge, and budget. These strategies
could be further distinguished by whether they attempt
to identify charge patterns that are in some way optimal,
or instead follow rules-of-thumb that seek to achieve
aggregate charging patterns that are reasonably close to
optimal.
2) Decentralized or distributed strategies allow individual
PEVs to determine their own charging pattern. Vehicle
charging decisions could, for example, be made on the
basis of time-of-day or electricity price. The outcome of
a decentralized approach may or may not be optimal, de-
pending on the information and methods used to deter-
mine local charging patterns. Care must be taken to ensure
charging strategies cannot inadvertently synchronize the
responses of large numbers of PEVs, as the resulting abrupt
changes in aggregate demand could potentially destabilize
grid operations [7].
Though the valley-filling charging pattern is conceptually
straightforward, the process of achieving it in a practical
manner with a large population of independent PEVs presents a
number of challenges. For example, a centralized approach may
not be palatable to consumers, who are accustomed to having
complete decision-making authority over their electricity con-
sumption patterns. On the other hand, a decentralized strategy
may preserve individual authority. However, as we will discuss
below, the challenge of coordinating many autonomous agents
to achieve an optimal or near-optimal outcome is non-trivial.
This paper explores the potential of decentralized strategies.
Because time-based and fixed schedule price-based strategies
have difficulty effectively filling the night-time valley [7], we
instead focus on methods that utilize real-time marginal elec-
tricity price information. We allow each PEV to choose and im-
plement its own local charging control strategy, with the aim of
minimizing its individual charging cost.
The rest of the paper is organized as follows. Section II
frames this work in the context of existing research, especially
in the area of game theory. In Section III we formulate charging
control problems for large populations of PEVs. Section IV de-
velops a decentralized control strategy for optimally charging
an infinite population of PEVs, and establishes existence
and uniqueness properties of the resulting Nash equilibrium.
Section V presents the control strategy as an algorithm that also
applies to finite systems. The valley filling property of the Nash
equilibrium is verified in Section VI. Finite population numer-
ical examples are presented in Section VII. In Section VIII, we
provide conclusions and suggest various extensions.
A summary of the notation used throughout the paper is pro-
vided in Table I.
II. RELATIONSHIP TO CONTROL AND
GAME-THEORETIC LITERATURE
The decentralized PEV charging control problem studied in
this paper is a form of non-cooperative game, where a large
number of selfish PEVs share electricity resources on a finite
collection of charging intervals. This charging game falls within
the class of potential games identified by Monderer and Shapley
[8]. It is mathematically equivalent to routing and flow control
games in telecommunications, where networks of parallel links
TAB L E I
LIST OF KEY SYMBOLS
are congested [9]. From this point of view, PEV charging games
are conceptually similar to network games [10].
Substantial work has been presented in the literature on the
computation of Nash equilibria, or -Nash equilibria, for poten-
tial games, especially in relation to network games. Research
on centralized mechanisms includes Christoudoulou et al. [11],
who consider two classes of potential games, selfish routing
games and cut games, and Even-Dar et al. [12] who study the
number of steps required to reach a Nash equilibrium in load
balancing games. Research on decentralized or distributed
mechanisms includes Berenbrink [13], who propose a strongly
distributed setting for load balancing games such that all agents
update their strategy simultaneously. Also, Even-Dar et al. [14]
present convergence results for an approximate -Nash equi-
librium under a non-centralized setting in routing games, and
Fischer et al. [15], [16] propose a distributed and concurrent
process for convergence to Wardrop equilibria [17] in adaptive
routing problems.
We note that non-cooperative game theory is widely used to
study the supply side of electricity markets, especially in the
context of imperfect competition (see, for example, [18]–[21]).
Some game theoretic work has been done to understand de-
mand-side behavior in the face of dynamic pricing tariffs, pri-
marily in the context of demand aggregators [22], [23]. Though
this paper also applies game-theoretic principles to understand
outcomes in electricity markets, to our knowledge it is unique
in that: 1) we are examining multi-period demand-side behavior
with a local energy constraint applied to the total energy con-
sumed across all periods and 2) we are analyzing the problem
from a decentralized perspective.
In this paper, PEVs are coupled through a common price
signal which is determined by the average charging strategy
of the PEV population. Therefore each PEV effectively inter-
acts with the average charging strategy of the rest of the PEV
population. As the population grows substantially, the influence
of each individual PEV on that average charging strategy be-
comes negligible. Accordingly, in the infinite population limit,
all PEVs will observe the same average strategy as they calcu-
late their optimal local strategy. In this situation, a collection of
MA et al.: DECENTRALIZED CHARGING CONTROL OF LARGE POPULATIONS OF PLUG-IN ELECTRIC VEHICLES 69
local charging controls is a Nash equilibrium, if the following
are met:
1) each PEV’s charging strategy is optimal with respect to
a single commonly observed charging trajectory;
2) the average of all the local optimal charging strategies is
equal to that common trajectory.
This paper presents a novel procedure that identifies a Nash
equilibrium by simultaneously updating each PEV’s best in-
dividual strategy with respect to the average charging strategy
of the whole population. It is proposed that this procedure
would be undertaken prior to the actual charging interval.
The computation time of this algorithm is unrelated to the
number of PEVs, since they simultaneously and independently
update their charging strategy. Under certain mild conditions,
the proposed decentralized charging control procedure drives
the system asymptotically to a unique Nash equilibrium that
is nearly globally optimal (valley filling). In the case of ho-
mogeneous populations, where all vehicles have identical
parameters, this unique Nash equilibrium becomes a perfect
valley-filling charging strategy.
The proposed algorithm converges to an -Nash equilibrium
for a finite population, and tends to zero as the population size
approaches infinity. In this limiting case, the Nash equilibrium
corresponds to a Wardrop equilibrium [17]. This method also
has connections with the Nash certainty equivalence principle
(or mean-field games), proposed by Huang et al. [24], [25] in the
context of large-scale games for sets of weakly coupled linear
stochastic control systems.
III. CHARGING CONTROL OF LARGE PEV POPULATIONS
In this section we introduce the basic PEV charging dy-
namics, and optimization models that are relevant for both the
centralized and decentralized framework. In the decentralized
case, we will also formally define the conditions for a Nash
equilibrium.
A. Model and Notation
We consider charging control of a significant PEV population
of size over charging horizon where
denotes the terminal charging instant. The population of PEVs
is denoted . For an individual PEV , we adopt
the notation of Table I. The state of charge (SOC) of PEV at
instant is given by , and the SOC dynamics are
described by the simplified model
(1)
with initial SOC , battery size and charger efficiency
.
The charging control trajectory of PEV , denoted
,isanadmissible charging control, if it belongs
to the set
(2)
where
(3)
The set of admissible charging controls for the entire PEV pop-
ulation is denoted by . It follows from the SOC dynamics de-
scribedin(1)that for any admissible control
.
Subject to an admissible charging control ,thecost
associated with delivering the total system demand is given by
(4)
where denotes the collection of PEV
charging rates at time ,is the electricity charging
price at instant ,and is the total inelastic non-PEV demand
at instant . We assume the electricity charging price
is determined by the ratio between the total demand and the total
generation capacity, so
(5)
where denotes the total generation capacity. The impor-
tance of the dependence of and on is discussed later
in this section.
Note that this definition of price differs from typical retail
electricity tariffs, which are either constant or change according
to a fixed (demand-independent) schedule in time. Instead, price
varies in proportion to total demand; this is how wholesale elec-
tricity market prices (and therefore the real-time marginal price
of electricity) vary. In this formulation, we assume for sim-
plicity that electricity price is a function only of instantaneous
demand. We are aware that in practice, price in any given hour
will be influenced by demand in all hours. This is because the
mix of units available in a given hour is determined by the unit
commitment process [26], which is a function of demand over
the entire commitment interval. We note that this price defini-
tion does not consider transmission congestion or losses, under
the assumption that they are negligible at night.
This paper studies systems where the number of PEVs is suf-
ficiently large that the action of each individual PEV on the
system is negligible, but the action of the aggregation of PEVs
may be significant. For this reason we will examine asymptotic
properties of the system in the large limit. To ensure that key
properties are preserved at that limit, we assume that non-PEV
demand and total generation capacity vary with the number of
PEVs and make the following asymptotic assumptions as PEV
population size approaches infinity
(6)
The implication inherent in (6) is that larger power systems, with
greater capacity and base demand, are required to support large
numbers of PEVs. Direct substitution into (5) gives
(7)
70 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013
where
(8)
Definition of Valley-Filling
We define valley-filling charging as follows:
(9)
for some constant .Inwords,inhourswhen this
strategy chooses total PEV demand such that system-wide de-
mand is equal to ; otherwise PEV demand is zero. The value
of uniquely determines the total energy supplied for charging.
This is illustrated in Fig. 1, where corresponds to the level of
the horizontal line.
The following lemma establishes sufficient conditions
for optimal centrally-determinedchargingstrategiestobe
valley-filling.
Lemma 3.1: If is convex and increasing on ,
then given non-PEV demand , the optimal charging strategy
is valley-filling.
Proof: DefineaLagrangian
where and are Lagrange multipliers. Since this is a
convex optimization problem the Karush-Kuhn-Tucker condi-
tions ensure the following optimality conditions:
(10)
(11)
(12)
(13)
where (12) holds with complementary slackness. The right-hand
side of (10) is independent of , and complementary slackness
requires that when . Therefore, because is
convex and increasing, must be constant for all when
.Itfollowsthat(9)musthold.
It will be shown in Section VI that in the case of a homoge-
neous population of PEVs, the proposed decentralized control
process achieves this same minimum-cost strategy.
B. Decentralized Charging Control of Large Populations of
PEVs
In the rest of the paper we will study a decentralized game-
based charging control strategy for large PEV populations. This
subsection develops the mathematical framework for this anal-
ysis, and establishes sufficient conditions for the decentralized
problem to achieve a Nash equilibrium.
Consider the local cost function for an individual PEV
subject to a collection of charging controls
(14)
The locally optimal charging control problem with respect to a
fixed collection of controls is given by the minimization
(15)
where ,inotherwords
denotes the collection of control strategies of all PEVs except
the th. If a minimizing function exists, it will be referred to as
an optimal control law for the local charging control problem.
We are now in a position to formalize the definition of a Nash
equilibrium in the context of PEV charging strategies.
Definition 3.1: A collection of PEV strategies
is a Nash equilibrium if each PEV cannot benefitbyunilater-
ally deviating from its individual strategy ,i.e.,
for all and all
In Section V, we propose an iterative algorithm for obtaining
the Nash equilibrium. At each iteration, every PEV optimizes its
strategy relative to determined in the previous iteration. As we
will show in Section VII, specifically Fig. 4, charging intervals
with high PEV demand at one iteration tend to induce low PEV
demand at the following iteration, and vice versa. This occurs
because PEVs move their charging requirements from expen-
sive to inexpensive intervals; the resulting changes in demand
reduce the marginal electricity price in the previously expensive
intervals and raise the price in previously inexpensive intervals.
This establishes an oscillatory pattern from one iteration to the
next, preventing convergence to the Nash equilibrium.
To mitigate these oscillations, the local cost function (14) is
modified to include a quadratic term that penalizes the deviation
of an individual control strategy from the population average:
(16)
where determines the magnitude of the penalty for deviating
from the mass average. It will be shown that the presence of the
squared deviation term ensures convergence to a unique collec-
tion of locally optimal charging strategies that is a Nash equi-
librium. Unfortunately, because of the penalty term, this Nash
equilibrium only coincides with the globally optimal strategy
(9) when all PEVs are identical (homogeneous). Nevertheless,
we will see that the cost added due to this term can be quite
small compared with the electricity price .
Given the formal definition of a Nash equilibrium provided
by Definition 3.1, and the local cost function (16) for each PEV,
we can now establish the conditions governing a Nash equilib-
rium for an infinite population of PEVs.
MA et al.: DECENTRALIZED CHARGING CONTROL OF LARGE POPULATIONS OF PLUG-IN ELECTRIC VEHICLES 71
Theorem 3.1: A collection of charging strategies for
an infinite population of PEVs is a Nash equilibrium, if the fol-
lowing occur:
1) for all ,minimizes the cost function
(17)
with respect to a fixed ;
2) ,forall ,i.e., can be reproduced by
averaging the individual optimal charging trajectories of
all PEVs.
Proof: Consider the collection of PEV charging strategies
where each minimizes its corresponding
cost function (17) with respect to the common, fixed trajectory
,and for all .
As the population size approaches infinity, each individual
PEV’s charging strategy has negligible influence on the pop-
ulation average . Therefore, for every
(18)
Usingthisrelationshipin(16)givesforevery
(19)
Each minimizes , and so by (19) also minimizes
. Hence, by Definition 3.1, is a Nash
equilibrium.
As mentioned earlier, PEV charging games are consistent
with the Nash certainty equivalence principle (also known as
mean-field games). The key similarity is that individual agents
do not consider the behavior of other individuals. Instead indi-
viduals are influenced by the so-called “mass effect”, i.e., the
overall effect of the population on a given agent. In the case
of PEV charging, the effect felt by all individuals is the elec-
tricity price, which we specify as a function of the mass average
charging trajectory .
IV. IMPLEMENTATION FOR INFINITE SYSTEMS
In this section we study the existence and uniqueness of the
Nash equilibrium for the decentralized charging optimization
defined by (16). Section IV-A derives the local optimum with
respect to an arbitrary mass average. Proofs of existence and
uniqueness are presented in Section IV-B.
To obtain these results we will assume the PEV population
size is infinite. Obviously, any implementation of the control
strategy must work for finite PEV populations. We will discuss
this issue further in Sections V and VII, and show that the infi-
nite population results apply to finite systems.
A. Locally Optimal Charging Control
Lemma 4.1 determines the optimal charging trajectory
for an individual PEV when it is subjected to a fixed
trajectory .
Lemma 4.1: Consider a fixed trajectory , and the control
trajectory defined by
for all (20)
For a particular value of , uniquely dependent upon and de-
noted , the trajectory (20) provides the unique optimal
control minimizing the cost function given in (17),
subject to the admissible control requirement de-
fined in (2).
Proof: We define a Lagrangian
where is the Lagrange multiplier. Since is convex
with respect to , the local charging control that minimizes
, subject to , must satisfy the following:
(i) ;
(ii) , with complementary slackness.
Condition (i) recovers the constraint . We can de-
rive from condition (ii)
when
otherwise
(21)
which is equivalent to (20).
The form of the dependence of on expressed in
(20) ensures that, for any fixed .
• There exists an such that for ,
.
•For ,is strictly increasing with
, with the relationship continuous, though not smooth.
Hence, is invertible over this domain.
Therefore a constraint defines a unique
for each fixed , which may be written .The
particular value of that ensures satisfaction of the constraint
shall be denoted . The resulting control tra-
jectory can be written .
Since is convex with respect to , the minimizing
control defined by (20) must be unique for a given .
We now turn to identifying conditions that guarantee ex-
istence and uniqueness of the Nash equilibrium identified in
Section III-C. Before doing so we will introduce additional
notation. We denote by
the charging strategy that minimizes the local cost func-
tion (17) with respect to a fixed . By Lemma 4.1, we have
.Wedefine another local control
strategy for PEV .This
describes a local charging control satisfying (20) with respect
to and . There is no guarantee that .
B. Existence and Uniqueness of the Nash Equilibrium
Lemma A.1, presented in Appendix A, establishes several im-
portant properties of the control trajectories ,and
72 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013
. In this section, we will apply this key technical lemma
to show existence and uniqueness of the Nash equilibrium.
Theorem 4.1: Assume is continuous on .Thenthere
exists a Nash equilibrium for the infinite population charging
control system.
Proof: From Lemma A.1, we have
where and .Thisim-
plies that is continuous in if is continuous in .
It follows that is continuous in , since the average of a
group of continuous functions is also continuous.
We define a convex compact set
such that
By the specifications of admissible controls given in (2), we
have andsobyextension . Therefore,
for any ,wehave ,so maps a convex
compact set to itself. Consequently, by the Brouwer fixed point
theorem [27], there must be a fixed point such that
.Since is the set of locally optimal
charging strategies, by Theorem 3.1 the fixed point
is a Nash equilibrium.
Theorem 4.2: The infinite population charging control
system possesses a unique Nash equilibrium if is continu-
ously differentiable and strictly increasing on ,and
(22)
for some in the range ,where and de-
note, respectively, the minimum and maximum possible over
the charging interval , subject to the admissible charging con-
trol set .
The proof is provided in Appendix B.
Theorem 4.2 establishes a sufficient condition for a range
of values of for which the system will converge to a unique
Nash equilibrium. It may be difficult to satisfy this condition
over a large demand range , especially if the higher
demand value approaches the capacity limits of the system
(the supply curve is usually very steep there). However, for
overnight charging this is less likely to be a binding factor.
Moreover, as we will show using a numerical example in
Section VII, convergence is still possible even when condition
(22) is slightly violated.
V. D ECENTRALIZED COMPUTATIONAL ALGORITHM
Assuming the technical conditions underpinning Theorem
4.2 are satisfied, is a contraction mapping with respect
to . This result motivates an iterative algorithm for computing
the unique Nash equilibrium associated with the decentralized
charging control system.
(S1) The utility broadcasts the prediction of non-PEV base
demand to all the PEVs.
(S2) Each of the PEVs proposes an optimal charging strategy
minimizing its charging cost with respect to a common
aggregate PEV demand broadcast by the utility.
(S3) The utility collects all the optimal charging strategies
proposed in (S2), and updates the aggregate PEV de-
mand. This updated aggregate PEV demand is rebroad-
cast to all PEVs.
(S4) Repeat (S2) and (S3) until the optimal strategies pro-
posed by all PEVs no longer change.
A more formal expression of this procedure is given by Al-
gorithm 1. At convergence, the collection of optimal charging
strategies is a Nash equilibrium. Some time later, when the ac-
tual charging period occurs, each PEV implements its optimal
strategy.
Algorithm 1: Implementation of decentralized charging
control.
Initialize a positive ,anddefine a tolerance required to
terminate iterations.
Provide an initial average charging control forecast ,and
set .
while do
Obtain optimal charging control ,w.r.t. ,forall ;
Set equal to ,where ;
Update ;
;
end
During iterations, the optimal charging trajectories proposed
by PEVs may result in large average demand at some
charging instants. Consequently, the demand at those
instants may exceed generation capacity , giving .This
would, however, only occur as the population iterated towards
the Nash equilibrium. It will be shown in Section VI that the
Nash equilibrium corresponds to a charging strategy that is
almost valley-filling, implying that large excursions in demand
are not likely at the equilibrium.
Implementation of the charging strategy must, of course,
work for finite groups of PEVs. To understand the consequences
of a finite population , we refer to Theorem 3.1. The infinite
population limit is required in (18) to establish equality in (19)
between , which quantifies the Nash equilibrium
concept in Definition 3.1, and , which underpins
Algorithm 1. For finite , the equality (18) reverts to the
approximation
Nevertheless, for large , this approximation is sufficiently ac-
curate.
MA et al.: DECENTRALIZED CHARGING CONTROL OF LARGE POPULATIONS OF PLUG-IN ELECTRIC VEHICLES 73
VI. VALLEY-FILLING PROPERTY OF THE NASH EQUILIBRIUM
Having proven existence, uniqueness and convergence for
the Nash equilibrium obtained from the PEV charging control
process of Algorithm 1, this section establishes that the Nash
equilibrium is valley filling. In its simplest form, the valley
filling property appears as in Fig. 1. However there are several
special cases to consider. The following key points capture the
various cases that are formalized in Theorem 6.1.
(i) For any pair of charging instants, the one with the smaller
non-PEV base demand is assigned a larger charging rate
(for individual PEVs as well as for the average over all
PEVs), and possesses an equal or lower total aggregate
demand.
(ii) The total demand, consisting of aggregate PEV charging
load together with non-PEV demand, is constant during
charging subintervals when all PEV charging rates are
strictly positive. This is also true of the demand obtained
by summing non-PEV demand with the charging load of
any individual PEV.
For a homogeneous population of PEVs, this second outcome
guarantees perfect valley filling. That is not the case, however,
for heterogeneous populations because there may be charging
subintervals when the charging rate for some PEVs is zero. The
examples of Section VII illustrate these outcomes.
Theorem 6.1: Suppose that the collection of charging trajec-
tories is a Nash equilibrium, and that
is strictly increasing on .Then and the average sat-
isfy the following valley filling properties for all :
when with (23a)
for some with (23b)
where for all .
The proof is provided in Appendix C.
In case of homogeneous PEV populations, each of the in-
dividual optimal strategies is coincident with their average
strategy . It follows that the properties of the Nash equilib-
rium specified in (23) are equivalent to
for some (24)
which is the normalized form of the optimal valley-filling
strategy given in (9). In other words, in the case of a homoge-
neous PEV population, the Nash equilibrium coincides with the
charging strategy given by centralized control, and is therefore
globally optimal.
VII. NUMERICAL EXAMPLES
A. Background
A range of examples will be used in this section to illustrate
the main results of the paper. In particular, we will consider
the conditions for convergence of Algorithm 1, and explore the
nature of valley filling.
The examples use the non-PEV demand profile of Fig. 1,
which shows the load of the Midwest ISO region for a typical
summer day during 2007. It is assumed that the total genera-
tion capacity is 1.2 kW. Furthermore, the simulations are
basedonassuming , which corresponds to roughly
30% of vehicles in the Midwest Independent System Operator
(MISO) footprint. This gives 12 kW, where
the superscript indicates finite population size. Also, we de-
fine ,with displayedinFig.1.
The following PEV population parameters will be used for all
the examples. All PEVs have an initial SOC of 15%, i.e.,
for all , and 85% charging efficiency, i.e., for
all . The charging interval covers the 12-hour period from
8:00 pm on one day to 8:00 am on the next. The continuously
differentiable and strictly increasing price function
(25)
is used in all cases.
Other parameters, such as PEV battery size and the
tracking cost parameter , are specified within each of the
examples.
B. Computation of Nash Equilibrium for Homogeneous PEV
Population
This section considers the computation of the Nash equilib-
rium for a homogeneous population of PEVs, each of which pos-
sesses an identical battery size of 10 kWh. First, it may
be verified from Fig. 1 that
To de t e rmine ,weassumetheentireenergyrequirement
is delivered over a single time step, so
Referring to (25), this gives
which can be satisfied for some in the range .
Therefore a tracking parameter exists such that condition (22)
of Theorem 4.2 holds.
Fig. 2 provides simulation results for the decentralized com-
putation algorithm of Section IV, for the homogeneous PEV
population of this section. The tracking parameter
was used for this case. Each line in the figure corresponds to
an iterate of the algorithm. We observe that convergence to the
Nash equilibrium (shown by the solid flat curve) is achieved in
a few cycles. This Nash equilibrium is clearly the globally op-
timal valley-filling strategy.
The condition on established in Theorem 4.2 is sufficient,
but not necessary. Fig. 3 confirms this. Here,
yet the system still converges to the same valley-filling solution
as in Fig. 2. As decreases, however, the process eventually
74 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013
Fig. 2. Convergence of Algorithm 1 for a homogenous PEV population, with
.
Fig. 3. Convergence of Algorithm 1 for a homogenous PEV population, with
. This violates the condition given in Theorem 4.2.
Fig. 4. Non-convergence of Algorithm 1 for a homogenous PEV population,
with . This significantly violates the condition given in Theorem 4.2.
ceases to converge. This can be observed in Fig. 4, whichshows
the iterations when .Inordertoavoidanunre
ason-
ably high charging rate during the non-PEV demand valley, we
have constrained the charging rate to a maximum of 3 kW. This
constraint does not effect the convergence property of the algo-
rithm.
Fig. 5. Convergence of Algorithm 1 for a heterogeneous PEV population.
C. Computation of Nash Equilibrium for Heterogeneous PEV
Populations
PEV populations are heterogeneous if vehicles do not have
identical charging requirements. To examine optimal charging
outcomes for heterogeneous populations, we constructed a sim-
plified case with PEVs having one of three charging energy re-
quirements: 10, 15, or 20 kWh. We further assumed that the
number of PEVs in each group accounted for about 50%, 30%
and 20% of the population, respectively. We can verify that
where denotes the energy delivery requirement
of each PEV averaged across the entire population. It follows
that
which can be satisfied for some in the range
. Fig. 5 shows the results for this heterogeneous case, with
tracking parameter . In particular the dashed curves
with marks show the optimal charging strategies for the first,
second, and third class of PEVs, and the solid curve provides
the average demand value across the entire population. Notice
that this curve of average demand is flat between 10 pm and 8
am, where all PEVs are charging, in accordance with Theorem
6.1.
VIII. CONCLUSION AND FUTURE RESEARCH
This paper introduces a class of decentralized charging con-
trol problems for large populations of PEVs. These problems are
formulated as large-population games on a finite charging in-
terval. We study the existence, uniqueness and optimality of the
Nash equilibrium of the charging problems. In particular, fol-
lowing the decentralized computational mechanism established
MA et al.: DECENTRALIZED CHARGING CONTROL OF LARGE POPULATIONS OF PLUG-IN ELECTRIC VEHICLES 75
in the paper, we show that, under certain mild conditions, the
large-population charging games will converge to a unique Nash
equilibrium which is either globally optimal (for homogeneous
populations) or nearly globally optimal (for the heterogeneous
case). These results are demonstrated with illustrative examples.
These examples demonstrate that convergence to the Nash
equilibrium occurs very quickly over a broad range of pa-
rameters. Therefore, the method may be particularly useful in
applications where fully centralized control is not possible,
yet optimal or near-optimal charging patterns are essential to
system operation. The results in this paper will be important
when PEV market penetration becomes sufficiently large that
electricity demand patterns change significantly with PEV
charging. This is because the algorithm allows users to choose
their own locally optimal charging pattern while still achieving
near-optimal global conditions. The strategy may improve PEV
market penetration, especially relative to centralized strategies
that could deter consumers who wish to independently deter-
mine their charging strategy.
The paper has only included two user-specific preferences in
the constrained optimization problem, namely to minimize local
electricity costs and to fully charge. This work should be ex-
tended to include other local considerations such as time con-
straints, a willingness to trade off gasoline versus electricity
costs (in plug-in hybrid vehicles), and battery state of health
concerns. This is the subject of ongoing research.
We have assumed that supply and non-PEV demand are de-
terministic and predictable. This is clearly not the case in prac-
tice, as demand is stochastic, conventional generators experi-
ence forced outages, and wind and solar generation is variable
and difficult to predict. Therefore a natural extension to this
work would incorporate stochastic supply and demand fore-
casting models into the optimization process. We have begun
work in this direction [28]. Along a similar line, PEVs may ul-
timately have “vehicle-to-grid” (V2G) capability, whereby they
could act as loads at certain times and sources at other times.
This function could be used to counterbalance load and gener-
ator supply forecast errors.
APPENDIX A
STATEMENT AND PROOF OF LEMMA A.1
Lemma A.1: Control trajectories ,and
satisfy the following inequalities for all :
for all (26)
(27)
where ,,and
denotes the norm of the associated vector.
Proof: For notational simplicity, we will use
throughout the proof.
(26) Proof: There are four cases to consider:
(i) . It follows immediately that
.
(ii) and . By (20), implies
,and
implies . Together these give
with the last term equal to
.
(iii) and . Similarly to (ii), we can derive
(iv) and . By (20), we have directly
(27) Proof: There are the following three cases to consider.
(i) . This equality ensures .
Also, charging control has the form (20) with
. Therefore, by Lemma 4.1, is the local optimal
control with respect to , and hence .Itfol-
lows that
(ii) . By (2) we have
. Therefore which,
together with (20) and the definitions of and ,
implies,
for all
Hence
with the last line a consequence of the triangle inequality
for norms, taking into account that for all
valid control trajectories. Then
(28)
(iii) . A similar argument to (ii) can be
used to show that (28) holds in this case.
APPENDIX B
PROOF OF THEOREM 4.2
Proof of Theorem 4.2: First notice that
. Therefore
76 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013
where the final inequality follows directly from (22).
This result, together with a similar argument in terms of
gives
(29)
Manipulation of (29) results in
(30)
Because is strictly increasing with ,
(30) can be rewritten
This inequality, in conjunction with (26) and (27) of Lemma
A.1, gives
and hence
Since , it follows that is a contraction map-
ping with respect to . It may be concluded from the contraction
mapping theorem [29] that the infinite population of PEVs pos-
sesses a unique fixed point which is the unique Nash equilibrium
for the infinite population charging control system.
APPENDIX C
PROOF OF THEOREM 6.1
Proof of Theorem 6.1: Consider any pair of time instants
, and denote by the set of charging controls
and that satisfy and .Let
(31)
so that ,and .Itfollowsthat
is equivalent to .
We proceed by writing the minimum of the cost function (17)
as a Bellman equation [30]. To do so, we define
s.t. for all
The minimum over the entire charging period can then be
written
(32)
In terms of and defined at (31), this becomes
(33)
where is an expression in that is unrelated to .Let
and denote the values of and associated with the optimal
controls and .Thenby(33), is a function of that
satisfies
(34)
with
(35)
It follows from (34) that
if (36a)
if (36b)
if (36c)
First Part of (23a): We prove this result by establishing a
contradiction. Suppose there exist two time instants and ,such
that and , which implies .Since is
strictly increasing on ,, and so from (35),
. It follows from (36) that .
Hence, ,forall ,
which implies . However is a Nash
equilibrium, so , hence a contradiction.
Second Part of (23a): Proof by contradiction is again used.
Suppose there exist two time instants and ,suchthat
when . It follows that ,andso
from (35), . But from (i.1), ,so
it follows from (36) that ,for
all , which implies . However
is a Nash equilibrium, so , hence a contradiction.
Third Part of (23a): Again consider two time instants
and ,where .From(i.1)and(i.2),wehave and
respectively. Therefore (35) implies ,so
we may conclude from (36) that . Hence
as desired.
(ii.1), First Part of (23b): Proof by contradiction will again
be used to establish this result. Suppose there exist two time
instants such that . Without lose of
generality, assume ,forsome .Then
there exist and ,suchthat .By
the definition of ,for and all . Therefore
there exists a sufficiently small such that .
Consider a revised charging strategy ,with
for
MA et al.: DECENTRALIZED CHARGING CONTROL OF LARGE POPULATIONS OF PLUG-IN ELECTRIC VEHICLES 77
For the cost function defined at (17), it follows that
Notice that . Also, because
and is strictly increasing,
. Therefore for sufficiently small
. However, is a Nash equilibrium, and therefore minimizes
. Hence a contradiction.
(ii.2), Second Part of (23b): The total energy delivered to
the th PEV over the period by the optimal charging strategy
is given by , for every .Provided
fixed energy is delivered over , variation of the trajectory
has no influence on the cost over the balance of
the charging period . The optimal choice for
is therefore given by
(37)
(38)
Accordingto(ii.1), ,forall . Therefore
the electricity charging price ,with ,
is a constant for all . This allows the cost function (37)
to be rewritten as , so the minimum
cost can be found from
subject to (38). Using Lagrange multipliers, optimality is
achieved when all are equal. In conjunction
with (ii.1), this gives for all .
REFERENCES
[1] D. Lemoine, D. Kammen, and A. Farrell, “An innovation and policy
agenda for commercially competitive plug-in hybrid electric vehicles,”
Environ. Res. Lett., vol. 3, p. 014003, 2008.
[2] S. Davis, S. Diegel, and R. Boundy, “Transportation energy data book,”
Oak Ridge National Lab., Oak Ridge, TN, 2010, Tech. Rep. ORNL-
6985.
[3] P. Denholm and W. Short, “An evaluation of utility system impacts
and benefits of optimally dispatched plug-in hybrid electric vehicles,”
National Renewable Energy Lab., Golden, CO, 2006, Tech. Rep.
NREL/TP-620-40293.
[4] S. Rahman and G. Shrestha, “An investigation into the impact of elec-
tric vehicle load on the electric utility distributio n system,” IEEE Trans.
Power Del., vol. 8, no. 2, pp. 591–597, 1993.
[5] F.KoyanagiandY.Uriu,“Modelingpowerconsumptionbyelectric
vehicles and its impact on power demand,” Elect. Eng. Japan, vol. 120,
no. 4, pp. 40–47, 1997.
[6] F.Koyanagi,T.Inuzuka,Y.Uriu,andR.Yokoyama,“MonteCarlo
simulation on the demand impact by quick chargers for electric vehi-
cles,” in Proc. IEEE Power Eng. Soc. Summer Meet., 1999, vol. 2, pp.
1031–1036.
[7] D. Callaway and I. Hiskens, “Achieving controllability of electric
loads,” Proc. IEEE, vol. 99, no. 1, pp. 184–199, Jan. 2011.
[8] D. Monderer and L. Shapley, “Potential games,” Games Economic Be-
havior, vol. 14, pp. 124–143, 1996.
[9] E.Altman,T.Başar, T. Jiménez, and N. Shimkin, “Routing into two
parallel links: Game-theoretic distributed algorithms,” J. Parallel Dis-
trib. Comput., vol. 61, no. 9, pp. 1367–1381, 2001.
[10] E. Altman, T. Boulogne, R. El-Azouzi, T. Jimenez, and L. Wynter, “A
survey on networking games in telecommunications,” Comput.Operat.
Res.,vol.33,pp.286–311,2006.
[11] G. Christodoulou, V. Mirrokni, and A. Sidiropoulos, “Convergence
and approximation in potential games,” in Proc. 23rd Symp. Theoret-
ical Aspects Comput. Sci. (STACS), 2006, pp. 349–360.
[12] E. Even-Dar, A. Kesselman, and Y. Mansour, “Convergence time to
Nash equilibirum in load balancing,” ACM Trans. Computat. Logic,
vol. 2, no. 3, pp. 111–132, 2001.
[13] P. Berenbrink, T. Friedetzky, L. Goldberg, P. Goldberg, Z. Hu, and
R. Martin, “Distributed selfish load balancing,” in Proc. 17th Annu.
ACM-SIAM Symp. Discrete Algorithms, 2006, pp. 354–363.
[14] A. Blum, E. Even-Dar, and K. Ligett, “Routing without regret: On con-
vergence to Nash equilibria of regret-minimizing algorithms in routing
games,” in Proc. 25th ACM Symp. Principles Distrib. Comput., 2006,
pp. 45–52.
[15] S. Fischer, L. Olbrich, and B. Vocking, “Approximating Wardrop equi-
libria with finitely many agents,” Distrib. Comput., vol. 21, no. 2, pp.
129–139, 2008.
[16] S. Fischer, H. Racke, and B. Vocking, “Fast convergence to Wardrop
equilibria by adaptive sampling methods,” in Proc. 38th Annu. ACM
Symp. Theory Comput., 2006, pp. 653–662.
[17] J. Wardrop, “Some theoretical aspects of road traffic research,” in Proc.
Inst. Civil Eng., Part 2, 1952, pp. 1: 325–378.
[18] S.Borenstein,J.Bushnell,andF.Wolak,“Measuringmarketinef-
ficiencies in California’s restructured wholesale electricity market,”
Amer. Econom. Rev., vol. 92, no. 5, pp. 1376–1405, 2002.
[19] J. Bushnell, E. Mansur, and C. Saravia, “Vertical arrangements, market
structure, and competition: An analysis of restructured US electricity
markets,” Amer. Econom. Rev., vol. 98, no. 1, pp. 237–266, 2008.
[20] B. Hobbs, “Linear complementarity models of Nash-Cournot compe-
tition in bilateral and POOLCO power markets,” IEEE Trans. Power
Syst., vol. 16, no. 2, pp. 194–202, May 2002.
[21] L. Cunningham, R. Baldick, and M. Baughman, “An empirical study
of applied game theory: Transmission constrained Cournot behavior,”
IEEE Trans. Power Syst., vol. 17, no. 1, pp. 166–172, Feb. 2002.
[22] E. Pettersen, “Managing end-user flexibility in electricity markets,”
Fakultet for samfunnsvitenskap og teknologiledelse, Institutt for indus-
triell konomi og teknologiledelse, NTNU, Trondheim, Norway, 2004.
[23] A. Philpott and E. Pettersen, “Optimizing demand-side bids in day-
ahead electricity markets,” IEEE Trans. Power Syst.,vol.21,no.2,pp.
488–498, May 2006.
[24] M. Huang, P. Caines, and R. Malhamé, “Individual and mass behaviour
in large population stochastic wireless power control problems: Cen-
tralized and Nash equilibrium solutions,” in Proc. 42th IEEE Int. Conf.
Decision Control, 2003, pp. 98–103.
[25] M. Huang, P. Caines, and R. Malhamé, “Large-population cost-cou-
pled LQG problems with non-uniform agents: Individual-mass
behaviour and decentralized epsilon-Nash equilibria,” IEEE Trans.
Autom. Control, vol. 52, no. 9, pp. 1560–1571, Sep. 2007.
[26] N. Padhy, “Unit commitment-A bibliographical survey,” IEEE Trans.
Power Syst., vol. 19, no. 2, pp. 1196–1205, May 2004.
[27] K. Border, Fixed Point Theorems with Applications to Economics and
Game Theory. Cambridge, U.K.: Cambridge University Press, 1985.
[28] Z. Ma, I. Hiskens, and D. Callaway, “Distributed MPC methods in
charging control of large populations of plug-in electric vehicles,” pre-
sented at the IFAC World Congr., Milano, Italy, 2011.
[29] D. Smart, Fixed Point Theorems. London, U.K.: Cambridge Univer-
sity Press, 1974.
[30] D. Bertsekas, Dynamic Programming and Optimal Control. Singa-
pore: Athena Scientific,1995,vol.I.
Zhongjing Ma received the B.Eng. degree from
Nankai University, Tianjin, China, in 1997, the
M.Eng. and Ph.D. degrees from McGill University,
Montreal, QC, Canada, in 2005 and 2009, respec-
tively, all in the area of systems and control.
After a period as a postdoctoral research fellow
with the Center of Sustainable Systems, the Univer-
sity of Michigan, Ann Arbor, he joined Beijing Insti-
tute of Technology, Beijing, China, in 2010, where
he is currently an Associate Professor. He was an
Engineer with the Institute of Automation Research,
Shanxi, China. His research interests lie in the areas of optimal control, sto-
chastic systems, and applications in the power and microgrid systems.
78 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 21, NO. 1, JANUARY 2013
Duncan S. Callaway (M’08) received the B.S. de-
gree in mechanical engineering from the University
of Rochester, Rochester, NY, in 1995, the Ph.D. de-
gree in theoretical and applied mechanics from Cor-
nell University, Ithaca, NY, in 2001.
He is currently an Assistant Professor of Energy
and Resources and Mechanical Engineering, Univer-
sity of California, Berkeley. Prior to joining the Uni-
versity of California, he was first an NSF Postdoc-
toral Fellow with the Department of Environmental
Science and Policy, University of California, Davis,
subsequently worked as a Senior Engineer at Davis Energy Group, Davis, CA,
and PowerLight Corporation, Berkeley CA, and was most recently a Research
Scientist with the University of Michigan. His current research interests include
the areas of power management, modeling and control of aggregated storage
devices, spatially distributed energy resources, and environmental impact as-
sessment of energy technologies.
Ian A. Hiskens (F’06) is the Vennema Professor
of engineering with the Department of Electrical
Engineering and Computer Science, University of
Michigan, Ann Arbor. He has held prior appoint-
ments in the electricity supply industry (for ten
years), and various universities in Australia and the
United States. His research focuses on power system
analysis, in particular the modelling, dynamics and
control of large-scale, networked, nonlinear systems.
His recent activities include integration of renewable
generation and new forms of load.
Prof. Hiskens is actively involved in various IEEE societies, and is Treasurer
of the IEEE Systems Council. He is a Fellow of Engineers Australia and a Char-
tered Professional Engineer in Australia.
