Power-Optimized Vertical Handover Scheme for Heterogeneous Wireless Networks

Abstract

This letter proposes a vertical handover scheme to minimize the total power consumption required in serving a traffic flow, while guaranteeing a particular service rate of different access networks. The proposed scheme is based on a Markov decision process (MDP) that uniquely captures the power consumption during the vertical handover execution as well as the transmission power and circuit power. Our scheme is capable of handling stochastic system behaviors while finding an optimal decision policy. Simulation results show that the proposed scheme can lead to a lower total power consumption compared to several existing schemes. For a special case where each traffic flow has a fixed number of frames, our results suggest that the number of decision epochs can be reduced to further conserve power.

Full text

IEEE COMMUNICATIONS LETTERS, OOOO 2013 1
Power-optimized Vertical Handover Scheme for
Heterogeneous Wireless Networks
Yujae Song, Peng-Yong Kong, Senior Member, IEEE, and Youngnam Han, Senior Member, IEEE
Abstract—This letter proposes a vertical handover scheme
to minimize the total power consumption required in serving
a traffic flow, while guaranteeing a particular service rate of
different access networks. The proposed scheme is based on a
Markov decision process (MDP) that uniquely captures the power
consumption during the vertical handover execution as well as
the transmission power and circuit power. Our scheme is capable
of handling stochastic system behaviors while finding an optimal
decision policy. Simulation results show that the proposed scheme
can lead to a lower total power consumption compared to several
existing schemes. For a special case where each traffic flow has
a fixed number of frames, our results suggest that the number
of decision epochs can be reduced to further conserve power.
Index Terms—Heterogeneous wireless networks, Markov deci-
sion process (MDP), vertical handover decision.
I. INT ROD UC TI ON
There is an exponential growth in the total data traffic
throughout the world, driven by the use of wireless de-
vices such as smart phones and tablet PCs and the development
of high-quality service applications [1]. To provide support for
the increasing traffic, deploying small base stations (BSs), such
as pico and femto BSs, seems to be a promising and economic
solution in conjunction with the evolution of mobile terminals
(MTs) with multiple network interfaces and deployment of IP-
based applications [2]. The utilization of small BSs together
with existing macro BSs not only offers a rich dimension for
increasing system capacity but also for coverage improvement
inside the initial deployment of macro cells.
In heterogeneous wireless networks with a mixture of macro
and small BSs, efficient vertical handover is necessary for
an MT to support seamless service among different access
networks. Vertical handover allows an MT to select the most
suitable connection among all available access networks. Un-
like conventional horizontal handover that mainly considers
received signal strength (RSS) as the only handover decision
parameter, various parameters, such as network connection
time, power consumption, and monetary cost, can also be taken
into consideration for vertical handover decisions to enhance
user satisfaction [3]. The IEEE 802.21 standard provides a
Manuscript received October 10, 2013. The associate editor coordinating
the review of this letter and approving it for publication was G. Giambene.
This research was funded by the MSIP (Ministry of Science, ICT & Future
Planning), Korea in the ICT R&D Program 2013.
Y. Song and Y. Han are with the Department of Electrical Engineering,
Korea Advanced Institute of Science and Technology, Daejeon, Korea (e-mail:
{ednb1008, ynhan}@kaist.ac.kr).
P.-Y. Kong is with the Department of Electrical and Computer Engineering,
Khalifa University of Science, Technology and Research (KUSTAR), Abu
Dhabi, United Arab Emirates (e-mail: pengyong.kong@kustar.ac.ae).
Digital Object Identifier 10.1109/LCOMM.2013.120713.132279
media-independent framework and related services to support
seamless vertical handover [4], but it does not provide specific
vertical handover algorithms for implementation. To fill this
gap, various vertical handover decision algorithms have been
studied. In [5], a vertical handover decision method based
on fuzzy multiple attribute decision making (MADM) is
proposed. This method consists of two phases. Fuzzy data
is first converted into a real number, and then a classical
MADM method is adapted to determine the ranking of can-
didate networks. However, that paper does not specify which
MADM method to use. The work in [6] presents two specific
MADM methods, namely simple additive weighting (SAW)
and technique for order preference by similarity to ideal
solution (TOPSIS). SAW determines the overall score of a
candidate network by the weighted sum of multi-attribute
values. It is a well-known and widely used method. TOPSIS
selects the network that is the closest to the ideal solution
and the farthest from the worst solution. In [7], the authors
take into account the service history of user traffic as one
of the vertical handover decision parameters, which plays a
key role in mitigating premature completion of a service and
unnecessarily frequent handover switching.
Unlike in the past when traditional mobile wireless network
operation mainly focuses on ubiquitous access and network
performance improvement, wireless service providers and re-
searchers are shifting their focus to power-efficient network
operation because power saving and environmental protection
become global demands and inevitable trends [8]. Especially,
the reason why BSs’ power-efficient operation is more im-
portant is that currently over 80% of the power in mobile
telecommunications is consumed in the radio access network,
more specifically at the base station [9]. However, none of
existing vertical handover schemes aims at minimizing power
consumption at BSs.
To meet these requirements, we propose a vertical handover
scheme to minimize total power consumption required in
serving a traffic flow without affecting the constant service
rate of each network by using dynamic power control. The
proposed scheme uses a Markov decision process (MDP) to
capture the system’s stochastic behaviors as well as the power
consumption during a handover execution. In the literature
[10], MDP has been used for optimal vertical handover de-
cision making. There, the MDP reflects the signaling cost
incurred for vertical handover execution and total network
resources consumed by an MT connection. The goal is to
determine the policy that maximizes expected total reward per
connection, where the reward is a function of QoS. However,
power consumption has not been considered.

IEEE COMMUNICATIONS LETTERS, OOOO 2013 2
II. SY ST EM M OD EL
We consider heterogeneous wireless networks where a tradi-
tional macro BS is collocated with smaller BSs. The different
types of BSs have different transmission powers and nominal
service rates. For example, a pico BS has lower transmission
power but a higher service rate compared to a macro BS.
Let Nbe the set of all accessible BSs for an MT such that
N=Nmacro ∪Nf emto ∪Npico ={1,2, ..., N }, where Nis
the total number of accessible BSs.
We define link nas a wireless channel that connects an MT
to BS n∈N. The wireless channels suffer from statistically
identical and independent slow Rayleigh fading. The time-
varying link quality is constant during service time of a frame
and can be quantized into Mdiscrete values by adopting
finite-state Markov channel (FSMC) model in [11]. Let qn
denotes the channel quality of link nand 0 = A0< A1<
... < AM−1< AM=∞denotes pre-determined received
signal to noise ratio (SNR) thresholds. Then, qnis said to
have the quality of Am, if Am≤δn(t)< Am+1, where
δn(t)is the actual received SNR of link nat time t. The
steady-state probability pmthat the link n’s channel quality
qnequals the pre-determined received SNR Amis given by
pm= P [qn=Am] =
Am+1

Am
fA(a)da = exp −Am
ν−
exp −Am+1
ν, where νis the mean received SNR, and
fA(a) = 1
νexp −a
νis the probability density function of
Awhich is exponential. The transition probability from a link
quality to another quality is represented by
pm,m+1 =Dm+1
µnpm
, m = 0,1, ..., M −2(1)
pm,m−1=Dm
µnpm
, m = 1,2, ..., M −1
p0,0= 1 −p0,1,
pM−1,M−1= 1 −pM−1,M −2,
pm,m = 1 −pm,m−1−pm,m+1 , m=1,2, ..., M −2
where Dm=2πAm
νfDexp −Am
νis the expected number
of downward crossings in received SNR when maximum
Doppler frequency is fD, and µnis a service rate of BS n.
Let each active MT be represented by a traffic flow that
consists of homogeneous K+1 frames. After transmitting the
K+ 1 frames, an active MT becomes inactive and leaves the
system before becoming active again. Each MT can connect
to only one BS at any time but may perform vertical handover
between the Navailable BSs from time to time. Vertical
handover can only be carried out before the start of each frame
after the first frame. Thus, there exists a total of Khandover
decision epochs for each active MT. We assume all handovers
are successful due to the existence of handover reservation
channels. Through the BSs’ dynamic power control, each BS
can provide an MT with a constant network service rate that is
different according to the type of network that BS belongs to.
At each decision epoch, a vertical handover decision is made
at the macro BSs for each MT using the MDP-based scheme
we propose in this letter.
III. MDP FRAMEWORK FOR POWER-OPTIMIZED VERTICAL
HA ND OVE R
An MDP is defined as a tuple T = (S, A, T, r), where S
is state space, Ais the set of all possible actions, T[s′|s, a]
is transition probability from state s∈Sto s′∈Safter
taking action a∈Ain state s, and r(s, a)is the cost of
performing action ain state s. First, we define the state of an
MT to provide information regarding which BS it is currently
connected to and the qualities of all its Nlinks. As such, the
state space Sbecomes:
S={1,2, ... , N } × Q1×Q2×... ×QN(2)
where Qnis the set of Mquantized link qualities for link n
described earlier and ×represents Cartesian product.
The action is to select one BS among Navailable BSs.
When the action is a, the MT will be attached to BS afor
transmission of next frame. The action set is given by A=
{1,2, ..., N }.
The cost r(s, a)stands for power consumption of chosen
BS ato transmit one frame to an MT and is defined as the
sum of transmission cost f(s, a)and handover cost g(s, a)as
follows:
r(s, a) = f(s, a) + g(s, a).(3)
The transmit cost function f(s, a)captures not only the
transmission power from BS ato the MT but also the circuit
power of BS awhich is independent with the transmission
power [12], as given below:
f(s, a) = bframe
1
µa
(Pa,tx +Pa,c),(4)
where Pa,tx is the transmission power from BS ato the MT,
Pa,c is the circuit power of BS a, and bframe is total number
of bits consisting of a homogeneous frame. Recall that BSs
perform dynamic power control to provide a constant network
service rate to MTs. With the power control, Pa,tx depends
on link quality and can be formulated as below:
Pa,tx =(pa,min−pa,max )
(qth−q0)(q−q0) + pa,max , q0≤q < qth
pa,min , qth ≤q≤qM−1
(5)
where pa,max and pa,min denotes maximum and minimum
transmission power supported by the chosen BS a, and qth
denotes the threshold level of link quality for power control,
respectively. The handover cost function g(s, a)reflects the
power consumption of BSs incurred by signaling exchanges
and processing loads during the vertical handover execution
phase, as given below:
g(s, a) = Ki,a , i ̸=a
0, i =a, (6)
where Ki,a is power consumption required for network switch-
ing from current BS ito BS a.
Finally, we have the state transition probability from state
s= [n, q1, q2, ... , qN]to state s′= [n′, q′
1, q′
2, ... , q′
N]given
as follows:
T[s′|s, a] = 


N

n=1
P[q′
n|qn], n′=a
0, otherwise .
(7)

IEEE COMMUNICATIONS LETTERS, OOOO 2013 3
Recall that our goal is to minimize the total power consump-
tion while providing the constant network service rate during
the lifetime of a flow. In the context of our proposed MDP
model, it is about choosing an action (BS) at each decision
epoch so that the expected total cost can be minimized. For a
given MDP policy πand initial state s, the expected total cost
is as follows:
vπ(s) = Eπ
sEKK

t=1
δt−1r(st, at) (8)
where Eπ
s[·]denotes the expectation under the policy πand
initial state s,δ∈[0,1) denotes the discount factor that de-
termines the importance of future cost to a current epoch, and
r(st, at)is the cost at decision epoch t. Also, EK[·]denotes
the expectation under random variable K, where Khas been
defined earlier as the number of decision epochs per traffic
flow. Reasonably, Kis a random variable because different
MTs may have different number of frames to transmit. We
assume Kis geometrically distributed with mean 1/(1 −λ).
Then, from Appendix A, the expected total cost vπ(s)can be
also represented as follows:
vπ(s)=Eπ
s∞

t=1
δt−1λt−1r(st, at)=Eπ
s∞

t=1
ςt−1r(st, at)
(9)
where ς=δλ ∈[0,1) can be interpreted as the discount factor
with random number of decision epochs per traffic flow.
Now, our challenge is to identify the optimal policy π∗
among all the feasible policies Πsuch that the total expected
cost is minimized as v(s)≡min
π∈Πvπ(s), where v(s)denotes
the minimum total expected cost.
The optimal policy can be determined using dynamic pro-
gramming through the Bellman optimality equation as follows:
v(s) = min
a∈Ar(s, a) + ς
s′∈S
T(s′|s, a)v(s′).(10)
In practice, we have determined the optimal policy using value
iteration algorithm described in [13].
IV. SIM UL ATION RESULTS
Consider heterogeneous wireless networks, where an MT
is located within the collocated area from two BSs and can
access both BSs during the service time of a traffic flow. For
simplicity, we assume both BSs have a same configuration
except a different service rate. In general, the maximum
transmission power of BSs determines their cell sizes. If the
maximum transmission power of BSs decreases, cell sizes will
become smaller; whereas the average received SNR on a link
will improve due to a shorter average link distance. To study
the effect of different cell sizes, we consider two scenarios:
Scenario 1 (large cell case) and Scenario 2 (small cell case).
The simulation parameters for both scenarios are summarized
in Table I.
We compare the performance between power-optimized
VHO algorithm (proposed algorithm), SNR-based VHO algo-
rithm [3], and one heuristic algorithm that is called NO-VHO
algorithm [10]. In each handover decision epoch, SNR-based
TABLE I
SIM ULATI ON PAR AM ETE RS
Symbol Scenario 1 Scenario 2
µa1 Mbps for a= 1
1.2 Mbps for a= 2
1 Mbps for a= 1
1.2 Mbps for a= 2
Pa, c 5 W, a={1,2}5 W, a={1,2}
pa,max 10 W, a={1,2}8 W, a={1,2}
pa,min 2 W, a={1,2}2 W, a={1,2}
qaqa∈ {0,4,8,12,∞} dB qa∈ {0,6,10,14,∞} dB
qth 10 dB 10 dB
ν8 dB 10 dB
VHO algorithm compares the SNR of current BS against the
others to make handover decision. This algorithm is widely
used due to its simplicity in hardware implementation. NO-
VHO algorithm does not perform vertical handover during
the service time of a traffic flow, which means that the MT
is always connected to a same BS. It can be acceptable
under the condition that the MT is within collocated area in
heterogeneous wireless networks.
In this work, performance metrics are the expected total
power consumption (cost), number of handover, and total
number of handover. Note that number of handover refers to
the sum of handover counts occurred at each handover decision
epoch for all possible initial states, whereas total number of
handover means the sum of handover counts occurred for the
entire lifetime of a traffic flow for all possible initial states.
Fig. 1(a) and 1(b) present the expected total power con-
sumption and total number of handover during the lifetime of
a traffic flow with respect to the changes in handover cost.
Discount factor of MDP framework ςis set to be 0.9. From
Fig. 1(a) , it is verified that as handover cost increases, the opti-
mal policy π∗from power-optimized VHO algorithm gives the
lowest expected total power consumption compared as SNR-
based algorithm and NO-VHO algorithm regardless of cell
sizes. From Fig. 1(b) , it is also verified that in case of power-
optimized VHO algorithm at both scenarios, as handover cost
increases, total number of handover is reduced. This implies
that, from the perspective of power saving, although the link
quality of current network is not good, maintaining the current
network while guaranteeing the network service rate through
dynamic power control can be a better strategy than perform-
ing handover to the network that has a relatively good link
quality. This is because each handover incurs a handover cost,
and the cost can be higher than the little transmission power
saving achieved through handover. In short, power-optimized
algorithm can accurately identify the diminishing potential
power saving at increasing handover cost. On the other hand,
SNR-based algorithm selects the BS depending only on the
link quality, and NO-VHO algorithm does not perform vertical
handover. Thus, total number of handover is constant for those
algorithms regardless of handover cost. Since the objective
is to minimize total power consumption while guaranteeing
constant network service rate, we can conclude that power-
optimized VHO algorithm gives best performance than other
algorithms under collocated heterogeneous wireless networks.
Fig. 1(c) shows the number of handover at each decision

IEEE COMMUNICATIONS LETTERS, OOOO 2013 4
02468
60
65
70
75
80
85
90
95
100
Handover Cost [Joule]
Expected total power consumption [Joule]
Power−optimized VHO (Scenario 1)
SNR−based VHO (Scenario 1)
NO−VHO for BS2 (Scenario 1)
Power−optimized VHO (Scenario 2)
SNR−based VHO (Scenario 2)
NO−VHO for BS2 (Scenario 2)
(a) Expected total power consumption under differ-
ent handover cost
02468
0
2
4
6
8
10
12
14
16
Handover Cost [Joule]
Total number of handover
Power−optimized VHO (Scenario 1)
SNR−based VHO (Scenario 1)
NO−VHO for BS2 (Scenario 1)
Power−optimized VHO (Scenario 2)
SNR−based VHO (Scenario 2)
NO−VHO for BS2 (Scenario 2)
(b) Total number of handover under different han-
dover cost
1 2 3 4 5 6 7 8 9
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Decision epochs [k]
Number of handover
Power−optimized VHO (δ = 0.9)
Power−optimized VHO (δ = 0.93)
Power−optimized VHO (δ = 0.96)
Power−optimized VHO (δ = 0.99)
(c) The number of handover at each handover de-
cision epoch under Scenario 1 when K= 9
Fig. 1. Performance comparisons of the proposed scheme against existing schemes in terms of total power consumption and number of handover. The figure
also shows the effect of discount factor on the proposed scheme.
epoch with respect to changes in discount factor δunder
Scenario 1 when handover cost is 16 Joule. In this case,
we assume that a traffic flow consists of 10 frames, which
means that Kis not random but designated. The reason we
consider the designated value of Kis to check the number
of handover at each decision epoch. From Fig. 1(c) , it is
shown that, for last two consecutive handover decision epochs,
there is no vertical handover although the discount factor δis
changed. Further, we can easily expect that as handover cost
is bigger, the number of these epochs that has no handover
will increase. From the results, we can propose an additional
vertical handover decision scheme: Based on power-optimized
VHO algorithm, vertical handover decision is not considered
for last some consecutive handover decision epochs depending
on the size of a traffic flow and network characteristic. BSs
should gather all the information required to identify the need
for handover at handover information gathering phase, and
determine whether and how to perform the vertical handover
by selecting the most suitable network at handover decision
phase. By minimizing the number of these two complex
processes, we can also reduce additional power consumption
in the process of vertical handover.
V. CONCLUSION
In this letter, we proposed a power-optimized vertical han-
dover decision algorithm. The objective was to minimize
the total power consumption during the lifetime of a traffic
flow. To achieve this objective, the decision algorithm was
formulated by a Markov decision process to capture sys-
tem’s stochastic behaviors. Simulation results showed that
the proposed power-optimized vertical handover algorithm
gives better performance than the SNR-based VHO algorithm
and NO-VHO algorithm. Further, by reducing the number of
handover decision per traffic flow when there is a fixed number
of frames in the flow, our work resulted in additional power
saving.
APP EN DI X A
In case that the total number of handover decision epochs
Kis a random variable, we assume that Kis geometrically
distributed with mean 1/(1 −λ)and with its probability mass
function (PMF) by P(K=k) = λk−1(1 −λ). After some
manipulations, (8) can be rewritten as follow:
vπ(s) = Eπ
s∞

k=1
k

t=1
δt−1r(Xt, Yt)λk−1(1 −λ)
=Eπ
s∞

t=1
δt−1r(Xt, Yt)(1 −λ)
∞

k=t
λk−1
=Eπ
s∞

t=1
δt−1λt−1r(Xt, Yt)
REF ER EN CE S
[1] A. Ghosh et al., “Heterogeneous cellular networks: from theory to
practice,” IEEE Commun. Mag., vol. 50, no. 6, pp. 54–64, 2012.
[2] V. Chandrasekhar, J. Andrews, and A. Gatherer, “Femtocell networks:
a survey,” IEEE Commun. Mag., vol. 46, no. 9, pp. 59–67, Sep. 2008.
[3] X. Yan, Y. Sekercioglu, and S. Narayana, “A survey of vertical han-
dover decision algorithms in forth generation heterogeneous wireless
networks,” Comput. Networks, vol. 54, no. 11, pp. 1848–1863, Aug.
2010.
[4] K. Taniuchi et al., “IEEE 802.21: Media independent handover: Features,
applicability, and realization,” IEEE Commun. Mag., vol. 47, no. 1, pp.
112–120, Jan. 2009.
[5] S.-J. Chen, C.-L. Hwang, and F. P. Hwang, “Fuzzy multiple attribute
decision making,” Springer-Verlag, pp. 465–486, 1992.
[6] W. Zhang, “Handover decision using fuzzy MADM in heterogeneous
networks,” in Proc. IEEE WCNC, Mar. 2004, pp. 653–658.
[7] T. Kim, S.-W. Han, and Y. Han, “A QoS-aware vertical handoff
algorithm based on service history information,” IEEE Commun. Lett.,
vol. 14, no. 6, pp. 527–529, 2010.
[8] Y. Chen et al., “Fundamental trade-offs on green wireless networks,”
IEEE Commun. Mag., vol. 49, no. 6, pp. 30–37, Jun. 2011.
[9] O. Arnold et al., “Power consumption modeling of different base station
types in heterogeneous cellular networks,” in Proc. Future Network and
Mobile Summit, Jun. 2010, pp. 1–8.
[10] E. Stevens-Navarro, Y. Lin, and V. Wong, “An MDP-based vertical
handoff decision algorithm for heterogeneous wireless networks,” IEEE
Trans. Veh. Technol., vol. 57, no. 2, pp. 1243–1254, Mar. 2008.
[11] H.-S. Wang and N. Moayeri, “Finite-state markov channel-a useful
model for radio communication channels,” IEEE Trans. Veh. Technol.,
vol. 44, no. 1, pp. 163–171, Feb. 1995.
[12] C. Xiong et al., “Energy- and spectral-efficiency tradeoff in downlink
OFDMA networks,” IEEE Trans. Veh. Technol., vol. 10, no. 11, pp.
3874–3886, Nov. 2011.
[13] M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dy-
namic Programming. John Wiley, 1994.