Content uploaded by Kok-Lim Alvin Yau
Author content
All content in this area was uploaded by Kok-Lim Alvin Yau on Nov 06, 2020
Content may be subject to copyright.
Cognitive Radio-based Wireless Sensor Networks:
Conceptual Design and Open Issues
Kok-Lim Alvin Yau, Peter Komisarczuk and Paul D. Teal
School of Engineering and Computer Science
Victoria University of Wellington
P.O. Box 600
Wellington 6140, New Zealand
{kok-lim.yau, peter.komisarczuk, paul.teal}@ecs.vuw.ac.nz
Abstract-Traditional static spectrum allocation policies have
been to grant each wireless service exclusive usage of certain
frequency bands, leaving several spectrum bands unlicensed
for industrial, scientific and medical purposes. The rapid
proliferation of low-cost wireless applications in unlicensed
spectrum bands has resulted in spectrum scarcity among those
bands. Since most applications in Wireless Sensor Networks
(WSNs) utilize the unlicensed spectrum, network-wide
performance of WSNs will inevitably degrade as their
popularity increases. Sharing of under-utilized licensed
spectrum among unlicensed devices is a promising solution to
the spectrum scarcity issue. Cognitive Radio (CR) is a new
paradigm in wireless communication that allows sensor nodes
as the unlicensed users or Secondary Users (SUs) to detect and
use the under-utilized licensed spectrum temporarily. Given
that the licensed or Primary Users (PUs) are oblivious to the
presence of SUs, the SUs access the licensed spectrum
opportunistically without interfering the PUs, while improving
their own performance. In this paper, we propose an approach
to build Cognitive Radio-based Wireless Sensor Networks
(CR-WSNs). We believe that CR-WSN is the next-generation
WSN. Realizing that both WSNs and CR present unique
challenges to the design of CR-WSNs, we provide an overview
and conceptual design of WSNs from the perspective of CR.
The open issues are discussed to motivate new research
interests in this field. We also present our method to achieving
context-awareness and intelligence, which are the key
components in CR networks, to address an open issue in CR-
WSN.
Keywords–Wireless sensor networks; cognitive radio
I. INTRODUCTION
Wireless Sensor Network (WSN) is one of the most
compelling technologies for performing monitoring and
surveillance tasks. It is a self-organizing ad hoc network
comprised of a large number of sensor nodes that are
systematically or randomly deployed. Generally, a wide
range of WSN applications utilize the unlicensed spectrum
bands such as Industrial, Scientific and Medical (ISM) and
Unlicensed National Information Infrastructure (UNII). In
practice, the precious and limited unlicensed radio spectrum
are shared by many wireless applications including
Bluetooth, WiFi, WiMAX, and Zigbee. Other applications
such as microwave oven and cordless phones also operate in
those bands. With the tremendous growth in ubiquitous low-
cost wireless applications that utilize the unlicensed
spectrum, we can expect network-wide performance of the
traditional WSNs to degrade as the demand for spectrum
increases because of increasing competitiveness in spectrum
utilization especially in populated urban areas.
Studies sponsored by the Federal Communications
Commission (FCC) pointed out that the current static
spectrum allocation has led to overall low spectrum
utilization where up to 70% of the allocated spectrum
remains unused (called white space) at any one time even in
a crowded area [1]. The white space is defined by time,
frequency and maximum transmission power at a particular
location. Consequently, Dynamic Spectrum Access (DSA)
has been proposed so that unlicensed spectrum users or
Secondary Users (SU)s are allowed to use the white space
(see Figure 1) of licensed users' or Primary Users (PU)s'
spectrum, conditional on the interference to the PU being
below an acceptable level. This function is realized using
Cognitive Radio (CR) [2] technology that enables an SU to
change its transmission and reception parameters including
operating frequencies. At present, research into CR is still in
its infancy. There are two prominent features of CR. Firstly,
sensing is performed across a wide range of spectrum to
identify the white space. Secondly, data packets are
allocated opportunistically to the white space at different
channels. This means that whenever a PU accesses a
channel which has been regarded as white space by the SUs,
the SUs must vacate the spectrum as soon as possible.
One of the most important concepts in the CR networks is
the Cognition Cycle (CC) [3] and it is shown in Figure 2.
The CC enables a host to achieve context-awareness and
intelligence so that it is able to be aware of its operating
environment in order to sense for the white spaces, and use
them in an intelligent and efficient manner. It is comprised
of six main states: observe, orient, act, decide, plan, and
learn. In the observe state, a CR host senses its operating
environment. Next, the orient state determines the
importance and priority of the sensing outcome, such that
immediate priority leads to an act state, urgent to decide,
and normal to plan. The decide state determines the next
Figure 1. An illustration of DSA. An SU exploits white spaces acro ss
various chann els. Each locat ion has different spectrum utilization by
Wh ite Spaces
955978-1-4244-4487-8/09/$25.00 ©2009 IEEE
The 2nd IEEE Workshop on Wireless and Internet Services (WISe 2009)
Zürich, Switzerland; 20-23 October 2009
action, while the plan state draws up a longer term course of
actions. In the act state, the chosen action is executed, and
its consequences are learnt during the learn state.
Using CR technology offers the SU sensor nodes a
number of benefits. Firstly, the SUs are expected to operate
over a wide range of non-contiguous frequency bands: 400-
800 MHz (UHF TV bands) and 3-10 GHz [4], where the
time scale of spectrum occupancy varies from milliseconds
to hours. Due to channel heterogeneity, the channel
properties vary with carrier frequency. For instance, an SU
sensor node can increase its transmission range using lower
frequency bands with the same transmission power because
of better signal propagation characteristic as shown in
Figure 3. The longer transmission range improves several
important factors in WSNs including network connectivity,
lifetime and end-to-end delay. For instance, in directed
diffusion [5], query or interests dissemination, initial
gradient setup, data delivery and gradient maintenance
along a reinforced path can be performed in an efficient and
timely manner. The gradient is an entry in an interest that
contains a data rate field requested by a particular neighbour
node. To date, research on CR networks applies a common
assumption of channel homogeneity [6]-[8]. Thus, current
research in CR is not suitable for our investigation on
Cognitive Radio-based Wireless Sensor Networks (CR-
WSNs). This means that the channels are assumed to have
the similar transmission range and channel quality.
Secondly, another advantage is data from sensor nodes that
are not spatially correlated or have low redundancy can be
transmitted to the sink (or resource center) simultaneously
in different channels using the CR technology. This enables
the sink to monitor a large number of non-spatially
correlated information in a real-time manner. This capability
is particularly useful in the new research area of Wireless
Multimedia Sensor Networks (WMSN) [9], which has high
level of bandwidth demand, that delivers multimedia
contents including data, images, audio and video streams
with predetermined level of Quality of Service (QoS).
Nevertheless, the advantages are achieved at the expense
of more complex WSN implementation. In this paper, an
approach to build CR-WSN is proposed to address the
spectrum scarcity problem while providing other
advantages. To the best of our knowledge, there is only
limited research on CR-WSN with several preliminary
results in [10]. Though CR-WSN has yet to attract research
interest, we believe that it is the next-generation WSN. Our
aim is to provide better understanding of the WSN from the
CR perspective and outline several open issues in realizing a
CR-WSN. Conceptual design of the CR-WSN is provided.
We also shows our method to achieving context-awareness
and intelligence, which are the key components in CR
networks, to address an open issue in CR-WSN. The rest of
this paper is organized as follows. Section II provides an
overview of WSN characteristics. Section III presents an
overview of CR functions. Section IV shows conceptual
design of a CR-WSN. Section V discusses open issues in
realizing a CR-WSN. Section VI presents our method in
achieving context-awareness and intelligence to address an
issue in CR-WSN. Section VII concludes this paper.
II. CHARACTERISTICS OF WSN
A sensor is a low-cost device capable of monitoring a
wide range of ambient conditions such as temperature, noise
levels, lighting conditions, and object movement. Equipped
with a low-level and low- power wireless communication
module, a large number of sensor nodes form a multi-hop
network that constitutes a WSN in order to deliver collected
data to a resource center or a sink (see Figure 4). Table I
presents important characteristics of WSNs.
TABLE I. CHARACTERISTIC S OF WIRELESS SENSOR NETWORKS
Characteristics Description
Low energy
consumption
Battery-driven sensor nodes are deployed in
inhospitable and inaccessible areas. Thus, network
lifetime maximization is imperative without
compromising on t he network quality.
Simple hardware
platform
Each sensor node has simple and low-cost hardware
implementation with low computation capability.
Self organization The sensor nodes, whic h are scattered randomly should
self-organize in order to form a multi-hop WSN that
improves connectivity, network lifetime and other QoS
metrics.
A clustered or hierarchical architecture has been widely
adopted for achieving QoS and improving energy efficiency
in WSN. A cluster-based distributed WSN, as shown in
Figure 4, is considered in this paper. In the cluster-based
network, some sensor nodes are elected as ClusterHead
(CH), which is the leader of a cluster. The GateWay (GW)
node is a bridge node that connects adjacent clusters. The
CH and GW nodes connect with each other to form a
backbone topology to the sink. The rest of the sensor nodes,
which are neither CH nor GW, are associated with at most
Figure 2. T he cognition cycle [3] .
Figure 3.Transmission range against frequency and transmission
power. A free space m odel is adopted. Receiver t hreshold is 5.8nW .
System loss, transmit ter an d receiver ant enna gain equal 1
956
one CH and they are called Member Nodes (MNs). The
backbone nodes perform data fusion and transmit packets to
the sink. Due to the importance of the backbone topology,
the efficiency of the WSN is greatly dependent on how well
the capacity is managed at the backbone nodes. For that
reason, any WSN should be conceived to cope with the
capacity consumption at the CHs and GWs. Previous
heuristics restrict cluster size to a certain number of hops or
member nodes; however, these are insufficient to ensure
ample amount of available bandwidth in each cluster
considering that all sensor nodes have to compete with other
wireless applications for spectrum usage. This condition
would result in longer end-to-end delay, significantly
deteriorating the performance of real-time detection and
monitoring. Such a condition motivates the necessity to
design a CR-WSN.
III. FUNCTIONS OF CR NETWORKS
One of the most important components for performance
enhancement in CR networks is the CC. In this paper,
Reinforcement Learning (RL) is employed to implement the
CC. We first provide an overview on RL; followed by a list
of functions in CR networks.
Q-learning [11], [12] is an on-line algorithm in RL that
determines an optimal policy without detailed modeling of
the operating environment. As shown in Figure 5, a CR
sensor node observes its operating environment, learns and
decides an appropriate action. In Q-learning, the learnt
action value or Q-value,
Qs,a indicates the
appropriateness of choosing action a in state
s
. The state
captures the characteristics of the operating environment. At
time
t
1, the Q-value of a chosen action in state
s
at
time
t
is updated as follows:
where 01 is a learning rate, 01 is a discount
factor, and rt1st1 is a delayed reward, which is
generally higher for appropriate actions for a particular
state. An optimal policy is being searched for in RL that
maximizes the value function V
st
so that accumulated
rewards are maximized as time goes by as shown below:
The update of the Q-value in (1) does not cater for the
actions that are not chosen. Choosing the best overall action
according to (2), or the greedy action, at all times is termed
exploiting. To improve the estimates of the other Q-values,
the other actions are chosen once in a while though they are
not known to be the best one so that better actions may be
discovered, which is a procedure called exploring. This
enables an SU sensor node to search for the best action for a
particular state as time goes by. In the -greedy approach
[11], an agent chooses the greedy action as its next action
with probability 1 , and random action with a small
probability .
Applying the RL method as a context-aware and
intelligence model provides several advantages. Firstly, RL
helps a sensor node to adapt to its dynamic and uncertain
operating environment. In reality, the radio resources,
topology, nodal availability and other factors affect a node's
performance in a complex manner. Rather than addressing a
single factor at a time, the RL enables a sensor node to
observe all these factors as a state and optimizes a general
goal as a whole, such as throughput and delay, with regard
to the state. Secondly, the RL adopts a simple modeling
approach. Thus, the complexity involved in modeling the
environment can be minimized. Assumptions on the
operating environment is not necessary, though it is
common in most schemes in WSN. For instance, a topology
control mechanism in [18] adopts free space and two-ray
ground propagation models. Using RL, a CR node performs
its action without modeling the channel characteristics
including channel selective fading, path loss, shadowing and
PU interference.
The CC is the key component to implement most CR
functions. The CR functions must be performed to mitigate
interference to the PU, as well as to improve the SU
performance. A list of the CR functions is shown next:
•Cooperative sensing: To improve the reliability of
spectrum sensing, decision fusion on the outcomes
collected from neighbour nodes is performed to
mitigate the effects of fading and shadowing. This
reduces hardware requirements such as the sensitivity
of sensing capability at each sensor node.
•Coordination in cooperative sensing: This task is
necessary for nodes to detect PU signals in the absence
of SUs' activities. Nodes cooperate with their
neighbour nodes to perform fast and fine sensing. The
fast sensing applies simple energy-based detection to
detect the presence of PU signals. The fine sensing
applies feature-based detection to categorize the PU
signal according to its signature in order to understand
the characteristics of the PU traffic. The fine sensing is
more advanced and takes longer duration. During the
Figure 4. A cluster-based WSN.
Sensor node
Sink
Cluster 1
Cluster 2 Cluster 3
CH2 CH3
CH2
CH1
GW
Vst=max
a∈A
Qtst,a
Qt1st,at1Qtst,a
t
rt1st1max
a∈A
Qtst1,a
Figure 5. Abstract view of RL in a CR sensor node.
(1)
(2)
Environment
Observe
Learn
Decide action
t→t+1
state,
event reward
agent
action
957
sensing task, all nodes keep quiet to sense for the PU
signal for a short period called the quiet period.
•Notification on PU detection: Once a PU signal is
detected, the SU nodes to communicate with each
other so that decision fusion on sensing outcomes is
performed as soon as possible. The outcome of the
decision fusion is disseminated to neighbour SUs.
•Dynamic Channel Selection (DCS): Channels are
selected in adaptation to channel availability for packet
transmission.
•Channel switching mechanism: When a channel is
reoccupied by a PU, the SU activities must be switched
to a backup channel. Sender and receiver nodes must
inform each other of the channel switching.
•Compliance with timing requirements: The SUs must
conform to the specifications imposed by the PUs. For
instance, an SU node must detect the PU's signal
within a Channel Detection Time (CDT) for signal
strength greater than the Incumbent Detection
Threshold (IDT). During notification and recovery, a
node must cease all transmission within a Channel
Move Time (CMT). In addition, the Channel Closing
Transmission Time (CCTT) defines the aggregated
transmission duration during CMT.
IV. CR-WSN: CONCEPTUAL DESIGN
The CR and WSN share a distinct similarity in the way
they operate: sensing tasks are performed to collect
information from the operating environment and respond
accordingly. However, due to the intrinsic characteristics of
sensor nodes as presented in Table I, none of the existing
CR designs can be directly applied to WSN. Various issues
in the existing CR methods must be addressed to realize a
CR-WSN. Conceptual design of a CR-WSN is proposed in
this section. The CR-WSN must adopt the characteristics of
a traditional WSN in Table I, and perform CR functions in
Section III. It is a daunting challenge as a CR-WSN must
mitigate interference to the PU while improving its own SU
network performance using battery-driven, low cost and low
computation capability sensor nodes only.
For simple hardware requirement, each sensor node is
equipped with a single transceiver. In general, spectrum
sensing is a complex and expensive task that requires each
node to be equipped with sensing hardware. To avoid
excessive load and energy consumption, as well as
expensive hardware implementation on each sensor node,
specialized spectrum sensing devices called coordinators are
deployed throughout the WSN [10] in Figure 4. Various
potential-field-based deployment approaches are discussed
in [13] to improve self-organization. For instance, the
coordinators are treated as virtual particles that repel among
each other by virtual forces in order to achieve an
equilibrium state where the coordinators are spread out to
maximize their coverage throughout the network. The
coordinators are equipped with better channel sensing
devices that provide long sensing range. The coordinator
manages and performs the CR functions. The RL-based CC
could exist in both coordinator and sensor nodes if context-
awareness and intelligence are necessary to be implemented
for performance enhancement. In short, the coordinators
perform and coordinate cooperative sensing, notify the
sensor nodes upon PU detection, perform channel selection
and channel switching. The CHs receives information
disseminated by the coordinators through single or multiple
hops, and notify its GWs and all MNs in its cluster. For
instance, a coordinator that detects PU activity in its
operating channel chooses the next operating channel and
disseminates this information to nearby CHs, which in turn,
notify their respective GWs and MNs.
Cross-layer paradigm is adopted at the coordinator and
sensor nodes to implement the CC. To the best of our
knowledge, there is lack of research into cross-layer
paradigm in CR networks. So why is the cross-layer
paradigm potentially important in CR-WSN? In CR-WSN,
an SU node must be aware of its operating environment
such as the DCS, which resides in the Medium Access
Control (MAC) layer in the protocol stack. In DCS, an SU
node must sense for white space across various channels and
choose a channel dynamically for data transmission. To
enable the functions at the upper layer to be aware of their
operating environment, functions such as routing and
topology management in the network layer must cooperate
with the DCS in the lower layer. As an example, Figure 6
illustrates Joint DCS and topology management design in a
ZigBee protocol stack. This joint design constructs
backbone topology, and assigns a channel with sufficient
transmission range and better quality (recall that we
consider channel heterogeneity), as well as lower channel
utilization from the PUs to the backbone nodes. The
connectivity in the backbone topology must be ensured to
alleviate congestion and packet loss. We assume that the
backbone topology is comprised of sensor nodes that has
ample amount of energy, and are better off compared to
their neighbour sensor nodes. This ensures that packets can
be transmitted to the sink along the high-quality backbone
topology using the best possible channels.
Other cross-layer designs are possible. Joint DCS and
routing design enables sensor nodes to choose a route with
the least level of utilization from PU. Joint DCS and
congestion control helps to ameliorate congestion locally at
each sensor node. In a single channel environment, if a node
experiences congestion, its neighbour nodes have the same
experience. This is not the case in CR environment where
multiple channels exist. Each operating frequency has a
certain level of channel quality and channel utilization from
the PUs. Without load-balancing among the channels, a
node may experience congestion, while its neighbour nodes
have more than ample amount of bandwidth.
V. CR-WSN: OPEN ISSU ES
In this section, various important open issues in CR-WSN
are raised. In general, the open issues incorporate the CR
functions in Section III into a WSN; while upholding the
important characteristics of a WSN in Table I. To the best of
our knowledge, there has been lack of research effort in
addressing these open issues. For each subsection, we
describe the open issue, its importance and motivation, and
the performance metrics that could be improved through
addressing the open issue.
958
A. Sleep Wake Strategy
In traditional WSN, the sensor nodes fall into sleep mode
to minimize their energy consumption without jeopardizing
the network connectivity. During the sleep mode, a node is
not aware of any event happening around itself, hence it
does not know what has happened in its operating
environment upon waking up. In CR-WSN, connectivity is
more important than it ever has been as it ensures that CR
information, such as the operating channel, quiet period, and
notification on PU detection, is disseminated to the CH, GW
and MNs in a timely fashion. Two types of sleep-wake
strategies are Scheduled Rendezvous (SR) and
Asynchronous Wakeup (AW) [14]. The SR requires
sleeping nodes to wake up simultaneously periodically. The
SR method ensures that all sensor nodes receive the CR
information; however, strict synchronization is imperative
because adjacent nodes might not discover each other, and
the CR information might lost even a slight clock drift
occurs. The AW does not require synchronization and each
node maintains its own wake-up schedule; however, this
implies that a node must wake-up more frequently, leading
to a challenging research question of “What is the best sleep
wake strategy so that energy consumption is reduced, while
the network connectivity among the sensor nodes and their
respective coordinator are well maintained without
compromising on the CR functions?” Failure to achieve this
would result in obsolete choice of operating channel among
sensor nodes which causes interference to the PU, as well as
overmitting. The overmitting is an event where energy is
wasted as a result of unpreparedness at the receiver sensor
node to switch to the appropriate channel and results in
retransmission. This also implies that a sensor node's wake-
up schedule is dependent on the PU traffic characteristics,
its utilization level and timing requirements. For instance, a
deterministic and low traffic PU traffic profile with long
CDT, such as the TV spectrum, would require less
communication from the coordinator. This enables th eSU
sensor node to sleep longer and conserve energy. In short, a
good sleep-wake strategy for the CR-WSN must achieve its
traditional purposes of reducing energy consumption and
preserving network connectivity without compromising on
the essential CR functions in Section III so that interference
to the PU can be alleviated. This improves the CR-WSN
network lifetime, and ameliorates interference to the PU.
B. Spectrum Sensing Algorithm
A new algorithm is necessary so that the coordinators and
sensor nodes perform spectrum sensing task in an efficient
manner. The purpose is to detect the PU signal within CDT
among the active sensor nodes that are communicating in
the respective channel. To reduce energy consumption, a
passive sensing approach senses the available channels only
when there is data for transmission; thus, real-time
transmission may not be possible. On the other hand, active
sensing senses the available channels periodically regardless
of its packet arrival; thus it supports real-time transmission;
however this is not energy efficient. This leads to a
challenging research question of “What is the spectrum
sensing algorithm so that energy consumption is reduced
without compromising on the CR functions?” A good
spectrum sensing algorithm for CR-WSN should detect the
PU activity using the least amount of energy. Again, this
improves the CR-WSN network lifetime, and ameliorates
interference to the PU.
C. Effects of Quiet Period on CR-WSN
Since the sensor nodes are not allowed to transmit packets
during the quiet periods in CR-WSN, the end-to-end delay
for a packet from a sensor node to its sink may seem to
increase compared to a traditional WSN. However, in CR-
WSN, the end-to-end delay can be reduced by reducing the
number of hops between the sensor node and its sink using
channels that provide higher transmission range. Thus, the
amount of time saved as a result of smaller number of hops
must offset the delay introduced by quiet period, otherwise
CR-WSN is counterproductive compared to a traditional
WSN. We show that this is possible using an example.
Suppose, the CDT is less than 2s in TV channels. Each fast
and fine sensing incurs approximately 1ms/channel and
25ms/channel respectively. Fine sensing is only performed
if fast sensing detects any signal. Suppose, a packet with
header and payload incurs 0.5ms of transmission delay with
negligible propagation delay for a single hop. Thus, 4000
similar packets can be transmitted within 2s. This means
that each packet incurs 0.25μs on average of fast sensing
delay. If a fine sensing is performed, each packet would
incur 6.5μs. In short, an average single hop transmission
incurs time duration of approximately 77 to 2000 times
greater than the average quiet period per packet. Therefore,
a packet end-to-end delay can be reduced even with the
reduction of a single hop compared to the traditional WSN.
The preceding example is applicable to long-term delay;
however, quiet period increases short-term delay. This issue,
if remain unsolved, would affect real-time performance.
This leads to a research question of “How to reduce the
number of hops for packet transmission using channels that
provide higher transmission range in order to mitigate the
effect of the quiet period in CR function?” This research
question is important to minimize the end-to-end delay of
packet transmission in CR-WSN.
D. Topology Management
Another way to improving the network lifetime is
transmission power adjustment. In general, the underlying
network topology changes as various channel frequencies
are applied for packet transmission. For the same
transmission power, higher frequency provides shorter
transmission range, and thus reduces interference to the
other sensor nodes; while lower frequency provides longer
transmission range, and thus reduces the number of hops
Figure 6. Cross-layer design in ZigBee proto col stack at coordinat or.
Medium Access Control (MAC) Layer
Network Layer
Application Layer
DCS
Physical Layer
Topology Management
959
(which reduces the end-to-end delay) to reach the sink and
conserves energy at the transmitter sensor nodes.
Table II shows the typical values of power incurred
during packet transmission and reception, idle state and
sleep mode for FreeScale MC 13192 SARD [15] that
applies IEEE 802.15.4, and Lucent Silver WaveLAN [16]
for IEEE 802.11. Note that the transmission power in Table
II is adjustable. With ni
is the number of single-hop
neighbours of node i and Ttr ans is the transmission duration,
the total amount of energy incurred for each hop of
transmission is:
Extending the transmission range by choosing lower
channel frequency increases ni. With larger number of nodes
overhearing the packet that is not destined for them, this
increases Eh,i as according to (3) that reduces the network
lifetime. Since both short and long transmission range
provide distinctive advantages, investigation into optimal
methods to ensure network connectivity using different
transmission frequencies is necessary. A solution is to
construct a Minimum Spanning Tree (MST) [17], [18] that
connects all nodes in an undirected graph with different
maximum edge lengths at different channel. This involves
the adjustment of both transmission power and operating
frequency channel. Nodes can sleep to avoid incurring
unnecessary reception energy consumption.
The research question is “What is the best topology
management strategy so that energy consumption and
number of hops for packet transmission are both reduced in
the CR environment where multiple operating channels are
available?” This research question is important to adjust the
tradeoff between network lifetime and end-to-end delay of
packet transmission.
E. Context-Aware Routing
Though channels with lower frequency have larger
transmission range, this does not mean that a node chooses a
channel merely dependent on this factor. In addition to
routing considerations in traditional WSN such as residual
energy at sensor node; other factors are channel utilization
by PU, and channel quality. These factors should be
incorporated into the routing protocol for WSN such as
geographic routing [19]. The research question is “What is
the best routing mechanism so that various factors
including channel utilization by the PU and channel quality
are being tackled in the CR environment?”
We show that the CR improves the existing routing
protocols in WSN. Geographic routing is a popular routing
protocol in WSNs that enables users to query a geographical
region rather than a single sensor node. In [19], the
performances of routing algorithms are compared using
dilation. The dilation represents the stretch factor of a
wireless network G(V,E) relative to an ideal wireless
network. Smaller value of dilation indicates a better route
because it is closer to an ideal route. With longer
transmission range, the geographic routing improves
because of lower dilation [15]. Two types of geographic
routing, namely Greedy Forwarding (GF) and Bounded
Voronoi Greedy Forwarding (BVGF) improves with
Rc/Rs≫2 [15]. The Rc is the sensor node
communication range. The Rs is the sensor node sensing
range. The extended Rc improves both GV and BVGF.
TABLE II. STATE AND ITS POWER
State Power (W)
FreeScale MC SARD Lucent WaveLAN
Transmissi on Ptrans 0.1404 1.3
Reception Precv 0.1404 0.9
Idle Pidle 0.0018 0.74
Sleep Psleep 0.000018 0.047
F. Deployment of Coordinators
A large number of sensor nodes, typically in the order of
hundreds, thousands or millions, may be deployed in CR-
WSN. An optimal number of coordinators (including
backup coordinators if necessary) avoids over-provisioning
and reduces cost. In the case where some of the coordinators
are out of order due to lack of power and hardware damage,
the sensor nodes must be capable of sustaining its
functionality. In short, a good coordinators deployment
mechanism provides a robust CR-WSN at reduced cost
without compromising on the cognitive functions.
VI. ADDRESSING AN OPEN ISSUE IN CR-WSN USING
REINFORCEMENT LEARNING
In this section, we present our preliminary results in using
the RL to address the open issue of topology management in
CR-WSN. The RL is applied in the joint DCS and topology
management discussed in Section IV. We assume that the
underlying backbone topology is readily available and our
focus in this section is the DCS scheme that assigns a
channel with sufficient transmission range and better
quality, as well as lower channel utilization from the PUs to
the backbone nodes, as shown in Figure 7. To simulate the
capability of the DCS to choose channels with sufficient
transmission range, another SU sensor node is moving
within the maximum transmission range in Figure 7. This
ensures that packets can be transmitted to the sink along the
high-quality backbone topology using the best possible
channels in the presence of channel heterogeneity.
A. RL Model for DCS
The performance of a communication from node i to node
j is indicated by packet transmission success probability in a
channel k, Ps,k
i, j, which is the proportion of packets
successfully transmitted to node j. The Ps,k
i, j is dependent
on many factors including the PU Channel Utilization Level
(PUL) and the Packet Error Rate (PER) in the channel.
The RL model for the DCS is shown in Table III. The
state SC has one component, which is node i's neighbour
nodes j, with Nn=∣Nbr i∣ as the cardinality of node i's
neighbour nodes. The action AC is to choose a channel for
packet transmission from K available channels. For every
successful packet transmission, there is a reward with
positive constant value
RW
, otherwise a cost with
(3)
=Ptrans TtransniPrecv Ttrans
Eh,i=EtransniErecv
960
negative constant value
CT is incurred. In practice, the
value of
RW
and CT are based on the amount of revenue
and cost that a sensor network operator earns or incurs for
each successful or unsuccessful packet transmission. Packet
packet transmission is successful when a link-layer
acknowledgment is received for the packet sent, else the
transmission is unsuccessful. In addition, if a chosen
channel is reoccupied by PU immediately before the packet
is sent, it is considered an unsuccessful transmission.
At each decision epoch t, node i chooses a channel for
data transmission. The node maintains a list of rules, U, that
matches a state sc
i to an action ac
i with a certainty value of
Ei
u
sc
i,ac
i
within an interval of [Emin ,Emax ]. The
Eiusc
i,ac
i indicates the appropriateness of taking ac
i
in sc
i and it is updated using Q-learning:
with the ma
x
a∈A
Q
t
u
s
t1,a
in (1) being omitted to denote
lack of dependency on future discounted rewards. Based on
(4), Emin=CR and Emax=RW . The greedy action is
chosen as follows:
At the beginning of every attempt to transmit a data
packet, a node chooses to either continue or change an
action or channel. In order to reduce the number of channel
switchings, a node does not switch channel unless the Q-
value of the other action is better than the current one. In
addition, the node may change its channel with probability
for exploration purpose.
B. Simulation Setup
We have implemented a CR-enabled environment in
INET framework for OMNeT++ [20]. Simulation
parameters are shown in Table IV. The values of Q-learning
parameters in the table,
RW
and
CT , were chosen
empirically to optimize simulation performance.
C. Simulation Performance Metrics
Our goal is to maximize throughput over different
heterogeneous channels with different PUL and PER. The
throughput of the RL-based DCS is compared with that of
Random DCS where an available channel is chosen for next
data packet transmission in a uniformly distributed random
manner. Graphs are presented with PUL and PER as
ordinate respectively. Each simulation result is for all
possible combinations of PUL or PER. As an example, a
PUL or PER of 0.2 for three available channels may indicate
[0.2,0.2,0.2], [0.2,0.3,0.1], [0.2,0,0.4], and others.
D. Simulation Results
The throughput achieved by the RL and Random scheme
is investigated for various levels of PUL. The PER for all
channels are set to 0.1 to show the effectiveness of the RL
method in choosing a channel with low PUL for packet
transmission. As shown in Figure 8, the RL scheme
outperforms the Random for all levels of PUL. Throughput
enhancement provided by the RL scheme is up to 4.39 times
at 0.9 PUL. Hence, the RL scheme learns well and helps the
SU sensor node to choose a channel with low PUL, as well
as suitable transmission range such that the packet
successful transmission rate is high.
TABLE III. RL MODEL FOR DCS
RL
Element
Description Representation
State Set of node i's neighbour
nodes j.
S
C
={sc=j}
j
={n1,n2,. ..
,
nNn}
Action Available channels for data
transmission.
AC
={
ac
=
c1,c2,
,cK
}
Reward Constant value to be
rewarded/incurred for
successful/unsuccessful data
packet transmission.
RC={rs,a}
={
{
RW if successful
CT otherwise }
TABLE IV. NOTATIONS AND P ARAMETER SETTINGS IN SIMULATI ON
Category Details Values
Scenario
Initializa
tion
Number of SU sensor nodes,
N
2
Number of available channels,
K
3
Channel frequency Channel 1: 5.7GHz
Channel 2: 800MHz
Channel 3: 400MHz
Packet error rate of each available
channel
[0, 0.9]
Total simulation time 500s
Second
ary user
Sensor node traffic model Always backlogged.
Data packet duration 5.44ms
Channel switching delay 100μs
Mobility characteristic One SU is static; Another
SU is moving randomly
at 20m/s.
Primary
user
Primary user traffi c model Stochastic chan nels with
exponentially distributed
ON and OFF times.
Data packet duration 5.44ms
Utilization of each PU traffic [0.1, 0.9]
Q-
learning
Learning rate of Q-learning,
0.2
Trade-off between exploration and
exploitation,
0.1
Initial certainty values 1
Reward, RW 15
Cost, CT 5
Next, the throughput achieved by the RL and Random
scheme is investigated for various levels of PER. The PUL
Et1
iusc,t
i,ac,t
i
1Et
iusc,t
i,ac,t
i×rt1
isc,t1
i(4)
ac,t
i=argmax
ai∈AC
Et
iusc,t
i,a
i (5)
Figure 7 . A se nso r SU and it s t ran smi ssion ran ges using dif fer ent
channels. Another sensor SU is moving within the maximum
tran smissio n r an
g
e. Each ch annel is licensed.
m1
m2
m3
961
for all channels are set to 0.1 to show the effectiveness of
the RL method in choosing a channel with low PER for
packet transmission. As shown in Figure 9, the RL scheme
outperforms the Random for all levels of PER except 0.9.
At 0.9 PER, the Q-values of all the channels converge to
-CT. When all the channels result in poor performance, the
RL simply chooses channel 1 (see Figure 7) that provides
the shortest transmission range resulting in failure
transmission for all attempts when the SUs move beyond the
transmission range of channel 1. This issue can be solved by
imposing a rule to transmit using channel that provides
larger transmission range when all Q-values converge to
-CT. Throughput enhancement provided by the RL scheme
is up to 2.33 times at 0.5 and 0.6 PER. Hence, the RL
scheme learns well and helps the SU sensor node to choose
a channel with low PER, as well as suitable transmission
range such that the packet successful transmission rate is
high.
E. Addressing Open Issues in CR-WSN using RL
The RL has been shown to achieve context-awareness and
intelligent in CR-WSN. To address the open issues raised in
Section V, it is necessary to model the open issues using RL
as shown in Section VI.A. As an example, in Section V.A,
we question “What is the best sleep wake strategy?”. The
state is the amount of white space in the operating
environment, and connectivity of the sensor nodes; the
action is to choose one of the actions in Table II: transmit,
receive, idle or sleep; and the reward the amount of energy
saved throughout the network lifetime. Other RL model for
the sleep wake strategy is possible.
CONCLUSIONS
In this paper, we have introduced a Cognitive Radio-
based Wireless Sensor Network (CR-WSN). It is our firm
believe that CR-WSN is the next-generation WSN. CR
technology can be incorporated into traditional WSN to
provide advantages including transmission range extension,
network connectivity and lifetime enhancement. Various
open issues in deploying a CR-WSN including sleep wake
strategy, spectrum sensing algorithm, effect of quiet period,
topology management, context-aware routing, and
deployment of coordinators are presented to intensify
research in this area. We have also shown our method to
achieve context-awareness and intelligence in CR-WSN
using Reinforcement Learning (RL). The RL method is
suitable to be applied to address the open issues in CR-
WSN. In our future work, we will present more results on
the application of RL in addressing some of the open issues
raised in this paper.
REFERENCES
[1] FCC Spectrum Policy Task Force, “Report of the spectrum efficienc y
working group,” FCC, Technical Report 02-155, Nov 2002.
[2] I. F. Akyildiz, W.-Y. Lee, M. C. Vuran, and S. Mohant y, “A survey
on spectrum management in cognitive radio networks,” IEEE Comm.
Magazine, April 20 08.
[3] J. Mitola III and G. Q. Maguire, “C ognitive radio: making software
radios more personal,” Personal Communications, IEEE, 1999.
[4] D. Cabric, S. M. Mishra, and R. W. Brodersen, “Implementation
issues in spectrum sensing for cognitive radios,” In Proc. Asilomar
Conf. on Signals, Systems and Computers, pp. 772-776, Nov. 2004.
[5] C. Intanagonwiwat, R. Govindan, and D. Estrin, “Directed Diffusion:
A Scalable and Robust Communication Paradigm for Sensor
Networks,” In Proc. of the 6th AICMCN, pp. 56-67, 2000.
[6] Hoyhtya M., Pollin S. , and Mammela A., “Performance improvement
with predictive channel selection for cognitive radios,” CogART First
Intl. Wo rkshop on, pp. 1-5, Feb. 2008.
[7] C. Xin, L. Ma, and C.-C. Shen, “Path-centric channel assignment in
cognitive radio wireless networks,” Cognitive Radio Oriented
Wireless Nwks and Comm. (CrownCom), pp. 313-320, Aug. 2007.
[8] K. Bian, and J.-M. Park, “Segment-based channel assignment in
cognitive radio ad hoc networks,” Cognitive Radio Oriented Wireless
Networks and Comm. (CrownCom'07), pp. 327-335, Aug. 2007.
[9] I. F. Akyildiz, T. Melodia, and K. R. Chowdury, “Wireless
multimedia sensor net works: A survey,” IEEE Wls Comm, Dec 2007.
[10] D. Cavalcanti, S. Das, J. Wang, and K. Challapali, “Cognitive radi o
based wireless sensor networks,” 17th ICCCN, August 2008.
[11] R. S. Sutton and A. G. Barto, Reinforcement learning: an
introduction. Cambridge MA, MIT Press, 1998.
[12] C. Watkins, “Learning from delayed rewards,” PhD thesis, University
of Cambridge, UK, 1989.
[13] A. Howard, M. J. Mataric, and G. S. Sukhatme, “Mobile sensor
networks deployment using potential fields: A distributed, scalable
solution to the area coverage problem,” 6th DARS, June 2002.
[14] R. Zheng, J. C. Hou, and L. Sha, “Asynchronous Wakeup for Ad Hoc
Networks,” ACM Il. Sp. Mob Ad Hoc Nwk & Com., pp. 35-45, 2003.
[15] A. Bachir, D. Barthel, M. Heusse, A. Duda, “A synthetic function for
energy-delay mapping in energy effi cient routing,” 3rd Annual Conf.
on Wireless On-demand Nwk S ys and Services (WONS), Jan. 2006.
[16] L. Feeney, and M. Nilson, “Investigating the energy consumption of a
wireless network interface in an ad hoc networking environment,”
IEEE Conf. on Comp. Comm. (INFOCOMM), April 2001.
[17] V. Rodoplu, and T. H. Meng, “Minimum energy mobile wireless
networks,” Sel. Ar. in Comm., IEEE, 14(8), pp. 1333-1344, Aug 1999.
[18] N. Li, J. C. Hou, and L. Sha, “Design and ana lysis of an MST-based
topology control algorithm,” INFOCOMM, April 2003.
[19] G. Xin g, C. Lu, R. Pless, and Q. Huang, “On greedy geographic
routing algorithms in sensing-covered networks,” ACM Il. Sp. Mob
Ad Hoc Nwk & Co m., pp. 31-42, 2004.
[20] INET Fwk OMNet++ r12.10.06 http://www.omnetpp.org/doc/INET/.
Figure 8. T he mean throughput of an SU sensor against mean of
chan nel utiliza tio n by PU for RL and random schem es in CR-WSN.
Figure 9. The mean throughput of an SU sensor against mean o
f
packet error rate for all channels for RL and random schemes in CR-
962
