Steady-state Markov chain analysis for heterogeneous cognitive radio networks

Abstract

Cognitive radio technology has been widely researched to improve the spectrum usage efficiency. Modeling of the spectrum occupancy in a cognitive framework including licensed and unlicensed users with various traffic conditions, is a prior requirement to do the system analysis. In this paper, we develop a continuous-time Markov chain model to describe the radio spectrum usage, and derive the transition rate matrix for this model. In addition, we perform steady-state analysis to analytically derive the probability state vector. The proposed model and derived expressions are compared to the existing models, and examined through numerical analysis.

Full text

Steady-State Markov Chain Analysis for
Heterogeneous Cognitive Radio Networks
Amir Sepasi Zahmati, Xavier Fernando
Department of Electrical and Computer Engineering
Ryerson University
Toronto, Canada
Email: {asepasi, fernando}@ee.ryerson.ca
Ali Grami
Faculty of Engineering and Applied Science
University of Ontario Institute of Technology
Oshawa, Canada
Email: ali.grami@uoit.ca
Abstract—Cognitive radio technology has been widely re-
searched to improve the spectrum usage efficiency. Modeling
of the spectrum occupancy in a cognitive framework including
licensed and unlicensed users with various traffic conditions, is
a prior requirement to do the system analysis. In this paper,
we develop a continuous-time Markov chain model to describe
the radio spectrum usage, and derive the transition rate matrix
for this model. In addition, we perform steady-state analysis to
analytically derive the probability state vector. The proposed
model and derived expressions are compared to the existing
models, and examined through numerical analysis.
Index Terms—cognitive radio networks; continuous-time
Markov chain; steady-state analysis; heterogeneous networks
I. INTRODUCTION
Recent measurements report that, at any given time and
location, generally less than 5% of the allocated radio spec-
trum is used in most wireless networks [1]. As a potential
solution for this inefficiency, cognitive radio (CR) technology
is considered by many researches to improve the spectrum
usage inefficiency [2-4]. The CR concept introduces a scenario
in which the unused spectrum by the licensed (primary) users
can be identified through spectrum sensing and reused by the
unlicensed (secondary) users.
Modeling of the spectrum occupancy by both primary and
secondary networks is a prior need to analyze a cognitive
radio network (CRN). Several research works have performed
Markov analysis in CRNs [5-9]. Particularly, [7] validates
the existence of a Markov chain by collecting real-time
measurements in the paging spectrum (928-948 MHz); the
same methodology can be applied to any other spectrum
band. Moreover, [10] sets up a continuous-time Markov chain
(CTMC) model for dynamic spectrum, where the radio sys-
tems attempt to operate in the same band.
Although existing Markov systems have modeled the cogni-
tive system accurately, most of them assume a simple cognitive
network. Authors in [11] assume a cognitive network with one
primary user and two secondary users, and propose a primary-
prioritized Markov approach through modeling the interactions
between the primary and the secondary users as a CTMC
model. In [3], a CTMC model is proposed to predict the
behavior of open (unutilized) spectrum access in unlicensed
bands. The model considers two types of secondary systems,
but the analysis does not include the primary user. Moreover,
[12] presents the CTMC model for a simple case of one
primary system and one secondary system.
In this paper, we assume a heterogeneous cognitive network
consisting of Nsecondary users (with various traffic condi-
tions) and one primary user. We describe the radio spectrum
occupancy as a CTMC model, and analytically derive the
transition rate matrix based on the model. Moreover, we
perform a steady-state analysis to derive the stationary state
probability (SSP) vector. We perform numerical analysis to
compare our model with the existing models, and to validate
the proposed model for some critical cases.
The rest of this paper is organized as follows: Section II
discusses the proposed system model and analytical work to
derive the transition rate matrix and SSP vector. In Section
III, we perform numerical analysis to compare our model with
the existing models in the literature. Finally, we conclude the
paper in Section IV.
II. SYSTEM MODEL
We study a CRN with one primary user and Nsecondary
users as shown in Fig. 1. We assume that the arrival and
departure of the primary and secondary users’ traffic are
independent, continuous time Poisson Processes. Therefore,
we model the spectrum access process by a continuous-time
Markov chain (CTMC) system. The primary users’ traffic is
modeled with two random processes. The service request is
modeled as a Poisson Process with arrival rate λPs−1, and
the service duration (access duration) is negative-exponentially
distributed with mean time 1/μPs, so the departure of the
primary user’s traffic is another Poisson Process with departure
rate μPs−1[11]. Similarly, the arrival and departure rates of
secondary user i(1 ≤i≤N)are modeled as independent
Poisson Processes with λis−1and μis−1respectively.
In a CRN where the secondary users opportunistically
access the available licensed spectrum, priority must be given
to the primary user. In other words, the secondary users have
to vacate the channel whenever the primary user appears.
Moreover, we assume that the spectrum cannot be occupied
by more than one user simultaneously, i.e., there is no overlap
between any user. Therefore, we can model the spectrum
access process by the model shown in Fig. 2. The state space

Fig. 1. The CRN with one primary user and Nsecondary users.
Fig. 2. The state transition diagram for the CTMC model.
vector xfor the model is,
x={O, S1,S
2,...,S
N,P}.(1)
In (1), state O(idle) means no user operates in the spectrum,
state Pmeans the primary user operates in the spectrum, and
state Simeans the secondary user ioperates in the spectrum
(1 ≤i≤N). We assume the initial state of the system to
be state O. The system enters state Pwith arrival rate λP
whenever the primary user appears, and it returns to state O
with departure rate μPwhen the primary user’s service is
complete. While the system is in state O, and the secondary
user ineeds to operate in the spectrum, the system enters state
Siwith arrival rate λi. In case the primary user arrives before
the secondary user i’s service is complete, the system transits
to state Pwith arrival rate λP. Otherwise, the secondary user
ireturns to state Owith departure rate μiwhen its service is
complete. In the CTMC, when the secondary users contend to
access the idle spectrum using CSMA, collisions occur only
when their service requests arrive exactly at the same time.
This case rarely happens for independent Poisson Process [11].
Therefore, in the CTMC model we omit the collision state of
the secondary users.
According to [14], a complete model of a CTMC is obtained
if the transition rate matrix Qand the state space vector (x)
are specified. The state space vector xpresented in (1) is a
Fig. 3. The Transition Rate Matrix of the CTMC model Q
(N+2) element vector. Therefore, Qis a square matrix of
order (N+2). The off-diagonal element of the transition rate
matrix (qij ,(i=j)) is equal to the Poisson Process rate of the
event causing transition from state ito state j. Moreover, the
diagonal element qii,1≤i≤N+2 is determined by solving
qii =−
allj, i=j
qij .(2)
We can therefore derive (Q)for the proposed model as
shown in Fig. 3.
One of the main objectives of Markov chain analysis is
the determination of the state probability vector Π(t)=
[Π0(t),Π1(t),Π2(t),...,ΠN(t),ΠP(t)].Theith element of
Π(t)is given by,
Πi(t)=P[x(t)=i],∀i∈x(3)
which implies the probability to find the chain at state iat
specific time instants [15]. Now, we perform the steady-state
analysis by assuming the system has been in the ON condition
long enough and all parameters have achieved their steady-
state values [14]. Therefore the steady-state probability of state
iis
Πi= lim
t→∞ Πi(t),(4)
where, Πirepresents the probability of being in state i.The
steady-state probability (SSP) vector (Π)is determined by
solving (5) and (6) as follows:
ΠQ =0(5)

∀j∈x
Πj=1 (6)
The matrix equation in (5) can be expanded to a system of
linear equations presented in (7-9) as follows:
−Π0(λ1+λ2+...+λN+λP)+
+Π1μ1+Π
2μ2+... +Π
NμN+Π
PμP=0 (7)
Π0λi−Πi(μi+λP)=0,(1 ≤i≤N)(8)
(Π0+Π
1+...+Π
N)λP−ΠPμP=0 (9)
Substituting ΠPfrom (7) into (6), we get
Π0+Π
1+...+Π
N+Π
P=1⇒
Π0+Π
1+...+Π
N+λP
μP
(Π0+Π
1+...+Π
N)=1⇒
Π0+Π
1+...+Π
N=μP
μP+λP
.(10)

Now, we substitute (8) into (10) to obtain Π0:
Π0+Π0λ1
μ1+λP
+...+Π0λN
μN+λP
=μP
μP+λP
⇒
Π0(1 + λ1
μ1+λP
+...+λN
μN+λP
)= μP
μP+λP
⇒
Π0=μP
(μP+λP)(1 + N
i=1
λi
μi+λP).(11)
Therefore, Π1,Π2,...,ΠNare given by (8), where, Π0is
presented in (11),
Πi=λi
μi+λP
Π0,1≤i≤N. (12)
Finally, we substitute (12) into (6) to derive ΠPas follows:
ΠP=1−Π0−Π1−...−ΠN
=1−Π0−λ1
μ1+λP
Π0−...−λN
μN+λP
Π0
=1−Π0(1 + λ1
μ1+λP
+...+λN
μN+λP
)
=1−μP(1 + λ1
μ1+λP+...+λN
μN+λP)
(μP+λP)(1 + λ1
μ1+λP+...+λN
μN+λP)
=1−μP
μP+λP
=λP
μP+λP
.(13)
Therefore, the SSP vector is determined by (11)-(13). As
expected, Πpis a function of the primary channel parameters
(λpand μp) only.
III. NUMERICAL ANALYSIS
In this section, we test our model for a number of critical
cases. First, we study a CRN with no secondary user (N=0),
and compare the proposed model with the existing model
in the literature. Second, we consider a CRN with three
secondary users (N=3), and examine the effects of the
secondary users’ arrival and departure rates on SSP vector.
Third, we study a secondary network with identical arrival and
departure rates for all secondary users. Finally, we examine a
case with a very large number of secondary users.
A. No Active Secondary User (N=0)
In this part, we evaluate our model for a basic scenario
with no secondary user (N=0). Fig. 4 (a) shows the state
transition diagram in this case, where the state space vector is
x={O, P }. The SSP vector Π=[Π
0,ΠP]is given by (11)
and (13) as follows:
Π0=μP
μP+λP
,(14)
ΠP=1−Π0=λP
μP+λP
.(15)
Our model in this case is similar to the model presented in
[13] and [16]. Authors in [13] model the primary channel as
Fig. 4. State transition diagram: (a) our model for N=0, (b) the existing
model. [13]
an ON-OFF source alternating between ON (busy) and OFF
(idle) periods. Fig. 4 (b) shows the state transition diagram for
this model. Furthermore, [13] defines channel utilization (u)
as the fraction of time in which the primary channel is in ON
state:
u=λOF F
λON +λOF F
,(16)
where, λOF F and λON are the mean values of TOFF and
TON respectively, i.e. E[TOF F ]= 1
λOFF and E[TON ]= 1
λON .
Therefore, the fraction of time in which the primary channel
is in OFF state is:
1−u=λON
λON +λOF F
.(17)
Therefore, the proposed model agrees with [13]. As it can be
seen in (14-17), ΠPand Π0in our model represent uand (1−
u)in [13] respectively. Furthermore, λOF F and λON in [13]
correspond to μPand λPrespectively. Table I summarizes the
above notations and results.
B. Three Secondary Users (N=3)
Now, we consider a CRN with three secondary users (N=
3), where the state space vector is x={O, S1,S
2,S
3,P}, and
the state transition diagram is shown in Fig. 5. The stationary
state probability Π0is calculated by (11):
Π0=μP
(μP+λP)(1 + λ1
μ1+λP+λ2
μ2+λP+λ3
μ3+λP).(18)
Moreover, the secondary users’ stationary state probabilities
(Π1,Π2,Π3)are given by (12), and ΠPis calculated by (13).
For this case, we set the primary user’s arrival and departure
TAB L E I
STATIONARY STATE PROBABILITIES:THE RESULTS ARE CONSISTENT WITH
THE MODEL IN [13].
The proposed model Existing model [13]
μPλOF F
λPλON
ΠP=λP
λP+μPu=λOFF
λON +λOFF
Π0=μP
λP+μP
1−u=λON
λON +λOFF

Fig. 5. The state transition diagram for N=3.
rates to be λP=2s−1and μP=4s−1respectively, and
investigate the effect of the secondary users’ parameters on
the SSP vector (Π)for three cases as follows:
1) Case-I: We assume different departure rates μ1<μ
2<
μ3, and assume identical arrival rates (λ1=λ2=λ3=λ).
Thus, (12) simplifies to
Πi=λ
μi+λP
Π0,1≤i≤3.(19)
From the above equation, μ1<μ
2<μ
3yields Π1>Π2>
Π3which confirms that for a fixed value of λ, there is a high
probability to find the system at a state with lower departure
rate (μ). Fig. 6 (a) shows the stationary state probabilities for
a secondary network with the following parameters, where the
secondary user iis denoted by SUi, and λiand μiareins
−1.
SU1:λ1=3
μ1=2 ,SU
2:λ2=3
μ2=4 ,SU
3:λ3=3
μ3=6
The SSP vector in this case is Π=
[0.2540,0.1905,0.1270,0.0952,0.3333].
2) Case-II: We assume different arrival rates λ1<λ
2<
λ3, but we fix the departure rates (μ1=μ2=μ3=μ).
Therefore, (12) simplifies to
Πi=λi
μ+λP
Π0,1≤i≤3.(20)
Asshownin(20),λ1<λ
2<λ
3yields Π1<Π2<Π3.
This confirms that for a fixed value of μ, there is a higher
probability to find the system at a state with larger arrival rate
(λ). The secondary users’ parameters are as follows, where
the secondary user iis denoted by SUi, and λiand μiare in
s−1.
SU1:λ1=2
μ1=3 ,SU
2:λ2=4
μ2=3 ,SU
3:λ3=6
μ3=3
Fig. 6. The stationary state probabilities for cases (a), (b), and (c).
The SSP vector in this case is Π=
[0.1961,0.0784,0.1569,0.2353,0.3333]. Fig. 6 (b) shows the
stationary state probabilities.
3) Case-III: In this part, we assume a secondary network,
where all secondary users have the same arrival rates (λ1=
λ2=... =λN=λ)and also have the same departure rates
(μ1=μ2=...=μN=μ). First, we derive Πfor a general
case with Nsecondary users. Then, we consider a secondary
network with N=3users, and sketch the stationary state
probabilities similar to the previous cases. Taking into account
the above assumptions, (11) simplifies to
Π0=μP
(μP+λP)(1 + λ
μ+λP+...+λ
μ+λP)
=μP
(μP+λP)(1 + Nλ
μ+λP).(21)
Therefore, all the secondary users have the same stationary
state probabilities:
Π=Π
1=Π
2=...=Π
N=λ
μ+λP
Π0
=λμP
(μP+λP)(μ+λP+Nλ).(22)
Therefore, for a CRN with N=3secondary users, and
parameters λ=4s−1and μ=6s−1, the SSP vector is Π=
[0.2667,0.1333,0.1333,0.1333,0.3333]. Fig. 6 (c) shows the
stationary state probabilities for this case.
Note that the primary user’s stationary state probability
(ΠP)presented in (13) is not a function of the number of
secondary users (N)and the secondary users’ parameters (λ
and μ). Therefore, in all the above cases, ΠPhas the same
value ΠP=0.3333.

Fig. 7. The stationary state probabilities in a secondary network with N=
100 secondary users.
C. Very Large Number of Secondary Users
In this part, we study a CRN with a very large number of
secondary users (N). Similar to the previous cases, secondary
users’ stationary state probabilities are determined by (11) and
(12), and ΠPis independent of N. Fig. 7 shows the stationary
state probabilities for a CRN with N= 100 secondary users,
where the secondary users’ arrival and departure rates are
selected randomly from 0.1≤λi≤5s−1and 0.1≤μi≤7.5
s−1, and the primary user’s parameters are set to be λP=2.5
s−1and μP=5s−1. Fig. 7 confirms that for a very large
number of secondary users, the probability to find the system
at each secondary state would be quite small.
IV. CONCLUSION
This paper presents a continuous-time Markov chain model
to describe the spectrum usage by both primary and secondary
networks in a CRN. The presented model, derivations, and
results can be used to model the radio spectrum occupation
in a heterogeneous CRN including Nsecondary users and
one primary user. In addition, it shows the analytical work
to derive the transition rate matrix and the probability state
vector. The proposed model and derived expressions were
examined through numerical analysis, and compared to the
existing models in the literature. As a part of future work, we
will add more complexity to the proposed model by assuming
that the secondary users can operate in the radio spectrum
simultaneously.
REFERENCES
[1] R. Tandra and A. Sahai, “Fundamental limits on detection in low SNR
under noise uncertainty,” in Proc. International Conference on Wireless
Networks, Communications and Mobile Computing, 2005.
[2] I. F. Akyildiz, W. Lee, and M. C. Vuran, Shantidev Mohanty, “NeXt
generation/dynamic spectrum access/cognitive radio wireless networks: a
survey,” Computer Networks, vol. 50, pp. 2127-2159, Sep. 2006.
[3] Y. Xing, R. Chanddramouli, S. Mangold, and S. Shankar, “Dynamic
spectrum access in open spectrum wireless networks,” IEEE J. Sel. Areas
Commun., vol. 24, pp. 626-636, Mar. 2006.
[4] T. Weiss and F. Jondral, “Spectrum pooling: an innovative strategy for
the enhancement of spectrum efficiency,” IEEE Commun. Mag., vol. 42,
pp. S8-S14, Mar. 2004.
[5] T. W. Rondeau, C. J. Rieser, T. M. Gallagher, and C. W. Bostian, “Online
modeling of wireless channels with hidden markov models and channel
impulse responses for cognitive radios,” 2004 IEEE MTT-S Digest,vol.
2, pp. 739-742, June 2004.
[6] K. Kim, I. A. Akbar, K. K. Bae, J-S. Um, C. M. Spooner, and J. H.
Reed “Cyclostationary approaches to signal detection and classification
in cognitive radio,” in Proc. of IEEE DySPAN 2007, pp. 212-215, April
2007.
[7] C. Ghosh, C. Cordeiro, D.P. Agrawal, and M.B. Rao, “Markov chain
existence and Hidden Markov models in spectrum sensing,” IEEE Inter-
national Conference on Pervasive Computing and Communications, pp.
1-6, March 2009.
[8] Q. Zhao and A. Swami, “A decision-theoretic framework for opportunistic
spectrum access,” IEEE Wireless Commun. Mag., vol. 14, no. 4, pp. 14-
20, August 2007.
[9] Q. Zhao, B. Krishnamachari, and K. Liu, “On myopic sensing for multi-
channel opportunistic access: structure, optimality, and performance,”
IEEE Trans. Wireless Commun., vol. 7, no. 12, Dec. 2008.
[10] L. Luo and S. Roy A two-stage sensing technique for dynamic spectrum
access IEEE Int. Conf. on Communications, pp. 4181-4185, May 2008.
[11] Beibei Wang, Zhu Ji, K. J. Ray Liu, and T. Charles Clancy, “Primary-
prioritized Markov approach for dynamic spectrum allocation dynamic
spectrum allocation,” IEEE Trans. Wireless Commun., vol. 8, no. 4, pp.
1854-1865, April 2009.
[12] X. Zhu, L. Shen and T-S.P. Yum “Analysis of cognitive radio spectrum
access with optimal channel reservation,” IEEE Commu. Letters, vol. 11,
issue 4, pp. 304-306, Apr. 2007.
[13] H. Kim and K.G. Shin, “Efficient discovery of spectrum opportunities
with MAC-layer sensing in cognitive radio networks,” IEEE Trans.
Mobile Comput., vol. 7, no. 5, pp. 533-545, May 2008.
[14] C. G. Cassandras and S. Lafortune, Introduction to discrete event
systems. Springer, 2009.
[15] V. G. Kulkarni, Modeling and analysis of stochastic systems. CRC Press,
1995.
[16] H. Kim and K.G. Shin, “Fast discovery of spectrum opportunities in
cognitive radio networks,” 3rd IEEE Symposium on New Frontiers in
Dynamic Spectrum Access Networks, pp. 1-12, Oct. 2008.