Content uploaded by Nadisanka Rupasinghe
Author content
All content in this area was uploaded by Nadisanka Rupasinghe on Feb 07, 2018
Content may be subject to copyright.
Reinforcement Learning for Licensed-Assisted
Access of LTE in the Unlicensed Spectrum
Nadisanka Rupasinghe and ˙
Ismail G¨
uvenc¸
Department of Electrical and Computer Engineering, Florida International University, Miami, FL 33174
Email: {rrupa001, iguvenc}@fiu.edu
Abstract—In order to coexist with the WiFi systems in the
unlicensed spectrum, Long Term Evolution (LTE) networks
can utilize periodically configured transmission gaps. In this
paper, considering a time division duplex (TDD)-LTE system,
we propose a Q-Learning based dynamic duty cycle selection
technique for configuring LTE transmission gaps, so that a
satisfactory throughput is maintained both for LTE and WiFi
systems. By explicitly taking the impact of IEEE 802.11n beacon
transmission mechanism into account, we evaluate the coexistence
performance of WiFi and LTE using the proposed technique.
Simulation results show that the proposed approach can enhance
the overall capacity performance by 19% and WiFi capacity
performance by 77%, hence enabling effective coexistence of LTE
and WiFi systems in the unlicensed band.
Index Terms—Beacon, licensed-assisted access (LAA), Q-
Learning, reinforcement learning, TDD-LTE, WiFi 802.11n.
I. INTRODUCTION
Use of Long Term Evolution (LTE) technology in the unli-
censed spectrum has been recently gaining significant attention
to enable higher throughput, cater insatiable traffic demand,
and allow a better quality of service for cellular users [1]–[4].
The unlicensed spectrum is traditionally occupied by wireless
communication technologies such as WiFi, bluetooth, and
radar. Since WiFi provides a higher user throughput, network
operators, at the moment, prefer using WiFi for expanding
their capacity by offloading traffic to WiFi. Due to its coor-
dinated deployment and operation, LTE has the potential to
provide higher capacity and better coverage than WiFi for the
same transmit power, while providing seamless connectivity
[3]. Motivated by this potential, the 3GPP standardization
group has recently initiated a study item on licensed-assisted
access (LAA) using LTE in the unlicensed spectrum [1].
To enable the operation of LTE in the unlicensed band,
coexistence with WiFi technology carries critical importance.
Different coexistence mechanisms between WiFi and LAA
have been studied in the literature. In [5], coexistence of
LAA and WiFi as secondary users in TV white space is
investigated, and two techniques are proposed to facilitate
interference management: 1) spectrum sensing (Listen-Before-
Talk (LBT)) by LAA, and 2) coexistence gap during which
LAA refrain from transmitting. LBT based approaches have
been considered also in [6], [7] for LTE systems, to fa-
cilitate operation in the unlicensed spectrum. In [6], carrier
aggregation for LTE from licensed and license-exempt bands
This research was supported in part by the U.S. National Science Founda-
tion under the grants CNS-1406968 and AST-1443999.
is proposed. In that, to access license-exempt band, LBT is
used by LTE systems along with request-to-send (RTS) and
clear-to-send (CTS) message exchange prior to starting the
original LTE transmission. LBT based approach proposed in
[7] considers handling of both inter-radio access technology
(RAT) interference and intra-RAT interference. To handle
inter-RAT interference, energy detection based LBT approach
is proposed, whereas to handle intra-RAT interference, LBT
based on cross correlation detection is proposed. Exchanging
spectrum allocation information between WiFi and LAA via
a common database is considered in [8] for enabling simulta-
neous access to unlicensed spectrum by LTE and WiFi.
In [9], blank sub frame allocation technique by LAA is
introduced to facilitate simultaneous WiFi and LTE operation
in the unlicensed spectrum. During silent subframes referred
to as blank subframes, LAA refrains from transmitting and as
a result WiFi gets more opportunities to access the channel.
Similar type of approach is considered in [10], in which LAA
allocates silent gaps with a predefined duty cycle to facilitate
better coexistence with WiFi. An uplink (UL) power control
based mechanism is evaluated for LAA systems in [11] to
allow simultaneous operation of WiFi and LTE in the unli-
censed spectrum. In that, LAA UL transmit power is reduced
in a controlled manner based on interference measurements,
generating more transmission opportunities to WiFi.
In this paper, we introduce a reinforcement learning based
dynamic duty cycle selection technique for LAA to facilitate
WiFi-LAA simultaneous operation in the unlicensed spectrum.
In particular, we use Q-Learning to dynamically configure
transmission gaps in LAA periodically, based on its learn-
ings from the environment. First, using a 3GPP-compliant
simulation setting, we evaluate the system performance under
different duty cycles of the transmission gaps. Then, the per-
formance of Q-Learning based dynamic duty cycle selection
technique is evaluated. The simulation results show that the
Q-Learning based approach improves overall system capacity
performance by 19% and WiFi capacity performance by 77%
compared to the scenario with fixed duty cycles that yields the
highest aggregate capacity.
The rest of the paper is organized as follows. In Section II,
we provide details of the considered system model. Section III
introduces the proposed Q-Learning based dynamic duty cycle
selection approach for LTE transmission gaps. Simulation
results with various parameter configurations are presented in
Section IV. Finally, Section V provides concluding remarks.
2015 IEEE Wireless Communications and Networking Conference (WCNC): - Track 3: Mobile and Wireless Networks
978-1-4799-8406-0/15/$31.00 ©2015 IEEE 1279
Fig. 1: WiFi APs and LAA BSs operating simultaneously in the
unlicensed spectrum.
II. SY ST EM M ODEL
A. Deployment Scenario
In order to evaluate the coexistence challenges and related
interference management methods for LAA operation of LTE,
we consider a scenario as shown in Fig. 1, where MLAA
base stations (BSs) and WiFi access points (APs) are oper-
ating simultaneously in the unlicensed band. Each WiFi AP
(LAA BS) consists of NWiFi stations (STAs) (LAA user
equipment (UEs)), which are uniformly randomly distributed
within the cell coverage area. Time division duplex (TDD)-
LTE is considered and it is assumed that LAA BSs and LAA
UEs are synchronized together all the time.
As shown in Fig. 1, due to simultaneous operation of WiFi
and LAA in the unlicensed spectrum, targeted WiFi STA
experiences interference from LAA DL/UL transmissions and
other WiFi DL/UL transmissions. This will result in degrading
the signal to interference plus noise ratio (SINR) at the targeted
WiFi STA. In the same way, for WiFi UL transmissions
and LAA DL/UL transmissions, WiFi and LAA simultaneous
operation will increase interference and hence reduce SINR
which will then degrade capacity performance. Due to carrier
sense multiple access with collision avoidance (CSMA/CA)
mechanism in WiFi [12], when coexisting with LTE, WiFi
transmissions get delayed, further degrading WiFi capacity
performance.
For both WiFi and LAA, we have considered a non full
buffer traffic model as given in 3GPP FTP traffic model-2 [13].
In order to evaluate the capacity of WiFi and LAA for different
simulation scenarios, a physical (PHY) layer abstraction is
used. In particular, Shannon capacity is calculated at the
granularity of each WiFi OFDM symbol duration (4µs) to
obtain the number of successfully received bits [14]. In all
the simulations, wireless channel is modeled according to
[13]. Both for WiFi and LAA, Indoor Hotspot (InH) scenario
is considered when determining path loss and shadowing
parameters used in the simulations.
B. Beacon Transmission Model
Beacon transmissions in WiFi networks are utilized by the
WiFi STAs to detect WiFi APs. Reception of beacon frame
Fig. 2: Beacon PPDU.
is important since it contains information such as beacon
interval, supported rates by the WiFi AP, and time stamp
to synchronize with WiFi AP for transmission/reception of
data to/from WiFi AP by a WiFi STA. Fig. 2 shows the
beacon physical protocol data unit (PPDU) considered in the
paper. Beacon is a management medium access control (MAC)
frame. In Fig. 2, beacon payload represents that MAC frame.
Beacon frame is always transmitted using BPSK modulation
with code rate of 1/2.
Beacon transmission based WiFi STA/AP association is
used in infrastructure basic service set (BSS)1with passive
scanning. In that, WiFi AP periodically broadcasts beacon
frames and WiFi STAs can associate with that WiFi AP if
they receive beacon frames properly [12]. Also, as shown in
Fig. 3, all the WiFi STAs that are associated with an WiFi AP
wait for beacon frame when target beacon transmission time
(TBTT), the time period for beacon transmission, is reached.
Before transmitting a beacon frame, WiFi AP waits for a
time duration specified by the point co-ordination function
inter-frame space (PIFS) to ensure medium is free. Successful
reception of beacon frame is important because, without that
it is not possible for an STA to transmit/receive data. In this
paper, we consider infrastructure BSS with passive scanning
for WiFi transmission as explained here.
Fig. 3: Beacon frame is expected to be transmitted at the end of
Beacon Interval. But this is not possible always as WiFi AP has to
wait for the completion of all ongoing WiFi transmissions.
As PHY layer abstraction is used in the paper to calculate
the capacity in WiFi and LTE transmissions, we implement
following method to identify successful reception of a beacon
PPDU (frame) at an STA. First, to determine whether an
orthogonal frequency division multiplexing (OFDM) symbol
carrying a portion of the beacon PPDU was received at an
STA, observed SINR of that OFDM symbol is compared with
a threshold; if it is larger than the threshold, it is assumed that
the information in that OFDM symbol was properly received
by the STA2. The same detection mechanism is used by the
STA for all the OFDM symbols belongs to a particular beacon
1A BSS is formed in IEEE 802.11 systems when an association is created
by STAs which are located within a certain coverage area.
2Each OFDM symbol is carrying a fixed number of symbols all the time
as modulation scheme and code rate are fixed for a beacon PPDU.
2015 IEEE Wireless Communications and Networking Conference (WCNC): - Track 3: Mobile and Wireless Networks
1280
PPDU. At the end of the beacon PPDU transmission, WiFi
STA calculates erroneously received beacon PPDU bits by
summing up bits in all the unsuccessfully received beacon
OFDM symbols. Then, the ratio (ρ) between erroneously
received bits to all the transmitted beacon bits (NB) of the
beacon PPDU is calculated as
ρ=Nerr ×NOFDM
B
NB
,(1)
where Nerr and NOFDM
Bare the number of erroneously
received beacon OFDM symbols and number of bits in a
beacon OFDM symbol, respectively. The ratio in (1) is then
compared with a predefined threshold for acceptable bit error
ratio of a beacon PPDU and determines whether the beacon
PPDU is successfully received at the WiFi STA.
C. Duty cycle implementation for LAA
To implement duty cycle based LAA transmission, we
consider a TDD configuration as shown in Fig. 4. The rationale
behind selecting this type of a TDD configuration is to keep
the UL to DL sub frame ratio constant irrespective of the
selected duty cycle. In particular, a sequence of four subframes
are assumed always to consist of two DL subframes, one UL
subframe and one guard subframe.
Fig. 4: 20 ms duty cycle period is considered with four duty cycles
(20%, 40%, 60%, 80%). Here x%represents the percentage of time
where the LTE network is transmitting.
Four different duty cycles are considered with a transmis-
sion gap duty cycle period of 20 ms. As shown in Fig. 4, LTE
transmits for xpercentage of time from the allocated duty
cycle period. For an example, if we consider 60% duty cycle,
LTE will transmit for 12 ms out of 20 ms duty cycle period.
When moving between adjacent duty cycles (i.e., from 20% to
40%), LTE transmission duration is increased/decreased with
a granularity of 4 ms. As the subframe pattern gets repeated
for every 4 ms, changing between duty cycles will add/remove
block(s) of considered subframe pattern while keeping DL to
UL subframe ratio constant.
III. Q-LEA RN IN G BA SE D DYNAM IC D UT Y CY CL E
SE LE CT IO N FOR LAA
In this section, we present Q-Learning based dynamic duty
cycle selection algorithm for LAA transmission. Dynamic
duty cycle selection is important since the network traffic is
bursty in realistic systems. Hence, the proposed approach can
help in enhancing LTE operation in the unlicensed spectrum
while providing more opportunities for WiFi transmission. As
proposed in [15], we consider a Q-Learning algorithm with
-greedy policy. In that, a pre-defined target capacity value
(Ctar) is set for LAA DL, and LAA BSs autonomously aim
to operate at a capacity close to Ctar by dynamically adjusting
their duty cycles.
When formulating the proposed Q-Learning algorithm, we
consider set of LAA BSs (B), as the players/agents of the
multi-agent system. Each player i∈ B has set of actions Ai=
{ai,1, ai,2, .., ai,|Ai|}and states Si={si,1, si,2, .., si,|Si|}
where ai,j and si,k represents a possible action and a state
of player i, respectively. In Q-Learning, each player i∈ B
keeps a Q-table with Q-values Qi(si,j, ai,k )for each state
si,j ∈ Si,1≤j≤ |Si|and action ai,k ∈ A,1≤k≤ |Ai|
pair. This Q-value provides an estimate for future costs, if the
player iselects the action ai,k when he is in the state si,j .
A player iin a particular state si,j , selects and deploys an
action ai,k. Then, based on the feedback from the environment,
the player learns about the outcome of the deployed action ai,k
in state si,j . This feedback is given as a cost value ci, i ∈ B,
which determines the absolute difference between the achieved
LAA DL capacity CLAA,i, i ∈ B, during the previous duty
cycle period and the target capacity Ctar. Using CLAA,i new
state of player i,si,l ∈ Si,1≤l≤ |Si|is also identified.
Then, using the identified next state si,l and calculated cost
value ci, Q-value of the current state (si,j) and action (ai,k )
pair is updated as follows:
Qi(si,j , ai,k)←(1 −α)Qi(si,j , ai,k )
+αci+γminai,m Qi(si,l, ai,m ),(2)
where, α,γare the learning rate and discount factor re-
spectively. As can be seen from (2), the new Q-value
of the current state/action pair depends on the current Q-
value of that state/action pair (Qi(si,j, ai,k )), calculated cost
(ci), and minimum Q-value of the identified next state,
minai,m Qi(si,l, ai,m ). In this way, learning is achieved in the
proposed algorithm.
The learning rate α(0 ≤α≤1) determines how quickly
the learning can occur. If αis too small, it will take long
time to complete the learning process, while if it is too high,
algorithm might not converge. The discount factor γ(0 ≤
γ≤1) controls the value placed on the future costs. If γ
is too small, learning will not depend on future costs much
and immediate costs are optimized. On the other hand, if it is
too high, learning will count on future costs heavily. Through
a careful selection of these two parameters, it is possible to
effectively control the learning process of the proposed Q-
Learning approach.
Once the Q-value of the current state (si,j) and action (ai,k )
pair is updated, an action ai,m ∈ Ai,1≤m≤ |Ai|is
selected for the next state si,l. A random number r∈ U (0,1) is
generated first and compared against the -greedy parameter
which is usually a very small value (0.01 ≤≤0.05). If
ris smaller than the -greedy parameter, an action will be
selected randomly. Otherwise, the action with the minimum
Q-value, (ai,m = argminai,mQi(si,l , ai,m )) in the identified
next state (si,l), is selected. The -greedy parameter allows
selecting an action in an exploratory way, and ensures that all
state/action pairs will be explored as the number of trials goes
2015 IEEE Wireless Communications and Networking Conference (WCNC): - Track 3: Mobile and Wireless Networks
1281
Algorithm 1 Q-Learning for duty cycle selection of LAA BS i∈ B
1: Initialize:
2: for each si,j ∈ Si,1≤j≤ |Si|,ai,k ∈ A,1≤k≤ |Ai|do
3: Initialize the Q-value representation mechanism Qi(si,j, ai,k )
4: end for
5: Evaluate the starting state s=si,j ∈ Si,1≤j≤ |Si|
6: Learning:
7: loop
8: Generate a random number r∈ U(0,1)
9: if (r < )then
10: Select action randomly
11: else
12: Select the action ai,m ∈ Aicharacterized by the min(Q-value)
13: end if
14: Execute ai,m
15: Receive an immediate capacity CLAA,i and cost ci
16: Observe the next state si,l ∈ Si,1≤l≤ |Si|
17: Update the Q-table entry as follows:
18: Qi(si,j , ai,k)←(1 −α)Qi(si,j , ai,k )
+α[ci+γminai,m Qi(si,l, ai,m )
19: s=si,l
20: end loop
to infinity. The proposed Q-Learning algorithm is summarized
in Algorithm 1.
Without any loss of generality, we consider that the action,
state and cost definitions in the proposed algorithm are defined
as follows.
•Action: Ai={20%,40%,60%,80%}.
•State:
si,j =
0, CLAA,i <1 Mbps
1,1 Mbps ≤CLAA,i <10 Mbps
2,10 Mbps ≤CLAA,i <20 Mbps
3,20 Mbps ≤CLAA,i <30 Mbps
4,30 Mbps ≤CLAA,i <40 Mbps
5, CLAA,i ≥40 Mbps
.(3)
•Cost:
ci=|Ctar −CLAA,i|,(4)
where CLAA,i is given by,
CLAA,i =NDC
Bits,i
TDC
Tx,i +TDC
Wait,i
.(5)
In (5), for LAA BS i∈ B,NDC
Bits,i represents number of
bits successfully transmitted during the previous duty cycle
period. TDC
Tx,i and TDC
Wait,i are the total transmitting time and the
waiting time due to silent subframe allocation3respectively,
during the previous duty cycle period.
IV. SIMULATION RESULTS
In simulations, we consider a two layer cell layout as shown
in Fig. 5. Each layer consists of M= 7 cells. There are
N= 10 WiFi STAs (LAA UEs) associated with each WiFi
AP (LAA BS). WiFi STAs (LAA UEs) move within the cell
with a speed of 3 km/h. WiFi and LAA traffic arrival rates,
λWiFi =λLAA = 2.5, are considered in all the simulations.
LTE and WiFi 802.11n MAC and PHY layers are implemented
as described in [14]. Round robin user scheduling is consid-
3LAA BS i∈ B has data to schedule in DL. However, due to silent
subframe allocation, it has to wait.
−50 0 50
−50
0
50
100
dist anc e (m)
dist anc e (m)
WiFi AP WiFi STA
LAA BS
LAA UE
Fig. 5: WiFi APs and LAA BSs in a two-layer cell layout.
TABLE I: LTE PHY/MAC parameters.
Parameter Value
Transmission Scheme OFDM
Bandwidth 20 MHz
DL Tx power 23 dBm
UL Tx power Path Loss based TPC
Frame duration 10 ms
Scheduling Round robin
P0-106 dBm
Path loss compensation factor (α) 1
Transmission time interval 1 ms
Traffic model FTP Traffic model-2 [13]
ered in LAA DL transmission and only one user is scheduled
during each transmission time interval (TTI). The LAA UEs
report the observed DL SINR value during a DL transmission
to the LAA BS, which is then used by the LAA BS to
determine the number of RBs to be allocated for the next DL
transmission. Based on the number of LAA UE requests for
UL transmission during one subframe, bandwidth is equally
divided between them. All the configuration parameters used
for LAA in simulations are given in Table I.
For WiFi, CSMA/CA is implemented with enhanced dis-
tributed channel access (EDCA) and clear channel assessment
(CCA) [14]. WiFi beacon transmission is implemented, as
discussed in Section II-B, for realistic performance evalua-
tions. All the STAs (including WiFi AP) having data in their
respective queues can compete for the channel access when
no transmission is going on in the cell. The WiFi STA (or the
WiFi AP) sensing the channel to be idle and having the shortest
back-off time will gain the access to the channel if it has
received the most recent beacon successfully. If the beacon has
not been received successfully, the WiFi STA can not initiate
any transmission or reception. All the configuration parameters
used for WiFi in simulations are summarized in Table II. In
all performance evaluations, we focus on the performance of
center cell in both WiFi and LAA cell layouts.
A. Performance analysis with WiFi beacon transmission
We evaluate WiFi and LAA performance with WiFi beacon
transmission considering TDD configuration 2. Fig. 6 shows
WiFi and LAA DL aggregate capacity, with/without beacon
transmission. There is an improvement in LAA DL capacity
and degradation in WiFi capacity when beacon transmission
2015 IEEE Wireless Communications and Networking Conference (WCNC): - Track 3: Mobile and Wireless Networks
1282
TABLE II: WiFi PHY/MAC parameters.
Parameter Value
Transmission scheme OFDM
Bandwidth 20 MHz
DL/UL Tx Power 23 dBm
Access Category Best Effort
MAC protocol EDCA
Slot time 9µs
CCA Carrier sensing threshold -82 dBm
CCA Energy detection threshold -62 dBm
No. of service bits in PPDU 16 bits
No. of tail bits in PPDU 12 bits
Contention window size U(0,31)
Noise figure 6 [12]
Beacon Interval 100 ms
Beacon OFDM symbol detection threshold 10 dB
Beacon error ratio threshold 15
Traffic model FTP Traffic model-2 [13]
0.5
1
1.5
2
2.5
3
3.5 x 107
Capaci ty (bit s/s)
0
0.5
1
1.5
2
2.5
3x 107
Capaci ty (bit s/s)
λLAA : 2.5
λWiFi : 2.5
λLAA : 2.5
λWiFi : 2.5
With Beacon With out Beac on
With out Beac on
Average Wi Fi c apac ity
With Beacon
Average L AA DL c apaci ty
Fig. 6: WiFi and LAA DL capacity variation with/without beacon
transmission.
exists. The reason for this is that, when a STA misses a beacon,
it can not transmit or receive until a beacon is received suc-
cessfully. Therefore, when WiFi beacon transmission exists,
number of simultaneous WiFi data transmissions reduces. As
a result, WiFi interference on LAA DL reduces and LAA DL
capacity improves. Moreover, missing a beacon at a WiFi STA
further delays WiFi transmission. This will result in increasing
WiFi waiting time, and hence reduces WiFi capacity.
Fig. 7 shows SINR distributions at WiFi and LAA DL
with/without WiFi beacon transmission. The LAA DL SINR
improves with WiFi beacon transmission, since the WiFi in-
terference on LAA reduces due to the reduction of the number
of simultaneous WiFi transmissions. WiFi SINR distribution
with/without WiFi beacon transmission is also shown in Fig. 7,
where an improvement in WiFi SINR can be seen with beacon
transmission. This is due to the lower WiFi interference with
the reduced number of simultaneous WiFi transmissions. Note
−20 −10 0 10 20 30 40 50 60 70
0
0.2
0.4
0.6
0.8
1
SINR (dB)
CDF
LAA SINR − wi th beaco n
LAA SINR − without beacon
WiFi SINR − wit h b eacon
WiFi SINR − wit ho ut b eaco n
WiFi DL SINR di st rib ut ions
with/w it hou t b eacon.
λWiFi : 2.5
λLAA : 2.5
LAA DL SINR d is tri bu tions
with/w it hou t b eacon.
Fig. 7: WiFi/LAA DL SINR distributions with/without WiFi beacon
transmission.
here that the SINR is captured during WiFi transmission and
this does not help much for improving WiFi capacity, as
waiting time for WiFi increases with missed beacons. That
is why we see a capacity reduction in Fig. 6 for WiFi, when
WiFi beacon transmission exists.
B. Performance analysis with different LAA duty cycles
In this section, we evaluate WiFi and LAA performance
under four different duty cycles considering TDD config-
uration presented in Section II-C. Fig. 8 shows WiFi and
LAA capacity variation under different LAA duty cycles.
While the WiFi capacity decreases with larger LAA duty
cycles, the LAA capacity increases. This is because, with
larger LAA duty cycles, LAA interference on WiFi increases
and as a result WiFi capacity decreases. On the other hand,
LAA capacity increases with higher duty cycles due to more
transmission opportunities. Note here that the rate of WiFi
capacity degradation reduces with LAA duty cycle. The reason
for this observation is, with higher duty cycles, number of
simultaneous WiFi transmissions reduces. Therefore, WiFi
interference is reduced, decreasing the WiFi capacity degra-
dation rate.
0.2 0.4 0.6 0.8 1
0.5
1
1.5
2
2.5
3
3.5
4
4.5 x 107
Duty cy cl e of L AA transm is sion
Capacity (bi ts/s )
Average Wi Fi c apaci ty
Average L AA DL cap aci ty
λWiFi : 2.5
λLAA : 2.5
Fig. 8: Average LAA DL and WiFi capacity variations with different
duty cycles for duty cycle period of 20 ms.
Fig. 9 captures WiFi DL SINR distributions with four dif-
ferent duty cycles. The results show that WiFi SINR degrades
with higher LAA duty cycles. This is because, interference
coming from LAA increases with higher LAA duty cycles.
A step like behavior can be observed in the WiFi DL SINR
distribution. This is due to the difference in LTE DL and UL
interference on WiFi [14].
−30 −20 −10 0 10 20 30 40 50 60 70
0
0.2
0.4
0.6
0.8
1
SINR (dB)
CDF
Duty Cyc le 0.2
Duty Cyc le 0.4
Duty Cyc le 0.6
Duty Cyc le 0.8
λWiFi : 2.5
λLAA : 2.5
When dut y c ycle is red uced , LA A i nter fer enc e
on WiFi r edu ces. Hence, Wi Fi DL
SINR impro ves .
Fig. 9: WiFi DL SINR distribution with different LAA duty cycles.
2015 IEEE Wireless Communications and Networking Conference (WCNC): - Track 3: Mobile and Wireless Networks
1283
−20 −15 −10 −5 0 5 10 15 20 25
0
0.2
0.4
0.6
0.8
1
SINR (dB)
CDF
Duty Cyc le 0.2
Duty Cyc le 0.4
Duty Cyc le 0.6
Duty Cyc le 0.8
λWiFi : 2.5
λLAA : 2.5
When dut y c ycle is lar ge, WiFi in ter feren ce
on LAA red uces . Henc e, LA A DL SINR
impro ves .
Fig. 10: LAA DL SINR distribution with different LAA duty cycles.
4.5
5
5.5
6
6.5
7x 107
Duty cycle o f LAA t ransmi ssion
Capaci ty (bit s/s)
λWiFi : 2.5
λLAA : 2.5
0.2 0.4 Dynami c
0.8 1
0.6
Fig. 11: Aggregate capacity (WiFi + LAA DL) variation with different
duty cycles and Q-Learning algorithm (Dynamic).
From Fig. 10, we can observe LAA DL SINR distribution
with four different duty cycles. In that, LAA SINR improves
with larger duty cycle. This is because of the lower WiFi
interference experienced due to the reduced number of simul-
taneous WiFi transmissions.
C. Performance analysis with Q-Learning based dynamic duty
cycle selection for LAA
In this section, we evaluate the performance of the proposed
Q-Learning based dynamic duty cycle selection technique. For
simulations, we consider α= 0.5,γ= 0.9,= 0.03, and
Ctar = 30 Mbps with the TDD configuration introduced in
Section II-C.
Fig. 11 shows the aggregate capacity variation (WiFi and
LAA DL) with different duty cycles and Q-Learning based
dynamic duty cycle selection technique. The Q-Learning based
dynamic duty cycle selection technique provides highest total
capacity when compared with fixed duty cycle and full LAA
transmission scenarios. The reason for this capacity gain is
that, as the LAA BSs dynamically adjust their operating
duty cycles based on the bursty traffic arrival given the
capacity constraint Ctar, WiFi gets fair amount of transmission
opportunities. As the medium sensing procedure in WiFi is
one of the main barriers which prevents WiFi from achieving
higher capacities, the proposed technique provides a solution
to overcome that problem. According to Fig. 11, the next
highest total capacity is achieved when operating without any
transmission gaps. However, as can be seen from Fig. 8,
achievable WiFi capacity is the lowest (21.45 Mbps) in this
case, whereas with Q-Learning based approach, WiFi capacity
of 39.7 Mbps could be achieved while keeping LAA capacity
around Ctar = 30 Mbps.
V. CONCLUDING REMARKS
In this paper, we have proposed a Q-Learning based dy-
namic duty cycle selection approach in which periodic trans-
mission gaps are configured by LAA, so as to effectively
coexist with WiFi systems in the unlicensed spectrum. First,
we evaluate WiFi and LAA performance with a fixed value of
the transmission gap. Then, the overall system performance
with the proposed Q-Learning based dynamic duty cycle
selection approach is evaluated. Simulation results show that
the proposed dynamic duty cycle selection approach for LAA
can effectively enhance the overall capacity performance.
ACK NOWLED GM EN T
The authors would like to thank Fujio Watanabe from
DOCOMO Innovations, Inc., for fruitful discussions and his
useful feedback on the final version of the manuscript.
REFERENCES
[1] “Study on Licensed-Assisted Access using LTE,” 3GPP Study Item -
RP-141397, Edinburgh, Scotland, Sep. 2014.
[2] A. Cavalcante, E. Almeida, R. Vieira, F. Chaves, R. Paiva, F. Abinader,
S. Choudhury, E. Tuomaala, and K. Doppler, “Performance Evaluation
of LTE and Wi-Fi Coexistence in Unlicensed Bands,” in Proc. IEEE
Vehi. Technol. Conf. (VTC), Jun. 2013, pp. 1–6.
[3] Qualcomm, “Extending LTE Advanced to unlicensed spectrum,” Dec.
2013, White Paper.
[4] T. Nihtila, V. Tykhomyrov, O. Alanen, M. Uusitalo, A. Sorri, M. Moisio,
S. Iraji, R. Ratasuk, and N. Mangalvedhe, “System performance of LTE
and IEEE 802.11 coexisting on a shared frequency band,” in Proc. IEEE
Wireless Commun. Networking Conf. (WCNC), Apr. 2013.
[5] M. Beluri, E. Bala, Y. Dai, R. Di Girolamo, M. Freda, J. Gauvreau,
S. Laughlin, D. Purkayastha, and A. Touag, “Mechanisms for LTE
Coexistence in TV White Space,” in Proc. IEEE Int. Symp. on Dynamic
Spectrum Access Networks (DYSPAN), Oct. 2012, pp. 317–326.
[6] R. Ratasuk, M. Uusitalo, N. Mangalvedhe, A. Sorri, S. Iraji, C. Wijting,
and A. Ghosh, “License-exempt LTE deployment in heterogeneous
network,” in Proc. Int. Symp. Wireless Commun. Sys. (ISWCS), Aug.
2012.
[7] NTT DOCOMO, “Views on LAA for Unlicensed Spectrum - Scenarios
and Initial Evaluation Results,” 3GPP RAN1 standard contribution -
RWS-140026, Sophia Antipolis, France, Jun. 2014.
[8] SONY, “Requirements and Coexistence Topics for LTE-U,” 3GPP
RAN1 standard contribution - RWS-140010, Sophia Antipolis, France,
Jun. 2014.
[9] E. Almeida, A. Cavalcante, R. Paiva, F. Chaves, F. Abinader, R. Vieira,
S. Choudhury, E. Tuomaala, and K. Doppler, “Enabling LTE/WiFi
Coexistence by LTE blank subframe allocation,” in Proc. IEEE Int. Conf.
on Commun. (ICC), Jun. 2013, pp. 5083–5088.
[10] CableLabs, “Cable Labs perspective on LTE-U Coexistence with Wi-Fi
and Operational Modes for LTE-U,” 3GPP RAN1 standard contribution
- RWS-140004, Sophia Antipolis, France, Jun. 2014.
[11] F. Chaves, E. Almeida, R. Vieira, A. Cavalcante, F. Abinader, S. Choud-
hury, and K. Doppler, “LTE UL Power Control for the Improvement of
LTE/Wi-Fi Coexistence,” in Proc. IEEE Vehic. Technol. Conf. (VTC),
Sep. 2013, pp. 1–6.
[12] E. Perahia and R. Stacey, Next Generation Wireless LANs: Throughput,
Robustness, and Reliability in 802.11n. Cambridge Univ. Press, 2008.
[13] “Evolved Universal Terrestrial Radio Access (E-UTRA); Further ad-
vancements for E-UTRA physical layer aspects (Release 9),” Tech. Rep.
3GPP TR36.814, V9.0.0, Mar. 2010.
[14] N. Rupasinghe and I. G¨
uvenc¸, “Licensed-Assisted Access for WiFi-
LTE Coexistence in the Unlicensed Spectrum,” in Proc. IEEE Global
Telecommun. Conf. (GLOBECOM) Workshops - Emerging Technologies
for 5G Wireless Cellular Networks, Dec. 2014.
[15] M. Simsek, A. Czylwik, A. Galindo Serrano, and L. Giupponi, “Im-
proved Decentralized Q-learning Algorithm for Interference Reduction
in LTE-femtocells,” in Proc. Wireless Adv., Jun. 2011, pp. 138–143.
2015 IEEE Wireless Communications and Networking Conference (WCNC): - Track 3: Mobile and Wireless Networks
1284
