Content uploaded by Taras Maksymyuk
Author content
All content in this area was uploaded by Taras Maksymyuk on Feb 22, 2019
Content may be subject to copyright.
The 4th IEEE International Symposium on Wireless Systems within the International Conferences on Intelligent Data Acquisit ion and Advanced Computing Systems
20-21 September, 2018, Lviv, Ukraine
Deep Learning Based Massive MIMO
Beamforming for 5G Mobile Network
Taras Maksymyuk1, Juraj Gazda2, Oleh Yaremko1, Denys Nevinskiy1
1Lviv Polytechnic National University, UKRAINE, Lviv
2Technical University of Košice, SLOVAKIA, Kosice
E-mail: taras.maksymyuk.ua@ieee.org, juraj.gazda@tuke.sk, oleg.yaremko.304@gmail.com, nevinskiy90@gmail.com
Abstract — The rapid increasing of the data volume in
mobile networks forces operators to look into different
options for capacity improvement. Thus, modern 5G
networks became more complex in terms of deployment and
management. Therefore, new approaches are needed to
simplify network design and management by enabling self-
organizing capabilities. In this paper, we propose a novel
intelligent algorithm for performance optimization of the
massive MIMO beamforming. The key novelty of the
proposed algorithm is in the combination of three neural
networks which cooperatively implement the deep
adversarial reinforcement learning workflow. In the
proposed system, one neural network is trained to generate
realistic user mobility patterns, which are then used by
second neural network to produce relevant antenna
diagram. Meanwhile, third neural network estimates the
efficiency of the generated antenna diagram returns
corresponding reward to both networks. The advantage of
the proposed approach is that it leans by itself and does not
require large training datasets.
Keywords — deep learning; massive MIMO; 5G;
beamforming; AI.
I. INTRODUCTION
Over the last few years, we experience a tremendous
growth of the data demand in the wireless networks driven
by the development of new services with high QoE
(Quality of Experience) requirements. According to the
Cisco Visual Index, global Internet traffic will reach 30
GB per capita by 2021, with the fraction of wireless and
mobile devices of more than 63 percent. In particular, this
growth is the result of the global trends of cloud
computing and Internet of Things, which tend to digitize
our world. It is expected that virtual transformation and
robotic will expand their presence in our life, which
requires intelligent and immersive QoE maintenance.
Numerous applications such as augmented reality, self-
driving cars, e-Health, e-Government, Industry 4.0 and
many others require high throughput, low latency as well
as good reliability. Moreover, in the era of Big Data,
Machine Learning and AI (Artificial Intelligence) we need
to offer scalable data transfer and management techniques
that can handle billion-object datasets within less than few
milliseconds [1].
Thus, development of the 5G mobile networks aims to
cope with the new challenging conditions. Despite
numerous research works on the 5G, there is no any single
view of the new standard, which makes 5G look like a
mix of solutions, which are partially compete and partially
supplement each other [2].
In this paper, we focus on the intelligent beamforming
based on the Massive MIMO (Multiple Input Multiple
Output) technology. The novelty of the proposed approach
is that deep learning is used to determine phase shift and
amplitude of each antenna element. Proposed solution
enables self-learning capabilities of the system that allows
to achieve higher capacity of the 5G mobile networks.
This paper is organized as follows. Section II covers the
brief overview of the existing achievements on 5G mobile
communications. Section III provides the description of
the system model and proposed beamforming algorithm.
Section IV concludes the paper.
II. OVERVIEW OF THE RECENT ACHIEVEMENTS IN
MOBILE COMMUNICATIOS
All of the solutions, proposed so far on the wireless
communications are designed for one of the three key
pillars of wireless communications:
• link spectral efficiency;
• available bandwidth;
• area spectral efficiency.
Previously, link spectral efficiency has been widely
considered as the most important factor of the wireless
communications. Spectral efficiency is the normalized
metric, what determines achievable throughput over
wireless channel per 1 Hz of occupied bandwidth for
specified transmission techniques such as modulation,
coding and multiplexing. Spectral efficiency of wireless
channel can be defined as following:
eff
bps C
S
Hz F
, (1)
where C – channel throughput, bps; ΔF – channel
bandwidth, Hz [3].
Numerous approaches have been proposed to improve
the link spectral efficiency on physical layer in order to
extend the capacity of wireless networks, without
purchase of additional piece of spectrum. All of them are
based on the advanced modulation and multiplexing
schemes. Nowadays, improvement of modulation schemes
has reached its threshold in terms of the tradeoff between
implementation cost and achieved gain, which makes it
less feasible option for overall network improvement.
Instead, modern solutions are focused mostly on the
aggregation of spectrum bands and improvement of the
bandwidth allocation per target coverage area [4].
One way to solve the problem is to deploy additional
small cells in the areas with high data demand, which
allows to increase the frequency reuse factor and area
spectral efficiency of mobile network. These additional
layers of small cells usually overlay the former coverage
of macro cell. Recent studies have shown that small cells
allow to increase area capacity by three orders of
magnitude comparing with conventional single tier
network deployment [4].
Multi-tier network coverage has enabled the feature of
multiple simultaneous connections to base stations of
different tiers. Thus, each user is able to aggregate
bandwidth from multiple connections into one logical
channel with much higher data rate. This approach
however has higher complexity, comparing to single tier
deployment. Another drawback of small cells is that their
performance is very sensitive to the instantaneous traffic
demand in the coverage area. Due to non-stationary
locations of mobile users, sometimes only few of them
appear in the small cell area, which result in low
bandwidth utilization and decreasing of the overall
network capacity.
Thus, small cells infrastructure should be redundant
with the possibility to turn off small cells with low
bandwidth utilization. However, redundant network
deployment requires additional capital expenditures. It
may not be feasible for operators to spend a lot of money
by deploying small cells with poor time utilization.
Therefore, Massive MIMO technology can be
considered as an alternative solution to increase the
capacity of mobile network, without redundant small cells
[5]. In particular, beamforming has been considered as a
promising approach to improve the energy allocation per
target coverage area. By using a large number of antennas
(up to few hundreds), base station can support multiple
spatially separated beams, which allows to reuse the same
spectrum band for each of them.
The main advantage of Massive MIMO, comparing to
conventional MIMO systems, is in the much higher
number of degrees of freedom for the base station, which
is similar to those in wireless sensor networks [6]. This,
in turn, allows to increase antenna resolution, i.e. capacity
gain from spatial multiplexing or beamforming precision.
In [5], authors proved that Massive MIMO
demonstrates better efficiency than small-cells for low
density of users, while for high users’ density small cells
shows significantly higher performance than Massive
MIMO. Thus, it is impossible to find the network
configuration with optimal trade-off between Massive
MIMO and small cells efficiency due to dynamic users’
density.
Hence, Massive MIMO systems along with small cells
should be considered as the key enabling combination for
5G design as shown in Fig. 1.
Massive
MIMO
Macro cell
Small cell
Small cell
Small cell
Small cell transceiver User equipment (UE)
D2D
channels
Figure 1. Heterogeneous architecture of 5G with combination of Massive MIMO and small cells.
III. DEEP LEARNING BASED MASSIVE MIMO
BEAMFORMING ALGORITHM
A. Beamforming in Massive MIMO Systems
Beamforming is the controlled interference of multiple
waves, which allows to increase the signal strength in the
target direction. Technically, this feature can be achieved
by using multiple transmitting antennas with different
phase shifts. Without beamforming all elements transmit
with the same phase, which result in the circular
irradiation pattern. However, circular (i.e. non-directed)
pattern can be effective only when traffic demand is
uniform, which is almost never the case. Therefore, it is
important to assess the instantaneous location of users to
determine the most suitable antenna irradiation pattern of
each base station.
In this paper, we consider the rectangular array of
antenna elements, which has the ability to tweak the
antenna pattern in the three dimensional space. The phase
shifts map for each antenna array is represented as
following [7]:
11 1
1
, ,
2 2
n
ij
n nn
Φ
(2)
Thus, the antenna diagram can be represented as:
11 11 1 1
'n1 1
cos cos
cos cos
n n
n nn nn
A t A t
A t A t
E
, (3)
where 2
f
, f is the carrier frequency, and Aij is the
amplitude of the irradiated wave, which is directly related
to the transmission power [8]. Thus, in order to change the
antenna diagram, we can adjust A and φ. Fig. 2 shows the
comparison of two different antenna diagrams, when
diagram in Fig. 2.a is more directed and the diagram in
Fig. 2.b is more wide in coverage.
(a) (b)
Figure 2. Comparison of two different antenna diagrams.
In addition, more precise beamforming can be
achieved by using sparse arrays where some of antenna
elements are inactive. Mathematically in can be
represented as a Hadamard product of matrix Φ and
identity matrix I:
ij ij
I
Φ Φ I
, (4)
It is obvious that higher number of antenna elements
provides better flexibility of beamforming due to more
degrees of freedom for adjusting parameters of antenna
array. However, when number of antenna elements is
getting higher the complexity of beamforming increases
exponentially, because more values need to be calculated
[9]. Therefore, it is impossible to scale the size of Massive
MIMO antenna array and provide real-time beamforming
simultaneously. Therefore, our aim in this paper is to
develop new approaches for Massive MIMO systems,
which will provide better tradeoff between complexity and
performance.
B. Deep Adversarial Reinforcement Learning Algorithm
for Massive MIMO Beamforming
Current achievements in the area of deep learning and
artificial intelligence enable the new level of the tasks
complexity for mobile network coverage optimization [10,
11]. However, such algorithms need to be trained by using
one of two possible options. First option is the supervised
learning, when system is trained according to the specific
training dataset. In this case, training process is done by
minimizing the root mean square error (RMSE) between
target data and obtained result [12]. Second option, called
reinforcement learning assumes that target dataset is not
known, but there is a reward function which provides
insights whether result is good or not. In this case, system
is trained by itself by trying to get as high reward as
possible [13].
In this paper, we propose new approach for
beamforming namely deep adversarial reinforcement
learning. The main idea is to use two competing neural
networks and one referee network, so that one network
will be trained by the other under supervision of third
network. First deep neural network is trained to generate
realistic user mobility patterns, second tries to response
with the most suitable antenna diagram by using all
available degrees of freedom and third evaluates the
efficiency of the result and returns reward to both
networks.
The inspiration of the proposed approach has been
proposed by Goodfellow et al in [14], where generative
adversarial networks (GAN) have been proposed first
time. In original GAN, generator produces random
samples of data, which try to mimic data from real world,
while discriminator tries to determine whether obtained
data sample is fake or real.
In our approach, we introduce second generator, which
produces the antenna diagram according to generated
location of users. In this case, discriminator implements
the workflow of deep reinforcement learning by returning
the reward to both generators. By reward we use the
aggregated throughput of all users, so that system will try
to improve it over the training time.
Below we describe the proposed training algorithm
step-by-step.
Step 1. First generator network produces a sample of
users’ location for the specific cell according to some
predefined probabilistic distribution.
Step 2. Second generator network reacts to the
obtained data sample and produces relevant antenna
diagram for the specific cell.
Step 3. Discriminator network evaluates the
instantaneous performance of the cell in terms of the total
aggregated throughput:
2
log 1
i i
i
C B SINR
, (6)
where Bi is the bandwidth of i-th user, and SINRi is the
SINR (signal-to-interference-plus-noise ratio) value
perceived by i-th user, expressed as:
2
1
,
i
j
i x i
iK
j x j
j
Ph PL
SINR i j
P h PL
, (7)
where Pi denotes the power of transmitted signal from
serving base station, Pj – denotes the power of transmitted
signal from interfering base station, hx – channel gain, σ2 is
additive white Gaussian noise, PL is the path loss of the
link between base station and user.
Step 4. Based on the throughput value, obtained from
previous step second generator network updates antenna
diagram (3) according to the following Q-function:
1
, 1 , max ,
t t t t t t t
a
Q s Q s C Q s
E E E , (8)
where st denotes the previous state, at denotes the previous
action of the second generator network, i.e. antenna
diagram before action, Ct is the current reward, expressed
by the throughput value, st+1 is the new state observed
after action, i.e. with updated antenna diagram, γ is the
discount factor, which determines how long algorithm can
expect the highest reward, e.g. γ=0 means that only
current reward is considered, while γ=1 means that
algorithm will be infinite.
Step 5. Check the obtained throughput value to assess
the convergence criteria:
max
C k C
, (9)
where Cmax is the total aggregated throughput for the most
ideal case when all users have the highest possible spectral
efficiency values, k is the factor from 0 to 1, which
reflects the accepted deviation from ideal case. If
condition (9) is satisfied, algorithm proceeds to step 1.
Otherwise, algorithm iterates steps 2-5 until condition (9)
will be satisfied.
Thus, by continuous iteration of the above mentioned
algorithm, network management system is able to acquire
knowledge about optimal network configurations for
different location of users. In addition, proposed algorithm
can be supplied with real-world data, so that obtained
statistical distributions will be used to improve the
efficiency of training [15].
IV. CONCLUSION
In this paper, we propose the intelligent beamforming
algorithm for massive MIMO based on the deep
adversarial reinforcement learning. Proposed algorithm
uses two competing neural networks and one referee
network to improve the training process. The advantage
of the proposed algorithm is that it provides nearly
optimal antenna diagrams for a large number of
scenarios, without solving mathematically complex
optimization problem.
ACKNOWLEDGEMENT
This research was supported by the project No.
0117U007177 “Designing the methods of adaptive radio
resource management in LTE-U mobile networks for
4G/5G development in Ukraine,” funded by Ukrainian
government and by the Slovak Research and
Development Agency project number APVV-15-0055.
REFERENCES
[1] M. Wollschlaeger et al. “The future of industrial communication:
Automation networks in the era of the internet of things and
industry 4.0,” IEEE Industrial Electronics Mag., vol. 11, no. 1, pp.
17-27, 2017.
[2] T. Maksymyuk et al., “Deployment strategies and standardization
perspectives for 5G mobile networks,” IEEE Int. Conf. on the
Modern Problems of Radio Engineering, Telecommunications and
Computer Science (TCSET’2016), Lviv-Slavske, Ukraine, pp.
953-956, Feb. 2016.
[3] I. Hwang, B. Song, S. Soliman, “A holistic view on hyper-dense
heterogeneous and small cell networks,” IEEE Communications
Magazine, vol. 51, no.6, pp. 20-27, 2013.
[4] N. Bhushan et al., “Network densification: the dominant theme for
wireless evolution into 5G,” IEEE Communications Magazine,
vol. 52, no. 2, pp. 82-89, 2014.
[5] W. Liu, S. Han, and C. Yang, “Massive MIMO or Small Cell
Network: Who is More Energy Efficient?” in Proc. WCNCN 13,
Shanghai, China, Apr. 2013, pp. 24–29
[6] V. Yatskiv et al., “The use of modified correction code based on
residue number system in WSN,” IEEE 7th International
Conference on Intelligent Data Acquisition and Advanced
Computing Systems, pp. 513-516, 2013.
[7] V. Inzillo et al, “A low energy consumption smart antenna
adaptive array system for mobile ad hoc networks,” International
Journal of Computing, vol. 16, no. 3, pp. 124-132, 2017.
[8] V. Kochan et al., “Energy-efficient method for controlling the
transmitters power of wireless sensor network,” IEEE Int. Conf.
on Electrical and Computer Engineering, pp. 1117-1120, 2017.
[9] S. Obadan, Z. Wang, “A hybrid optimization approach for
complex nonlinear objective functions,” International Journal of
Computing, vol. 17, no. 2, pp. 102-112, 2018.
[10] S. Bezobrazov et al., “The methods of artificial intelligence for
malicious applications detection in Android OS,” International
Journal of Computing, vol. 15, no. 3, pp. 184-190, 2016.
[11] J. Gazda, et al., "Unsupervised Learning Algorithm for Intelligent
Coverage Planning and Performance Optimization of Multitier
Heterogeneous Network," IEEE Access, vol. 6, pp. 39807-39819
2018
[12] S. Pan et al., “A survey on transfer learning,” IEEE Transactions
on Knowledge and Data Engineering, vol. 22, no. 10, 2010.
[13] R. Razavi et al., “A fuzzy reinforcement learning approach for
self-optimization of coverage in LTE networks,” Bell Labs
Tachnical Journal, vol. 15, pp. 153–175, 2010.
[14] I. Goodfellow et al., “Generative adversarial nets,” In Advances in
neural information processing systems, pp. 2672-2680, 2014.
[15] T. Maksymyuk et al., “An IoT based Monitoring Framework for
Software Defined 5G Mobile Networks”, Proceedings of the 11th
ACM Int. Conf. on Ubiquitous Information Management and
Communication (IMCOM’2017), article #5-4, Jan. 5-7, 2017.
