Risk-Aware Resource Allocation for URLLC: Challenges and Strategies with Machine Learning

Abstract

Supporting ultra-reliable low-latency communications (URLLC) is a major challenge of 5G wireless networks. Stringent delay and reliability requirements need to be satisfied for both scheduled and non-scheduled URLLC traffic to enable a diverse set of 5G applications. Although physical and media access control layer solutions have been investigated to satisfy only scheduled URLLC traffic, there is a lack of study on enabling transmission of non-scheduled URLLC traffic, especially in coexistence with the scheduled URLLC traffic. Machine learning (ML) is an important enabler for such a co-existence scenario due to its ability to exploit spatial/temporal correlation in user behaviors and use of radio resources. Hence, in this paper, we first study the coexistence design challenges, especially the radio resource management (RRM) problem and propose a distributed risk-aware ML solution for RRM. The proposed solution benefits from hybrid orthogonal/non-orthogonal radio resource slicing, and proactively regulates the spectrum needed for satisfying delay/reliability requirement of each URLLC traffic type. A case study is introduced to investigate the potential of the proposed RRM in serving coexisting URLLC traffic types. The results further provide insights on the benefits of leveraging intelligent RRM, e.g. a 75% increase in data rate with respect to the conservative design approach for the scheduled traffic is achieved, while the 99.99% reliability of both scheduled and nonscheduled traffic types is satisfied.

Full text

1
Risk-Aware Resource Allocation for URLLC:
Challenges and Strategies with Machine Learning
Amin Azari, Mustafa Ozger, and Cicek Cavdar
Abstract—Supporting ultra-reliable low-latency communica-
tions (URLLC) is a major challenge of 5G wireless networks.
Stringent delay and reliability requirements need to be satisfied
for both scheduled and non-scheduled URLLC traffic to enable
a diverse set of 5G applications. Although physical and media
access control layer solutions have been investigated to satisfy
only scheduled URLLC traffic, there is a lack of study on
enabling transmission of non-scheduled URLLC traffic, especially
in coexistence with the scheduled URLLC traffic. Machine
learning (ML) is an important enabler for such a co-existence
scenario due to its ability to exploit spatial/temporal correlation
in user behaviors and use of radio resources. Hence, in this
paper, we first study the coexistence design challenges, especially
the radio resource management (RRM) problem and propose
a distributed risk- aware ML solution for RRM. The proposed
solution benefits from hybrid orthogonal/non-orthogonal radio
resource slicing, and proactively regulates the spectrum needed
for satisfying delay/reliability requirement of each URLLC traffic
type. A case study is introduced to investigate the potential of
the proposed RRM in serving coexisting URLLC traffic types.
The results further provide insights on the benefits of leveraging
intelligent RRM, e.g. a 75% increase in data rate with respect
to the conservative design approach for the scheduled traffic is
achieved, while the 99.99% reliability of both scheduled and non-
scheduled traffic types is satisfied.
Index Terms—5G, non-scheduled traffic, IoT, machine learn-
ing, proactive resource provisioning, URLLC.
I. INTRODUCTION
THE fifth generation of wireless networks (5G) has tar-
geted a diverse set of wireless services, from enhanced
mobile broadband (eMBB) to the Internet of Things (IoT) [1].
The latter itself could be categorized into two distinct service
types, i.e. massive machine type communications (mMTC),
which aims at connecting everything that benefits from be-
ing connected, and ultra-reliable low-latency communications
(URLLC), which requires successful data transmission within
a strictly bounded short time interval [2]. URRLC is consid-
ered as an essential prerequisite of a new wave of services
including drone-based delivery, smart factory, remote control,
and intelligent transportation systems (ITS) [3].
Different URLLC applications may have different setup
delay tolerances, where the setup delay is defined as the time
from generation of the first URLLC packet at the devices
until when the packet is transmitted. Serving URLLC traffic
with very-low setup delay tolerance is extremely challenging
because this type of traffic cannot wait to be scheduled, e.g.
an accident report by a vehicle. Let us denote this category
The authors are with the School of Electrical Engineering and Computer
Science, KTH - The Royal Institute of Technology, Stockholm, Sweden (e-
mail: {aazari, ozger, cavdar}@kth.se).
of traffic as non-scheduled URLLC traffic. This could be (i)
from connected/disconnected devices which have critical data,
usually of short payload size; (ii) from devices which ask
access reservation for subsequent scheduled transmissions of
critical data; or (iii) critical device-to-device communications.
In the uplink direction, enabling URLLC mandates guaran-
teeing delay and reliability requirements for both scheduled
URLLC transmissions, e.g. continuous control of a drone,
as well as the non-scheduled URLLC transmissions, e.g.
immediate accident reports. The coexistence of scheduled and
non-scheduled traffic could happen in many scenarios like
vehicular networks, and monitoring and alarm systems [3, 4].
Then, the set of resources allocated to URLLC should be
managed to be used for both scheduled and non-scheduled
URLLC traffic. Isolation among services via allocating orthog-
onal slices of resources is a common practice when dealing
with ergodic objectives like throughput. However, in URLLC
applications, due to the limitations of time-diversity and cru-
cial need to large spectrum bands for providing frequency
diversity, isolation through orthogonal slicing is challenging
[3, 5]. Recently, serving coexisting eMBB/URLLC services
over limited radio resources has attracted attentions, and non-
orthogonal RRM for serving URLLC has been proposed
for spectrum efficiency [1, 6]. While the orthogonal RRM is
itself a complex problem, the non-orthogonal RRM, which
requires specifying shared/dedicated resources and scheduling
rules over shared ones, is a much more complex problem.
Furthermore, to comply with URLLC delay requirements, this
problem is needed to be solved in very short time scales.
Among candidate enablers for solving such complex, dy-
namic and time-limited RRM problem, artificial intelligence
(AI) is promising [7]. In the light of recent advances in
computing and storage technologies, AI is making the leap
from traditional pattern recognition use cases to governing
complex systems through advanced machine learning (ML)
approaches. While in previous decades, ML, as a major branch
of AI, has experienced several up and down periods, this time
its technology readiness level is so high that it has already
penetrated to the design of many complex systems [7]. This
motivates us to investigate usefulness of leveraging ML in
RRM for serving the URLLC traffic. While the traditional
ML consists in a single node which makes decisions in a cen-
tralized manner, the delay/reliability requirements of URLLC
call for novel, scalable and distributed learning approaches.
In this paper, our main focus is on serving urgent
short packet URLLC transmissions along with the sched-
uled URLLC transmissions. The results derived within this
work could be easily extended to the coexistence of URLLC
arXiv:1901.04292v1 [cs.NI] 22 Dec 2018

2
Power
Frequency
20$120$·5DGLRXVHUPRELOLW\PDSV·,QWHUIHUHQFHDQGFKDQQHOTXDOLW\PHDVXUHPHQWVFrequency
120$20$hƌŐĞŶƚƚƌĂĨĨŝĐ
6WDWH6WDWHRIWKHQHWZRUN^
$FWLRQVFKHGXOLQJRYHUVKDUHGUHVRXUFHVFrequency
0/SRZHUHG+0$U: Urgent WUDIILFS : Scheduled WUDIILFUU
S
U
S
Power
5DGLR(QYLURQPHQW,QWHOOLJHQW553556DQG558([DPSOHWRRO4OHDUQLQJIRU556Traditional
550120$20$3URSRVHG5500/SRZHUHG+0$
^ĐŚĞĚƵůĞĚƚƌĂĨĨŝĐ
Power
Fig. 1: A graphical illustration of the proposed intelligent
RRM solution. The network status, including measurements
in the radio environment, is fed to the intelligent RRM
modules, which enable more efficient use of radio resources
for scheduled and non-scheduled URLLC traffic types.
and non-URLLC traffic by modifying the delay/reliability
constraints of the scheduled traffic. The main contributions
of this work include: (i) introduce coexistence of sched-
uled and non-scheduled URLLC traffic as a challenge to be
tackled for realization of URLLC; (ii) introduce a hybrid
orthogonal/non-orthogonal multiple access (OMA/NOMA),
called HMA scheme, to serve coexisting URLLC traffic;
(iii) investigate a distributed learning-powered RRM solution,
including intelligent resource provisioning (RRP), scheduling
(RRS), and utilizing (RRU) modules, for spectrum efficient yet
reliable coexistence management; and (iv) investigate different
sources of uncertainty in proactive RRM, design of a risk-
aware ML-powered RRM for compensating such uncertainties,
and figuring out the amount of radio resources which are
sacrificed because of such uncertainties. Fig. 1 presents a
graphical illustration of the proposed RRM solution and how
ML can be mapped to RRM problem and HMA to satisfy
URLLC requirements for both scheduled and non-scheduled
traffic. We have also present open challenges and potential
solutions in RRM for URLLC coexistence.
II. ENABLING URLLC: THE RO LE O F EFFIC IE NT RA DI O
RESOURCE MANAGEMENT
In URLLC applications, we usually deal with infrequent
bursty traffic, where the arrival time of the burst could
not be predicted [1, 6]. Due to stringent delay constraints,
once a URLLC packet is generated, the packet must be
transmitted immediately without any delay [3, 6]. This abrupt
transmission usually having a short payload size, could be an
alarm message, or the first packet transmission (FPT) which
will be followed by upcoming scheduled transmissions. Apart
from the non-scheduled URLLC traffic, we also need serving
scheduled URLLC traffic which is not necessary of short
payload size [3, 10]. The resource allocation to this traffic
type should guarantee no packet drop at the device due to
the expiration of data [10].
Fig. 2(a) represents a graphical illustration of the scheduled
and non-scheduled URLLC traffic coexistence in the ITS use-
case. In this use case, intelligent vehicles communicate with
each other (V2V) as well as with the communication infras-
tructure (V2I) for traffic efficiency and safety. One observes
that non-scheduled traffic transmissions include (i) FPT of
devices which transmit critical data and reserve access for
further messages [3] (refer to FPT-NS in Fig. 2(b)); (ii) critical
message from a device to the access network [6] (refer to
V2I-NS in Fig. 2(b)); and (iii) critical V2V communications
[4] (refer to V2V-NS in Fig. 2(b)). One further observes that
non-scheduled traffic transmissions are infrequent and could
have temporal correlation with the scheduled transmissions
(the FPT-NS in Fig. 2(b)).
Based on the above discussions, enabling URLLC requires
tackling three major issues: (i) enabling ultra-reliability in
communications, which on the other hand requires diversity;
(ii) enabling low-latency communications, which limits time
diversity and enforces frequency diversity; and (iii) coexis-
tence management of scheduled and non-scheduled traffic,
which is due to the fact that frequency resources could not
be scaled with the amount of URLLC traffic. The coexistence
management calls for an efficient resource management strat-
egy, without which, scalability of communications systems in
serving URLLC service is challenging. While in prior studies,
the first two issues have been investigated, there is lack of
research on the coexistence management within the URLLC
services [6, 10]. Towards serving a coexistence of scheduled
and non- scheduled URLLC traffic, not only the queuing delay
violation for scheduled traffic should be minimized, but also
the delay and packet error probability for the first transmission
(non-scheduled traffic) should be reduced dramatically. Es-
pecially, serving non-scheduled traffic requires revolutionary
multiplexing schemes breaking the barrier of interference-free
orthogonal transmissions [1, 6]. The need for prompt access
may occur in any time instant, e.g. while resources have been
reserved for the scheduled-users, hence, serving these two
traffic types together poses coexistence challenges.
A. The Hybrid Multiple Access Solution
The straightforward way of co-serving scheduled and non-
scheduled traffic is to allocate them orthogonal sets of radio
resources, i.e. OMA. The OMA can be very suboptimal in
spectral efficiency because the set of allocated resources to
non-scheduled traffic could be unused most of the time due
to the bursty nature of such safety-critical messages [1]. On
the other hand, insufficient resource allocation to this traffic
type results in reliability degradation. In order to compensate
spectral inefficiency of OMA, NOMA multiplexes different
traffic types over a set of shared resources to overcome
spectral inefficiency of OMA. However, keeping isolation
among URLLC services with NOMA is challenging due to
potential temporal overloading in one service.

3
!
"
#
$
BS!
(a) V2X communications within the ITS use case
BS
!
!
!
!
signaling
scheduled
transmission
V2V-NS
V2I-NS
FPT-NS
signaling
scheduled
transmission
V2V-NS
downlink data /control/ACK
(b) Communications exchanges
Fig. 2: An illustrative example of system model in the ITS use case, including scheduled (green dashed arrows) and
non-scheduled (red dotted arrows) URLLC communications.
Here, we consider a hybrid of OMA/NOMA, namely
HMA, in which, frequency resources, are divided into three
groups: (i) resources allocated to serving the non-scheduled
URLLC traffic; (ii) resources allocated to serving the sched-
uled URLLC traffic; and (iii) resources shared among them.
The HMA could be reduced to OMA and NOMA by tuning
the amount of shared resources. Over the shared resources,
we need to limit the aggregated received interference from the
scheduled users. Furthermore, in order to prevent performance
degradation for the scheduled traffic over the shared resources,
we need to assure reliability of scheduled data transmission
even in presence of non-scheduled URLLC traffic. Towards
this end, proactive or reactive strategies could be used [15].
The former aims at making the transmission robust, e.g. by
planning for occurring erasure in the channel (erasure coding),
while the latter consists in cancellation of received interference
from the non-scheduled traffic. Regarding the stringent delay
requirement of URLLC, the reactive strategy increases the
delay violation probability for the scheduled traffic. Thus, risk-
aware proactive RRM for minimizing reliability degradation is
required.
B. RRM for the Hybrid Multiple Access
To make the HMA solution working, the radio access
network (RAN) control center needs to control the traffic load
dynamically, and adjust proactively the size of resource pools
associated to each traffic stream. Furthermore, coordination
among the BSs is needed to make sure the aggregated intra-
and inter-cell interference will not cause an outage. Then,
the RRM must be done in a multi-cell level to compensate
the inter-cell interference impact on the outage probability.
While a centralized solution seems promising for satisfying the
reliability constraint even in cell-edges; the required signaling,
including data gathering, forwarding to the center, and decision
feedback, can potentially violate the URLLC delay constraints
[7]. Even in a single cell scenario, due to the potential
inter-dependence of the scheduled and non-scheduled traffic,
keeping the outage risk for both traffic types lower than the
URLLC requirement is not an easy task. All in all, regard-
ing the high-mobility of users, their time/location-dependent
distributions within the service area, and the bursty nature of
the service requests, the RRM for assuring a minimum service
requirement over a wide service area is highly challenging [5,
7, 10].
Action:
RRP: (i) inter-service/cell
resource allocation, (ii) BSs’
activity management.
Learning:
(i) risk-aware online learning for RRP update,
(ii) inter-BS transfer learning for RRS,
(iii) federated learning of mobility pattern, etc.
Action:
RRU: (i) intelligent resource
utilization for data transmission
and reception.
Action:
RRS: (i) inter and intra service
resource scheduling.
Learning:
(i) risk-aware online learning for RRS update,
(ii) regression-based user/radio map update.
Learning:
(i) risk-aware online learning for RRU update.
RAN
control
center
Intelligent RRP at the RAN control center
Intelligent RRS at the BSs
Intelligent RRU at the end-devices
Fig. 3: Distributed ML for radio resource management. The
proposed RRM includes (i) RRP at the RAN control center,
(ii) RRS at the BSs, and (iii) RRU at the end-devices.
III. ML IN SU PP ORT O F EFFIC IE NT RRM
By broadening the set of services and resources in 5G,
control and troubleshooting of radio access network (RAN) by
human resources become too complicated and costly [7]. Fur-
thermore, the stringent delay/reliability requirements of newly
established URLLC services make them more challenging.
Moreover, many decisions taken in serving users’ requests in
legacy networks have temporal/spatial correlations, i.e. they
are repeated in different locations and time instances. Then,
it is wise to leverage the huge volumes of available data
in cellular networks, and extract patterns of request arrivals,
channel variations of users along mobility patterns, and outage
statistics [7–9]. The derived patterns could be beneficial in
reliable resource provisioning for future service requests. This
brings the idea of leveraging ML tools in design and operation
control of future wireless networks. ML aims at estimating a
function, e.g. a rule or pattern, from a set of noisy data which
has been generated by the true function. The recent advances
in centralized and distributed computing, storage, and learning
algorithms have improved the position of ML as a design and
optimization tool significantly [7].

4
The Proposed ML-Powered RRM
To manage network resources for serving scheduled and
non-scheduled URLLC traffic, one needs to identify the set of
dedicated resources to non-scheduled traffic, the set of shared
resources, and the scheduling policy over the shared and
dedicated resources. Once the set of resources and scheduling
rules are determined, the scheduler allocates resources to users
based on their rankings. The rank of each user depends on its
delay budget and the expected level of received interference
from the user over the shared resources. We propose to
break the intelligent RRM problem into 3 sub-problems: radio
resource provisioning (RRP), radio resource scheduling (RRS),
and radio resource utilizing (RRU).
Fig. 3 represents a graphical representation of the proposed
hierarchical learning solution. In the proposed solution, a
proactive resource provisioning based on training and pre-
diction is done at the RAN control center. RRP at the RAN
control center benefits from (i) transfer learning, i.e. making
decision for a service area based on decisions taken in neigh-
boring areas or previous time instances [11]; (ii) federated
learning, i.e. leveraging the huge volumes of available records
of requests/responses at the edges for extracting common rules
and policy sets [7]; and (iii) risk-aware learning from past
RRP decisions for outage minimization of coexisting services
and traffic types. Distributing the learning task between BSs
and the RAN control center aims at compensating partial
observability of BSs in cellular networks. As BSs have access
to local information, i.e. they observe only a limited part of
the service area, and URLLC users could be highly mobile,
resource provisioning at the BS-level could be less effective.
For example, a BS cannot predict the arriving of a vehicle pla-
toon which is currently in the neighboring service area. Then,
distributed learning can further enable transfer knowledge of
arrival of such demanding users, and sharing of respective
effective resource scheduling policies. As an example, such
learning may result in the knowledge for the RAN control
center to activate some dormant BSs to balance the scheduled
and non-scheduled traffic serving responsibilities among BSs,
and hence, to minimize the probability of outage.
Furthermore, in the proposed intelligent RRM solution, BSs
decide how to control access of the scheduled users over
the shared and dedicated resources to minimize the queuing
delay violation for them, while complying with the maximum
allowed interference constraint over the shared resources. This
distributed control allows BSs to react to instant changes in the
channel quality of connected users, and schedule them based
on their remaining delay budgets. Finally, the intelligent usage
of radio resources for reliable data transmission/reception is
done at the device side. Devices can decide how to allocate
their power resources to the set of shared and dedicated
frequency resources to ensure the highest possible reliability.
This is especially the case in non-scheduled URLLC trans-
missions, either to the BS or to another device/vehicle, where
devices can benefit from reinforcement learning for improving
reliability of their transmissions [12].
The proposed RRM requires a set of ML solutions at the
RAN control center, BS, and device-side for addressing the
challenges posed in serving the URLLC traffic, as described
in Section I. In the following section, we describe some of
these solutions, as well as the open problems. These solutions
have been also depicted in the learning boxes of Fig. 3.
IV. ML S OLUTIONS EMBEDDED IN THE PROPOSED RRM
A. Federated Learning of User and Radio Maps for RRP
The RRP for URLLC traffic at the RAN control center
requires knowledge about the outage risk of potential users.
When we consider no a priori information about the users,
the design is based on the worst case scenario, and hence,
huge volumes of frequency resources are required to satisfy a
level of QoS. Here, we investigate the combined use of radio
and user maps in achieving an estimate of the outage risk of
URLLC users.
1) Radio Map: They are expected to be an essential 5G
component for enabling agile yet efficient resource allocation
for mobile use cases like autonomous driving [13]. For a given
service area, radio map includes information like radio signal
strength, delay spread, and coexisting interference level. Re-
garding the inevitable changes in the environment, radio maps
need to be updated once a time. In the proposed hierarchical
RRM solution, this is the role of the RAN control center to
coordinate continuous update of the radio map, and to manage
location-based inter-update time intervals. Furthermore, there
might be time periods during which, no update of signal
strength from a specific region is sent to the BSs. This could
be due to temporal absence of users in a location for an
extended period of time. In such cases, this is the role of
the involving BSs to predict and update the received signal
strength in that location, e.g. by leveraging temporal/spatial
regression as described in [13].
2) User Map: With the ever increasing number of con-
nected devices and the volume of online data, the locations
and mobility patterns of many vehicles, specially public trans-
portation systems, are known or could be learnt [14]. Further
than offline learning of mobility patterns, the RAN control
center can coordinate information exchange among BSs in a
service area to let them collaboratively construct and update
a common user map. Having a map of users and mobility
patterns, enables prediction of presence of users in different
regions of the service area for an extended period of time [14].
The map of users requires continuous update to be effective
in URLLC applications. In our proposed structure, this is the
role of the RAN control center to keep the consistency of the
global user map, and coordinate the inter-update time intervals
among the involving BSs.
Once the position/speed/direction information of a user is
known, and the radio map of the respective environment
is given, we are able to estimate path-loss and shadowing
components of its wireless communication channel. Then, the
uncertainty of the channel scales down to the small-scale
fading. Thus, we will be able to predict the received power
at the BS as a function of transmit power of users. Within
the context of URLLC, this type of information is useful
for early identification of a user which is expected to enter
a crowded/low-coverage area. Thus, the RAN would be able

5
to configure the network -manage the radio/energy resources-
to lower the risk of communication outage for the respective
users. The radio maps will further enable user management
based on regional characteristics. This means that once devices
enter a specific region, they could be granted higher-level
support, e.g. dual connectivity, or extra resources for their
transmissions could be reserved, or their urgent transmissions
could be handed over to multiple neighboring BSs.
B. Risk-Aware Learning for RRS
Introduction of URLLC into cellular networks mandates a
transition from average-utility network design into risk-aware
network design, where a rare event may incur a huge loss [2].
Instead of legacy ML algorithms, which aim at maximizing
the average return of agents, or equivalently minimizing the
average regret of agents, we aim at minimizing the risk of huge
loss. The loss could be defined in different ways as a function
of the return. When serving solely one traffic type, the risk-
aware learning-powered scheduling enforces maximizing the
first moment of the return, i.e. reliability, while minimizing
its higher moments, e.g. the variance. Minimizing higher
moments of the return, i.e. optimizing the worst case return,
could be achieved by defining the utility function as a non-
linear function of the return, R. A well-known example is
exponential utility function in which, instead of maximizing
the expected value of R, the objective is to maximize the
expected value of exp(-R)[2]. In our proposed RRM solution,
scheduling of shared radio resources is done by leveraging
risk-aware learning at the BSs. In other words, the scheduling
task is performed such that the risk of outage for scheduled
and non-scheduled traffic remains meets the URLLC’s require-
ments.
C. Open Challenges and Potential Solutions
The main challenge in utilizing intelligent RRM consists
in large dimensionality and complexity. The heterogeneity
of services, QoS requirements, and diverse set of available
resources make the RRM a very complex problem. This com-
plexity has a significant impact on the convergence time of the
learning algorithms. Furthermore, the high mobility of users
demanding URLLC service, especially in the ITS use case,
and the partial observability of the network by each BS, put
further constraints to the RRM problem. In order to overcome
the shortcomings of ML in timely finding optimal decisions,
it is important to compare it against the way human beings
learn. Human beings benefit from planning and reasoning for
reducing the search space and explore the solution in most
probable areas. In contrast, ML approaches treat the search
space unbiased, and in the exploration phase, select all actions
randomly to get an estimate of their respective payoffs. One
way to reduce the search space, and speed up the learning
process, is to add smart planning to the learning algorithms.
Towards this end, one may benefit from the human-resources,
and integrate them in the learning loop, to reduce the search
space, and prevent BSs in taking risky/time-consuming actions
during the exploration process. Another major challenge in
utilizing intelligent RRM consists in exchanging information
among different network entities, e.g. from a BS to the RAN
control center, which is energy and radio resource consuming.
Towards addressing this problem, designing communication-
efficient learning algorithms, e.g. federated learning in which
only the learnt models are communicated [7], are of crucial
importance.
V. ML-POWE RE D RR M: A C AS E STU DY
In this section, we demonstrate how to take advantage of
intelligent RRM for spectrum-efficient yet reliable serving
of scheduled and non-scheduled URLLC traffic. First, we
investigate performance of intelligent RRP and RRS (Fig.
4(a)). Consider a single-cell scenario in which URLLC users
generate and transmit non-scheduled URLLC traffic (infre-
quent) as well as scheduled URLLC traffic. The minimum
and maximum experienced path-loss in the service area are
denoted by -70 and -120 dB, respectively. Other simulation
parameters could be found in Table I. Based on the outage
risk of the most critical device with potential non-scheduled
URLLC traffic, the required outage probability for the non-
scheduled traffic, and the learnt risk of outage in the service
area, the ML-powered RRP and RRS modules are imple-
mented and used for resource management, as described in
Section IV and Table I. Fig. 4(a) represents the level of
reliable data-rate achieved for the scheduled traffic, versus
the required outage probability for the non-scheduled traffic.
The dashed-curve represents a conservative RRM solution in
which, resource allocation is done based on the risk to the
cell-edge user. For easier interpretation of the results, the
decoupled gains of RRM with OMA, NOMA, and HMA
have been presented. OMA represents orthogonal allocation of
RRBs to scheduled/non-scheduled traffic based on the online
outage risk-level, in contrast with the conservative design
(the dashed curve). NOMA represents the case in which, all
RRBs could be reused by the scheduled traffic, and hence,
there is no dedicated RRBs for the non-scheduled traffic.
HMA represents the advanced form of NOMA in which, once
reliability requirement of the non-scheduled traffic increases,
it dedicates some resources to the non-scheduled traffic. To
quantify the achieved gain by intelligent RRM, let us focus
on the 99.99% reliability requirement for both scheduled and
non-scheduled traffic types. One observes that 75% increase
in reliable data rate for the scheduled traffic is achieved by
TABLE I: Parameters of the case study.
Parameters Values
Service area Circular area, radius 3 Km, BS at the center
Traffic Scheduled: saturated; Non-scheduled: exp(0.01)
Radio resources 5 radio resource blocks (RBBs), each 180 KHz
Transmit power Scheduled: 21 dBm, Non-scheduled: 23 dBm
Intelligent RRM Q-learning: learning rate=0.1, exploration
rate=0.85, Q(S,A): value of state-action
combinations
Q-learning for
RRP and RRS
S: state of the network including reliability require-
ments and position of users, A: scheduling of a user
over shared resources
Q-learning for
RRU
S: state of the network including interference level
on RRBs, A: power allocation over RRBs

6
1e-1 1e-2 1e-3 1e-4 1e-5 1e-6 1e-7
Target outage probaility for urgent traffic
0
0.5
1
1.5
2
2.5
3
0.9999-reliable datarate (b/s), sched. traffic
106
No-learning (conservative design)
Intelligent RRP and RRS (OMA)
Intelligent RRP and RRS (NOMA)
Intelligent RRP and RRS (HMA)
(a) Intelligent RRP and RRS: reliable data rate for the scheduled
traffic versus the outage probability for the non-scheduled traffic
(single-shot transmissions). Increase in the required reliability
highlights the merits of intelligent RRM and HMA.
0 0.2 0.4 0.6 0.8 1
α: fraction of allocated power to RRB1
10-7
10-6
10-5
10-4
10-3
10-2
Outage probability for non-sched traffic
PA= [1,0,0]
PA= [α, 1-α ,0]
PA= [α, (1-α)/2, (1- α)/2]
PA= [α, 1-α, 0]
PA= [α, (1-α)/2, (1- α)/2]
PSD of Interference=
[0,-164,-164] dBm/Hz
PSD of Interference=
[0,-172,-172] dBm/Hz
(b) Intelligent RRU: outage probability for different power
allocation (PA) decisions. PA=[α, x, y]represents a decision in
which, αfraction of power is used for the dedicated RRB, and x
and yfraction is used for the other two shared RRBs.
Fig. 4: Performance evaluation of the proposed RRM.
intelligent RRM, while the reliability of both traffic types is
satisfied. One further observes, the higher the outage risk,
the higher room for spectrum efficiency by intelligent RRM.
Also, we see that ultra-reliable serving of the non-scheduled
traffic mandates dedicating some RRBs, and beyond the target
reliability level of 99%, the NOMA scheme fails in serving
the non-scheduled traffic. Furthermore, the benefits of HMA
increase by increase in the required reliability level. Finally,
we observe that uncertainty in the wireless channel, as well
as user demand, mandates reserving more radio resources for
guaranteeing reliability performance, which on the other hand
reduces the overall spectrum efficiency.
Now, let us examine the benefit of intelligent radio resource
utilization. We assume a typical device with non-scheduled
URLLC traffic experiences a distance-dependent path-loss of
-80 dB, and needs to combat small-scale fading as well as
path-loss for achieving reliable communications. Out of 5
RRBs, 3could be used for urgent transmission of this device,
where the power spectral density (PSD) of scheduled traffic’s
interference over them is denoted by [0,-172,-172] dBm/Hz.
In other words, the first one is dedicated and the other two
are shared with the scheduled traffic. Fig. 4(b) represents
the outage probability versus the fraction of allocated power
to the dedicated RRB, i.e. α. In this figure, PA=[α, x, y]
represents the power allocation decision over the RRBs by
HMA, which reduces to OMA and NOMA when PA= [1,0,0]
and PA=[1/3,1/3,1/3], respectively. One sees that the best
action is to distribute power equally on all RRBs. This is
because the PSD of interference is comparable with the PSD
of noise, i.e. -174 dBm/Hz. On the other hand, one observe
that when the PSD of interference is [0,-164,-164] dBm/Hz,
i.e. interference is much stronger than the noise, the best action
is to use half of the power over the dedicated RRB, and divide
the other half among the two shared RRBs. In both cases, one
observes that an intelligent selection of α, could bring a huge
gain for HMA in comparison with the OMA and NOMA.
VI. CONCLUSION
Regarding the stringent reliability and delay requirements
of URLLC, there is still lack of communications solutions
enabling the URLLC service. One major challenge consists in
addressing URLLCs scalability and co-existence with other
URLLC/non-URLLC services. Here, we have investigated
coexistence of scheduled and non-scheduled URLLC services
and discussed challenges to be addressed for satisfying their
stringent requirements in co-existence scenarios. Then, we
have proposed a hybrid multiple access (HMA) solution for
addressing such issues in a spectral/energy efficient way. In
the heart of our proposed solution, we have further leveraged
a distributed hierarchical ML approach for proactive RRM to
different URLLC traffic streams. Finally, we have provided a
case study to demonstrate the potential of leveraging ML in
serving coexisting URLLC traffic over limited radio resources.
The results indicated that even for target outage probability
of 10−7for non-scheduled URLLC traffic, approximately
more than 6times data rate is achieved in comparison to
conservative design for scheduled URLLC traffic with 99.99%
reliability thanks to our ML-powered HMA solution.
ACKNOWLEDGMENT
This work is supported in part by the Celtic Plus Project
SooGreen (Service Oriented Optimization of Green Mobile
Networks).
REFERENCES
[1] P. Popovski, et al., “5G Wireless Network Slicing for eMBB, URLLC,
and mMTC: A Communication-Theoretic View,” IEEE Access, vol. 6, pp.
55765-55779, Sept. 2018.
[2] M. Bennis, et al., “Ultra-reliable and Low-latency Wireless Communica-
tion: Tail, risk and scale,” Proceedings of the IEEE, vol. 106, no. 10, pp.
1834-1853, Oct. 2018.
[3] I. Parvez, et al., “A Survey on Low Latency towards 5G: RAN, core
network and caching solutions,” IEEE Communications Surveys and
Tutorials, vol. 10, no. 2, pp. 36-49, May. 2018.

7
[4] H. Ye and G. Li, “Deep Reinforcement Learning for Resource Allocation
in V2V Communications,” in Proc. of IEEE ICC, May. 2018.
[5] A. Kasgari, et al., “Stochastic Optimization and Control Famework for
5G Network Slicing with Effective Isolation,” in Proc. of IEEE CISS,
Mar. 2018.
[6] H. Ji, et al., “Ultra-Reliable and Low-Latency Communications in 5G
Downlink: Physical Layer Aspects,” IEEE Wireless Communications, vol.
25, no. 3, pp. 124-130, June 2018.
[7] F. Calabrese, et al., “Learning Radio Resource Management in RANs:
Framework, Opportunities, and Challenges,” IEEE Communications Mag-
azine, vol. 56, no. 9, pp. 138-145, Sept. 2018.
[8] R. Li, et al., “The Prediction Analysis of Cellular Radio Access Network
Traffic: From entropy theory to networking practice,” IEEE Communica-
tions Magazine, vol. 52, no. 6, pp. 234-240, June 2014.
[9] C. Yuan, et al., “On the Emerging of Scaling Law, Fractality and Small-
World in Cellular Networks,” in Proc. of IEEE VTC-Spring, June 2017.
[10] C. She, C. Yang, and T. Quek, “Radio Resource Management for
Ultra-reliable and Low-latency Communications,” IEEE Communications
Magazine, vol. 55, no. 6, pp. 72-78, June. 2017.
[11] K. Weiss, et al., “A Survey of Transfer Learning,” Journal of Big Data,
vol. 3, no. 1, pp. 1-40, May 2016.
[12] A. Azari and C. Cavdar, “Self-organized Low-power IoT Networks: A
distributed learning approach,” in Proc. of IEEE GLOBECOM, Dec. 2018.
[13] V. Chowdappa, et al., “Distributed Radio Map Reconstruction for 5G
Automotive,” IEEE Intelligent Transportation Systems Magazine, vol.
10, no. 2, pp. 36-49, Apr. 2018.
[14] R. Taranto, et al., “Location-aware Communications for 5G Networks,”
IEEE Signal Processing Magazine, vol. 31, no. 6, pp. 102-112, Oct. 2014.
[15] R. Alieiev, et al., “Predictive Communication and its Application to
Vehicular Environments: Doppler-shift compensation,” IEEE Transactions
on Vehicular Technology, May. 2018.
Amin Azari received the B.Sc. and M.Sc. degrees in electrical and communi-
cation systems engineering from University of Tehran, Iran in 2011 and 2013,
respectively, and the Ph.D. degree in information and communication technol-
ogy from KTH Royal Institute of Technology, Sweden in 2018. Currently, he
is a postdoctoral researcher at Stockholm University. His research interests
include 5G radio access network design, Internet of Things and machine
learning.
Mustafa Ozger received his B.Sc. degree in electrical and electronics
engineering from Middle East Technical University, Ankara, Turkey, in 2011,
and his M.Sc. and Ph.D. degrees in electrical and electronics engineering from
Koc University, Istanbul, Turkey, in 2013 and 2017, respectively. Currently,
he is a postdoctoral researcher at KTH Royal Institute of Technology. His
research interests include wireless communications and the Internet of Things.
Cicek Cavdar is an assistant professor at the School of EECS at KTH in
Sweden. She has been leading the Intelligent Network Systems research group
under the Radio Systems Lab focusing on design and planning of intel-
ligent network architectures, direct air-to-ground communications, and IoT
connectivity platforms. She has been coordinating the EU EIT Digital project
Seamless DA2GC in Europe, which has resulted in successful technology
transfer cases to industry. She served as Symposium Chair for IEEE ICC
GCSN 2017.