Content uploaded by Iacovos Ioannou
Author content
All content in this area was uploaded by Iacovos Ioannou on Jun 04, 2020
Content may be subject to copyright.
1
Distributed Artificial Intelligence Solution for D2D
Communication in 5G Networks
Iacovos Ioannou, Vasos Vassiliou, Christophoros Christophorou, and Andreas Pitsillides
Department of Computer Science, University of Cyprus and
RISE Center of Excellence on Interactive Media, Smart Systems and Emerging Technologies
Nicosia, Cyprus
Abstract—Device to Device (D2D) Communication is one of
the technology components of the evolving 5G architecture, as it
promises improvements in energy efficiency, spectral efficiency,
overall system capacity, and higher data rates. The above
noted improvements in network performance spearheaded a vast
amount of research in D2D, which have identified significant
challenges that need to be addressed before realizing their full
potential in emerging 5G Networks. Towards this end, this
paper proposes the use of a distributed intelligent approach
to control the generation of D2D networks. More precisely, the
proposed approach uses Belief-Desire-Intention (BDI) intelligent
agents with extended capabilities (BDIx) to manage each D2D
node independently and autonomously, without the help of the
Base Station. This paper proposes the DAIS algorithm for the
decision of transmission mode in D2D, which maximizes the
data rate and minimizes the power consumption, while taking
into consideration the computational load. Simulations show the
applicability of BDI agents in solving D2D challenges.
Index Terms—5G, D2D, D2D challenges, Artificial Intelligence,
BDI Agents, Distributed Artificial Intelligence, Multi-Agent Sys-
tems
I. INTRODUCTION
Device to Device (D2D) Communication is expected to be
a core part of the forthcoming 5G mobile communication
networks. D2D can operate both in the licensed and unlicensed
spectrum and is generally transparent to the cellular network
as it allows adjacent user equipment (UE) to bypass the base
station (BS) and establish direct links between them, to either
share their connection and act as relay stations, or directly
communicate and exchange information. D2D can be used
to implement many of the 5G requirements, because it can
support high bit rates and minimize the delay between D2D
UEs. The gains of D2D communications in spectral efficiency,
resource reallocation, and reduction of interference [1], [2] can
potentially improve throughput, energy efficiency, delay, and
fairness [3], [4]. In addition, due to the shorter communication
distance, D2D can offer lower power consumption for the
communicating D2D devices. D2D can enable mobile traffic
offloading, so overall one can anticipate that the non-D2D UEs
can also benefit from the mobile traffic offloading because they
will, as a result, have access to more bandwidth for the com-
munication between them (non-D2D UEs) and the BS, as well
This research is part of a project that has received funding from the
European Union’s Horizon 2020 research and innovation programme under
grant agreement Nº739578 and the government of the Republic of Cyprus
through the Directorate General for European Programmes, Coordination and
Development.
as less interference [3], [4]. However, in order to fully realize
D2D, several challenges need to be resolved, including device
discovery, mode selection, interference management, power
control, security, radio resource allocation, cell densification
& offloading, Quality of Service (QoS) & path selection,
use of mmWave communication, non-cooperative users, and
handover management [24], [34], [35].
This work investigates the idea that the D2D communication
is not a global problem that must be solved centrally, but it is
an optimization problem that should be solved in a distributed
fashion with the use of artificial intelligence. To address that,
the paper proposes that the control is handled by the UEs,
locally, in order to form communication links in shorter time
[5], [6], [7], [8], [9], [10], [11], [12], [13]. We consider that the
use of distributed artificial intelligence (AI) control is the most
suitable in the challenging and dynamic environment of D2D
communication. To the best of our knowledge, there are no
solutions in the literature that jointly satisfy all of the D2D
requirements in one approach. We chose intelligent agents
because of their ability to concurrently solve multiple complex
problems, as it was shown in [38].
In this paper we are making the following contributions:
(a) we propose a solution using Belief-Desire-Intention
(BDI) software agents with extended capabilities (BDIx),
to collectively satisfy the challenges identified for D2D
communication,
(b) we provide a proof-of-concept algorithm that encom-
passes the use of intelligent agents for selecting the
D2D transmission mode, while ensuring a high spectral
efficiency and low computational load,
(c) we propose the use of a new parameter called Weighted
Data Rate (WDR) for the decision of D2D transmission
mode, and
(d) we evaluate the proposed solution under varying scenarios
and provide insights into its operation.
The rest of the paper is structured as follows. Section II
provides background information on D2D communications
and intelligent agents. Section III discusses related work in AI
techniques for communications and D2D. Section IV presents
the proposed solution of distributed control in D2D through
BDIx agents and describes the DAIS algorithm. Section V
discusses the evaluation of the proposed approach, and lastly,
Section VI contains our conclusions and ideas for future work.
2
II. BACKGROU ND O N D2D AN D BDI AGE NT S
A. Background on D2D
1) Control of D2D Communication: We can categorize the
solutions on D2D communication based on the type of control,
as follows:
•Centralized: In centralized techniques the BS completely
manages the UE nodes, even when they (UEs) are com-
municating directly. The controller manages all aspects if
interference/connections/path etc., between cell and D2D
UEs.
•Distributed: In a distributed scheme, the procedure of
D2D node management does not require a central entity,
but it is performed autonomously by the UEs them-
selves. The distributed scheme decreases the control and
computational overhead. This tactic is more suitable for
large size D2D networks. In such a system, all control
processes are run in parallel and start at the same time.
•Semi Distributed: In spite of the fact that both central-
ized and distributed schemes have their strong points,
tradeoffs can be accomplished between them. Such D2D
management schemes are referred to as semi-distributed”
or ”hybrid”.
2) Transmission Mode in D2D Communication: There exist
different modes for D2D communication, based on how UEs
interact with the BS and other D2D nodes (see Fig. 1).
•D2D Direct: Two UEs connect to each other by using
licensed or unlicensed spectrum. The two D2D UEs only
communicate with each other (also called Full-Duplex
D2D).
•D2D Single-hop Relay: Sharing of bandwidth between a
UE and other UEs. In D2D Single-hop Relay mode one
of the D2D UEs is connected to a BS or access point
(AP) and provides access to another D2D UE. [29].
•D2D Multi-hop Relay: The single-hop mode is extended
by enabling the connection of more D2D UEs in chain.
Both backhaul and D2D transmissions are performed in
an uplink with other D2D relay node (as a bridge) and
they are subject to the control of the other D2D relay
node [30].
•D2D Cluster: D2D cluster is a group of UEs connected
to a D2D relay node acting as a Cluster Head (CH).
The D2D relay node acts as an intermediate router to
the network though an access point or BS. Clustering is
suitable in high user densities [31], [32], [33].
B. Research Challenges in D2D
In order for D2D to mature and shape the D2D communica-
tion for the upcoming 5G and beyond wireless communication
networks, some technical issues must be resolved [34], [35].
Each of these challenges is further elaborated below.
1) Device Discovery: In order for two devices (i.e., UEs) to
directly communicate with one another, they must first perform
a device discovery process to identify that they are close to
each other and in range for D2D communication [2], [18].
Fig. 1: Transmission in D2D Communication
2) Mode Selection: When a pair of D2D candidates identify
each other for possible future communication, mode selection
is performed. Mode selection implies that a decision is made
whether the D2D candidates will communicate directly or via
the conventional cellular network [18]. The communication
mode selection should be carefully chosen in order not to
impact on the interference in the network. This communication
mode decision is categorized in the following way:
(a) Inband D2D communication:
•Reuse/Underlay: D2D communication shares the same
resources with existing Cellular UEs. This mode can
achieve high spectral efficiency; however, it may cause
interference to other Cellular and D2D UEs using the
cellular resources.
•Dedicated/Overlay: The cellular network has abun-
dant channel resources so that the D2D UEs can use
dedicated resources that are orthogonal to cellular UEs.
•Cellular: The two UEs will communicate with each
other via the cellular network as traditional cellular
UEs.
(b) Outband D2D Communication:
•Controlled: In the controlled mode the device has two
interfaces. On the first interface it uses unlicensed spec-
trum to share with its peers. On the second interface
it uses licensed spectrum to connect to the mobile
network.
•Autonomous: In autonomous mode, the device can only
use and communicate with other devices under the
unlicensed spectrum, without accessing BS.
3) Interference Management: The communication mode
selection has a direct impact on the interference in the network.
For example when the Reuse/Underlay resource-sharing mode
is selected, high spectral efficiency can be achieved. However,
since many D2D and cellular users will use the same portion
of spectrum, interference may become a problem. Therefore,
interference management must be used [18].
4) Power Control: Although high transmission power can
provide wider coverage and better signal quality during D2D
communication, it can, at the same time, drain the battery
of D2D UEs and cause interference to the network. Thus,
3
proper power control during D2D communication is vital for
controlling the transmission power levels of D2D UEs so as
to deal with the interference generated by the D2D UEs and
improve spectral efficiency, system capacity, coverage, and
reduce energy consumption [19], [20], [21].
5) Security Concerns: In D2D communications, the routing
of users’ data is done through other users’ devices. This
makes the D2D communication network vulnerable to many
security risks and malicious attacks that could breach the data
privacy and confidentiality. Thus, providing efficient security
is a major issue in order to facilitate D2D communication in
cellular networks [22], [23], [12].
6) Radio Resource Allocation: Radio resource allocation
mainly addresses the issues of how to assign the frequency
resources to a group of D2D pairs, or all the D2D pairs,
targeting an optimal use of the radio resources focusing also
on the interference control and management between D2D
and cellular links and the efficient reuse the radio resources
whenever the interference is small [18].
7) Cell Densification and Offloading: Providing high sys-
tem capacity and high per-user data rates – requirements for
the creation of a 5G network – will require a densification
of the radio access network or the deployment of additional
network nodes. In general, the need of network densification
[24] for performance enhancement dictates the deployment of
small coverage cells [18].
8) QoS / Path Selection (Routing): During D2D communi-
cation it is essential to ensure that the QoS requirements of
the communication links are satisfied. To achieve this a major
issue to handle is the selection of the optimum routing path,
otherwise excess resources/power/link usage (bandwidth) will
be wasted [20], [25], [26].
9) D2D in mmWave communication: Communication using
the mmWave band has recently received significant attention
for 5G cellular networks and D2D communication, as it
operates at a higher frequency band (30-300 GHZ) and allows
a significant increase in data rates (multi-Gbps) and network
capacity [27].
10) Handover of D2D device: In order to keep the com-
munication between two D2D devices when these are moving
away from each other, handover should be performed. More
specifically, when a D2D device is moving away from the
access point (e.g., a D2D Relay or a D2D Cluster Head) it
is assigned to, then the problem of handing it over to another
access point (e.g., another D2D Relay or D2D Cluster Head)
with a shared medium should be dealt with [20].
11) Non-cooperative Users: An issue to consider for D2D
data delivery is that the data delivery in non-cooperative D2D
communication may be unfair or compromised. In the real
world, some rational nodes may have strategic interactions
and may act selfishly for various reasons (such as resource
limitations, the lack of interest in data, or social preferences)
or even malicious nodes that they may use the data relay to
attack anonymously [28].
Fig. 2: BDI Agent Architecture
C. Background on Intelligent Agents and Belief-Desire-
Intention Agents
1) Intelligent Agents: An intelligent agent (IA) is an au-
tonomous unit, which observes an environment using sensors
and acts upon it using actuators, coordinating its activity in
the direction of achieving goals (i.e. it is ”rational”, as defined
in economics) [14]. Agent theory is concerned with the use
of mathematical formalisms for representing reasoning and
the properties of agents. Software agents are characterized as
computer software that display flexible autonomous behavior,
which infers that these systems are capable of independent,
autonomous action in order to satisfy their design objectives.
Agents are utilized in a lot of applications. For instance,
autonomous programs used for operator assistance or data
mining (in some cases referred as bots) are also called ”in-
telligent agents”.
2) Belief-Desire-Intention Agents: This work makes use
of Belief-Desire-Intention (BDI) software agents, which are
agents with three key mental structures (see Fig. 2): informa-
tive states of mind around the world (beliefs or convictions),
motivational approaches on what to do (desires or wants) and
planned responsibilities to take action (intentions or expecta-
tions). The BDI model fundamentally relies on two principle
forms: thought and mean send thinking. With the thought
processes the agent produces its goals on the premise of its
convictions and desires, while mean send thinking comprises
of a succession of activities to execute, as an endeavor to
satisfy desires [15].
Unique features of BDI agents [16]:
(a) Beliefs: Beliefs correspond to the informational state
of the agent. Beliefs can also include inference rules,
allowing advance chaining to guide to new beliefs.
(b) Desires: Desires correspond to the motivational state of
the agent. They characterize objectives or situations that
the agent would like to fulfil or bring about.
(c) Intentions: Intentions correspond to the deliberative state
of the agent. This is what the agent has chosen to perform.
Intentions are desires to which the agent has, to some
extent, committed.
A BDI agent decides its actions based on beliefs, which
either contribute to the achievement of its goals, or react to its
received (or perceived) events and messages. [17]. BDI agents
can also cooperate and form a multi-agent system. Multi-
agent systems are systems composed of multiple interacting
computing elements capable of autonomously deciding what
4
actions they require to perform in order to satisfy their design
objectives. In multi-agent systems, the entities are interacting
with other agents, not only by exchanging information, but
also by applying analogues of the type of social activity
that people engage in every day, like cooperation, coordi-
nation, and negotiation [17]. In multi-agent systems, there
are two important issues to consider: (a) since agents are
anticipated to be autonomous, it is usually expected that
the synchronization and coordination structures in a multi-
agent system are not hard-wired at design time, as they
normally are in standard concurrent/distributed systems. In this
manner, mechanisms are needed in order to allow agents to
synchronize and coordinate their activities at runtime; and (b)
the encounters that occur between computing elements in a
multi-agent system are financial encounters, in the sense that
they are encounters between self-interested entities. In a classic
distributed/concurrent system, all the computing elements are
implicitly expected to share the common goal of making the
overall system function correctly. In multi-agent systems, it
is assumed instead, that agents are primarily concerned with
their own welfare, although of course, they will be acting on
behalf of some user/owner [17].
In addition, we can say the BDI agents have foundations in
the Algorithmic, Game-Theoretic, and Logical theories [17].
All the features discussed above make, in our opinion, BDI
agents suitable for solving the challenges of D2D.
III. REL ATED WORK
A. Related Work on AI Techniques for Communications
There is a wealth of research on the use of artificial
intelligence (AI) and machine learning (ML) techniques for
communication and networking issues. In this section we
include a few examples that deal with the use of multi-
agent systems and BDI agents in general communication
problems and at the end focus on AI approaches for D2D
communication.
B. Multi-agent Approaches for Wireless and Mobile Commu-
nications
The authors in [38] address the problem of energy consump-
tion and communication latency in wireless sensor networks
(WSNs). More specifically, the authors propose a system with
a single mobile agent (MA) travelling freely within the net-
work and performing data collection. This behavior improves
data delivery to the sink, and reduces energy consumption.
The specific work utilizes deep neural network for learning,
in which the input is the state of the wireless sensor network
and the output is the optimal route path. The route planning
can be done with the usage of the locations of each node in
the environment that act as input for the intelligent agent. The
intelligent agent architecture selected is the actor network and
a critic network. The information used comes from the whole
network, but the decision is taken locally.
Another work that uses reinforcement learning is [36],
which deals with the problem of discovering low-level wire-
less communication schemes between two agents in a fully
decentralized system. This is the type of problem considered
in the DARPA Spectrum Collaboration Challenge (SC2). The
proposed method employs policy gradients to learn an ideal
bi-directional communication scheme. The approach places
two agents against each other and show that the two actors
are able to learn modulation schemes for communication
while sharing only limited information and having no domain-
specific knowledge about the task.
C. BDI Agents for Wireless and Mobile Communications
The authors in [37] utilize a multi-agent software design,
dynamic analysis, and decentralized control in order to imple-
ment solutions for the complex distributed systems of WSNs.
The paper’s purpose is to create an autonomic system design
for distributed nodes in a diverse and changing environment,
that interact on top of a wireless communication channel
for decentralized problem solving. Due to hardware limita-
tions, the multi-agent system techniques and especially nodes
(agents) are not deliberative (or strong) reasoning systems.
The BDI agent model is used. The paper’s authors implement
two simple WSN test scenarios and show that BDI agents can
perform basic WSN functions. In addition, the agents succeed
in imitating some recognizable aspects of the system and show
that the solution is adaptable to different scenarios. In the
scenarios, five different agents are discussed. A problem of
this approach is that a better method is needed for managing
the size of the belief-base used in each agent, as this turns out
to expand unboundedly in a case such as flooding.
Another class of wireless networks built dynamically in
an ad hoc network manner with a large mobile user base
is found in vehicular ad-hoc networks (VANETs). The work
presented in [15] tackles the problem of routing in VANETs.
Routing in VANETs is critical because of limitations such as
unpredictable network topology, frequent disconnections, and
varying network densities. The authors in this paper proposed
a multi-agent scheme-based routing scheme that comprises of
static agent and mobile agents for vehicle-to-vehicle communi-
cation (V2V), where they address the challenge of how to route
the data with short communication delay, overhead, and the
complexity. The proposed algorithm has the following steps:
i) establish a connectivity pattern between the vehicles; ii)
create a set of beliefs; iii) develop the desires, and iv) execute
the intentions.
D. Artificial Intelligence Approaches for D2D
In the last decade we have seen many approaches for solving
the D2D challenges using AI and ML [46]. The authors in [20]
proposed EHSD -Exemplary Handover Scheme During D2D
Communication- a framework describing a handover scheme
that is based on software-defined radio (SDR) decentralization
by using fuzzy logic. In [39], the authors proposed a learning-
based resource allocation approach for D2D communications
with QoS and fairness considerations by using Q-Learning.
In addition, in [40] the authors proposed a Hierarchical Ex-
treme Learning Machine (H-ELM) Neural Network in order
to manage the severe interference in D2D communications.
Another paper, [41], proposed a genetic algorithm (GA)-based
scheme for Fair Joint Channel Allocation and Power Control
5
for Underlaying D2D Multicast Communications. Also in [42],
the authors proposed an approach for power control in two-
tier orthogonal frequency division multiple access (OFDMA)
femtocell networks by using particle swarm optimization
(PSO). Another intelligent technique is presented in [43],
where the authors used an ant-colony optimization (ACO)-
based resource allocation scheme for solving the problem
of swarm intelligence-based radio resource management for
D2D-based V2V communication.
In the evaluation section we will compare our results with
those of [44]. The authors in [44] use a low complexity method
for matching D2D links with cellular UEs to form partners
for spectrum sharing. Another work we will compare with is
[45], which investigates the gain that cooperative multicast
transmission provides when used to boost the data rate in
D2D communication, enabling data sharing among users by
implementing clusters.
All of the solutions discussed above solve only one of
the many challenges identified in II-B, with the exception of
[47], which solves a joint sub-carrier assignment and power
allocation problem. There is also a yet unpublished work by
[48], which claims to be offering a solution in joint network
admission control, mode assignment and power allocation in
energy-harvesting D2D networks.
To the best of our knowledge there is currently no other
work addressing 5G D2D communication issues using BDI
agents with extended AI capabilities (BDIx).
IV. DISTRIBUTED CON TROL IN D 2D T HR OUGH BDIX
AGE NT S
In this Section we are describing both the new framework
we are proposing for using BDI agents for D2D communi-
cation and we are also describing the DAIS Algorithm for
selecting a node’s transmission mode.
The flowchart in Figure 3 shows the operation of a BDIx
agent from the point it receives a message from the environ-
ment, until it selects and executes a plan.
After perceiving a change in its world, the agent checks
if the Intention must be satisfied or must be changed. If the
Intention is not changed then it continues with the execution
of the Intention plan. Otherwise, the agent selects another
Intention from the list that it has the higher priority and then
it selects a Plan that will satisfy the selected Intention. After
this it continues to execute the plan.
A. Assumptions and Constraints
The assumptions used in the design of the BDIx agents
framework are the following:
•The information needed by BDIx agents is the following:
frequencies used, IP addresses, remaining energy, trans-
mission mode (D2D Relay/D2D multi-hop/D2D cluster),
etc (see Section II-A2).
•Location is known at the agent (all known devices have
GPS).
•Location information and signals can be obtained within
an operator’s network.
Fig. 3: Flowchart of BDIx Agent Operation
•Each agent must be either a D2D Relay Node (D2D-R),
a Multi-hop Relay Node (D2D-MHR), or a D2D Cluster
Head (D2D-CH), or a ”client” D2D node, i.e. at the edge
of the communication path. So a D2D node can either
serve or be served, not both. The agent will decide its
role based on the beliefs and the events it has.
•A frequency should exist for the outband inter-
communication between the BDIx agents.
•A threshold should be preset on Signal Quality (Received
Signal Strength and Bit Error Rates)
•All D2D UEs that are in D2D-R or D2D-MHR transmis-
sion modes know their link and path rates and they can
broadcast them over LTE proximity services.
•A BDIx Agent always accepts proposals from other BDIx
Agents (e.g. a D2D UE to D2D-CH or D2D-MHR request
is always granted).
•A BDIx Agent always selects unused RB (Resource
Block in OFDMA). This is done for simplicity. The
resource management and interference management will
be done in future work.
•The UE device has two mobile interfaces or is using
full duplex interface split equally between uplink and
downlink.
•The UE device has one WiFi interfaces (like all mobiles).
6
B. Sum Rate and Weighted Data Rate
One of the most common metrics for the evaluation of D2D
solutions is Sum-Rate. The Sum-Rate is the total throughput
in a network calculated as the sum of the data rates that are
delivered to all UEs and D2D UEs in a network [49], [50].
Variations on Sum Rate exist, such as Weighted Sum-Rate in
[51], which considers certain links to be of more importance
and gives different weights to the links based on the mode
of transmission (direct, relay, etc). We introduce a new metric
called ”Weighted Data Rate” (WDR). The WDR is defined
at each node as the minimum data rate in the path that the
UE selected. The minimum data rate of a path is the data
rate of the weakest edge in the path. Our aim is, essentially,
to maximize the WDR, i.e WDR = max(min(Link Rate) for
each path. The choice for using WDR instead of sum-rate is
mainly for reducing the computational load of the BDI agent.
The benefits will be shown clearly in the next section.
C. The DAIS Algorithm for Transmission Mode Selection
The following terms are used in the DAIS algorithm:
•D2D-R - D2D Relay node.
•D2D-MHR - D2D Multihop Relay node.
•D2D-CH - D2D Cluster Head.
•WDR - Weighted Data Rate.
•MAXUsersCH - Maximum Users Supported by CH =
255. This is based on WiFi Direct limits.
•MAXQueryD2DRelayDistance - Maximum distance to
query D2D Relay UEs = 200m. It is the maximum
distance of WiFi Direct (200m) or the maximum distance
of LTE Direct (1000m).
•MAXDistancetoFormCluster - Maximum distance
threshold to accept connection to the node if the UE
is CH. This is the pre-defined maximum radius range
between D2D UEs in order to form a D2D Cluster. This
will be calculated based on the technology used (WiFi
Direct or LTE Direct). It can be calculated from WiFi
Direct range/2 (100m) or LTE Direct range/2 (500m).
•MAXSpeedToFormBackhauling - Maximum speed that
a node is moving in order to be D2D Relay or D2D Multi
Hop Relay = 1.5 m/s (pedestrian).
•MAXDistanceMultiHop - Maximum distance threshold
for a UE from the nearest D2D Relay in order to act as
D2D multi-hop relay. In order for the UE to select to be
multi-hop, the device must have the weighted data rate
to an existing D2D Relay greater than the weighted data
rate of the existing D2D Relay.
•MAXDistanceMoveAway - Maximum Distance to move
away from the current position in order to recalculate. It
can be calculated from WiFi Direct range/2 (100m) or
LTE Direct range/2 (500m).
•PERCDataRate - Percentage of difference of Data Rate
in order to make D2D Relay connect from UE D2D
multihop Relay to Gateway =20
•DeviceBatteryThreshold - The minimum battery per-
centage in order of the D2D device to act as D2D-R
or D2D-MHR is 70%
•The D2D device power is calculated randomly and it is
following a Gaussian distribution with mean of 0.6 and
variance of 0.4.
In our approach the D2D-R/D2D-MHR are using proximity
services to broadcast the connection information (i.e. WDR,
coordinates).
The notation and mathematical representation of symbols
used in the DAIS algorithm are presented in Table I. The
plan of execution of transmission mode selection is shown
in Algorithm 1. This is executed at the startup phase of
the BDIx Agent. The computational complexity of such an
algorithm is O(n) because the algorithm calculates the values
in Table I only once. In addition, the algorithm is quick
because decisions are made locally and do not rely on global
information. Since routes are created instantaneously and
incrementally by each agent, by identifying local D2D-R and
D2D-MHR using proximity services, the complexity is based
on the actual number of D2D-R and D2D-MHR that the agent
in each device must communicate with, whenever it is needed
(e.g. in order to become D2D-R by connecting to an existing
D2D-MHR in our algorithm).
V. PERFORMANCE EVAL UATIO N OF T HE DAIS
ALGORITHM
In this section, we investigate the performance of the
proposed DAIS algorithm. The simulations are done using
Java and Matlab. We consider scenarios with one BS and a
number of UEs ranging from 10 to 1000, over an area of
1000x1000 meters. The BS in the simulations is in the center
of the grid. The simulation parameters are shown in Table II.
The parameters are taken from the standards for WiFi Direct
[53], LTE Direct [52], and LTE communication [54], [55].
The first thing we examine is the spectral efficiency of
the proposed solution. Figure 4 shows that our proposed
solution has a better performance compared to a random
clustering solution and when no-D2D communication is used.
The realized benefits are in the order of 30%. The most
interesting result is that random clustering results in spectral
efficiency even worse than direct UE-BS communication.
Considering the power needed to realize the communication
of the nodes, it is not surprising to see that clustering indeed
requires less power. However, the proposed solution still
outperforms the second best by about 25%.
Within the proposed framework we have the ability to easily
interchange metrics and parameters. In Section IV-B we have
argued on the feasibility of using WDR instead of Sum-Rate
in our calculations. Figure 6 shows that the use of WDR does
not reduce the spectral efficiency of the system. The same
happens if we consider an option in which a UE participates in
the D2D communication depending on the remaining battery
it has. Figure 7 shows no difference in spectral efficiency.
On the contrary, by utilizing a battery threshold we are
slightly increasing the required power for the communication,
as evident by the slight differences shown in Fig. 8.
A significant result, which validates our choice of WDR
is that the computational time needed to perform sum-rate
calculations is up to five (5) times greater than the constant
7
TABLE I: Algorithm Notations and Mathematical Representations
Notations Mathematical Representation
d»(UEx1−D2Dx2)2+ (U Ey1−D2Dy2)2
maxD2DR D2Djwhere WD RD2Dj= (MAX (W DRD2Di)∃D2Diwhere d ≥MAX DistancetoF ormC luster
∧W DRD2Di≥(W DRU Ei+P ERC DataRate ∗W DRU Ei)∧i∈D2DR
∧COU N T (D2Dig
W HE RE g servedby i)<=D)
maxD2DMHRNoConnections D2Djwhere WD RD2Dj= (MAX (W DRD2Di)∃D2Diwhere d ≥MAX DistancetoF ormC luster∧
W DRD2Di≥(W DRU Ei+P ERC DataRate ∗W DRU Ei)∧i∈D2DMHR ∧C OU NT (D2Dig
W HE RE g servedby i) = 0)
maxD2DRNoConnectionsToBeD2DMHR D2Djwhere W DRD2Dj= (M AX (W DRD2Di)∃D2Diwhere d ≥MAX DistancetoF ormC luster∧
d≤MAXQueryD2DRelayD istance ∧W DRD2Di≥(W DRU Ei+P ERC DataRate ∗W D RUEi)∧
i∈D2DR ∧CO UN T (D2DigW H ERE g servedby i) = 0) ∧
D2DDeviceP ow eri≥DeviceBatter yT hreshold
maxD2DRToUseUED2DMHR D2Djw here W D RD2Dj= (MAX (W DRD2Di)∃D2Diwhere d ≥MAX DistancetoF ormC luster ∧
d≤MAXQueryD2DRelayD istance ∧W DRD2Di(W DRU Ei−P ERC DataRate ∗W D RUEi)
∧i∈D2DR ∧D2DDev iceP oweri≥DeviceB atteryT hreshold
maxD2DMHRToUseAsMultiHop D2Djw here W D RD2Dj= (MAX (W DRD2Di)∃D2Diwhere d ≥MAX QueryD 2DRelayDistance∧
d≤MAXDistanceM ultihop ∧W DRD2Di≥(W D RUEi+P ER CDataRate ∗W DRU Ei)∧
i∈D2DM HR ∧C OU NT (D2DigW H ERE g servedby i) = 0) ∧
D2DDeviceP ow eri≥DeviceBatter yT hreshold
Fig. 4: Spectral Efficiency of Different Transmission Modes Fig. 5: Power Savings of Different Transmission Modes
computation needed when we perform WDR calculations
locally. This is ascribed to the fact that sum-rate needs to
check all links in the network every time it needs to decide the
transmission mode of a UE. As the number of UEs increases
the computational time increases as well. In our case, the time
to form a cluster is 100ms for any device density, because the
D2D UEs have all their link rates precalculated, so that WDR
for the new connection is easily computed.
By comparing the results of our approach with those in [45]
we observe that for 50 UEs (maximum number considered in
that work) we have the same number of clusters (seven) and
the same amount of average UEs per cluster. However, we
have no way of knowing if the solution in [45] can scale,
whereas our approach is shown to scale well for at least up to
1000 UEs. In our approach the energy gained by the BS when
we apply clustering is the same as in the work in comparison.
However, in our case we can have 1000 UEs clustering at the
almost instantaneous time of 100ms. Another work that lends
itself for comparison is [44], when considered for similar BS
and UE power as well as node density. The max number of
UEs and D2D links used in that work is, again, in the order
of 50. In the best case scenario analyzed in [44], the spectral
efficiency reaches 220 b/s/Hz for N=30 UEs when all of them
are D2D linked. The performance goes down to 180 b/s/Hz
as the D2D links are reduced to twenty (20). By comparison,
in our work, a similar number of UEs (N=30) and D2D links
the corresponding spectral efficiency is 296 b/s/Hz. If we test
it with 30 D2D links and 10 UE links the BDI solutions has
a rate of 405 b/s/Hz which outperforms the 260 b/s/Hz of the
paper in comparison.
8
Algorithm 1: DAIS Algorithm for Transmission Mode Selection Plan in BDIx Agents
1connect to BS (GateWay) ;
/*Check to find D2D Relay to connect as client */
/*Check if a D2D Relay device exists near the D2D UE with the maximum WDR */
2if exists maxD2DR then
/*Check to find D2D Relay to connect as client */
3Connect UE as D2D Client to maxD2DR using WiFi Direct;
/*Check if a D2D-MHR exists near the D2D UE with the maximum WDR and convert it to D2D-R */
4else if exists maxD2DMHRNoConnections then
/*Check to find D2D Multihop Relay that no one connects to, make it D2D Relay, and connect to it
as D2D Client */
5Request from maxD2DMHRNoConnections UE to be D2DR;
6Connect UE as D2D Client to maxD2DMHRNoConnections using WiFi Direct;
/*Now the D2D-MHR is D2D-R */
/*Connect as D2D-R or Optimize a Path */
/*Check if a D2D-R device exists far from the D2D UE with maximum WDR and not have connections other
than path to BS in order to connect to it as D2D-R (The device will convert to D2D-MHR) */
7else if exists maxD2DRNoConnectionsToBeD2DMHR then
/*Check to find D2D-R that no one connects to and make it D2D-MHR and connect to it as D2D-R */
8Request from DMHRNoConnections UE to be D2D-MHR ;
9Connect UE as D2D-R to maxD2DRNoConnectionsToBeD2DMHR using LTE Direct;
/*Now the D2D-R is D2D-MHR and UE is D2D-R */
/*Check if a D2D-R device exist far from the D2D UE with maximum WDR worse than the UE and with no
connections other than a path to BS in order to connect to it as D2D-MHR (The device will connect
as D2D-R to the new UE that is going to be D2D-MHR) */
10 else if exists maxD2DRToUseUED2DMHR then
/*Check to find D2D-R that no one connects to, with worse WDR than the UE and make UE as D2D-MHR
and ask the device D2D-R to connect to UE */
11 Set UE as D2D-MHR ;
12 Connect maxD2DRToUseUED2DMHR as D2D Relay to UE using LTE Direct;
/*Now the UE is D2D-MHR */
/*Check if a D2DMHR device exist from the D2D UE with maximum WDR and no connections other than path
to BS in order to connect to it as D2D-R */
13 else if exists maxD2DMHRToUseAsMultiHop then
/*Check to find D2D-MHR that no one connects to, make UE as D2D-R and connect to it */
14 Set UE as D2D-R ;
15 UE.TransmissionMode=D2D Relay ;
16 Connect UE as D2D-R to maxD2DMHRToUseAsMultiHop using LTE Direct;
/*Now the UE is D2D-R */
17 else
18 set UE as D2D-MHR; Stay connected to BS ;
Simulation Parameters Value
D2D power 130 mW [54], [55]
UE power 260 mW [54], [55]
WiFi Direct Radius 200 m [52]
LTE Direct Radius 1000 m [53]
BS Range 1000 m [54], [55]
Path loss exponent (Urban Area) 3.5
BS Antenna gain 40 dB [54], [55]
UE/D2D antenna gain 2 dB [54], [55]
N0 (White Noise) 0.0001
D (WiFi Direct max clients) 200 [52]
N (no of UEs) 10-1000
Shadowing Log-normal
Mobility Static scenario
TABLE II: Simulation Parameters
VI. CONCLUSIONS AND FUTURE WO RK
Device to Device (D2D) Communication is expected to be
a core part of the forthcoming 5G Mobile Communication
Networks. To achieve that goal, several challenges, like in-
terference management, power control, and routing, among
others, need to be addressed. The paper investigates the prob-
lem of solving multiple D2D communication requirements
in one framework by using BDI agents. Such agents can
be implemented at the UEs and there is no need to change
how BSs operate or to change the hardware at BSs or UEs.
The current work focuses on the definition of a joint solution
of D2D requirements. To that extend it contains a detailed
proof-of-concept algorithm. which works towards deciding the
9
Fig. 6: Spectral Efficiency of Different Rate Options Fig. 7: Spectral Efficiency of Different Power Options
Fig. 8: Power Saved
Fig. 9: Computational Complexity
transmission mode of each UE and forms the best possible
paths towards the base station using relays and clusters.
Through simulations the solution was found to ensure a high
spectral efficiency and low computational load. In future work
we will focus on the utilization of more AI approaches under
our BDIx framework and we will evaluate a more dynamic
environment by considering mobile UEs.
REFERENCES
[1] K. Doppler, M. P. Rinne, P. Janis, C. Ribeiro, and K. Hugl, ”Device-
to-device communications; Functional prospects for LTE-advanced net-
works,” in Proc. IEEE Int. Conf. Commun. Work., ICC Workshops, Jun.
2009, pp. 1-6.
[2] G. Fodor, E. Dahlman, G. Mildh, S. Parkvall, N. Reider, G. Mikl´
os,
and Z. Tur´
anyi, ”Design aspects of network assisted device-to-device
communications,” IEEE Commun. Mag., vol. 50, no. 3, pp. 170-177,
Mar. 2012.
[3] P. Gandotra and R. K. Jha, ”Device-to-Device Communication in Cel-
lular Networks: A Survey,” J. Netw. Comput. Appl., vol. 71, no. 4, pp.
1801-1819, Aug. 2016.
[4] M. Ahmad, M. Azam, M. Naeem, M. Iqbal, A. Anpalagan, and M. Ha-
neef, ”Resource management in D2D communication: An optimization
perspective,” J. Netw. Comput. Appl., vol. 93, pp. 51-75, Sep. 2017.
[5] S. Wen, X. Zhu, Z. Lin, X. Zhang, and D. Yang, ”Distributed resource
management for Device-to-Device (D2D) communication underlay cel-
lular networks,” in IEEE Int. Symp. Pers. Indoor Mob. Radio Commun.
PIMRC 2013, pp. 1624-1628.
[6] D. H. Lee, K. W. Choi, W. S. Jeon, and D. G. Jeong, ”Two-stage semi-
distributed resource management for device-to-device communication
in cellular networks,” IEEE Trans. Wirel. Commun., vol. 13, no. 4, pp.
1908-1920, April 2014.
[7] D. Wu, Y. Cai, R. Q. Hu, and Y. Qian, ”Dynamic Distributed Resource
Sharing for Mobile D2D Communications,” IEEE Trans. Wirel. Com-
mun., vol. 14, no. 10, pp. 5417-5429, Oct. 2015.
[8] G. Fodor and N. Reider, ”A Distributed Power Control Scheme for
Cellular Network Assisted D2D Communications,” in IEEE Global
Telecommunications Conference - GLOBECOM, Dec. 2011, pp. 1-6.
[9] F. Librino and G. Quer, ”Distributed Mode and Power Selection for a
Stochastic Approach,” IEEE Trans. Cogn. Commun. Netw., vol. 4, no.
2, pp. 232-243, Feb. 2018.
[10] J. Kim, J. Park, J. Noh, and S. Cho, ”Completely Distributed Power
Allocation using Deep Neural Network for Device to Device communi-
cation Underlaying LTE,” Feb. 2018, arXiv:1802.02736.
[11] Y. Cai, H. Chen, D. Wu, W. Yang, and L. Zhou, ”A distributed resource
management scheme for D2D communications based on coalition forma-
tion game,” in Proc. IEEE Int. Conf. Commun. Work., ICC Workshops,
Jun. 2014, pp. 355-359.
[12] H. Nguyen, M. Hasegawa, and W. Hwang, ”Distributed Resource Al-
location for D2D Communications Underlay Cellular Networks,” IEEE
Commun. Lett., vol. 20, no. 5, pp. 942-945, May 2016.
[13] R. Yin, G. Yu, C. Zhong, and Z. Zhang, ”Distributed resource allocation
for D2D communication underlaying cellular networks,” in Proc. IEEE
Int. Conf. Commun. Work., ICC Workshops, 2013, pp. 138-143.
[14] J. R. Stuart and N. Peter, Artificial Intelligence a Modern Approach 3rd
Edition, Prentice Hall, 2009.
[15] M. S. Kakkasageri, M. J. Sataraddi, P. M. Chanal, and G. S. Kori,
”BDI Agent Based Routing Scheme in VANETs,” Int. Conf. on Wireless
Commun., Sign. Proc. and Networking (WiSPNET), 2017, pp. 129-133.
[16] A. S. Rao and M. P. Georgeff, ”BDI agents: From theory to practice,”
in First Intl. Conf. on Multiagent Systems, 1995, pp. 312-315.
[17] Y. Shoham and K. Leyton-Brown, ”Multiagent systems: Algorithmic,
Game-Theoretic, and logical foundations,” Cambridge University Press,
1 edition, Dec. 2008.
[18] M. Noura and R. Nordin, ”A survey on interference management
for Device-to-Device (D2D) communication and its challenges in 5G
networks,” J. Netw. Comput. Appl., vol. 71, pp. 130-150, 2016.
10
[19] K. Kuntz, ”Performance of Macro- and Co-channel Femtocells in a
Hierarchical Cell Structure,” Der Spiegel, 2015.
[20] H. B. Valiveti and P. T. Rao, ”EHSD: An Exemplary Handover Scheme
During D2D Communication Based on Decentralization of SDN,” Wirel.
Pers. Commun., vol. 94, no. 4, pp. 2393-2416, 2017.
[21] F. Wang, C. Xu, L. Song, and Z. Han, ”Energy-efficient resource
allocation for device-to-device underlay communication,” IEEE Trans.
Wirel. Commun., vol. 14, no. 4, pp. 2082-2092, 2015.
[22] J. Yue, C. Ma, H. Yu, and W. Zhou, ”Secrecy-based access control for
device-to-device communication underlaying cellular networks,” IEEE
Commun. Lett., vol. 17, no. 11, pp. 2068-2071, Nov. 2013.
[23] C. Ma, W. Wu, Y. Cui, and X. Wang, ”On the performance of successive
interference cancellation in D2D-enabled cellular networks,” in 2015
IEEE Conference on Computer Communications (INFOCOM), 2015,
pp. 37-45.
[24] E. Hossain and M. Hasan, ”5G cellular: key enabling technologies and
research challenges,” IEEE Instrum. Meas. Mag., vol. 18, no. 3, pp.
11-21, Jun. 2015.
[25] J. Liu, S. Zhang, N. Kato, H. Ujikawa, and K. Suzuki, ”Device-to-Device
Communications for Enhancing Quality of Experience in Software
Defined Multi-Tier LTE-A Networks,” IEEE Network, vol. 29, no. 4,
pp. 46-52, August 2015.
[26] M. Sadik, N. Akkari, and G. Aldabbagh, ”SDN-based handover scheme
for multi-tier LTE/Femto and D2D networks,” Comput. Networks, vol.
142, pp. 142-153, 2018.
[27] Y. Niu, Y. Li, D. Jin, L. Su, and A. V. Vasilakos, ”A survey of millimeter
wave communications (mmWave) for 5G: opportunities and challenges,”
Wirel. Networks, vol. 21, no. 8, pp. 2657-2676, Nov. 2015.
[28] B. Jedari, F. Xia, and Z. Ning, ”A Survey on Human-centric Communi-
cations in Non-cooperative Wireless Relay Networks,” IEEE Commun.
Surv. Tutorials, vol. 20, no. 2, pp. 914-944, 2018.
[29] J. Deng, A. A. Dowhuszko, R. Freij, and O. Tirkkonen, ”Relay selection
and resource allocation for D2D-relaying under uplink cellular power
control,” 2015 IEEE Globecom Work. (GLOBECOM Wkshps), Dec.
2015, pp. 1-6.
[30] G. Steri, G. Baldini, I. N. Fovino, R. Neisse, and L. Goratti, ”A novel
multi-hop secure LTE-D2D communication protocol for IoT scenarios,”
23rd Int. Conf. Telecommun. ICT, May 2016, pp. 1-6.
[31] L. Song, D. Niyato, Z. Han, and E. Hossain, ”Game-theoretic resource
allocation methods for device-to-device communication,” IEEE Wirel.
Commun., vol. 21, no. 3, pp. 136-144, 2014.
[32] T. Koskela, S. Hakola, T. Chen, and J. Lehtom¨
aki, ”Clustering concept
using device-to-device communication in cellular system,” IEEE Wirel.
Commun. Netw. Conf. WCNC, 2010.
[33] B. Peng, T. Peng, Z. Liu, Y. Yang, and C. Hu, ”Cluster-based multicast
transmission for device-to-device (D2D) communication,” IEEE Veh.
Technol. Conf., Sep. 2013, pp. 1-6.
[34] I. F. Akyildiz, S. Nie, S. C. Lin, and M. Chandrasekaran, ”5G roadmap:
10 key enabling technologies,” Comput. Networks, vol. 106, pp. 17-48,
2016.
[35] S. Chen and J. Zhao, ”The requirements, challenges, and technologies
for 5G of terrestrial mobile telecommunication,” IEEE Commun. Mag.,
vol. 52, no. 5, pp. 36-43, 2014.
[36] C. de Vrieze, S. Barratt, D. Tsai, and A. Sahai, ”Cooperative Multi-
Agent Reinforcement Learning for Low-Level Wireless Communica-
tion,” Jan 2018, arXiv:1801.04541.
[37] A. Morris, P. Giorgini and S. Abdel-Naby, ”Simulating BDI-Based
Wireless Sensor Networks,” 2009 IEEE/WIC/ACM International Joint
Conference on Web Intelligence and Intelligent Agent Technology,
Milan, Italy, 2009, pp. 78-81.
[38] J. Lu, L. Feng, J. Yang, M. M. Hassan, A. Alelaiwi, and I. Humar,
”Artificial agent: The fusion of artificial intelligence and a mobile agent
for energy-efficient traffic control in wireless sensor networks,” Futur.
Gener. Comput. Syst., vol. 95, pp. 45-51, 2019.
[39] S. Kazemi Rashed, R. Shahbazian, and S. A. Ghorashi, ”Learning-
based resource allocation in D2D communications with QoS and fairness
considerations,” Trans. Emerg. Telecommun. Technol., vol. 29, no. 1, pp.
1-20, 2018.
[40] J. Xu, X. Gu, and Z. Fan, ”D2D Power Control Based on Hierarchical
Extreme Learning Machine,” 2018 IEEE 29th Annu. Int. Symp. Pers.
Indoor Mob. Radio Commun., pp. 1–7, 2018.
[41] M. Hamdi, D. Yuan, and M. Zaied, ”GA-based scheme for fair joint
channel allocation and power control for underlaying D2D multi-
cast communications,” 13th Int. Wirel. Commun. Mob. Comput. Conf.
IWCMC 2017, pp. 446-451, 2017.
[42] Z. Huang, Z. Zeng, H. Xia, and J. Shi, ”Power control in two-tier
OFDMA femtocell networks with Particle Swarm Optimization,” IEEE
Veh. Technol. Conf., pp. 1-5, 2011.
[43] S. Feki, A. Masmoudi, A. Belghith, F. Zarai, and M. S. Obaidat,
”Swarm intelligence-based radio resource management for D2D-based
V2V communication,” Int. J. Commun. Syst., no. May, pp. 1–16, 2018.
[44] L. Wang and H. Wu, ”Fast pairing of device-to-device link underlay for
spectrum sharing with cellular users,” IEEE Commun. Lett., vol. 18, no.
10, pp. 1803–1806, 2014.
[45] S. Doumiati, H. Artail, and K. Kabalan, ”A framework for clustering
LTE devices for implementing group D2D communication and multicast
capability,” 8th Int. Conf. Inf. Commun. Syst. ICICS 2017, pp. 216-221,
2017.
[46] K. Zia, N. Javed, M. N. Sial, S. Ahmed, H. Iram, and A. A. Pirzada, ”A
Survey of Conventional and Artificial Intelligence / Learning based Re-
source Allocation and Interference Mitigation Schemes in D2D Enabled
Networks,” Sep. 2018, arXiv:1809.08748.
[47] U. Saleem, S. Jangsher, H. K. Qureshi, and S. A. Hassan, ”Joint
subcarrier and power allocation in the energyharvesting-aided d2d
communication,” IEEE Trans. on Industrial Informatics, vol. 14, no.
6, pp. 2608-2617, June 2018.
[48] A. Y. Awan, M. Ali, M. Naeem, F. Qamar and M. N. Sial, ”Joint Network
Admission Control, Mode Assignment and Power Allocation in Energy
Harvesting aided D2D Communication,” in IEEE Trans. on Industrial
Informatics. doi: 10.1109/TII.2019.2922667.
[49] C. Yang, X. Xu, J. Han, and X. Tao, ”GA based user matching with
optimal power allocation in D2D underlaying network,” in IEEE Veh.
Technol. Conf., pp. 1-5, January 2014.
[50] C. Xu et al., “Efficiency resource allocation for device-to-device under-
lay communication systems: A reverse iterative combinatorial auction
based approach,” IEEE J. Sel. Areas Commun., vol. 31, no. 9, pp. 348-
358, 2013.
[51] R. Wang, J. Zhang, S. H. Song, and K. B. Letaief, ”QoS-Aware channel
assignment for weighted sum-rate maximization in D2D communica-
tions,” IEEE Glob. Commun. Conf. GLOBECOM 2015, no. pp. 1-6,
2015.
[52] Qualcomm, “LTE Direct Overview”,
”https://www.qualcomm.com/documents/lte-direct-overview”
[53] WiFi Alliance, “WiFi Direct - The worldwide network of companies that
brings you Wi-Fi”, ”https://www.wi-fi.org/discover-wi-fi/wi-fi-direct”
[54] Xiao, S., Feng, D., Yuan-wu, Y., Li, G. Y., Guo, W., & Li, S. (2015). Op-
timal Mobile Association in Device-to-Device-Enabled Heterogeneous
Networks. (2014).
[55] 3GPP TR 36-942 - V14.0.0 - LTE Evolved Universal Terrestrial Radio
Access (E-UTRA); Radio Frequency (RF) system scenarios.
