Content uploaded by Mohamed Rahouti
Author content
All content in this area was uploaded by Mohamed Rahouti on Jun 25, 2018
Content may be subject to copyright.
Applying Software-Defined Networking to Minimize the
End-to-End Delay of Network Services
Tommy Chin
Rochester Institute of Tech.
Rochester, NY 14623 USA
tommy.chin@ieee.org
Mohamed Rahouti
University of South Florida
Tampa, FL 33620 USA
mrahouti@mail.usf.edu
Kaiqi Xiong
University of South Florida
Tampa, FL 33620 USA
xiongk@usf.edu
ABSTRACT
There is a grand challenge to transfer the gigantic volume of
data generated from a variety of smart cities applications.
It is an urgent need to efficiently and effectively transmit
such time-sensitive data in a wide-area communication. In
this paper, we propose an approach by leveraging Software-
Defined Networking (SDN) to develop a communication so-
lution for data transfers in smart city applications. Our
approach is Quality of Service (QoS)-aware that minimizes
the end-to-end delay for data transmission in an SDN in-
frastructure by using a Timestamp Recording method to
compare the arrival and departure of flows and packets over
a period. Finally, we evaluate our solution on the Global
Environment for Network Innovations (GENI), a real-world
federated testbed. We first compare the Timestamp Record-
ing to other common delay measurement techniques. Our
analysis demonstrates its effectiveness at scale. Addition-
ally, we carefully examine the accuracy and efficiency in
minimizing end-to-end delay delivered by our SDN solution
with contrast to a non-optimized SDN environment. Our ex-
periments demonstrate the proposed solution effective and
efficient in delivering data with an optimal delay. Partic-
ularly, our solution reduces about 22% in end-to-end-delay
compared to a non-optimized SDN environment.
CCS Concepts
•Networks →Network experimentation; Network
performance analysis; Network measurement;
Keywords
End-to-End Delay; Path Selection; Priority Traffic; Software-
Defined Networking (SDN);
1. INTRODUCTION
Building an efficient communication system is not only im-
portant but also necessary in a variety of application services
in smart cities. For instance, a next-generation power grid
system, as one of the most critical applications in smart
cities, requires an efficient networking infrastructure to sup-
port the efficacious generation, transmission, and distribu-
Copyright is held by the authors. This work is based on an ear-
lier work: RACS’17 Proceedings of the 2017 ACM Research in Adap-
tive and Convergent Systems, Copyright 2017 ACM 978-1-4503-5027-3.
http://dx.doi.org/10.1145/3129676.3129731
tion of Phasor Measurement Unit (PMU) data. Further-
more, in such a communication system, the end-to-end de-
lay can affect the performance of entire grid systems, causing
power losses, and possibly equipment damage [1, 23].
The end-to-end delay plays a critical role in the delivery
or transfer of data between the source and the destination
within end-to-end devices. The information exchanged be-
tween these devices is often beneficial or valid just for specific
time slot window. Hence, if the delay associated with the
communication system surpasses a predefined time window,
the information could no longer serve its objective. Thus,
the key components to efficient network traffic forwarding
in an environment are the resiliency of a network infrastruc-
ture to flash crowd traffic (burst), the ability to minimize an
end-to-end delay and to provide the highest priority where
Quality-of-Service (QoS) is guaranteed [18, 19, 24]. As users
are broadly interested in network service processing time in-
stead of network throughput, the end-to-end communication
among end-user devices is required to guarantee a maximum
QoS level. Thus, accurately predicting the perceived QoS for
a user allows service providers to not only assure QoS but
also avoid over-provisioning to meet a user’s requirement.
Because of a variable burst of network traffic derived from
customer requests, the dynamic assurance of a minimal end-
to-end communication delay will not be an easy task.
Unlike a traditional network, a Software-Defined Network-
ing (SDN) environment utilizes SDN controllers to provide
a centralized and holistic view of the network [7]. Besides,
the key features of SDN include, but are not limited to, flex-
ibility and programmability [14, 15, 17]. These features are
used to ensure different QoS requirements. Policy Cop [5],
VSDN [16], and other QoS-based routing frameworks [10,
21] utilize a centralized routing approach to improving video
delivery within end-to-end entities. Herein, the limitation
of these applications is that they implement solely those
QoS management functions that are already available in the
newest versions of some SDN controllers (such as Floodlight
and OpenDaylight). In this paper, we leverage the flexibil-
ity of an SDN controller to implement a Timestamp-based
framework that takes into account the shortest path selec-
tion. This framework measures and optimizes the end-to-
end delay for a real-time communication system in SDN en-
vironments, where such a QoS metric is of great importance
to realize end-to-end applications, for instance, e-learning
[12]. Although the proper development of such end-to-end
applications is required to control the communication delay
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
30
between end-to-end devices [25], there are still two signifi-
cant challenges to take into account. The first challenge is
the simultaneousness, implying that all entities need to be
time-synchronized. The second challenge is timeliness where
the transmission of data has a precise and sensitive deadline.
Furthermore, prioritizing application services is also impor-
tant. There have been many solutions aimed to minimize
end-to-end delay in a traditional network environment such
as OpenQOS [9], MCUP [26], and an SDN controller for
cloud gaming [2]. However, to the best of our knowledge,
none of them has addressed the end-to-end delay for a net-
work that handles different priority traffic at burst intervals
for network traffic with different priorities.
Efficiently measuring the end-to-end delay between two end
devices is critical in this research. Since ‘Traditional Ping’
is a basic approach to calculating delay within network-
enabled entities by deploying Internet Control Message Pro-
tocol (ICMP/Ping), we consider this approach as a baseline
value due to ICMP’s fundamental nature in the end-to-end
delay measurement. The delay between a source and a desti-
nation is computed as one-half of the round trip time (RTT)
of a message from and returned to the source. We further
consider ‘Packet Probing’ as our second approach for the
calculation where the end-to-end delay is computed based
on the Round-Trip Time (RTT) of Link Layer Discovery
Protocol (LLDP) and Broadcast Domain Discovery Proto-
col (BDDP) accordingly. However, there is a significant
fault associated with Packet Probing, which is the excess
of network traffic generated by these two network protocols
and the period to examine delay. Our third approach is
Timestamp Recording where Open vSwitch records the ar-
rival and departure time of a per-packet flow where the delay
is calculated base on the sending and receiving time over a
set of packets. We extensively examine the effectiveness of
Timestamp Recording, Traditional Ping, and Packet Prob-
ing on the ExoGENI [4], one of the Global Environment
for Network Innovation (GENI) testbeds [6]. Our experi-
mental evaluation demonstrates that Timestamp Recording
outperforms the other two measurement approaches regard-
ing measurement accuracy. In the end, we study and exam-
ine Dijkstra’s algorithm on an SDN controller for prioritized
application services using Timestamp Recording where we
aim to select an optimal path in the network. Our Exo-
GENI experiments show the applicability and effectiveness
of Timestamp Recording in the optimal path selection.
The rest of our paper is organized as follows. Section 2 dis-
cusses related work. Section 3 gives our research method-
ology. Section 4 outlines the experimental results of this
research. Finally, we conclude our work with future work in
Section 5.
2. RELATED WORK
QoS is the non-functional attribute of a service such as an
end-to-end communication delay [3]. Satisfying such a QoS
metric is of great importance towards customizing an ef-
ficient and reliable network in smart city applications. For
example, in power grid systems [3], the communication delay
between a power grid substation and its controlled electronic
device needs to be in milliseconds to maintain the balance
of power flow among power lines.
The end-to-end network delay has been widely studied [9,
20, 22]. While Xiong [20] studied the bandwidth allocation
to ensure the guarantee of QoS including the end-to-end
delay, Xiong, et al. [22] presented an approach to measuring
the QoS metric in cloud networks. Likewise, OpenQoS [9]
leveraged OpenFlow to deliver multimedia network traffic.
Although it can deal with the dynamics of network traffic
and provide the shortest path in terms of the QoS metric,
the implementation of OpenQoS [9] is limited to a very small
scale local area network.
In OpenQoS [9], SDN controllers repeatedly update their
data planes to optimize network utilization and provide dy-
namic QoS routing for data delivery. However, these repeti-
tive updates in OpenFlow networks by SDN controllers max-
imize network utilization and may cause serious transient
congestion and packet loss. In MCUP [26], authors formu-
lated the update of data planes as an optimization problem
and solved it heuristically. However, they only evaluated
their SDN-based approach through Mininet simulation.
Amiri, et al. [2] employed SDN to adaptively disperse the
cloud game traffic load among different routing paths based
on end-to-end delays and jitters for cloud players. Under
QoSflow [13], researchers examined the way to minimize end-
to-end delay from the perspective of packet schedulers using
Linux kernels. However, their approach provided the ability
to individually fine-tune each SDN device from a system-
level point of view, and to optimize QoS mechanisms. Al-
though QoS provides the ability to manage traffic type re-
garding priority, OpenTCP [11] fine tunes TCP communi-
cation on SDN topology. Under OpenTCP [11], QoS was
not addressed in the performance of their design by real-
world experiments conducted under their analysis. Instead,
we leverage end-to-end delay measurement approaches with
SDN and implement them in a real-world at-scale virtual
network using GENI computing resources across multiple
sites, through the Internet.
This paper is an extension of our previous conference pa-
per [8]. It gives a comprehensive examination of the pro-
posed SDN framework for ensuring the end-to-end delay
minimization of network services. Furthermore, we investi-
gate the optimal path selection in our SDN framework and
the behavior of the optimized SDN controller under such a
scenario with varying types and quantities of network traf-
fic.
3. DELAY MINIMIZATION METHODS
AND PATH SELECTION
A computer network supports and facilitates the transporta-
tion of data, ranging from medical information to multi-
media TV shows. The elevation of urgency or the sense
of prioritization of information has much consideration in
the design of a network and includes, but not limited to,
throughput, latency, and end-to-end delay. This study eval-
uates three methods of end-to-end delay: Traditional Ping,
Packet Probing, and Timestamp Recording with considera-
tions modeled from the previous three key factors. SDN pro-
vides two major contributors to the design and implementa-
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
31
Figure 1. A sample network topology with the
three methods of end-to-end delay measurement,
Traditional Ping, Packet Probing, and Timestamp
Recording. The visual representation of the three
methods is for adequate demonstration purposes
and do not represent the behavior of each scheme.
Figure 2. Delay is estimated based on ICMP packets
transmitted between two clients as represented by
the blue arrows. The measurements are taken based
on one-half the delay of ICMP.
tion of an infrastructure: global-view and programmability.
Leveraging the two previously listed contributors in the de-
velopment of end-to-end delay minimization approach, SDN
facilitates the measurements and analysis of the three meth-
ods described in this study.
3.1 Controlling a Network
A controller in an SDN environment projects and establishes
the final choice in determining path selection, optimizing a
network, and handling prioritized data. Other abilities and
tasks an SDN controller executes include, but not limited to,
the calculation of end-to-end delay and selecting the optimal
path in a complex network environment. The three meth-
ods of end-to-end delay measurement show a common trait
of packet flow analysis using either ICMP, LLDP, BDDP,
or statistical calculation in time stamping. The controller
in the SDN environment utilizes either one or a combina-
tion of multiple end-to-end delay measurement techniques
to efficiently perform the optimal path selection for network
communication and traffic direction. One challenge to ad-
dress in the analysis of each end-to-end delay measurement
is a comparison to the baseline value.
The solution to the challenge above is to build a non-optimi-
zed controller where the SDN system does not consider any
delay measurements, and all links in the network are treated
equally without considerations of traffic congestions, bottle-
necks, or bandwidth. Floodlight, the deployed open-source
SDN controller in our work, in its default configuration adop-
ts the behaviors of Packet Probing. The use of LLDP and
BDDP facilitates the discovery and latency measurement of
paths and links in a network. The non-optimized controller
removes the procedure to evaluate link latency and treats
all paths equally while utilizing the discovery processes to
determine and explore the network topology shape. Figure 1
shows a sample representation of an SDN infrastructure de-
ploying the three techniques of end-to-end delay measure-
ments evaluated in our paper.
3.1.1 The Traditional State Control:
Round-Trip-Time
ICMP or ping is a ubiquitous technique to measure the la-
tency between two given points on a network. The Round-
Trip-Time (RTT) is the measurement of latency, typically
in the form of milliseconds, where a sender records the nec-
essary time for a message to be sent to a recipient and then
returned with a particular ICMP response. Figure 1 demon-
strates an example configuration where two nodes ping one
another. The latency on the link between the two switch-
ing devices is defined as one-half the RTT (i.e., RT T /2).
Notably, one flaw in this configuration is the consideration
of bi-directional or full-duplex traffic for path selection and
that the latency measurement defines both directions for the
controller. Additionally, this technique does not consider the
latency between end-point devices and the switch, and that
halving the RTT value will be considered the end-to-end
delay.
Under Figure 2, nodes N1 and N2 are clients in a small net-
work topology, S1 and S2 are SDN switches, and Controller
is an SDN Controller managing S1 and S2. Within the fig-
ure, N1 transmits an ICMP message destined to N2. This
ICMP message is received by N2 after a series of packet
forwarding by S1 and S2. The ICMP message is then re-
turned to N1 by N2 through S2 and S1. Node N1 calcu-
lates the delay based on one half of the ICMP measurement
value. Although RTT using ICMP is a fundamental method
of measuring end-to-end delay, Packet Probing provides a
more accurate measurement through its use of LLDP and
BDDP.
3.1.2 The Common State Control: Packet Probing
Packet Probing is one of the more ubiquitous methods to
measure end-to-end delay as one of the essential charac-
teristics is the ability to identify the shape of a network
with latency measurements of each network path. Floodlight
adopts Packet Probing as its defacto method for link discov-
ery and path selection. In an SDN environment, Floodlight
jointly deploys both LLDP and BDDP for identifying net-
working links. Additionally, LLDP and BDDP use multicast
and broadcast respectively to identify neighboring network
devices from SDN switches.
In Figure 4, an SDN topology is depicted where nodes S1
and S2 are SDN switches, and Controller is an SDN Con-
troller. In the initial transmission, an LLDP or BDDP
packet is encapsulated into an OpenFlow Type 13 message
and forwarded to S1 for processing from Controller. Within
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
32
Figure 3. A visual representation of the experiments conducted on ExoGENI testbed for the proposed SDN
framework. All the nodes in this figure are instantiated across multiple hypervisors located in distant regions.
Dashed lines represent controller communication and they are individual links to the SDN controller from
their respective SDN switches for OpenFlow messages.
Figure 4. Delay is estimated based on Packet Prob-
ing techniques where measurements are indepen-
dent of network client activity.
S1, the OpenFlow Type 13 message is de-encapsulated, and
the LLDP or BDDP is sent out to S2. Depending on the
configuration of the SDN environment, the LLDP or BDDP
packet maybe dropped upon transmission at the POSTROU-
TING phase of the SDN switch. Netfilter hooks will show
the transmitted packet and logged for statistical analysis,
but the receiving device will not see the LLDP or BDDP.
Instead, OVS will retransmit the LLDP or BDDP packet
as a 0x8942 packet with a payload of the LLDP or BDDP
content. The receiving device, S2, will interpret LLDP or
BDDP messages and directly forward the information to the
Controller, encapsulated in an OpenFlow Type 10 message.
Delay is calculated based on the trip time of the entire pro-
cess from S1 to S2 as represented in equation (1).
Delay(S1, S 2) = TTotal −TS1/2−TS2/2 (1)
Additionally, delay measurements between Controller to S1
and Controller to S2 are subtracted from the trip time in
order to obtain the pure latency value of the network link.
To clarify our approach, Figure 5 expresses a packet diagram
of the Packet Probing approach.
As shown in Figure 5, an Optional Type-Length-Value (TLV)
is implemented for Packet Probing where epoch time is in-
serted into each LLDP packet. This epoch time is calcu-
lated from the perspective of the SDN-Controller, and the
packet will traverse through two SDN switching devices be-
fore returning to the SDN-Controller. Using this method,
we can obtain the inter-switch latency for each link. BDDP
will have a similar approach for a topology that have mixed
switching devices of SDN to non-SDN peripherals.
Within (1), TT otal is obtained through Packet Probing while
TS1and TS2is measured on the response of OpenFlow mes-
sages. Essentially, as the Controller and respective SDN
switching device communicates using OpenFlow messages,
TCP-based communication is established. Delay is mea-
sured based on the initial SYN flagged TCP message and
the responding SYN-ACK flagged TCP message. We divide
the value by two to obtain an estimated one-way delay mea-
surement on a bi-directional network path for the links TS1
and TS2. Upon subtracting TS1and TS2from TT otal, we can
obtain the inter-switch latency. BDDP is an asset in Flood-
light for identifying hybrid networking paths of both SDN
and non-SDN switching devices. Following a similar scheme
to LLDP, BDDP facilitates the discovery of external links,
or links that traverse over multiple physical network envi-
ronments. The joint configuration of LLDP and BDDP in
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
33
Figure 5. An LLDP packet with Packet Probing
implemented as an Optional TLV.
Figure 6. Estimating delay using Timestamp
Recording by examining network traffic. Delay is
calculated based on the quantity of traffic over a pe-
riod of time.
an SDN environment aids in constructing the map of a net-
work to calculate path selection optimally. Figure 1 demon-
strates an example of BDDP usage for physical switching
devices where a network path traverses through a non-SDN
region. Although Packet Probing utilizes existing protocols
to measure the delay between two switches, it is vulnerable
to security threats where interception of LLDP or BDDP can
occur. Unlike Packet Probing, Timestamp Recording does
not interact with end-user devices and therefore utilizes a
trusted system between the Controller and SDN device.
3.1.3 The Timestamping Control
To identify our approach of Timestamp Recording exami-
nation, Figure 6 shows an example topology that illustrates
the process. Herein, the delay between two SDN switch-
ing devices is calculated based on the time difference be-
tween the sending and receiving switch, S1 and S2, respec-
tively. To calculate such time measurement, we synchro-
nize all SDN switches using Network Time Protocol (NTP).
It is a challenge to synchronize time as a network topol-
ogy can be stitched within multiple regions and hence dif-
ferent time zones. Wide-area communication environments
at scale do not limit themselves to one localized area, and
can at times be in different time zones. Such characteris-
tics introduce problems with timezone where switching de-
vices can report varying time differences. With the intro-
duction of inter-connected regions, a network topology may
span across multiple time zones and report erroneously cal-
culated end-to-end delay measurements. The simple case to
correct this calculation error would be to synchronize each
system to UTC timezone, therefore, removing time shifting
errors while the more advanced case would be to factor time
differences, based on the origin of the transmitting device.
Within the figure, nodes N1 and N2 are clients, S1 and S2
represent SDN switches, and Controller is an SDN Con-
troller. As depicted, N1 transmits a message to N2, and
the flows are injected into their respective SDN switch for
forwarding purposes. As the flow is injected into an SDN
switch, the arrival and departure times (tRand tS) are taken
based on the number of packets, P1..n , transmitted over a
period of time between the two switching devices, S1 and
S2, as depicted in equation (2). For bi-directional traffic
(full duplex), the measurement is taken twice per network
link, one for each direction.
Delay(S, R ) = [tR−(tRZ −tSZ)] −tS(2)
W here :tR=1
n
n
X
k=1
tk
R, tS=1
n
n
X
k=1
tk
S
Under equation(2), tRand tSare the received and sent
timestamps at SDN switching devices per network link. De-
lay is measured where nis the number of packets on a
given network stream, TRZ and TS Z are timezone values
for the sending and receiving switching devices. Moreover,
the interval of time to examine latency of each link is pre-
established administratively. Specifically, to update the met-
ric of delay per link within the SDN environment, an inter-
val value in seconds will need to be set to measure the delay.
To implement such design, we will examine modifying key
components to our Floodlight-based SDN Controller with
regards to per link latency.
3.2 Path Selection: Dijkstra’s Algorithm
Path selection is a key component to end-to-end delay. To
select the optimal path for our SDN-based services, we uti-
lize Dijkstra’s algorithm. Under the approach, a network
topology has a finite number of combinations constructed
from Dijkstra’s algorithm. The calculation time to compute
the number of combinations can be strenuous to any de-
vice with regards to system resources and performance. To
improve the feasibility of minimizing end-to-end delay, we
use Dijkstra’s algorithm on a tuple approach. Under this
approach, only the top three paths will be selected to deter-
mine the optimal path and therefore reducing the amount
of system resources needed to calculate an entire network
at scale. To construct the tuple for Dijkstra, we will ini-
tially sort the set of all link combinations to the sum of
lowest latency value. For each new flow injected into the
SDN topology, Dijkstra will be recalculated based upon the
end-to-end delay measurement technique. Quality of Service
(QoS) plays a significant role in minimizing delay. When a
particular traffic flow is identified as a highest priority, the
traffic flow will be processed with the lowest latency.
3.3 Prioritizing Network Services
Inter-domain and local network communication present a
challenge when traffic of many natures, whether high (time
sensitive) or low priority, are mixed in a network. Enabling
QoS on suitable SDN devices provides an examination of
data communication origins with identification processes to
determine whether a particular traffic set should be con-
sidered time sensitive. The support of QoS has limitations
based on the version of OVS, and specifically the OpenFlow
protocol. The construction of a topology to evaluate the
three methods of end-to-end delay measurement this study
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
34
presents has considerations to factor time-sensitive commu-
nication.
4. EVALUATION
Figure 7. Wide-area communication at scale does
not limit itself to a single geographical region. GENI
testbed allows inter-network domains for inter-
region communication to aid in conducting real-
world experiments over public Internet traffic.
Figure 8. A comparative analysis of the non-
optimized network, Packet Probing, and Timestamp
Recording for node N4under the presented scenario.
Conducting a comprehensive experimental evaluation requir-
es the construction of a topology with three properties. The
outline of the properties includes, but not limited to, (1) an
environment with SDN switching and routing devices, (2) a
software module that emulates network traffic, and (3) a net-
work with multiple links and paths with changeable through-
put and loss. This study employs Global Environment for
Network Innovation (GENI) [6], a real-world federated, het-
erogeneous testbed solution. ExoGENI, a component within
GENI, provides the ability to construct a topology with the
previously listed factors with an additional benefit to allow
inter-domain network communication over the Internet to
address areas of realism.
To evaluate the end-to-end delay among multiple sites, we
selected ExoGENI nodes from Lexington, Kentucky, Clem-
son, South Carlina and Salt Lake City, Utah as shown in
Figure 7 and built using a stitched topology. The stitch-
ing feature in ExoGENI testbed allows connecting resources
provided by multiple aggregates, such as cities, into a coher-
ent network. The architectural design of the topology uses
Open vSwitch (OVS) that supports OpenFlow 1.3. We fur-
ther select Floodlight 1.2 as our SDN controller since it sup-
ports both physical and virtual OpenFlow switches, has the
capabilities to handle both OpenFlow and non-OpenFlow
networks and provides QoS features.
4.1 The Uncontrolled Network State
In our initial experiments, we conducted numerous tests
without any optimization or end-to-end delay algorithm im-
plemented to examine the resiliency of the network, during
scenarios of a massive burst of network traffic. We utilized
the designed non-optimized controller that was previously
described in Section 3.
For the baseline study of the non-optimized configuration,
Figure 10 demonstrates the intercommunication of two nodes
where one is a client, and the other is a server. Communica-
tion in this analysis was conducted using iPerf and hping3
as normal and burst traffic, respectively. At approximately
1.5 seconds after the initial communication with node N8,
a massive burst of traffic occurred with full link saturation.
This scenario resulted in two devices, N1 and N7, losing
communication in the non-optimized SDN environment. An
analysis of the communication loss identified that the net-
work becomes overburdened in congestion and that roughly
100.13 seconds after the initiation of the experiment was the
time approximation for the event above.
4.2 The Controlled State: End-to-End
Table 1. Percentage improvement of the average
end-to-end delay in our optimized framework in re-
gard to non-optimized framework for nodes N1,N2,
N4, and N6communicating with N8simultaneously.
Statistic N1N2N4N6
Mean 8.29 % 7.86 % 17.13 % 21.85 %
Median 9.33 % 7.75 % 18.40 % 20.75 %
The results of the evaluation of a non-optimized SDN con-
troller in the previous subsection showed that a network
without any optimization lead to communication failure.
Next, we examine and analyze the effectiveness of end-to-
end delay amongst Traditional Ping, Packet Probing, and
Timestamp Recording. Providing a fair comparison, we con-
ducted the same scenario once again, but with one method
implemented per execution. A deep investigation of the Tra-
ditional Ping, Packet Probing, and Timestamp Recording
demonstrated their effectiveness in combating the commu-
nication loss. Figure 9 illustrates cumulative distribution
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
35
Figure 9. A comparison of the three approaches where low priority nodes N1and N6and high priority nodes
N2and N4nodes communicate with N8. TS, PP, and Ping represents Timestamp Recording, Packet Probing,
and Traditional Ping, respectively.
Figure 10. Client N1communicated to server N8for
approximately six minutes. A scheduled event exe-
cuted a massive burst of normal traffic, transmitted
from N2,N4, and N7to the previously mentioned
server (N8). The loss of communication, as shown
by the discontinuity in the graph, was due to an
overly congested link in the infrastructure.
functions (CDFs) of the average results of each end-to-end
delay method under the scenario conducted from Section
4.1.
The analysis of the experiments projected that approximately
60 seconds after starting the experiment, the Traditional
Ping technique performed as expected during scenarios of
massive traffic burst for the situation where nodes N2 and
N4 transmitted prioritized traffic to server N8 while N6
communicated normal networking traffic to N8. The Tradi-
tional Ping approach showed concerning performance results
for node N6 where communication loss occurred due to net-
work congestion as projected by the discontinuity on the
graph. Node N1 presented a similar concern for commu-
nication loss, but through analysis, the event occurred due
to the recalculation of the end-to-end scheme. Moreover,
under the same experimental scenario discussed above, Ta-
ble 1 represents percentage improvements that reflect the
reduction in end-to-end delay provided by our optimized
SDN framework with regard to a non-optimized SDN en-
vironment. The table shows that the proposed SDN-based
solution reduces the end-to-end delay from 8.29% to 22 % for
the different nodes communicating with server N8. Finally,
Figure 8 depicts the comparisons of CDF between the non-
optimized SDN environment and three delay measurement
method. Table 2 shows the average results of each mea-
surement between each node and server N8. The obtained
results demonstrate that our proposed SDN framework can
deliver network traffic with significantly lower end-to-end
delay in comparison to the non-optimized SDN environment
and packet probing.
Table 2. An average end-to-end delay analysis over
a period of five minutes under the configuration
of mixed communication of prioritized and non-
prioritized traffic.
Client Non-Optimize Ping Packet Probe Timestamp
N1 4.33 ms 3.37 ms 3.22 ms 3.21 ms
N2 4.06 ms 2.78 ms 3.05 ms 2.41 ms
N4 4.29 ms 3.08 ms 3.18 ms 2.57 ms
N6 2.17 ms 3.19 ms 2.88 ms 2.83 ms
4.3 Examination of Dijkstra’s Algorithm
From the given results in previous subsections, Timestamp
Recording appeared to have the least impact (i.e. commu-
nication loss and end-to-end delay increase) to burst traf-
fic when an emergency occurred. Additionally, Timestamp
Recording was able to minimize end-to-end delay for nodes
transmitting emergency data with an overall lower latency
as previously described. To investigate in further detail on
the approach of minimizing end-to-end delay, we examine
Dijkstra’s algorithm during the event of the initial burst of
emergency data to see the effects of path selection. Under
Figure 11, we see the outcome of Dijkstra algorithm during
the initial burst where node N4 communicates with server
N8. As described in the figure, three distinguishable paths
are depicted (one, two, and three). Table 3 represents the
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
36
Figure 11. Implementation of Dijkstra’s algorithm
from N4to N8where we utilize only three paths in
the calculation. A visual representation of Dijkstra’s
algorithm where D1and D2contain an undefined
quantity of both emergency and normal traffic and
they are being transmitted to server N8.
latency of the network topology prior to N4 communication.
In Table 3, L1, and L8 are unknown latency measurements
from an end-user device to an OVS device. Additionally,
client nodes attached to L1 and L8 are considered edge
nodes and therefore, communication is limited to the sole
path (edge link) provided. As depicted, the transmission
and return latency are of the same value, and this is due to
the data type we transmitted (TCP). To better understand
the behavior of Dijkstra’s algorithm, we increase the amount
of traffic transmitted to 5GB of UDP data.
Although, using a combination of TCP and UDP did not
appear to influence the given network topology links in com-
parison to pure TCP-based communication. D1 showed sub-
tle latency differences where the transmitting and receiving
latencies were 17ms and 22ms, respectively. Additionally,
because there were minimal differences with regards to the
directionality of traffic, the calculation of Dijkstra had a
similar setup. Lastly, even though we only utilize UDP traf-
fic as a one direction method to examine latency per link,
normal switch management traffic such as LLDP, BDDP,
ARP, and other various protocols were still present within
our configured topology.
4.4 Quality-of-Service Examination
The end-to-end delay and traffic prioritization are key com-
ponents to QoS delivery. There are numerous ways to pro-
vide QoS for a network topology ranging from the Class of
Service (CoS) to the examination of the source and desti-
nation IP and MAC addresses. For our brief QoS analysis,
we utilize source addresses, destination addresses, and SDN
switch ports as our method to identify traffic for priority
Table 3. Delay measurements using Timestamp
Recording where node N4is transmitting traffic to
server N8during an emergency event. L1and L8are
stated UNKNOWN as they represent edge links in
our ExoGENI topology. We are unable to determine
the latency of edge links as the SDN Controller has
no communication to end nodes (i.e., user devices).
Lastly, the top three paths for Dijkstra were calcu-
lated and expressed at the bottom of the table.
Link Transmit Latency Return Latency
L1 UNKNOWN UNKNOWN
L2 52ms 52ms
L3 39ms 39ms
L4 25ms 25ms
L5 5ms 5ms
L6 15ms 15ms
L7 13ms 13ms
L8 UNKNOWN UNKNOWN
L3,4,7 77ms 77ms
L2,7 65ms 65ms
L3,5,6 59ms 59ms
Figure 12. N1and N2communicating to N8where
N2transmitted a massive burst of prioritized traffic
and N1transmitted normal traffic.
consideration. The following describes the parameters of
our QoS analysis:
Source Address Matching: The source address per SDN
flow should match a high priority traffic-based node. Nodes
may send a combination of high and low-priority traffic, and
to distinguish a difference, further matching is necessary.
The result of using a single approach is that all traffic would
be considered high priority.
Destination Address Matching: Given that network
communication may have a particular prioritization, depend-
ing on QoS configurations, traffic destined to a particular
system can be feasibly prioritized based on the destination
address field per an SDN flow.
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
37
Table 4. Timestamp Recording from node N4trans-
mitting to N8during the events of an emergency
where 5GB of data was transmitted using UDP and
TCP amongst other nodes.
Link Transmit Latency Return Latency
L1 UNKNOWN UNKNOWN
L2 34ms 34ms
L3 18ms 18ms
L4 65ms 65ms
L5 2ms 2ms
L6 3ms 3ms
L7 2ms 2ms
L8 UNKNOWN UNKNOWN
L3,4,7 85ms 85ms
L2,7 36ms 36ms
L3,5,6 23ms 23ms
SDN Switch Port Matching: Using switch port match-
ing as a way to identify traffic provides an additional verifi-
cation process in determining that a particular network com-
munication truly originates from a prioritized node. Given
a network port or interface that is known to only have a
critical system, high valued communication nodes, or an in-
frastructure dependent resource, a simple QoS policy can be
inserted to the network scheme to elevate the traffic commu-
nication.
Figure 12 presents our QoS examination for the Timestamp
Recording method where N2, the high priority traffic node,
was able to ascertain and maintain 30 ms less end-to-end
delay in comparison to node N1 that transmitted lower pri-
ority traffic.
5. CONCLUSIONS AND FUTURE WORK
In this paper, we have proposed a timely SDN-based frame-
work for handling network traffic in end-to-end communica-
tion systems that are typical of prioritized application ser-
vices in smart cities. As the end-to-end delay in the net-
work environment plays a key role in the successful delivery
of such services, we have presented efficient schemes that
identify routing paths with optimal (minimal) end-to-end
delay and prioritize network traffic depending on the prior-
ity of data transmitted. To better measure the end-to-end
delay within end-to-end entities, we carefully investigated
and examined the three techniques of calculating end-to-
end delay between two network-enabled entities, the Tra-
ditional Ping, Packet Probing, and Timestamp Recording.
We have implemented these three methods in an SDN frame-
work consisting of virtual networks across multiple sites on
ExoGENI through the Internet. Furthermore, to study path
selection for prioritized application services in this type of
communication networks, we have implemented Dijkstra’s
algorithm on an SDN controller and leveraged it with Times-
tamp Recording approach for an optimal path selection and
delay minimization.
Finally, We have evaluated and compared the efficiency of
the three studied techniques and our proposed framework vs.
a non-optimized SDN framework. Our experimental findings
have proved that our SDN-based framework is an effective
solution to handle burst-type network traffic where multi-
ple networking devices simultaneously communicate with a
varied data priority. In our future work, we will consider a
deep examination of different metrics of QoS policy such as
identifying the traffic of varied priority and address critical
security concerns that might affect the end-to-end QoS dis-
cussed in this paper. Moreover, we will expand our proposed
approach into a variety of applications in smart cities
6. ACKNOWLEDGMENTS
We acknowledge National Science Foundation (NSF) to par-
tially sponsor the work under grants #1633978, #1620871,
#1620862, #1620868, and BBN/GPO project #1936 via
an NSF/CNS grant. We also thank FC2 for its seed grant.
The views and conclusions contained herein are those of the
authors and should not be interpreted as necessarily repre-
senting the official policies, either expressed or implied of
NSF.
7. REFERENCES
[1] H. Ali and D. Dasgupta. Effects of time delays in the
electric power grid. In Proceedings of the International
Conference on Critical Infrastructure Protection,
pages 139–154. Springer, 2012.
[2] M. Amiri, H. Al Osman, S. Shirmohammadi, and
M. Abdallah. An SDN controller for delay and jitter
reduction in cloud gaming. In Proceedings of the 23rd
ACM international conference on Multimedia, pages
1043–1046. ACM, 2015.
[3] D. E. Bakken, R. E. Schantz, and R. D. Tucker. Smart
grid communications: QoS stovepipes or QoS
interoperability? Proceedings of Grid-Interop 2009,
pages 17–19, 2009.
[4] I. Baldine et al. ExoGENI: a multi-domain
infrastructure-as-a-service testbed. In Testbeds and
Research Infrastructure. Development of Networks and
Communities, volume 44 of LNCS, Social Informatics
and Telecommunications Engineering, pages 97–113.
Springer Berlin Heidelberg, 2012.
[5] M. F. Bari, S. R. Chowdhury, R. Ahmed, and
R. Boutaba. Policycop: An autonomic QoS policy
enforcement framework for software defined networks.
In Proceedings of the SDN For Future Networks and
Services (SDN4FNS), pages 1–7. IEEE, 2013.
[6] M. Berman, J. S. Chase, L. Landweber, A. Nakao,
M. Ott, D. Raychaudhuri, R. Ricci, and I. Seskar.
GENI: A federated testbed for innovative network
experiments. Computer Networks, 61:5–23, 2014.
[7] T. Chin, X. Mountrouidou, X. Li, and K. Xiong. An
sdn-supported collaborative approach for ddos
flooding detection and containment. In Proceedings of
the Premier International Conference for Military
Communications (Milcom). IEEE, 2015.
[8] T. Chin, M. Rahouti, and K. Xiong. End-to-end delay
minimization approaches using software-defined
networking. In Proceedings of the International
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
38
Conference on Research in Adaptive and Convergent
Systems, pages 184–189. ACM, 2017.
[9] H. E. Egilmez, S. T. Dane, K. T. Bagci, and A. M.
Tekalp. OpenQoS: An OpenFlow controller design for
multimedia delivery with end-to-end quality of service
over software-defined networks. In Proceedings of the
Signal & Information processing association annual
summit and conference (APSIPA ASC), 2012
Asia-Pacific, pages 1–8. IEEE.
[10] H. E. Egilmez, B. Gorkemli, A. M. Tekalp, and
S. Civanlar. Scalable video streaming over openflow
networks: An optimization framework for QoS
routing. In Proceedings of the 18th IEEE International
Conference on Image Processing (ICIP), pages
2241–2244. IEEE, 2011.
[11] M. Ghobadi, S. H. Yeganeh, and Y. Ganjali.
Rethinking end-to-end congestion control in
software-defined networks. In Proceeding of the
Workshop on Hot Topics in Networks, HotNets-XI,
pages 61–66, Redmond, WA, USA, October 2012.
ACM.
[12] S. Gorlatch, T. Humernbrum, and F. Glinka.
Improving QoS in real-time internet applications:
from best-effort to software-defined networks. In
Proceedings of the International Conference on
Computing, Networking and Communications (ICNC),
pages 189–193. IEEE, 2014.
[13] A. Ishimori, F. Farias, E. Cerqueira, and A. Abel´em.
Control of multiple packet schedulers for improving
QoS on OpenFlow/SDN networking. In Proceedings of
the Second European Workshop on Software Defined
Networks (EWSDN), pages 81–86. IEEE, 2013.
[14] M. Jarschel, T. Zinner, T. Hoßfeld, P. Tran-Gia, and
W. Kellerer. Interfaces, attributes, and use cases: A
compass for SDN. IEEE Communications Magazine,
52(6):210–217, 2014.
[15] N. McKeown, T. Anderson, H. Balakrishnan,
G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and
J. Turner. OpenFlow: enabling innovation in campus
networks. ACM SIGCOMM Computer
Communication Review, 38(2):69–74, 2008.
[16] H. Owens II and A. Durresi. Video over
software-defined networking (VSDN). Computer
Networks, Elsevier, 2015, pages 341–356.
[17] S. J. Vaughan-Nichols. OpenFlow: The next
generation of the network? Computer, 44(8):13–15,
2011.
[18] K. Xiong. Multiple priority customer service
guarantees in cluster computing. In International
Symposium on Parallel & Distributed Processing,
pages 1–12. IEEE, 2009.
[19] K. Xiong. Resource Optimization and Security for
Cloud Services. ISTE Ltd and John Wiley & Sons,
Inc., London SW19 4EU, UK and Hoboken, NJ 07030,
USA, 2014.
[20] K. Xiong. Secure network bandwidth provisioning for
quality of services (QoS) guarantees. In The 12th
International Conference on Ubiquitous Intelligence
and Computing and the 15th International Conference
on Autonomic and Trusted Computing and the 12th
International Conference on Scalable Computing and
Communications and Its Associated Workshops, pages
444–451. IEEE, 2015.
[21] K. Xiong and X. Chen. Ensuring cloud service
guarantees via service level agreement (SLA)-based
resource allocation. In Proceedings of the IIEEE 35th
International Conference on Distributed Computing
Systems Workshops (ICDCSW). IEEE, 2015.
[22] K. Xiong and M. Makati. Assessing end-to-end
performance and security in cloud computing. In
Proceedings of the Symposium on Applied Computing,
pages 405–410. ACM, 2017.
[23] K. Xiong and P. Ning. Cost-efficient and
attack-resilient approaches for state estimation in
power grids. In Proceedings of the 30th Annual ACM
Symposium on Applied Computing, pages 2192–2197.
ACM, 2015.
[24] K. Xiong and H. Perros. Computer resource
optimization for differentiated customer services. In
International Symposium on Modeling, Analysis, and
Simulation of Computer and Telecommunication
Systems, pages 226–238. IEEE, 2006.
[25] W. Zhao, D. Olshefski, and H. Schulzrinne. Internet
quality of service: An overview. Columbia University,
New York, Technical Report CUCS-003-00, 2000.
[26] J. Zheng, H. Xu, G. Chen, and H. Dai. Minimizing
transient congestion during network update in data
centers. In Proceedings of the International Conference
on Network Protocols, pages 1–10. IEEE, 2015.
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
39
ABOUT THE AUTHORS:
Tommy Chin received a B.S. degree in Applied Networking and System
Administration in 2013 and is currently pursuing an M.S. degree in Computing
Security at Rochester Institute of Technology. Tommy holds numerous industry
level certifications, awards, and achievements. His current research focuses on
Software-Defined Networking (SDN), Wireless Sensor Networks (WSN), and
network security.
Mohamed Rahouti received an M.S. degree in Statistics and is currently pursuing a
Ph.D. degree in Electrical Engineering at the University of South Florida. His
current research focuses on computer networking and security with applications to
smart cities.
Kaiqi Xiong is an Associate Professor at the University of South Florida. He
received his Ph.D. degree in Computer Science from the North Carolina State
University. Before returning to academia, he had worked in IT industry for several
years. He received the Best Demo Award at the 22nd GENI Engineering Conference
(GEC22) and the US Ignite Application Summit with his team in 2015. National
Science Foundation (NSF), NSF/BBN, Air Force Research Laboratory (AFRL),
Amazon AWS, Florida Center for Cybersecurity (FC2), and Office of Naval
Research (ONR) have recently supported my research. His research includes
security, networking, and data analytics with applications cyber-physical systems,
cloud computing, sensor networks, and the Internet of Things.
APPLIED COMPUTING REVIEW MAR. 2018, VOL. 18, NO. 1
40
