Content uploaded by Evangelos Markakis
Author content
All content in this area was uploaded by Evangelos Markakis on Mar 27, 2018
Content may be subject to copyright.
EXEGESIS: Extreme edge resources harvesting for a virtualized Fog
environment
Evangelos K. Markakis, Kimon Karras, Nikolaos Zotos, Anargyros Sideris, Theoharris Moysiadis, Angelo
Corsaro, George Alexiou, Charalabos Skianis, George Mastorakis, Constandinos X. Mavromoustakis,
Evangelos Pallis
Abstract –
Currently there is an active debate about how the existing cloud paradigm can cope with the volume,
variety and velocity of the data generated by the end devices (e.g. Internet of Things sensors)—it is expected
to have over 50 billion of these devices by 2020, which will create more than two Exabyte’s worth of data
each day. Additionally, the vast number of edge devices create a huge ocean of digital resources close to
the data source which, however, remain so far unexploited to their full extent. EXEGESIS proposes to
harness these unutilized resources via a three-layer architecture that encompasses the mist, fog and cloud.
The mist network is located at the very bottom, where interconnected objects (Internet of Things devices,
Small Servers, etc.) create neighborhoods of objects. This arrangement is enhanced by a virtual fog layer
which allows for dynamic, ad-hoc interconnections among the various neighborhoods. At the top layer
resides the cloud with its abundant resources that can also be included in one or more virtual fog
neighborhoods. Thus, this paper complements and leverages existing cloud architectures enabling them to
interact with this new edge-centric ecosystem of devices/resources, and benefit from the fact that critical
data are available where they can add the most value.
INTRODUCTION AND CONTEXT
Nowadays a lot of discussion is being going on regarding the way the cloud paradigm can cope with the
volume, variety and velocity of the data generated by the end devices (e.g. Internet of Things (IoT) sensors).
It is expected to have over 50 billion of these end-devices [1], currently referred to as the “τhings”, by 2020,
which will be creating more than two εxabyte’s worth of data each day. It is clear that shipping all of that
Evangelos K. Markakis, George Mastorakis and Evangelos Pallis are with the Technological Educational Institute of
Crete.
Kimon Karras, Nikolaos Zotos, Anargyros Sideris and Theoharris Moysiadis are with Future Intelligence Ltd.
Angelo Corsaro is with PrismTech Corp.
Charalabos Skianis is with the University of the Aegean.
Constandinos X. Mavromoustakis is with the University of Nicosia.
data to the cloud, processing and storing them there, as the current paradigm dictates, can run into significant
bottlenecks, in terms of latency and network capacity. On the other hand, it is hard to miss that the vast
number of end-devices, most of them utilizing some form of processing power, storage space and network
connectivity, could constitute a pristine “ocean” of digital resources, which could be harnessed and used to
address the bottlenecks of the current cloud paradigm by processing and storing data close to where they
were created.
In this context, EXEGESIS, building upon and extending existing concepts [3], [8] such us as the micro
datacenters, cloudlets, mobile edge computing (MEC) and fog computing [7], proposes a novel three
layered architecture that is able not only to reap the resources of the end-users’ devices but also couple them
to the cloud by providing a cross-layer orchestration platform able to deploy services that have a cloud and
mist component and to provide a distributed marketplace where these resources can be traded off by any
EXEGESIS stakeholder; a local authority in Athens, an Small Medium Enterprise (SME) in Madrid, a
corporation in Brussels. In this way, EXEGESIS envisages that it can steer new and innovative services and
process efficiencies not possible with cloud computing alone.
Figure 1. High level view of EXEGESIS concept
The EXEGESIS high level architecture is composed of three layers (Figure 1). At the very bottom, the
mist network is located, where interconnected objects (probes, sensors, cell phones, home appliance devices,
small servers, small cell controllers, etc.) create a neighborhood. This arrangement is enhanced by the virtual
Fog (vFog) layer which allows for dynamic, ad-hoc interconnections among the various neighborhoods
allowing sub-groupings called “suburbs” to be formed. At the top layer resides the conventional cloud with
its abundant resources that can also be included in one or more “suburbs” in order to provide compute
resources and facilitate the interconnection of the various vFog elements. In this context, EXEGESIS
complements and even leverages existing cloud architectures as it enables them to interact with this new
edge-centric ecosystem of devices/resources and benefit from the fact that critical data are available where
they can add the most value.
The key idea and challenge here is to be able to partition the three-layer infrastructure consisting of the
mist, vFog and cloud layers into logical networks whose membership can partially overlap with that of other
such logical networks and to be able to dynamically remold this partitioning to ensure optimal performance
and utilization of the available resources
Furthermore, EXEGESIS aims to enable business innovation via the deployment and use of suburb-
based marketplaces named “AGORAs”, stemming from a Greek word which means the place where all
social and economic activity take place. The “AGORA” is for EXEGESIS the system through which every
infrastructure/platform provider offers over the top (OTT) and on-demand accelerated
service/network/connectivity applications to the requesting entities.
In other words, EXEGESIS aims to radically reshape the mist, fog and cloud landscapes by merging
them into one coherent whole and then slicing and dicing that into logical entities in order to achieve optimal
performance and resource utilization.
BACKGROUND AND RELATED WORK
Concept
EXEGESIS, building on the concepts of “edge computing” [2], “frugality of resources” [5] and
“democratization of the digital economy” [6], envisages a future where the processing, storage and
networking resources of the devices residing at the edge of the network can be harnessed and integrated
seamlessly and dynamically in a flexible system architecture. In making this a reality, EXEGESIS, provides
the means for establishing virtual fogs; overlays of interconnected end-devices—which can be intertwined
with cloud resources—forming ad-hoc isles of connectivity and compute, setting the basis for a common
marketplace where services can easily be deployed across all layers.
The following sections describe the methodology that EXEGESIS follows in order to reach its objectives
as well as the technological aspects utilized for realizing these objectives
Technical Approach
EXEGESIS proposes a new interaction ecosystem composed of three layers. At the very bottom, the
mist layer is located, where interconnected objects create a neighborhood. This arrangement is enhanced by
the vFog layer which allows for dynamic, ad-hoc interconnections among various mist elements allowing
sub-groupings called “suburbs” to be formed. Cloud layer resources can also be included in a suburb in
order to provide resources and facilitate the interconnection of the various elements. The key idea here is to
be able to partition the three-layer infrastructure consisting of the mist, fog and cloud layers into logical
virtual networks whose membership can partially overlap with that of other vFog networks and to be able
to dynamically remold this partitioning to ensure optimal utilization of the available resources.
Mist for EXEGESIS is the unified extreme edge playground where a variety of end user — end user can
be also a company that utilizes EXEGESIS solutions — devices cooperate towards abstracting, in a common
virtual pool, their available resources and as such enable any legitimate entity to use these resources for
hosting a variety of compute and networking tasks. The EXEGESIS mist overlay “copies” the hybrid P2P's
approach where a peer can be "primus inter pares"
1
. In this context, the EXEGESIS mist network has two
classes of peers (see Figure 2), regular mist nodes (RMNs) and super mist nodes (SMNs).
A RMN can be any end device having at least some processing and communication capabilities that will
allow EXEGESIS to deploy its solution on it and thus transform the device to a fully operational EXEGESIS
mist node. An RMN is able to interact with its corresponding SMN, first to inform it about the device's
available resources and second to receive and carry out the assigned computational and/or networking tasks.
To that end, a special kind of software, called the vFog agent, runs on each RMN. An RMN can be any
physical or virtual entity having even a "pinch" of processing and communication capabilities.
An SMN plays two roles inside the EXEGESIS ecosystem, namely the role of the mist's intra-manager
and the role of the mist's envoy to the vFog orchestrator. As an intra-manager, an SMN:
oversees the formation of the mist network by performing operations such as the (de)registering
of Mist nodes.
queries the registered Mist nodes about their state and their available resources.
creates a logical topology of the mist network along with a virtual pool of the RMNs available
resources.
As an envoy, an SMN interacts with the vFog orchestrator towards:
(de)registering a Mist network to the vFog overlay.
1
First among equals
providing a "copy" of the SMNs virtual pool of resources, therefore enabling the vFog
orchestrator to have a clear image about the available resources across the whole vFog overlay.
mediating between vFog orchestrator and RMNs for reserving resources, assigning
computational tasks or even deploying network function virtualization infrastructure (NFVI)
elements.
Following hybrid P2P's paradigm, an SMN is elected from the currently running RMNs taking into
account several attributes like processing and memory capabilities, network capacity, power level/type
among others. Acknowledging that the uncontrolled participation of mist nodes in the election process could
pose security threats, EXEGESIS provides the means for “screening” the candidates list based on the
EXEGESIS stakeholder’s policies. The SMN is elected from the existing RMNs; it manages RMNs and it
is the point of contact to the vFog orchestrator.
Figure 2. Two vFog neighborhoods accommodating two mist networks each
The tremendous number and the vast heterogeneity of the devices living on the edge of the network,
poses a significant challenge for EXEGESIS towards forming manageable and efficiently operating mist
networks. To handle this challenge, EXEGESIS proposes the development and exploitation of a middleware
solution that will sit on top of each device’s operating system (OS). The middleware utilizes a southbound
application programming interface (API) for interacting with the OS and acquiring access to the device’s
actual resources and a northbound API for communicating with its vFog orchestrator. A hypervisor will be
exploited for deploying in containerized form—it reduces the system’s footprint and increases services
deploy ability—the RMN/SMN module and, if assigned from the vFog orchestrator, other software units
that carry out computational tasks or realize a service.
EXEGESIS proposes the idea of a vFog for managing the underlying mist networks and harnessing their
available resources. As the name implies a vFog assumes the operations of a conventional Fog network –
for example coordination of the fog nodes, provisioning of the available resources to third parties,
management operations– but is not deployed over dedicated equipment pre-installed at specific places; a
vFog lives on the top of mist networks as an overlaid virtual entity (see Figure 2). In these configurations,
the underlying SMNs will be the vFog nodes utilizing an election protocol to select, based on a set of
predefined criteria (e.g. processing capabilities, storage space, network capacity, power level, etc.), the SMN
that will undertake the role of the vFog orchestrator; the mind and heart of vFog's overlay. In a nutshell, the
vFog orchestrator will carry out the following key tasks:
Perform the vertical managerial operations needed to form and maintain the vFog overlay network.
Query the underlying mist nodes for available resources and create an abstract pool of them.
Provide information about the available resources to any authorized third-party (including the
Agora)
Handle horizontal communication operations (e.g. with other vFogs and/or conventional fogs)
Exchange data with any clouds with which it belongs to the same suburb.
Accept and forward requests for computational tasks, storage space and deployment of services to
the vFog nodes based on the needed and the available resources.
Deploy the Agora across the vFog network
One of the key issues that EXEGESIS attempts to tackle is to stem the tide of data flowing into and out
of the cloud. This is done by injecting SMNs into the vFog network which have increased processing
capabilities. These nodes will then expose their resources to the orchestration environment so they can be
used for pre-processing and filtering of data. That processing might lead to direct decision making or to a
whittled down version of it being uploaded to the cloud for further elaboration. At the core of this process
are heterogeneous, programmable logic-based nodes, which are located in the vFog network and which will
be used both for processing and for vFog suburb management. Programmable logic was selected because it
offers the critical combination of high performance, low power and complete flexibility which is necessary
to successfully meet the challenges of this role.
A heterogeneous vFog node within the context of EXEGESIS will consist of a field programmable gate
array (FPGA) System-on-Chip (SoC), which is an integrated circuit that combines processors,
programmable fabric and, potentially, additional logic. This combination allows us to optimally balance the
task load by allowing the processors to handle control-dominated tasks, like managing a vFog network and
delegating all compute-intensive tasks to the programmable logic. To accomplish this, the programmable
fabric needs to be virtualized so that the orchestration environment can deploy the appropriate application
on it at any given time. This is accomplished by executing cloud software on the processors of the FPGA
SoC which, together with the specialized hardware, enables the deployment of hardware virtual machines
on the programmable logic.
Abstraction of Resources in EXEGESIS
Starting at the mist layer (see Figure 3a), each RMN, during its registration process or upon a status
update, informs the SMN about the amount and type of physical resources it is willing to provide to the
EXEGESIS platform. The SMN in turn abstracts this information towards constructing a virtual resources
pool aggregating the physical resources of all the mist network nodes. Following the same paradigm, each
SMN after registering as a vFog node delegates information about its virtual resources pool to the vFog
orchestrator. At the same time, the vFog orchestrator can request and bind, if needed, more resources from
a conventional cloud. In this way, the vFog orchestrator forges a new virtual pool that holds in abstracted
form the physical resources across the whole vFog network.
Figure 3. a) Abstraction of resources in EXEGESIS b) Deployment framework for tasks and
services
Deployment of Services and Tasks in EXEGESIS
EXEGESIS will deploy services and perform computational tasks following a hybrid operational
scheme (see Figure 3b). In such a scheme, the vFog orchestrators can receive the requests for computational
tasks and service deployment. Upon that, the orchestrator based on the vFog’s available resources and
policies and also by taking into account the incoming task/service requirements can assign each task or
service to one or more vFog nodes (including itself if appropriate). In doing so, the orchestrator will utilize
and extend existing work to optimize task allocation [9], [10]. In turn, each vFog node passes the request
to its SMN module and based on the mist’s resources and the assigned operation’s requirements forwards
the tasks to itself and also, if needed, to the appropriate RMNs. It is noted here that if a task exceeds the
capacity of a vFog the orchestrator can forward the task to another vFog or assign it to cloud computing
resources. EXEGESIS’ deployment framework has segmentation of tasks and services at its core. In this
way, barring any security policies or specific task requirements, EXEGESIS can optimally fragment and
distribute tasks to resources as required to ensure that performance targets are met.
USE CASES
Security cameras, mobile phones, machine sensors, environmental sensors, and so on are just a few of
the items in daily use that create data that can be mined and analyzed. Add to it the data created at smart
cities, manufacturing plants, financial institutions, oil and gas drilling platforms, pipelines and processing
plants, and it’s not hard to understand that the deluge of streaming and IoT sensor data can — and will —
very quickly overwhelm today’s traditional data analytics tools. Organizations are beginning to look to edge
computing as the answer. Edge computing exploits vFog and mist and promotes data thinning at the edge
that can dramatically reduce the amount of data that needs to be transmitted to a data center or cloud
infrastructure. Without having to move unnecessary data to a central location, analytics or distributed
processes at the edge can simplify and drastically speed up analysis while also cutting costs. This drastic
shift in data processing paradigm propounded in EXEGESIS can be utilized in many, diverse use cases. The
proposed concept thus includes and investigates two concrete use cases where the proposed architecture can
prove to be a game changer compared to the currently available infrastructure. These use cases among others
are illustrated on Figure 4, which demonstrates one possible example of an EXEGESIS architectural
configuration where the four scenarios presented in the following sections are served by three vFog suburbs,
each with its own mist node neighborhood. All three suburbs share a common cloud infrastructure, while
each use case runs different tasks that are executed on their respective suburbs.
Figure 4. EXEGESIS use cases playground
Enabling and Enhancing Services for Smart Cities
Cameras are ubiquitous in modern cities and they can be used for various purposes, among which is
traffic management and surveillance. Both of these applications can benefit from acceleration in the form
of advanced image processing but require that different algorithms be executed (e.g. traffic management
requires that the number of cars per lane or the number of cars violating traffic laws are counted whereas
surveillance demands that specific individual must be identified).
The smart city is going to be one of the major revolutions of the coming decades, with large urban areas,
under ever-increasing pressure to accommodate a busy, fast-paced life for their citizens, turning to the
Internet of Things to optimize the use of their infrastructure and thus save on cost and enable new services.
This entails everything from smart lightning, smart water supply, smart security and others.
There are two issues where today’s architecture is lacking: The reuse of existing infrastructure and the
complexity in implementing data analysis solutions over that infrastructure. The former means that a set of
input devices, say cameras in this scenario, is installed in order to be used only for one function, for instance
traffic monitoring. That function can’t be changed unless the infrastructure itself is physically altered,
replaced or duplicated. The latter refers to the fact that retrieving the data from the input devices, analyzing,
reaching a decision and applying that decision is prohibitively slow and complex since all city infrastructure
today is purpose built.
The architecture proposed in this paper can solve both issues by creating two separate fog segments both
of which share the same FPGA-accelerated node, through which the data pass and which performs the
appropriate analysis. The orchestrator platform makes sure the accelerated node executes the required
functionality at any given time. The switch between the two tasks can be performed very swiftly which will
allow the node to perform both tasks seemingly at the same time much like a typical central processing unit
(CPU) appears to parallelize thread execution. The results of this analysis can then be either sent on for
further processing (e.g. after identifying suspicious activity) to the cloud or trigger automatic reactions in
other systems (e.g. manipulating traffic signals when detecting an accident and notifying emergency
services automatically).
Even within the narrower confines of a smart traffic management, fog computing improves the
performance of the application in terms of response time and bandwidth consumption. A smart traffic
management system can be realized by a set of stream queries executing on data generated by sensors
deployed throughout the city. Typical examples of such queries are real time calculations of congestion (for
route planning), or detection of traffic incidents. One possible case study, further elaborated on later in this
paepr, could compare the performance of a DETECT_TRAFFIC_INCIDENT query on fog infrastructure
[4] vs. the typical cloud implementation. In the query, the sensors deployed on roads send the speed of each
crossing vehicle to the query processing engine. The operator Average Speed Calculation calculates the
average speed of the vehicles from the sensor readings over a given time frame and sends this information
to the next operator. The operator Congestion Calculation calculates the level of congestion in each lane
based on the average speed of vehicles in that lane. The operator Incident Detection, based on the average
level of congestion, detects whether an incident has occurred or not. This process will be implemented and
executed on both fog- as well as cloud-based stream query processing engine, which will highlight the faster
response times and bandwidth savings offered by the fog-based alternative.
Smart Industrial Automation
The new trend in automation is that of virtualizing as much as possible the operational technologies
(OT) side of the system over contemporary IT infrastructure. The idea is simple, as virtual machines have
virtualized hardware in IT, the automation industry is trying to virtualize OT hardware such as
programmable logic controllers and run them over, more or less, traditional IT infrastructure.
The automation industry has been challenged for several years by the difference in innovation cycles
and obsolescence rate existing between the OT and the IT. The result of this divergence in change rates has
left the automation floor replete with obsolete IT technologies that have often introduced security breaches
and that in general reduce the productivity and usability of the entire system.
Fog and mist computing has been identified as the most natural approach to leverage the benefits of
functions virtualization while maintaining the performance constraints typical of OT systems. This however
is one side of the coin as companies also like to leverage the advantage of the cloud, namely large storage
and massive data analytics to identify issues and bottlenecks in production and flesh them out.
The EXEGESIS platform provides the ideal deployment target for software defined automation as it can
enable (1) mist computing to address the deployment and management of virtualized OT functions and
services over industrial hardware, and (2) fog computing to address the consolidation of higher level control
and analytics on more computationally capable hardware deployed on the edge of the system.
PRELIMINARY EVALUATION
This section provides an initial investigation into how the EXEGESIS edge compute paradigm
influences the amount of data flowing throughout a network. This is accomplished by simulating a simple
scenario similar to the traffic camera use case described in the previous section. In order to perform the
evaluation we use an open source fog environment simulator called iFogSim [11]. We tested three separate
scenarios, all of them comprising a camera which collects information, a programmable-logic accelerated
gateway device which connects the camera to the cloud, an actuator that receives commands after analysis
of the camera data and performs the appropriate actions and finally the cloud itself as shown on Figure 5:
In the first scenario the camera input stream is forwarded through the gateway to the cloud,
which performs the analysis and decision making and returns the decision to the actuator. This
scenario is most akin to the current paradigm.
The second scenario performs motion detection in the fog using the gateway device but sends
the clip to the cloud for detailed analysis and decision making, representing a middle ground
between a pure cloud and a pure edge approach.
The third scenario implements all the processing including motion detection, analysis and
decision making at the edge on the gateway device and only sends a notification of actions taken
to the cloud.
Figure 5. Overview of the simulated scenario
We evaluate two important parameters for all three scenarios. The first is normalized network usage (Figure
6a) and the second is the energy consumption for the entire system (Figure 6b).
Figure 6. Simulation results showing: a) normalized network usage and b) system energy consumption
It is plainly evident that the edge compute variant (scenario 3) is clearly superior in both metrics. Energy
usage reduction is to be attributed to the advantages of using programmable logic to perform the
computation at the edge but also at the constrained network traffic which also factors into energy use.
Network traffic is whittled down by performing all the processing close to the source and only sending a
small action report to the cloud instead of an entire camera stream. These results underpin the claim that the
EXEGESIS architecture can yield important potential benefits in multiple areas if realized at scale.
CONCLUSIONS
Future 5G networks are being viewed as the key technology that will allow for the realization of a "hyper-
connected society" where billions of IoT devices will be able to exchange data and offer/receive services at
a high quality of service (QoS) level. Towards this, 5G aims to support high data speed at the networks'
edges (1-10 Gb/s) and achieve ultra-low end to end latency (~1ms); however, these alone may not be
enough, especially with highly heterogeneous and fragmented network environments, a vast number and
huge variety of the devices residing at the network edges and the colossal amount of generated data which
are slowly coming to the foreground. To overcome this, EXEGESIS exploits and advances the fog and mist
paradigms to propose a beyond 5G ecosystem where heterogeneous fixed and mobile edge nodes (e.g. home
gateways, small cells, smart phones, SME Servers, IoT devices, Vehicles) will form an archipelago of
interconnected islands of resources (e.g. storage, computing, network) where each island can be viewed as
the successor of a small-cell and the archipelago as the evolution of the macro-cell. A preliminary
simulation-based investigation hinted at the significant benefits that can be derived from moving to the
edge-centric EXEGESIS architecture. Future work will involve the implementation of a real-life prototype
and the validation of the EXEGESIS paradigm in real-life scenarios.
REFERENCES
[1] Fog Computing and the Internet of Things: Extend the Cloud to Where the Things
Arehttp://www.cisco.com/c/dam/en_us/solutions/trends/iot/docs/computing-overview.pdf Retrieved Mar. 2016
Retrieved July 2016
[2] Chiang, Mung. "Fog networking: An overview on research opportunities." arXiv preprint arXiv:1601.00835
(2016).
[3] Klas, G.I., 2016. Edge Cloud to Cloud Integration for IoT.
[4] Y. Nikoloudakis, S. Panagiotakis, E. Markakis, E. Pallis, G. Mastorakis, C. X. Mavromoustakis and C. Dobre. "A
Fog-based Emergency System for Smart Enhanced Living Environments." IEEE Cloud Computing magazine to
be published Nov./Dec. 2016.
[5] Vaquero, L. M., and Rodero-Merino, L. (2014). Finding your way in the fog: Towards a comprehensive definition
of fog computing. ACM SIGCOMM Computer Communication Review, 44(5), 27-32.
[6] https://ec.europa.eu/digital-single-market/en/digital-single-market Retrieved July 2016
[7] http://www.openfogconsortium.org/news Retrieved July 2016
[8] A. Poenaru, R. Istrate and F. Pop “AFT: Adaptive and fault tolerant peer-to-peer overlay – A user-centric solution
for data sharing”, In Press, Future Generation Computer Systems, May 2016
[9] A. Sfrent, F. Pop “Asymptotic scheduling for many task computing in Big Data platforms”, Information Sciences
Journal, Vol. 319, pp. 71-91, Oct. 2015
[10] J.F. Riera et al., "TeNOR: Steps towards an orchestration platform for multi-PoP NFV deployment," 2016 IEEE
NetSoft Conference and Workshops (NetSoft), Seoul, 2016, pp. 243-250.
doi: 10.1109/NETSOFT.2016.7502419
[11] H. Gupta, A.V. Dastjerdi, S. K. Ghosh, R. Buyya, “iFogSim: A Toolkit for Modelling and Simulation of Resource
Management Techniques in Internet of Things, Edge and Fog Computing Environments”, CoRR, vol.
abs/1606.02007, June 2016