Conference PaperPDF Available

Hierarchical Video Surveillance Architecture: A Chassis for Video Big Data Analytics and Exploration

Authors:

Abstract and Figures

There is increasing reliance on video surveillance systems for systematic derivation, analysis and interpretation of the data needed for predicting, planning, evaluating and implementing public safety. This is evident from the massive number of surveillance cameras deployed across public locations. For example, in July 2013, the British Security Industry Association (BSIA) reported that over 4 million CCTV cameras had been installed in Britain alone. The BSIA also reveal that only 1.5% of these are state owned. In this paper, we propose a framework that allows access to data from privately owned cameras, with the aim of increasing the efficiency and accuracy of public safety planning, security activities, and decision support systems that are based on video integrated surveillance systems. The accuracy of results obtained from government-owned public safety infrastructure would improve greatly if privately owned surveillance systems 'expose' relevant video-generated metadata events, such as triggered alerts and also permit query of a metadata repository. Subsequently, a police officer, for example, with an appropriate level of system permission can query unified video systems across a large geographical area such as a city or a country to predict the location of an interesting entity, such as a pedestrian or a vehicle. This becomes possible with our proposed novel hierarchical architecture, the Fused Video Surveillance Architecture (FVSA). At the high level, FVSA comprises of a hardware framework that is supported by a multi-layer abstraction software interface. It presents video surveillance systems as an adapted computational grid of intelligent services, which is integration-enabled to communicate with other compatible systems in the Internet of Things (IoT).
Content may be subject to copyright.
Hierarchical Video Surveillance Architecture -
A Chassis for Video Big Data Analytics and Exploration
Sola O. Ajiboye*, Philip Birch, Christopher Chatwin, Rupert Young
Department of Engineering and Design
University of Sussex, Falmer-Brighton, United Kingdom
ABSTRACT
There is increasing reliance on video surveillance systems for systematic derivation, analysis and interpretation of the
data needed for predicting, planning, evaluating and implementing public safety. This is evident from the massive
number of surveillance cameras deployed across public locations. For example, in July 2013, the British Security
Industry Association (BSIA) reported that over 4 million CCTV cameras had been installed in Britain alone. The BSIA
also reveal that only 1.5% of these are state owned. In this paper, we propose a framework that allows access to data
from privately owned cameras, with the aim of increasing the efficiency and accuracy of public safety planning, security
activities, and decision support systems that are based on video integrated surveillance systems.
The accuracy of results obtained from government-owned public safety infrastructure would improve greatly if privately
owned surveillance systems ‘exposerelevant video-generated metadata events, such as triggered alerts and also permit
query of a metadata repository. Subsequently, a police officer, for example, with an appropriate level of system
permission can query unified video systems across a large geographical area such as a city or a country to predict the
location of an interesting entity, such as a pedestrian or a vehicle. This becomes possible with our proposed novel
hierarchical architecture, the Fused Video Surveillance Architecture (FVSA). At the high level, FVSA comprises of a
hardware framework that is supported by a multi-layer abstraction software interface. It presents video surveillance
systems as an adapted computational grid of intelligent services, which is integration-enabled to communicate with other
compatible systems in the Internet of Things (IoT).
1 INTRODUCTION
Video surveillance systems capture and utilise data that will systematically predict, plan, evaluate and implement the
protection of citizens and properties in both the private and public domains. In most cases, the cameras act as a physical
deterrent but their data provide undeniable evidence in identifying and prosecuting offenders. It is common to install a
significant number of surveillance systems in important public places. Because surveillance video often contains
sensitive information and peoples’ identity, it is imperative to manage and protect video surveillance systems and their
data from unsolicited access.
Nonetheless, metadata generated from the surveillance systems can provide meaningful information without revealing
the full identity of captured objectsmetadata analytics is beneficial to all concerned parties. For organisations that have
video surveillance systems in multiple locations, unifying their systems will reduce the total cost of ownership, improve
scalability, and enhance maintenance since all systems have been unified into a single framework that can be managed
from a single point. For public safety organisations such as the police, it provides a means for leveraging privately
owned surveillance systems in the planning, prediction and investigation of crime.
In this paper, we propose a novel framework that supports automated generation of surveillance metadata and a
controlled access to the metadata from any permitted system, with the aim of improving the accuracy of security alerts,
public safety planning, and decision support systems that are based on state-owned video surveillance systems. Existing
research into video metadata has focused on the generation and accessing of metadata by the administrative owner of the
system. Our solution, the FVSA presents video surveillance systems as an adapted computational grid of intelligent
services, which is integration-enabled to communicate with other compatible systems in the Internet of Things (IoT).
* sola.ajiboye@sussex.ac.uk
Now we will attempt to define computational grid, Internet of Things and, a unified system. Computational grid is a term
used to describe a large-scale computing environment where high-powered intelligent devices and services (such as
computers, storage services, sensor devices) are integrated to communicate for the purpose of leveraging their capability
to mutually increase efficiency in terms of processing power, speed and input capacity. The computing resources of a
computational grid are usually distributed across different geographic locations, with independent administrative
ownership and management [2][3]. Internet of Things is a term that is popularly used to describe the ability to access
features and administration of a digital device (or system) over the Internet - it describes virtual representation of
uniquely identifiable devices in an internet-like architecture [4][5]. Lastly, this paper describes a unified system as the
result when independent systems provide interfaces for sharing limited information. The administrative ownership and
management of unified systems are independent.
A notable implementation of a computational grid based on the IoT is smart cities, which is a complex system
comprising several unrelated lifeline services such as environmental information system, smart energy grid, travel
information, waste management, urban planning, smart meters, emergency response, and smart events, which are being
integrated across a common framework, (usually by implementing big data technology stack) [6] [7]. However, despite
progressive trends of integrating systems across industries, as in smart city, video surveillance systems are still chiefly
deployed and administered as standalone systems. Video data originates from each surveillance camera in large volumes
without means to aggregately explore the embedded information. This is mainly because of complexities that are
technical, financial, socio-cultural, security and ethically inclined, such as the following:
Data protection owners of video surveillance systems have a sense of responsibility to protect the privacy of
the people captured in their data.
Data ownership fear of loss of full ownership and/or control over data if shared outside their own network
facilities.
Heavy cost and investment - surveillance systems were usually installed into the building structure; replacement
may disrupt many other services, the financial cost can seem unrealistic or unreasonable.
System incompatibility based on manufacturer/vendor configuration and video encoding, video from each
camera has a format that does not necessarily make it readily compatible with video from another camera.
Unprofitable bandwidth usagecontinuous and consecutive transmission of video by several cameras across
the network, where many video frames may not contain interesting events.
This paper presents and describes how we resolve the complexities described above. The rest of this paper is organised
as follows - section 2 reviews existing progress in improving accessibility to video surveillance data, focusing on current
state of the art. Section 3 outlines our assumptions, goals and design considerations while section 4 describes our
proposed architecture, the FVSA. In section 5, we suggest a sample implementation of the FVSA in a smart city
network. The last section concludes this paper - we discussed relevance, strengths and envisaged challenges of our
proposition and future direction for video surveillance systems based on our proposition.
2 RELATED WORK
High-end NVRs are already equipped with fast video processing capabilities. For example the BW® NVR5216-P (with
16 channels) runs a dualcore CPU and ample buffer memory allocation. It ships with surveillance applications and
services including email service, intruder detection and alert generation but these NVRs are predominantly isolated
systems, serving as an intelligent hub for all connected cameras. In our proposed model, multiple intelligent NVRs can
be connected to jointly make up a surveillance network.
Intelligent data storage systems have been suggested for video surveillance systems with some capable of compressing
the data before storing it [8]. In another work, Dey et al. proposed a solution capable of continuous I/O manipulations,
read/write mix, random vs. sequential access with supporting variety of input sources [7]. Others have suggested storing
video data in the cloud where growth becomes elastic and affordable [9]. However, while cloud storage is profitable and
realistic solutions in most cases for extremely sensitive and/or massive data environments such as defence, cloud storage
is not an option. As mentioned earlier, video from several surveillance cameras would consume massive bandwidth and
storage resources, and the video data can be highly sensitive. It would appear beneficial to persist video surveillance data
within the local network with support for accessibility via a cloud based application layer.
Other notable works include metadata generation and analysis of the internal processing of surveillance systems. The
works of Dian et. al. focused on the internal transactions in a video surveillance system including remote play, request
and response flow [10]. Several works involved the systematic approaches to designing, deploying and implementing
automated and event-based metadata from video surveillance systems including ontology and validation of events
systems [11][12][13][14]. Metadata persists abstracted structures and content that users can query to retrieve meaningful
information such as event detection and object tracking. Metadata can be queried independently of the video images -
this can technically solve the problem of data protection.
The FVSA is established on the reality of video metadata with access authorisation implemented, surveillance systems
can expose aspects of metadata. The exposed data can solely provide means for matching or comparing interesting
events, making the data useful beyond the political and economic boundaries of the system owners and simultaneously
protecting the privacy of the people in the video. A similar concept has been implemented in health informatics where
patients’ personal health records are de-identified and released for research the de-identified data can be re-identified
in the future for comparative analyticsthe process is termed pseudonymisation [15].
3 DESIGN GOALS AND ASSUMPTIONS
We provide justification and reasoning for the design of the FVSA: in section 3.1, we briefly review the state of the art in
video surveillance architectures; in sections 3.2, we discuss our aims and objectives while we discuss our design
considerations and assumptions in section 3.3.
3.1 Current Systems
It is noted that current video surveillance architectures have been successful in the sense that they deter vandalism and
provide a level of security to their administrative owners/managers [10] [16]. Figure 1a below is a common process flow
in video surveillance systems. It shows that anyone with access to the computer screen or TV can view data from any
camera on the network. A typical business model places a security officer in front of multiple screens where the officer
attentively monitors video from the cameras in order to detect, investigate and raise alarms in the event of unwanted or
unexpected scenes. Some of these systems provide the capability to watch real-time video from any camera on the
network permission to view the data is normally assumed since only authorised officers have physical access to the
CCTV rooms. In recent years, as mentioned above in section 2, some of these systems are configurable to trigger alarms
by sending email or SMS in the event of unwanted or unexpected events.
(a) (b)
Figure 1: Process flow for streaming video on a surveillance system (a) common process Flow in a current Surveillance
Systems (b) process flow model in FVSA. In 1a, a user must be located in the control room to stream video from any camera on
the network. In 1b, the user can stream video from any device running the system portal, which we described in section 4.
3.2 Design Goals
Our fundamental objective in this paper is to optimise the video surveillance systems, with a view to improving the
quality and accuracy of information derived from them. The FVSA aims to analyse the events from video metadata as
they are generated from cameras on the network. It provides authentication and authorisation to ensure that only
permitted users can access the system where each user only has access as appropriate for his/her role. For example, while
a security officer in a train station has been granted permission to view all surveillance data including real-time video, a
police officer, may only have access to alerts that are triggered from the station. Similarly, a permitted police officer is
conceptually aware of all video surveillance systems in town (through the directory server in section 4) and can seek
permission to query them.
Figure 2 below is a map of the areas surrounding University of Sussex, UK. It is a page from the system’s application
portal, as seen by a city police officer using the FVSA. The map shows the FVSA deployed at four locations: a
university campus, a stadium, Southern Water, and a train station. A city police officer has selected to view full details of
the element of the Sussex FVSA system. An overview of the FVSA is provided in section 4 below.
Figure 2: Topology of the video surveillance systems in a City A Conceptual Police View
As noted earlier, surveillance data is the property and responsibility of the system owner. However a safety officer can
be granted limited permission (time-limited or access-limited) to stream video data, which can help towards an
investigation. Our proposal seems fit for purpose when deployed as a component of the bigger network such as a smart
city. Our goals revolve around the need to optimise the video surveillance systems as technology advances towards
aggregated analytics in the sense of the IoT, smart city, and hierarchical communications - we explain this further in
section 5. Summarily, a video surveillance system based on FVSA will satisfy the following requirements:
To reduce the cost of investigationthe police currently appeal for evidence from the public when investigating
incidents. The FVSA can make data readily available for such investigations, so police can automatically query
any ‘open’ video surveillance systems to build up evidence.
To unify the data mining interface of independent video surveillance systems through a robust API.
Surveillance system can interoperate in existing computational grids system, such as in a smart city or Cisco
Service-Oriented Network Architecture (SONA) [17].
Potential integration point for further sources of surveillance data such as satellite images, social media, which
can provide useful information.
To increase the accuracy of results obtained by public safety departments while the owners of independent
surveillance system still protects their ‘real’ video data.
Autonomous and continuous identification, tracking and investigation of objects from any camera on the
network. And to generate statistical information for informed decision-making
Apply a level of authorisation and authentication on the data to prevent fraudulent access.
Perform high data compression on the video data so they are cheaper to store for a reasonable length of time.
3.3 Considerations and Assumptions
Our main assumptions are highlighted in Fig. 3 below:
Public safety departments will be interested in using video from privately owned surveillance systems.
We assume that current video systems can be preserved while the new architecture is implemented. However a
new video surveillance system will benefit immensely from this new structure.
We assume that owners or managers of CCTV systems will find our proposal more profitable and more
beneficial.
We assume cameras are unintelligent recording device; so all processing is achieved within the i-NVR.
4 THE FUSED VIDEO SURVEILLANCE ARCHITECTURE
Figure 3: High Level Conceptual model of the FVSA, with system services in modular view. In practice, some of the modules
depicted are merged for example, the web services, metadata server (excluding storage), and queue services are all installed
on the analytics server, which is ideally an implementation of a big data platform such as the Apache Hadoop platform.
4.1 Overview of the FVSA
Figure 3 above is a high-level architecture of the FVSA - it presents the following modules (i) cameras, (ii) intelligent
Network Video Recorders (i-NVR), (iii) a queue service, (iv) a metadata server (MDS), (v) a metadata warehouse, (vi)
an analytics server (vii) web services (viii) a global directory server (ix) user computer system. The operation of each
module is explained below. It is worth noting that the framework described above can be set up flexibly, depending on
the number of installed cameras and budget. If we consider the case of a small storeowner who requires only 1 camera
the camera can be equipped to perform the functions of an i-NVR in addition to capturing objects.
Camera Farm
The number of cameras on a system can be 1 or several thousand cameras. In small systems comprising only few
cameras, an intelligent camera can perform the combined operations of a simple camera plus an i-NVR. However in
large systems, all video processing can be achieved at the i-NVR, while unintelligent but high-resolution cameras can be
used to capture data. System administrator can configure several cameras onto the same surveillance network even when
they are deployed in different geographical locations, as in different cities/countries. For an organisation with branches
across various cities and/or countries, the FVSA can be leveraged to administer all the CCTV systems from all location.
This can be achieved by setting up the i-NVR hierarchically as described in the next section.
Analytics Server
The analytics server is responsible for analytics and exploration of the metadata, it is responsible for running queries,
generating trends, alerts, predicting future events, based on learning of earlier events. This system is ideally an
implementation of a reliable Big Data platform such as an Apache Hadoop stack. A Big Data platform can be deployed
on commodity computers, so that the cost of hardware can be kept low for smaller systems, with ease of scalability for
larger specs. It hosts compatible database engines/solution for storing and managing the metadata.
Storage (Video Storage, Metadata Database and Warehouse)
The intelligent video storage is empowered to transiently compress, decompress, and archive video data. It compresses
data before persisting it for as long as configured but it can decompress and transmit a specified block of video on
request. When the configured time lapses, the storage solution deletes old videos to provide space for more recent data.
Metadata contains information that was extracted from the video frames including camera identity, captured objects, and
system owner. Data exploration and analytics are carried out on the metadata, so accuracy of results and reports depends
on the quality of the metadata. The Metadata Server (MDS) must be included in any implementation of this architecture
irrespective of the network size - it indexes and stores the metadata and is responsible for the following operations:
Knowledge of all the cameras on the network (it receives data from them).
Metadata is the main integrated resource in this architecture all surveillance querying/investigation is carried
out on the metadata through the API.
It acts as network identifier as described in the next section
Intelligent Network Video Recorders (i-NVR)
In addition to connecting several cameras, the i-NVR encodes the video files and generates metadata before sending both
to their storage solution(s).
Queuing System
On a large network with several cameras, bottlenecks and deadlock is expected when transmitting data. The queuing
service is included to protect data integrity and manage deadlocks.
Web services
The web services, a RESTful service, manage all incoming and outgoing traffic to the system. These include system
security in the sense of authentication, authorisation, trust and session management and system audit for establishing
how data is being accessed. It also automatically discovers and registers or updates the directory service.
Directory Server
This service discovers, validates and organises a unique identity for all deployed instances of video surveillance systems
that connect to it. The service is responsible for cataloguing available systems details, and contact details. The high-level
functionality of this service is described in the next section. In practice, security firms and public safety departments
such as the police will own and administer these services, and surveillance system owners can configure their systems as
private (data will not be shared with any directory service) or public, where the system registers with the directory
service.
User System
This comprises of a the user portal and devices such as a desktop computer, tablet, mobile phones and remote sensing
devices such as satellite cameras, road traffic cameras, and mobile devices used by public safety officers. The portal
provides an interface for capturing data from different devices and for requesting and responding to user actions such as
uploading data, playing video and querying the metadata.
4.2 Hierarchies, System Scope and Visibility
A network architecture based on a flat design, which is one where all routing devices have full knowledge of the
network, can only grow to a limited size where the limitation is dictated by the capacity of the routers’ memory size,
processing power and transmission speeds. In order to build large networks where both inter-network and intra-network
routing can scale efficiently, there is a need for hierarchical design [18]. A hierarchical network is partitioned into areas
(or sub-networks) where each routing device only has full knowledge of its own local area. For each sub-network, there
is an inter-network router, which has knowledge of neighbouring sub-networks. In practice, sub-networks are usually
based on network ownership, geographical area covered or overall size of the network. Examples sub-networks are based
on floor sub-networks, departmental networks, overall company networks, and city networks. Although these partitions
are usually political and ownership defined, they enhance scalability, performance, security and efficiency of the bigger
network.
The FVSA depicts video surveillance systems as a hierarchical system, where subsystem boundaries are based on
administrative ownership and geographical location. Additionally, metadata servers (MDS) handle routing activities as
discussed below. They are configurable as intra-system (local scope) or inter-system (global scope). An MDS in the
local scope has full knowledge of the topological details of all the cameras in the system but does not have any
knowledge about any external camera. However in the global scope, an MDS provides connectivity to an external
surveillance system through the Directory service as described below.
In Figure 4 below, an L-MDS only has knowledge of cameras that directly connect to it, and those that connect through
an i-NVR and those that are connected to neighbour L-MDSs. Any G-MDS knows how to contact any other G-MDS that
is connected to the directory server, however the level of access or visibility depends on the role of the user. For
example, in Figure 4, the various L-MDS in the city mall system represents various FVSA systems in the mall, where
different shops own and independently manage their own surveillance system. The mall’s authority however provides a
G-MDS, which every shop can connect to. The mall authority manages the G-MDS and at the same time, the G-MDS
can provide connectivity to the city police. With this in place, the mall authority can provide evidence of events without
the police physically visiting the mall.
Figure 4: Global and local scope of the MDS the G-MDS connects to other G-MDS while each L-MDS can only administer
cameras within its own system boundary. The MDS in system C, which depicts the city hospital surveillance system, is
configured for local use only. The cameras and data in system C are therefore not available outside the hospital network.
Authorisation and Resource Visibility
Any information destined outside the system has to be initiated by a G-MDS, provided the user meets authentication and
authorisation requirements. Only the local administrator has full authorisation on all system services. Any user that is not
local to the system has to be granted authorisation to use a specific service. For example, by default, a police office can
view a system overview of any connected surveillance system but to play video or query such a system, the system
owner must first authorise the access. In Table 1 below, it is noted that all external users are not allowed access to the
service but public safety officers such as the police may be given authorisation to access some services.
Table 1: Visibility and authorization of system services.
Services in system A (Figure 4
above)
An admin
of system A
An admin
of system B
Views system overview: cameras, and
contact information.
Yes
No
Plays recorded video.
Yes
No
Queries System
Yes
No
Receives feeds and alerts
Yes
No
Configures/updates system or
cameras.
Yes
No
5 PROTOTYPE AND RELEVANCE
Figure 5 below shows the surveillance system in a smart city network - it is noted that each FVSA layer is relevant to a
layer in other grid computing platforms, such as smart cities. The layers (or hierarchies) in this view of the architecture
fall into either hardware domain (physical and network layers) or software domain (services and application layers). The
physical layer comprises all the devices that capture video such as cameras. The network layer includes all network and
switching devices such as the routers, MDS, and mobile antennas. The services layer comprises of network-based data
solutions and service APIs such as cloud storage. The application layer comprises of client applications and services
through which users interact with the system such as video player and query browser.
Figure 5 Layered architecture of the FVSA showing its relevance to other IoT compatible architecture (based on earlier works
such as [4] [5] [6]) – a view of the hierarchical design where each layer is depicted as a layer of the overall system architecture.
Physical and Network layers are hardware based while Application and Services layers are software implementation.
6 CONCLUSION
We have offered solutions for the problems described and highlight major areas that are still work in progress. The
solutions proposed by the FVSA include unification of independent surveillance systems. As described in section 4, each
implementation of the FVSA is independent while several instances can integrate to form a larger system (or a unified
system), such as a city’s surveillance system. The same section also introduced the directory server, which is the
integration catalogue for unifying the systems. With this in place, section 3 introduced how authorised public safety
officers can ‘browse’ all connected surveillance systems within their jurisdiction, with latent ability to review alerts and
video from any camera. In section 5, we demonstrate FVSA’s compatibility with other hierarchical network solutions
such as a smart city.
Ultimately, we suggest a hierarchical design and a high-level configuration for video surveillance devices and services,
making it possible to approach video networks in layers such as internal system (local) or external system (global).
Hierarchical design is an approach engineers employ to abstract complex multifaceted problems/requirements into
granular manageable subsystems. The framework of our solution is compatible with the hierarchical structure of
computer networks and emerging technologies.
REFERENCES
[1] S. Adcock and P. Norstrom, “Just 1 in 70 CCTV Cameras are State-Owned: Survey Revelation by the British
Security Industry Association (BSIA),” Press Conference, London, 2013. [Online]. Available:
http://www.bsia.co.uk/home/bsia-cctv-number-of-cameras-in-uk. [Accessed: 17-Sep-2014].
[2] S. Zikos and H. D. Karatza, “Clairvoyant site allocation of jobs with highly variable service demands in a
computational grid,” in 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops
and Phd Forum (IPDPSW), 2010, pp. 18.
[3] A. Pradesh, “A Novel Fault-tolerant Task Scheduling Algorithm for Computational Grids,” in 2013 15th
International Conference on Advanced Computing Technologies (ICACT), 2013, pp. 16.
[4] L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A survey,Comput. Networks, vol. 54, no. 15, pp.
27872805, Oct. 2010.
[5] S. Fang, L. Da Xu, S. Member, Y. Zhu, J. Ahati, H. Pei, J. Yan, and Z. Liu, “An Integrated System for Regional
Environmental Monitoring and Management Based on Internet of Things,” in IEEE TRANSACTIONS ON
INDUSTRIAL INFORMATICS, 2014, vol. 10, no. 2, pp. 15961605.
[6] C. Tao, X. Ling, S. Guofeng, Y. Hongyong, and H. Quanyi, “Architecture for Monitoring Urban Infrastructure
and Analysis Method for a Smart-Safe City,” in 2014 Sixth International Conference on Measuring Technology
and Mechatronics Automation, 2014, pp. 151154.
[7] S. Dey, A. Chakraborty, S. Naskar, and P. Misra, “Smart city surveillance: Leveraging benefits of cloud data
stores,” in goSMART 2012, Clearwater, 2012, no. 978–1–467321303, pp. 868876.
[8] R. Xue, Z.-S. Wu, and A.-N. Bai, “Application of Cloud Storage in Traffic Video Detection,” 2011 Seventh Int.
Conf. Comput. Intell. Secur., pp. 12941297, Dec. 2011.
[9] Y. Huo, H. Wang, and L. Hu, “A Cloud Storage Architecture Model for Data- Intensive Applications,” in 2011
International Conference on Computer and Management (CAMAN), 2011, no. 61073009, pp. 2629.
[10] D. Chu, C. Jiang, Z. Hao, and W. Jiang, “The Design and Implementation of Video Surveillance System Based
on H.264, SIP, RTP/RTCP and RTSP,” in 2013 Sixth International Symposium on Computational Intelligence
and Design, 2013, vol. 2, pp. 3943.
[11] J. R. Smith, R. J. Alexandre, J. Hobbs, and R. C. Bolles, “VERL!: An Ontology Framework for Representing and
Annotating Video Events,” MultiMedia, IEEE, vol. 12, no. 4, pp. 7686, 2005.
[12] R. Nevatia, J. Hobbs, B. Bolles, and M. Rey, “An Ontology for Video Event Representation,” in Proceedings of
the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
(CVPRW’04), 2004.
[13] H. Zhou and G. K. H. Pang, “Metadata extraction and organization for intelligent video surveillance system,” in
International Conference on Mechatronics and Automation (ICMA), 2010, 2010, pp. 489494.
[14] H. Zhou, L. Jia, and Y. Qin, “Metadata Specification of Railway Video Information and its application in Video
Monitoring System for Qinghai-Tibet Railway,” in International Symposium on Computer Network and
Multimedia Technology, 2009. CNMT 2009., 2009, no. 600332020.
[15] R. Rawassizadeh, J. Heurix, S. Khosravipour, and a. M. Tjoa, “LiDSec- A Lightweight Pseudonymization
Approach for Privacy-Preserving Publishing of Textual Personal Information,” in 2011 Sixth International
Conference on Availability, Reliability and Security, 2011, pp. 603608.
[16] X. Zhu, H. Deng, Z. Chen, and H. Yang, “Design of Large-Scale Video Surveillance System Based on P2P
Streaming,” in 2011 3rd International Workshop on Intelligent Systems and Applications, 2011, pp. 14.
[17] W. Paper, “The Cisco SONA Architectural Model in Unified Communications: A Solid Foundation for the
Collaborative Innovative Enterprise,” 2008. [Online]. Available:
http://www.cisco.com/c/en/us/solutions/collateral/enterprise-networks/branch/White_paper_C11-473760.html.
[18] R. Perlman and C. Kaufman, “Hierarchical networks with Byzantine Robustness,” in 2011 Third International
Conference on Communication Systems and Networks (COMSNETS 2011), 2011, pp. 111.

Supplementary resource (1)

Data
February 2015
Sola O Ajiboye · Phil Birch · Chris R Chatwin · Rupert C D Young
... The selected and related work papers proposed novel paradigms, which we distinguished from the set of the whole related research papers we have studied. Each column represents a piece of paper, and each color represents a characteristic of the IoT (Data Privacy, Quality of Communication, Transmission Speed, Easy Installation, Security, Efficiency) [6,10,[61][62][63][64]. Also, Figure 2 shows that the majority of architecture proposals address the quality of communication. ...
... Comparative analysis of major components of architecture comparison and contribution[6,10,[61][62][63][64]. ...
Article
Full-text available
The Internet of Things (IoT) was introduced as a recently developed technology in the telecommunications field. It is a network made up of real-world objects, things, and gadgets that are enabled by sensors and software that can communicate data with one another. Systems for monitoring gather, exchange, and process video and image data captured by sensors and cameras across a network. Furthermore, the novel concept of Digital Twin offers new opportunities so that new proposed systems can work virtually, but without differing in operation from a “real” system. This paper is a meticulous survey of the IoT and monitoring systems to illustrate how their combination will improve certain types of the Monitoring systems of Healthcare–IoT in the Cloud. To achieve this goal, we discuss the characteristics of the IoT that improve the use of the types of monitoring systems over a Multimedia Transmission System in the Cloud. The paper also discusses some technical challenges of Multimedia in IoT, based on Healthcare data. Finally, it shows how the Mobile Cloud Computing (MCC) technology, settled as base technology, enhances the functionality of the IoT and has an impact on various types of monitoring technology, and also it proposes an algorithm approach to transmitting and processing video/image data through a Cloud-based Monitoring system. To gather pertinent data about the validity of our proposal in a more safe and useful way, we have implemented our proposal in a Digital Twin scenario of a Smart Healthcare system. The operation of the suggested scenario as a Digital Twin scenario offers a more sustainable and energy-efficient system and experimental findings ultimately demonstrate that the proposed system is more reliable and secure. Experimental results show the impact of our proposed model depicts the efficiency of the usage of a Cloud Management System operated over a Digital Twin scenario, using real-time large-scale data produced from the connected IoT system. Through these scenarios, we can observe that our proposal remains the best choice regardless of the time difference or energy load.
... This work focused on using surveillance and sousveillance for data acquisition in Citizen Profiling setting. Recent works show a wide range of problems that are supposed to be solved through surveillance and computer vision [7][8][9][10][11][12][13][14]. Studies in Surveillance have focused more on computer vision in which the datasets contain a collection of images that can create storage overheads. ...
... Studies in Surveillance have focused more on computer vision in which the datasets contain a collection of images that can create storage overheads. In [7] authors discussed video surveillance, which were employed across Britain by BSIA (British Security Industry Association). The authors proposed a novel approach for dealing with hierarchical architecture using FVSA (Fused Video Surveillance Architecture) approach that includes integration of multilayer abstraction towards Internet of Things (IOT) based systems. ...
Article
Full-text available
In this work we have proposed a model for Citizen Profiling. It uses veillance (Surveillance and Sousveillance) for data acquisition. For representation of Citizen Profile Temporal Knowledge Graph has been used through which we can answer semantic queries. Previously, most of the work lacks representation of Citizen Profile and have used surveillance for data acquisition. Our contribution is towards enriching the data acquisition process by adding sousveillance mechanism and facilitating semantic queries through representation of Citizen Profiles using Temporal Knowledge Graphs. Our proposed solution is storage efficient as we have only stored data logs for Citizen Profiling instead of storing images, audio, and video for profiling purposes. Our proposed system can be extended to Smart City, Smart Traffic Management, Workplace profiling etc. Agent based mechanism can be used for data acquisition where each Citizen has its own agent. Another improvement can be to incorporate a decentralized version of database for maintaining Citizen profile.
... A video analytics system comprises multiple cameras installed at locations of interest, connected to edge nodes, which in turn are connected to the decision making core, usually hosted in an on-premises data-center or on the cloud [1][2][3]. The edge node is considered to have relatively low computational power, sufficient enough for basic pre-processing and forwarding the video streams to the core. ...
Article
Full-text available
Video surveillance and analytics solutions based on Artificial Intelligence (AI) are increasingly being deployed across industries, including academia. There are a number of use-cases for campus-wide video analytics applications. Detecting events of interest in real-time and generating alerts is a core requirement for such applications, making them both network and compute intensive. Thus, the underlying framework needs to be resource optimized in terms of latency, compute and storage requirements for a multitude of video applications. Increasingly privacy concerns have been voiced against the pervasive deployment of video analytics-based applications. Thus, protecting the privacy of students and staff in a campus setting shall be a major design consideration for such systems going forward. This paper presents a resource optimized and privacy preserving framework for campus-wide video analytics applications. Several use-cases are presented and early results from the deployment of the proposed framework establish its feasibility and effectiveness.
... In this direction, Ajiboye, S.O. et al. [305] stated that the network video recorder is already equipped with intelligent video processing capabilities but complained about its limitations, i.e., isolation, and scalability. To resolve such issues, they proposed a general high-level theoretical architecture called Fused Video Surveillance Architecture (FVSA). ...
Preprint
On the rise of distributed computing technologies, video big data analytics in the cloud have attracted researchers and practitioners' attention. The current technology and market trends demand an efficient framework for video big data analytics. However, the current work is too limited to provide an architecture on video big data analytics in the cloud, including managing and analyzing video big data, the challenges, and opportunities. This study proposes a service-oriented layered reference architecture for intelligent video big data analytics in the cloud. Finally, we identify and articulate several open research issues and challenges, which have been raised by the deployment of big data technologies in the cloud for video big data analytics. This paper provides the research studies and technologies advancing video analyses in the era of big data and cloud computing. This is the first study that presents the generalized view of the video big data analytics in the cloud to the best of our knowledge.
... In this direction, Ajiboye, S.O. et al. [306] stated that the network video recorder is already equipped with intelligent video processing capabil-ities but complained about its limitations, i.e., isolation, and scalability. To resolve such issues, they proposed a general high-level theoretical architecture called Fused Video Surveillance Architecture (FVSA). ...
Article
Full-text available
The proliferation of multimedia devices over the Internet of Things (IoT) generates an unprecedented amount of data. Consequently, the world has stepped into the era of big data. Recently, on the rise of distributed computing technologies, video big data analytics in the cloud has attracted the attention of researchers and practitioners. The current technology and market trends demand an efficient framework for video big data analytics. However, the current work is too limited to provide a complete survey of recent research work on video big data analytics in the cloud, including the management and analysis of a large amount of video data, the challenges, opportunities, and promising research directions. To serve this purpose, we present this study, which conducts a broad overview of the state-of-the-art literature on video big data analytics in the cloud. It also aims to bridge the gap among large-scale video analytics challenges, big data solutions, and cloud computing. In this study, we clarify the basic nomenclatures that govern the video analytics domain and the characteristics of video big data while establishing its relationship with cloud computing.We propose a service-oriented layered reference architecture for intelligent video big data analytics in the cloud. Then, a comprehensive and keen review has been conducted to examine cutting-edge research trends in video big data analytics. Finally, we identify and articulate several open research issues and challenges, which have been raised by the deployment of big data technologies in the cloud for video big data analytics. To the best of our knowledge, this is the first study that presents the generalized view of the video big data analytics in the cloud. This paper provides the research studies and technologies advancing the video analyses in the era of big data and cloud computing.
... Recently, the deployment of distributed computing technologies in the cloud for video big data analytics has been the center of attracted in academics and industry. In literature, efforts have been made to propose cloud-based service-oriented video analytics systems (CVAS), e.g., [1][2][3]. ...
Article
Full-text available
Quality of service, that is, the waiting time that customers must endure in order to receive a service, is a critical performance aspect in private and public service organizations. P1 voiding good service quality is particularly important in highly competitive sectors where similar services exist. In this paper, focusing on banking sector, we propose an artificial intelligence system for building a model for the prediction of service quality. While the traditional approach used for building analytical models relies on theories and assumptions about the problem at hand, we propose a novel approach for learning models from actual data. Thus, the proposed approach is not biased by the knowledge that experts may have about the problem, but it is completely based on the available data. The system is based on a recently defined variant of genetic programming that allows practitioners to include the concept of semantics in the search process. This will have beneficial effects on the search process and will produce analytical models that are based only on the data and not on domain-dependent knowledge.
Article
The principle point of this paper is to talk about the Video surveillance system plays major role in the security applications now a days. The need for surveillance systems are increasing drastically. Internet of things makes Video surveillance more effective. Internet of Things is an interrelated communication network for several systems and having the ability to transfer the information through the network without human or computer interaction. This paper reviews the network traffic due to Video surveillance and its management with Software Defined Networking. This paper also reviews the applications of Internet of things in Video surveillance system.
Conference Paper
Full-text available
A computational grid is a hardware and software infrastructure that provides consistent, dependable, pervasive and expensive access to high-end computational capabilities in a multi-institutional virtual organization. Computational grids provide computing power needed for execution of tasks. Scheduling the task in computing grid is an important problem. To select and assign the best resources for task, we need a good scheduling algorithm in grids. As grids typically consist of strongly varying and geographically distributed resources, choosing a fault-tolerant computational resource is an important issue. The main scheduling strategy of most fault-tolerant scheduling algorithms depends on the response time and fault indicator when selecting a resource to execute a task. In this paper, a scheduling algorithm is proposed to select the resource, which depends on a new factor called Scheduling Success indicator (SSI). This factor consists of the response time, success rate and the predicted Experience of grid resources. Whenever a grid scheduler has tasks to schedule on grid resources, it uses the Scheduling Success indicator to generate the scheduling decisions. The main scheduling strategy of the Fault-tolerant algorithm is to select resources that have lowest tendency to fail and having more experience in task execution. Extensive experiment simulations are conducted to quantify the performance of the proposed algorithm on GridSim. GridSim is a Java based discrete-event Grid simulation toolkit. Experiments have shown that the proposed algorithm can considerably improve grid performance in terms of throughput, failure tendency and worth.
Article
Full-text available
Climate change and environmental monitoring and management have received much attention recently, and an integrated information system (IIS) is considered highly valuable. This paper introduces a novel IIS that combines Internet of Things (IoT), Cloud Computing, Geoinformatics [remote sensing (RS), geographical information system (GIS), and global positioning system (GPS)], and e-Science for environmental monitoring and management, with a case study on regional climate change and its ecological effects. Multi-sensors and web services were used to collect data and other information for the perception layer; both public networks and private networks were used to access and transport mass data and other information in the network layer. The key technologies and tools include real-time operational database (RODB); extraction–transformation–loading (ETL); on-line analytical processing (OLAP) and relational OLAP (ROLAP); naming, addressing, and profile server (NAPS); application gateway (AG); application software for different platforms and tasks (APPs); IoT application infrastructure (IoT-AI); GIS and e-Science platforms; and representational state transfer/Java database connectivity (RESTful/JDBC). Application Program Interfaces (APIs) were implemented in the middleware layer of the IIS. The application layer provides the functions of storing, organizing, processing, and sharing of data and other information, as well as the functions of applications in environmental monitoring and management. The results from the case study show that there is a visible increasing trend of the air temperature in Xinjiang over the last 50 years (1962–2011) and an apparent increasing trend of the precipitation since the early 1980s. Furthermore, from the correlation between ecological indicators [gross primary production (GPP), net primary production (NPP), and leaf area index (LAI)] and meteorological elements (air temperature and precipitation), water res- urce availability is the decisive factor with regard to the terrestrial ecosystem in the area. The study shows that the research work is greatly benefited from such an IIS, not only in data collection supported by IoT, but also in Web services and applications based on cloud computing and e-Science platforms, and the effectiveness of monitoring processes and decision-making can be obviously improved. This paper provides a prototype IIS for environmental monitoring and management, and it also provides a new paradigm for the future research and practice; especially in the era of big data and IoT.
Conference Paper
Full-text available
Sharing personal information benefits both data providers and data consumers in many ways. Recent advances in sensor networks and personal archives enable users to record personal information including emails, social networking activities, or life events (life logging). These information objects are usually privacy sensitive and thus need to be protected adequately when being shared. In this work, we present a lightweight pseudonymization framework which allows users to benefit from sharing their personal information while still preserving their privacy. Furthermore, this approach increases the data owners' awareness of what information they are sharing, thus rendering data publishing more transparent.
Conference Paper
Full-text available
The research for metadata extraction originates from the intelligent video surveillance system, which is widely used in outdoor and indoor environment for the aims of traffic monitor, security guard, and intelligent robot. Various features are extracted from the surveillance image sequences such as target detection, target tracking, object's shape and activities. However, the trend of more and more features being used and shared in video surveillance system calls for more attention to bridge the gap between specific analysis algorithms and end-user's expectation. This paper proposes a three-layer object oriented model to extract the surveillance metadata including shape, motion speed, and trajectory of the object emerging in image sequence. Meanwhile, the high-level semantic metadata including entry/exit point, object duration time is organized and stored which are provided for the further end-user queries. The paper also presents the experiment results in different indoor and outdoor surveillance scenarios. At last, a comparative analysis with another traditional method is presented.
Conference Paper
The streaming media protocols including SIP(Session Initiation Protocol), RTP(Real-time Transport Protocol), RTCP(RTP Control Protocol) and RTSP(Real-Time Streaming Protocol) is the basic technology of the video surveillance system. A video surveillance system based on SIP, RTP/RTCP and RTSP is designed and implemented in this paper. The video surveillance system uses the SIP protocol to connect the client and the server, and provides the service of the multimedia data transmission between them with the RTP/RTCP and RTSP protocol. The video images of the surveillance system is encoded by H.264 video coding standards. The play of the video data is based on the technology of Direct show. The experiment results show that the surveillance system provides good video quality and adaptability of the network conditions.
Conference Paper
City is a complicated system with several kinds of lifeline infrastructure such as water, electricity, and gas system as the basic element. The safety of city lifeline systems has characteristics of complex network, multiple areas, numerous operation parameters, and chain reaction. The Internet of Things technology used in monitoring city lifeline can effectively help people to acquire the concerned data that can be used to analyze the operation safety status, and even respond to disasters. This paper studies the general architecture of monitoring typical lifeline systems, identifies the key parameters and proposed the systematic safety analysis methodologies.
Conference Paper
The smart cities of future need to have a robust and scalable video surveillance infrastructure. In addition it may also make use of citizen contributed video feeds, images and sound clips for surveillance purposes. Multimedia data from various sources need to be stored in large scalable data stores for compulsory retention period, on-line, off-line analytics and archival. Multimedia feeds related to surveillance are voluminous and varied in nature. Apart from large multimedia files, events detected using video analytics and associated metadata needs to be stored. The underlying data storage infrastructure therefore needs to be designed for mainly continuous streaming writes from video cameras and some variety in terms of I/O sizes, read-write mix, random vs. sequential access. As of now, the video surveillance storage domain is mostly dominated by iSCSI based storage systems. Cloud based storage is also provided by some vendors. Taking in account the need for scalability, reliability and data center cost minimization, it is worth investigating if large scale video surveillance backend can be integrated to the open source cloud based data stores available in the “big data” trend. We developed a multimedia surveillance backend system architecture based on the Sensor Web Enablement framework and cloud based “key-value” stores. Our framework gets data from camera/ edge device simulators, splits media files and metadata and stores those in a segregated way in cloud based data stores hosted on Amazons EC2. We have benchmarked performances of a few cloud based key-value stores under large scale video surveillance workload and demonstrated that those perform satisfactorily, bringing in inherent scalability and reliability of a cloud based storage system to a video surveillance system for a smart safe city. With a case study of the storage of video surveillance system, we show in this paper that with the availability of several cloud based d- stributed data stores and benchmarking tools, an application's data management needs can be served using hybrid cloud based data stores and selection of such stores can be facilitated using benchmark tools if the application workload characteristics are known.
Article
Video surveillance becomes more and more popular as it is a high-tech method to protect personal safety and public property. Recently it is not the privilege of police and common people can view videos of many public places lively or by on- demand way. Traditional client-server based video streaming solutions incur expensive bandwidth provision cost on the server and are not scale well. Peer-to-Peer (P2P) streaming is a new paradigm to build distributed networking video surveillance applications. Recently, several P2P-TV systems have been deployed to provide live and on-demand video streaming services on the Internet at low server cost. But it is seldom to use p2p streaming in video surveillance. In this paper, we provide the design of large scale video surveillance system client based on p2p streaming. Given the particularity of video surveillance, such as high churn and the heterogeneity of user access network and delay-tolerant, we construct the end-hosts into mesh application layer topology and adopt pull-push mode to delivery the data. Based on test on our previous implement of e-Touch video surveillance system, it shows that it is feasibly to realize video client using p2p streaming technology and simulation results shows single streaming media server can support tens of thousands of users simultaneously participating in the system.
Article
The lack of homogeneity in description elements and description method leads to video information sharing and management in the relative video application and management system for more management departments at all levels hampered. Based on the relevant metadata standards and the metadata technique, we put forward the railway video metadata specification, which consisted of mandatory elements, optional elements and extension elements, and the feasible method of key-frame and time-point. Then we apply the metadata specification in video monitoring system for Qinghai-Tibet railway. In this system, combined and matched the real-time video capture with the existing cataloging video, the video information is described accurately and displayed immediately. These works lay the foundation for building a normative video information management system and establish norms of video metadata cataloging to meet individual application requirements, improving the resource efficiency of the national railway video.
Conference Paper
Scalability and I/O efficiency are two of the biggest challenges in building cloud storage system, especially to the data-intensive applications that will run on cloud. To cloud computing, constructing a suitable architecture for every kind of cloud service is the key to succeed. But very little work has been done to create a valid and standard cloud storage architecture. In this paper, a four-layer cloud storage architecture is proposed. To meet the requirements of data-intensive applications, an efficient and scalable node organization model is introduced to the cloud storage system.