Video analysis system architecture An important aspect of the proposed system is the issue of data transmission. A layered approach was devised, based on TCP/IP protocols suite, which enables the sensitive data to be transferred by means of open Internet, regardless of the presence of Network Address Translators or firewalls which typically interfere with establishment of multimedia communication sessions. At the foundation of the developed solution lays the Extensible Messaging and Presence Protocol (XMPP), which is designed for building Instant Messaging systems, but thanks to its virtually-unlimited extensibility, it is an increasingly popular tool for general-purpose application servers and distributed applications. Its use provides an added value of security and message integrity layer (based on TLS standard), authorization, addressing scheme and a container for information structuring within self-describing, XML- 

Context in source publication

Context 1

... monitoring systems are a necessity in the modern times. Although some people object the idea of ‘being watched’, surveillance systems actually improve the level of public security, allowing the system operators to detect threats and the security forces to react in time. Surveillance systems evolved in the recent years from simple CCTV systems into complex structures, containing numerous cameras and advanced monitoring centers, equipped with sophisticated hardware and software. However, the future of surveillance systems belongs to automatic tools that assist the system operator and notice him on the detected security threats. This is important, because in complex systems consisting of tens or hundreds of cameras, the operator is not able to notice all the events. In the last few years many publications regarding automatic video content analysis have been presented. However, these systems are usually focused on a single type of human or vehicle activity. No complex approach to the problem of automatic video surveillance system has been proposed so far. In order to address this problem, the authors designed a framework that analyses camera images on multiple levels, from basic detection of moving objects to advanced object recognition and automatic detection of important events. The proposed system has a flexible structure, with functional modules that may be selected so that the system suits the need of a particular application. These modules are based on algorithms proposed by various authors, adapted to the needs of the presented framework and enhanced by the authors in order to provide an efficient solution for automatic detection of important security threats in video monitoring systems. The chapter is organized as follows. Section 2 presents the general structure of the proposed framework and a method of data exchange between system elements. Section 3 is describing the low-level analysis modules for detection and tracking of moving objects. In Section 4 we present the object classification module. Sections 5 and 6 describe specialized modules for detection and recognition of faces and license plates, respectively. In section 7 we discuss how video analysis results provided by other modules may be used for automatic detection of events related to possible security threats. The chapter ends with conclusions and discussion of future framework development. The system for intelligent video analysis has a distributed architecture (Fig. 1), consisting of multiple node stations, one central station and operator stations. Node stations are placed in the monitored area, close to video acquisition sensors (i.e. cameras). They are responsible for camera management and for automatic analysis of images from all cameras in their vicinity. Each node station contains a small-factor PC running Linux operating system and equipped with video analysis software. The computer is enclosed in a weather-proof casing which makes possible to mount a node station outdoors. Results of video analysis are sent from node stations to the central station for storing, evaluating and notifying operators. Central station is also responsible for aggregation results coming from multiple node station in order to detect large-scale, global threats. Such configuration makes possible to use wide-band, short-distance cable connections between cameras and node stations to transfer high-quality video streams and wireless communication medium to send results of analysis to the central station. The system operator has access to the analysis results and camera images from the whole system through the operator station which consists of a monitor set, controllers and computers with the specialized software. Depending on the network throughput available, there is also a possibility to view live video streams from any camera in the system from an operator station. based messages. Furthermore, XMPP grants access to a plethora of the protocol extensions developed by its mature community, which may be found surprisingly useful within the context of surveillance solution. Such particular extension, which is very important for the proposed system, is so-called Jingle protocol, which is a tool for establishing multimedia communication sessions. This forms the core of the system’s audio and video streaming functionality. In fact Jingle is a session-control (signaling) protocol while the actual multimedia data transfer is performed out-of-band of XMPP connection due to performance reasons. For this purpose encrypted Realtime Transfer Protocol (RTP) sessions are utilized. As a consequence, the initiation of multimedia streaming may be problematic in the presence of NAT devices or firewall on the route between transmission endpoints. Therefore, an additional proxy service has been implemented within the system, which allows for the efficient multimedia transmission between any connected terminals regardless of their network conditions. Two types of digital cameras are employed in the video monitoring system. Stationary (fixed), wide-angle cameras, especially megapixel ones, offer a wide field of view and are used for video content analysis and event detection. The other type – pan-tilt-zoom (PTZ) cameras – allow for adjusting their field of view as required and they are used for automatic tracking of objects, selected either manually by an operator or automatically by the event detection system. PTZ cameras provide an operator with a detailed view of the situation. Video analysis performed in the node station is a multi-stage process (Fig. 2). It consists of low-level image processing modules and high-level event detection modules. First, all moving object present in a fixed camera field of view are detected in each video frame, independently. Then, all moving objects are tracked in the adjacent video frames, as long as they stay in the camera field of view, in order to obtain characteristics of their movements. Various static (e.g. shape, texture) and dynamic (e.g. location, speed, heading) object features are used to classify them into a few groups (e.g. humans, cars). Object features are used in the final, high-level analysis stage for automatic detection of important events. Event detection is supplemented by additional, specific modules, such as face detection and recognition, license plate recognition and others. The main functional modules of the framework will be presented in detail further in this chapter. Detection of moving objects is usually the first stage of video processing chain and its results are used by further processing modules. Most video segmentation algorithms usually employ spatial and/or temporal information in order to generate binary masks of objects (Li & Ngan, 2007; Liu & Zheng, 2005; Konrad, 2007). However, simple time-averaging of video frames is insufficient for a surveillance system because of limited adapting capabilities. The solution implemented in the framework utilizes spatial segmentation for detection of moving objects in video sequences, using background subtraction algorithm (Yang et al., 2004). This approach is based on modeling pixels as mixtures of Gaussians and using an on-line approximation to update the model (Elgammal et al., 2000; Stauffer & Grimson, 2000). This method proved to be useful in many applications, as it is able to cope with illumination changes and to adapt to the background model accordingly to the changes in the scene, e.g. when motionless foreground objects eventually become a part of the background. Furthermore, the background model can be multi-modal, allowing regular changes in the pixel color. This makes it possible to model such events as trees swinging in the wind or traffic light sequences. Background modeling is used to model current background of the scene and to differentiate foreground pixels of moving objects from the background (Dalka, 2006; Czyzewski & Dalka, 2007). Each pixel in the image is modeled with a mixture of K Gaussian distributions for this purpose. The probability that a pixel has the value x t at the time t is given ...