ArticlePDF Available

Dealing Background Issues in Object Detection using GMM: A Survey

Authors:

Abstract and Figures

object detection is critical task in video analytics. Gaussian Mixture Model (GMM) based background subtraction is widely popular technique for moving object detection due to its robustness to multimodality and lighting changes. This paper presents the critical survey about various GMM based approaches for handling critical background situations. This survey describes various challenges faced by background subtraction such as shadow, sudden and slow light changes, multimodal background, bootstrap, camouflage, foreground aperture, camera jitter etc. and study of various modifications or extensions of GMM to handle these issues. This study helps researcher to select appropriate GMM version based on critical background condition.
Content may be subject to copyright.
International Journal of Computer Applications (0975 8887)
Volume 150 No.5, September 2016
50
Dealing Background Issues in Object Detection using
GMM: A Survey
Lajari Alandkar
Ph.D. Scholar,
Department of Electronics,
Walchand Institute of Technology,
MH, India
Sachin R. Gengaje
HOD, Department of Electronics,
Walchand Institute of Technology,
MH, India
ABSTRACT
Moving object detection is critical task in video analytics.
Gaussian Mixture Model (GMM) based background
subtraction is widely popular technique for moving object
detection due to its robustness to multimodality and lighting
changes. This paper presents the critical survey about various
GMM based approaches for handling critical background
situations. This survey describes various challenges faced by
background subtraction such as shadow, sudden and slow
light changes, multimodal background, bootstrap, camouflage,
foreground aperture, camera jitter etc. and study of various
modifications or extensions of GMM to handle these issues.
This study helps researcher to select appropriate GMM
version based on critical background condition.
General Terms
Pattern Recognition, Computer Vision, video surveillance
Keywords
Object Detection, Background Subtraction, Gaussian Mixture
Model, Background challenges
1. INTRODUCTION
Moving object detection is important task in video analytics.
Accuracy at object detection level significantly affects the
high level image sequence analysis [1].Lots of research has
been carried out to improve the performance of object
detection. However, accurate object detection is still
challenging due to critical dynamic background conditions
[3]. These critical background situations may include
flickering monitor, snow, rain, waving tree, shadow, sudden
illumination change due to light on/off, slower illumination
change during sunrise and sunset, foreground aperture, busy
background, object with color same as background, camera
jitter etc. Efficient moving object detection system must cope
up with these conditions and extract region of interest
accurately. Various basic techniques have been defined to
detect object such as background subtraction, optical flow,
frame differencing [2].Among these, background subtraction
is commonly used method due its low memory requirement,
simplicity and easiness in implementation [3].
First step in Background subtraction is background modelling.
Mean filter, Approximate Median, Kernel Density estimation,
single Gaussian, Gaussian mixtures are few background
modelling techniques [4]. Among these, Gaussian mixture
modelling (GMM) defined by Stauffer and Grimson[5] is
widely popular due to its robustness in handling multimodal
background and lighting changes. However, scientist still
exploring and innovating the established research for
performance improvement of object detection.
The aim of this paper is to summarize all the study about
GMM based object detection according to different
background situations which they can handle. This survey will
help researcher to select appropriate GMM version according
to their application. The paper is divided in four subsections.
Second section describes original GMM in brief along with its
challenging background situations. Third section summarizes
the study of various GMM versions w.r.t. different
background situations. Paper is concluded in section four.
2. GMM BASED BACKGROUND
SUBTRACTION
GMM is probabilistic approach for background modelling.
Each pixel in the scene is modeled by a mixture of K
Gaussian distributions [5]. The probability that a certain pixel
has a value of at time t can be written as
() = ,(
=1 ,,,,) (1)
Where , is the weight, , is the mean value, and , is
the covariance matrix for the ith Gaussian distribution at time t.
where η is a Gaussian probability density function.
,,=1
(2)/21/2 1
2()1() (2)
The K distributions are ordered based on ratio w/σ and the
first B distributions are used as a model of the background of
the scene.[5] Where B is estimated as
=
=1
> (3)
The threshold T is the minimum fraction of the background
model. Background subtraction is performed by marking a
foreground pixel any pixel that is more than 2.5 standard
deviations away from any of the B distributions [5]. The first
Gaussian component that matches the test value will be
updated by the following update equations, The prior weights
of the K distributions at time t, ω(k,t), are adjusted as
follows[5].
,=1,1+(,) (4)
Where α is the learning rate and , is 1 for the model
which matched and 0 for the remaining models. After this
approximation, the weights are renormalized. 1/α defines the
time constant which determines the speed at which the
distribution’s parameters change. The μ and σ parameters for
unmatched distributions remain the same. The parameters of
the distribution which matches the new observation are
updated as follows.
International Journal of Computer Applications (0975 8887)
Volume 150 No.5, September 2016
51
=11+ (5)
2=11
2+() (6)
Where, =,
Performance of this system to be satisfactory the appropriate
initial tuning of parameters α and T is important [5, 26]. Two
main facts about GMM which helps to effectively deals with
some background situations are as follows. First, learning rate
determines the speed of adaptation to illumination changes
which permits to handle the gradual illumination change in
the background [3, 5]. Second, use of mixture model allows
more than one color to be included in to background model,
which is big advantage when background situation is
multimodal [3, 5]. Despite these issues, there are many more
issues arises in real background scene and that couldn't be
handled by original GMM. These issues include shadow,
sudden illumination change, camera jitter, foreground
aperture, camouflage, bootstrap etc. [3]. Next section includes
extensive survey about various GMM versions which is
established to deal with these challenges along with
multimodality and illumination change.
3. BACKGROUND CHALLENGES AND
GMM VERSIONS
Background can be static or dynamic. Object detection in
static background is simple. It requires less memory and
computational time. Implementation is easy for such system.
In the case of dynamic background, required object detection
system is complex. Object detection system has to adapt with
dynamic background and perform satisfactorily in real time.
Various background situations and solution to handle these
situations using GMM based object detection system are
discussed in next subsections.
3.1 Shadow
Shadow is mostly explored background challenge, since this
is common issue in indoor and outdoor scene. It arises
certainly if foreground object is present in the scene. Incorrect
labeling to shadows as foreground pixels may cause failure in
applications such as tracking, video surveillance, motion
segmentation, etc.
Wang et al[6] used mixed color space to suppress shadow.
The color space adopted as (r, g, I) while r, g are normalized
chromaticity coordinates and I is the intensity coordinates.
The shadow is suppressed using criteria β ≤ It/ μ i γ. Where
(rt, gt, It) is observed value at the pixel in frame t. μi, σi is
mean and standard variance of the ith Gaussian distribution.
During low intensity r, g components are noisy. This may
affect the performance. Hence the solution for this problem is
mixed color space
=,,  >
,,  < (7)
Where Itd is a threshold and R, G are red and green color
component. This modification improves the results, especially
for video sequences including dark scenes, of background
modeling.
(a) (b) (c)
Fig. 1. (a) Image of a person and shadows;(b) Detection
result using RGB; (c) using (r, g, I).
Porikili et al[7], assumes that shadow effects on luminance
and saturation while maintaining hue constant. Their method
adapted luminance difference and saturation difference.
Additionally, authors defined shadow color range as a conic
cylinder around the background color vector. This approach
improves the detection accuracy.
Kristensen et al[8] studied seven different color space to
observe the behavior of shadow covered pixel. In the case of
YCbCr color space, Y will always be smaller when shaded
and that Cb and Cr will go towards 128, i.e. the origin. With
this information and considering noise three rules are
developed. All three assume that a negative change in Y has
already been detected. Rules are shown in Fig.1
Fig.2. Three different shadow detection rules. The gray
areas represent the part of the CbCr plane where a new
pixel is ruled to be a potential shadow. The area location is
based on the stored mean of Cb and Cr (Cb Cr)
Tian et al[9] improved the original GMM for shadow removal
by integrating the intensity information. The normalized
cross-correlation (NCC) of the intensities is calculated at each
pixel of the foreground region between the current frame and
the background image. The pixel is detected as shadow if
NCC is greater than predefined threshold Ts and intensity of
pixel is greater than predefined threshold Ti. This approach
allows shadow detection in bright areas only.
Mazeed et al[10], described a statistical disturbance technique
for shadow detection. The algorithm initially uses N frames to
form the background model. From these frames, the mean and
the variance is computed for each color band (RGB) in each
pixel. The brightness distortion, β, is computed between the
background model and a new pixel. If this distortion is
negative then new pixel is considered as shadow and
classified as background.
3.2 Sudden and Slower Illumination
Change
Sudden and slower illumination change is quite common issue
in indoor and outdoor scene respectively. Learning rate
parameter in original GMM determines the adaptation rate.
Therefore, it’s proper tuning helps GMM to deal with slow
illumination change in the background. Whereas, GMM
performance degrades in the sudden light change due to
selected learning rate value is insufficient to adapt with
sudden change. If learning rate is kept high for such
background case, then foreground object will merge in
background. Various modification and extension for GMM is
defined in order to improve the performance in slow and
sudden light change.
Teixeira et al [11] proposed cascade of change detection tests
including noise-induced, illumination variation and structural
changes. Pixels are removed from set of candidate foreground
pixel if they pass one of these tests. Illumination variation
change detection is carried out by simple co-linearity test. The
test consists evaluation of the angle between the current pixel
color vector and the reference color vector .
cos =.
(8)
International Journal of Computer Applications (0975 8887)
Volume 150 No.5, September 2016
52
If cos(θ) is greater than a predefined threshold TI , vectors are
considered to be collinear and the test is validated. Such
pixels are marked as background pixel and removed from
candidate foreground pixel set. This approach deals
effectively with illumination changes as shown in fig 3.
Fig.3 Test sequences and detection outputs for GMM
based on cascaded change detection
Porikily et al.[7] applied learning rate adaptation based
illumination change score. In this approach, the reference
image is updated if only a lighting change occurs in the scene.
Gaussian models with low variance (high confidence) are not
updated in this case. For this propose, an illumination change
score λ(t)is computed for a set of randomly selected pixels
that do not correspond to an object in the previous frame. If
illumination change score λ (t) is larger than a threshold τ the
learning parameter is adjusted as
= 0.01 + (t)
< (t) (9)
Where, c is the number of pixels in the pixel set Q. The value
of is determined empirically, and it controls the agility of
the update mechanism. This method improved the adaptation
performance of the original GMM by observing the amount of
illumination change in the background and updating a second
learning coefficient accordingly. This improvement
significantly reduces the computational load by minimizing
unnecessary model updates.
Javed et al. [12] developed a hierarchical approach that
combines color and gradient information to solve the problem
about rapid intensity changes. Javed et al. [12] adopted the kth,
highest weighted Gaussian component of GMM at each pixel
to obtain the gradient information to build the gradient-based
background model. However, choosing the highest weighted
Gaussian component of GMM leads to the loss of the short
term tendencies of background changes. Whenever a new
Gaussian distribution is added into the background model, it is
not selected owing to its low weighting value for a long
period of time. Consequently, the accuracy of the gradient-
based background model is reduced for that the gradient
information is not suitable for representing the current
gradient information.
To solve this problem Hu at al[13], selected the value of k
using Short Term Color Background Model(STCBM) and
Long Term Color Background Model(LTCBM). It helps to
develop a more robust gradient-based background model and
maintain the sensitivity to short-term changes.
Wang et al [6], adjusted the learning rate to deal with sudden
illumination change. In this case, If the pixel number of
detected foreground pixels is larger than a threshold (e.g.,
70% of the whole image pixels as in Wallflower), learning
rate adjusted to a high value; otherwise, it sets to a low value.
3.3 Multimodal Background
Multimodal situation arises when motion is present in the
background. For e.g. if scene contains one or more of
situations like waving tree, flickering monitors, rippling
water, snow or rain; then scene is called as multimodal scene.
Original GMM robustly handles multimodal background.
However, parameter K is fixed experimentally and constant
over time which is not optimal in terms of detection and
computational time [5].
Zivkovic [14] uses recursive equation with dirichlet prior for
appropriate selection of number of component for each pixel.
GMM initialization start with one component centered on the
first sample and new components are added based on
following condition.
=min
> (1 )
=1 (9)
Where, cf is a measure of the maximum portion of the data
that can belong to foreground objects without influencing the
background model. The Dirichlet prior with negative weights
will suppress the components that are not supported by the
data and component will be discarded if its weight becomes
negative. This improved GMM reduces the processing time.
In the same way, Lee [15] presented an online EM learning
algorithm for training adaptive Gaussian mixtures. Set of
recursive parameter update equations is derived based on
short term sufficient statistics. These parameters are computed
without additional storage of auxiliary variables.
Experimental result showed superior efficiency and
robustness on large simulations as well as real video data.
Chen and yang et al [16] proposed an approach to construct
background models directly from compressed video. It utilizes
the information from DCT coefficients at block level to
construct accurate background models at pixel level. In this
case, algorithm models DCT coefficients of each block in
DCT domain as a mixture of Gaussians. Each pixel block is
processed using Euclidian distance as matching function. A
threshold is associated with it to determine if the current block
matches a Gaussian, and the threshold will be updated. This
approach has much lower computational cost, compact model
storage without affecting the performance of original GMM.
In terms of improved detection accuracy, Zhao et al [17]
proposed a novel background modeling Method based on
Type-2 Fuzzy Gaussian Mixture Model (T2-FGMM) and
Markov Random Field (MRF), for motion detection in
dynamic scene. Spatial-temporal constraints are introduced
into the T2-FGMM by a Bayesian framework. The evaluation
results show that this approach performs better than the sound
Gaussian Mixture Model (GMM) in typical dynamic
backgrounds such as waving trees and water rippling.
(a) (b) (c)
Fig. 4 (a) Test sequence containing rippling water (b)
GMM (b) T2-FGMM foreground mask.
3.4 Camera Jitter
Camera shake is called as camera jitter. It results in specific
frequent change in background scene. The original GMM is
initialized using a training sequence. If this sequence is noisy
and/or insufficient to model background correctly, then it
generates false classification in the foreground detection mask
due to the related uncertainty.
Bouwmans et at [18] proposed Type-2 Fuzzy Mixture of
Gaussians Model (T2FMGM) to account for the uncertainty in
the background. Uncertain mean vector or covariance matrix
International Journal of Computer Applications (0975 8887)
Volume 150 No.5, September 2016
53
is used to produce the T2 FMGM with uncertain mean vector
(T2 FMGM-UM) or uncertain variance (T2 FMGM-UV).
Fig. 5(a) the Gaussian primary MF with uncertain mean.
b): At the right, the Gaussian primary MF with uncertain
std having uniform possibilities. The shaded region is the
footprint of uncertainty. The thick solid and dashed lines
denote the lower and upper MFs.
Uncertainty mean is [ ,] .and uncertainty in variance
is [ ,]. The factor km and kv control the intervals in
which the parameter vary as follows:
=, =+, 0,3 (10)
=, =1
, [0.3,1] (11)
This approach works better for camera jitter as compared to
original GMM as can be seen from Fig 6
(a) (b) (c) (d)
Fig.6 (a) Test sequence (b) original GMM (c) T2 FMGM-
UM (d) T2 FMGM-UV Foreground Mask.
Huijun Di et al. [19] said that original GMM assumes the
correspondence among the pixels in concurrent frames,
therefore cannot handle the case which contains camera jitter.
They developed a new background model by introducing
correspondence into it. Based on this model, they formulated
the foreground segmentation and correspondence estimation
as a labeling problem. Spatial context is enforced to every
pixel based on tree structure. This allows using dynamic
programming (DP) technique to compute global optima
efficiently. Finally, background model is updated based on
estimated optimal correspondence. This method reduces
parallax effect and registration errors significantly refer fig 7
(a) (b)
Fig. 7 (a) Test Sequence (b) detection output
Achkar [20] uses hysteresis based component matching to
improve the object blob quality at object detection stage.
Camera perturbation is addressed at frame level depending on
percentage of pixels classified as foreground.
3.5 Bootstrap
This condition arises if background is continuously busy with
foreground objects. Two cases are possible in such scene.
First case, Background is too busy and no single frame will
available without object. Second case, background is busy still
some initial frames without objects are available.
Amintoosi et al [21], handled first case using QR
decomposition method. Here, Background is identified using
QR decomposition. R-values are taken from QR
Decomposition and then applied to decompose a given system
to indicate the degree of the significance of the decomposed
parts. Then image is split into small blocks and background
blocks are selected based on weakest contribution, according
to the assigned R values. This improves object detection
accuracy significantly in busy environment.
Harville et al [22] proposed adaptive Gaussian mixture per
pixel in the combined input space of depth and luminance
invariant color. This model is further improved for bootstrap
challenge by adapting the learning rate based on scene
activity. Proposed method robustly handles second kind of
bootstrap challenge. It is also suited for real time
environment.
3.6 Camouflage
This kind of scenario may contain foreground object with
color similar to background. It causes ambiguity during
detection process about decision with foreground or
background. Original GMM requires fine tuning of important
parameters but still it tends to increase in false alarm.
Cristani et al [23] proposed joint pixel-region analysis which
is called as adaptive spatio-temporal neighborhood analysis
(ASTNA). In this method, each pixel went through BGPP
(Background per Pixel) test. If this test becomes true for given
pixel then it is classified as background else considered as not
background. Pixels those are marked as 'not background' in
this test need to carry BGPR (Background Per region) test. If
this test is validate for given pixel then it will classify as
background otherwise marked as foreground.
Harville et al [22] proposed adaptive Gaussian mixture per
pixel in the combined input space of depth and luminance
invariant color. This model is improved for camouflage by
making color based segmentation criteria dependent on depth
observations. This modification significantly reduces the
misclassification caused due to camouflage.
Darell et al [24] proposed the multidimensional clustering
based on range and color at image pixel. Range based
segmentation is largely independent of color, and hence not
affected by problems of shadows and camouflage. However,
range alone is also not sufficient for the good segmentation:
depth measurements are rarely available at all pixels in the
scene, and foreground objects may be indistinguishable in
depth when they are close to the background. Color
segmentation is complementary in these cases. Their
combinational clustering robustly deals with camouflage
situation.
3.7 Foreground Aperture
It is the phenomena in which parts of large moving
homogenous regions become part of the background instead
of being selected as moving pixels. Very less research has
been taken place for this challenge. However, it must carry
more attention as one of the important common issue in
background.
Utasi et al [25] uses separate foreground model with single
Gaussian distribution to represent large homogenous
foreground areas. This model is updated in same way as
background model in original GMM. Foreground pixels are
investigated for similar foreground neighbor, to deal with
foreground aperture challenge. If the pixel has foreground
component variance within a range of the investigated pixel
then neighbor’s deviation is increased. It is known as deviance
International Journal of Computer Applications (0975 8887)
Volume 150 No.5, September 2016
54
flooding. It is checked for pixels within fix radius and marked
as foreground pixel.
(a) (b) (c)
Fig 8 (a) Test sequence (b) original GMM(C) Utasi model
Foreground Mask
Table 1 summarizes all GMM approaches based on
background challenges.
Table 1 Summary of GMM approaches corresponds to background challenges
Sr. No.
Background
Challenges
GMM Modifications/Extensions
Authors
1
Shadow
1. Mixed Color Space
2. Adaptation of luminance and saturation difference
3. YCbCr color space with shadow detection rules
4. Normalized cross correlation
5. Statistical Disturbance Technique
1. Wang et al [6]
2. Porikilly et al [7]
3. Kristensen et al [8]
4. Tian et al [9]
5. Mazeed et al [10]
2
Sudden and Slower
Illumination
change
1. Cascaded GMM
2. Learning rate adaptation based on illumination
change score
3. Combination of color and gradient information
4. Long term and short term tendencies of change
5. Learning rate adjustment if sudden change
1. Teixeira et al [11]
2. Porikily et al [7]
3. Javed et al [12]
4. Hu et al [13]
5. Wang et al [6]
3
Mutimodality
1. Dirichlet Prior
2. Online EM algorithm
3. Background construction using DCT
4. Type-2 Fuzzy Gaussian Mixture Model (T2-FGMM)
and Markov Random Field (MRF)
1. Zivcovic[14]
2. Lee at al [15]
3. Chen et al [16]
4. Zhao et al [17]
4
Camera Jitter
1. T2 FMGM-UM and T2 FMGM-UV
2. Background model with correspondence
3. Hysteresis based component matching
1. Bouwmans et at [18]
2. Huijun Di et al.[19]
3. Achkar [20]
5
Bootstrap
1. QR decomposition
2. Learning rate adaptation based scene activity
1. Amintoosi et al [21]
2. Harville et al [22]
6
Camouflage
1. Adaptive Spatio-Temporal Neighborhood
Analysis(ASTNA)
2. Object segmentation based on color and depth
3. Multidimensional clustering based on range and
color
1. Cristani et al [23]
2. Harville et al [22]
3. Darell et al [24]
7
Foreground
Aperture
1. Separate foreground model with single Gaussian
distribution
1. Utasi et al [25]
4. CONCLUSION
This paper presents the extensive survey of various GMM
approaches to deal with different background challenges. It
also provides brief description of GMM approaches.
Background challenges that can be tackled by various GMM
versions include shadow, slow and sudden illumination
change, multimodality, camera jitter, bootstrap, camouflage,
foreground aperture. Among these issues, shadow,
illumination change are extensively explored by researcher
while foreground aperture remains less attentive. Various
GMM versions can handle multiple issues concurrently. This
study helps researcher to select appropriate version of GMM
based on their application. This survey on GMM approaches
with rich bibliography content can give valuable insight into
this important background challenges and encourage new
research.
5. REFERENCES
[1] Jun-Wei Hsieh, Shih-Hao Yu, Yung-Sheng Chen, An
Automatic Traffic Surveillance System for Vehicle
Tracking and Classification, IEEE Transactions on
Intelligent Transportation Systems, Vol. 7,
[2] S. Kannan, A. Sivasankar, Moving Object Detection
Based on Fuzzy Color Histogram Features and Dynamic
Threshold Optimization , The International Journal of
Science & Technoledge, Vol 2 Issue 3 March, 2014
[3] Thierry Bouwmans, Fida El Baf, Bertrand Vachon.
“Background Modeling using Mixture of Gaussians for
Foreground Detection - A Survey”. Recent Patents on
Computer Science, Bentham Science Publishers, 2008, 1
(3), pp.219-237.
[4] Yannick Benezeth, Pierre-Marc Jodoin, Bruno Emile,
Helene Laurent, Christophe Rosenberger, “Comparative
study of background subtraction algorithms”. Journal of
Electronic Imaging, Society of Photo-optical
Instrumentation Engineers, 2010
[5] Stauffer C, Grimson W. “Adaptive background mixture
models for real-time tracking”. Proc. IEEE Conf. on
Comp Vision and Patt. Recog.(CVPR 1999) 1999; 246-
252
[6] Wang H, Suter D. A Re-Evaluation of Mixture-of-
Gaussian background modeling. 30th IEEE Int Conf on
International Journal of Computer Applications (0975 8887)
Volume 150 No.5, September 2016
55
Acoustics, Speech, and Signal Processing (ICASSP 20
05 ), Pennsylvania, USA, March 2005, 1017-1020.
[7] Porikli F. Human body tracking by adaptive background
models and mean-shift analysis. IEEE Int Workshop on
Performance Evaluation of Tracking and Surveillance
(PETS 2003), March 2003.
[8] Kristensen F, Nilsson P, Öwall V. Background
segmentation beyond RGB. ACCV 2006, Hyderabad,
Indian, 2006, 602 -612.
[9] Tian Y, Lu M, Hampapur A. Robust and efficient
foreground analysis for real-time video surveillance.
CVPR 2005, San Diego, USA, June 2005, 1182-1187.
[10] Al-Mazeed A, Nixon M, Gunn S. Classifiers
combination for improved motion segmentation. Proc of
Int Conf on Image Analysis and Recognition (ICIAR
2004), Porto, Portugal, 2004; 363-371.
[11] Teixeira L, Cardoso J, Corte-Real L. Object
segmentation using background modelling and cascaded
change detection. J Multimedia 2007; 2(5): 55-65.
[12] O. Javed, K. Shafique, and M. Shah, “A hierarchical
approach to robust background subtraction using color
and gradient information,” in Proceedings of IEEE
Workshop on Motion and Video Computing (MOTION
’02), pp. 22–27, Orlando, Fla, USA, December 2002.
[13] Hu J, Su T. Robust background subtraction with shadow
and highlight removal for indoor surveillance. J Adv
Signal Proc 2007; 2007: 1-14.
[14] Zivkovic Z. Improved adaptive Gaussian mixture model
for background subtraction. Int Conf Pattern Recognition
(ICPR 2004), 2004, 2: 28-31.
[15] Lee D. Improved adaptive mixture learning for robust
video background modeling. IAPR Workshop on
Machine Vision for Application (MVA 2002), Nara,
Japan, December 2002; 443-446
[16] Wang W, Gao W, Yang J, Chen D. Modeling
background from compressed video. The Second Joint
IEEE Int Workshop on Visual Surveillance and
Performance Evaluation of Tracking and Surveillance, in
conjunction with the Tenth IEEE Int Conf on Computer
Vision (ICCV 2005), Beijing, China, October 2005; 161-
168.
[17] Zhao, Z., Bouwmans, T., Zhang, X. and Fang, Y., 2012.
A fuzzy background modeling approach for motion
detection in dynamic backgrounds. In Multimedia and
Signal Processing (pp. 177-185). Springer Berlin
Heidelberg.
[18] Fida El Baf, Thierry Bouwmans, Bertrand Vachon.
Type-2 Fuzzy Mixture of Gaussians Model: Application
to Background Modeling. G. Bebis et al. (Eds). ISVC
2008, Dec 2008, Las Vegas, United States. Springer-
Verlag, LNCS 5358 (Part I.), pp.772-781, 2008.
[19] Rao N, Di H, Xu G. Joint correspondence and
background modeling based on tree dynamic
programming. Int Conf on Pattern Recognition (ICPR
2006), 2006, 425-428.
[20] Achkar F, Amer A. Hysteresis-based selective Gaussian
mixture models for real-time background maintenance.
SPIE Symposium on Electronic Imaging, Conf on Visual
Communications and Image, Proc, San Jose, CA, USA,
January 2007.
[21] Amintoosi M, Farbiz F, Fathy M, Analoui M, Mozayani
N. QR decomposition-based algorithm for background
subtraction. ICASSP 2007, April 2007, 1: 1093-1096.
[22] Harville M, Gordon G, Woodfill J. Foreground
segmentation using adaptive mixture models in color and
depth. Proc of the IEEE Workshop on Detection and
Recognition of Events in Video, Vancouver, Canada,
July 2001
[23] Cristani M, Bicego M, Murino V. Integrated Region- and
Pixelbased approach to background modeling. Proc of
IEEE Workshop on Motion and Video Computing
(MOTION 2002), 2002; 3-8.
[24] Gordon G, Darrell T, Harville M, Woodfill J.
Background estimation and removal based on range and
color. Proc of the IEEE Conf on Computer Vision and
Pattern Recognition (CVPR 1999), June 1999, 2: 459-
464
[25] Utasi A, Czúni L. Reducing the foreground aperture
problem in mixture of gaussians based motion detection.
6th EURASIP Conf Focused on Speech and Image
Processing, Multimedia Communications and Services
EC-SIPMCS 2007, Maribor, Slovenia, 2007.
[26] KaewTraKulPong P, Bowden R. An improved adaptive
background mixture model for real-time tracking with
shadow detection. Proceedings 2nd European Workshop
on Advanced Video Based Surveillance Systems (AVBS
2001) , Kingston, UK, September 2001.
IJCATM : www.ijcaonline.org
... Each pixel is modelled as a mixture of k normal distributions. Detail of GMM can be studied in [12]. ...
... Time constant is achieved by taking the reciprocal of the learning rate which helps in determining the distribution parameter variation. GMM helps in dealing effectively with background situations because the learning rate determines the speed of adaptation to the illumination changes in the background [12]. Fig. 3, shows the results of background subtraction achieved through GMM when applied to input video frame in Fig. 2. ...
... In this paper, a framework for vehicle detection, tracking and counting was proposed. Its key functions are detecting vehicles which form the foreground part of a frame using Gaussian Mixture model [11], [12] which gives a binary mask. This binary mask is then subjected to morphological operators [23]. ...
... Irene's research in [7] performed a combination of Mean-shift and Kalman filters. This combination is assembled to obtain the power of the two algorithms aiming the mean-shift to perform tracking when there are no objects blocking the movement. ...
Conference Paper
Full-text available
Based on Type-2 Fuzzy Gaussian Mixture Model (T2-FGMM) and Markov Random Field (MRF), we propose a novel background modeling method for motion detection in dynamic scenes. The key idea of the proposed approach is the successful introduction of the spatial-temporal constraints into the T2-FGMM by a Bayesian framework. The evaluation results in pixel level demonstrate that the proposed method performs better than the sound Gaussian Mixture Model (GMM) and T2-FGMM in such typical dynamic backgrounds as waving trees and water rippling.
Article
Full-text available
We present a comparative study of several state-of-the-art background subtraction methods. Approaches ranging from simple background subtraction with global thresholding to more sophisticated statistical methods have been implemented and tested on different videos with ground truth. The goal is to provide a solid analytic ground to underscore the strengths and weaknesses of the most widely implemented motion detection methods. The methods are compared based on their robustness to different types of video, their memory requirements, and the computational effort they require. The impact of a Markovian prior as well as some postprocessing operators are also evaluated. Most of the videos used come from state-of-the-art benchmark databases and represent different challenges such as poor SNR, multimodal background motion, and camera jitter. Overall, we not only help to better understand for which type of videos each method best suits but also estimate how, sophisticated methods are better compared to basic background subtraction methods.
Article
Full-text available
Mixture of Gaussians (MOG) has been widely used for robustly modeling complicated backgrounds, especially those with small repetitive movements (such as leaves, bushes, rotating fan, ocean waves, rain). The performance of MOG can be greatly improved by tackling several practical issues. In this paper, we quantitatively evaluate (using the Wallflower benchmarks) the performance of the MOG with and without our modifications. The experimental results show that the MOG, with our modifications, can achieve much better results - even outperforming other state-of-the-art methods.
Conference Paper
Full-text available
Foreground segmentation with moving camera is a challenging task due to the presence of parallax effect, registration error, scene variations in out door, and etc. Currently, background modeling techniques either assumes correspondence among pixels in concurrent frames or do not model it explicitly. The contribution by this paper is in two folds. First, we achieve a new background model by introducing correspondence into it. Second, we pose foreground segmentation and correspondence estimation as a labeling problem. Spatial context is enforced in shape of tree structure and global optimal label at each node is computed using dynamic programming. Finally, based on the optimal correspondence, background model is updated. Resultantly, parallax effect and registration error are reduced significantly. Primary experiments proved our algorithm to be robust in performance
Conference Paper
Full-text available
This work describes a new 3D cone-shape illumination model (CSIM) and a robust background subtraction scheme involving shadow and highlight removal for indoor-environmental surveillance. Foreground objects can be precisely extracted for various post-processing procedures such as recognition. Gaussian mixture model (GMM) is applied to construct a color-based probabilistic background model (CBM) that contains the short-term color-based background model (STCBM) and the long-term color-based background model (LTCBM). STCBM and LTCBM are then proposed to build the gradient-based version of the probabilistic background model (GBM) and the CSIM. In the CSIM, a new dynamic cone-shape boundary in the RGB color space is proposed to distinguish pixels among shadow, highlight and foreground. Furthermore, CBM can be used to determine the threshold values of CSIM. A novel scheme combining the CBM, GBM and CSIM is proposed to determine the background. The effectiveness of the proposed method is demonstrated via experiments in a complex indoor environment
Article
We propose a novel Mixture of Gaussian (MOG)-based real-time background update technique. The proposed technique consists of a new selective matching scheme based on the combined approaches of component ordering and winner-takes-all. This matching scheme not only selects the most probable component for the first matching with new pixel data, greatly improving performance, but also simplifies pixel classification and component replacement in case of no match. Further performance improvement achieved by using a new simple and functional component variance adaptation formula. Also in this technique, the proposed new hysteresis-based component matching and temporal motion history schemes improve segmentation quality. Component hysteresis matching improves detected foreground object blobs by reducing the amount of cracks and added shadows, while motion history preserves the integrity of moving objects boundaries, both with minimum computational overhead. The proposed background update technique implicitly handles both gradual illumination change and temporal clutter problems. The problem of shadows and ghosts is partially addressed by the proposed hysteresis-based matching scheme. The problem of persistent sudden illumination changes and camera movements are addressed at frame level depending on the percentage of pixels classified as foreground. We implemented three different state-of-the-art background update techniques and compared their segmentation quality and computational performance with those of the proposed technique. Experimental results on reference outdoor sequences and real traffic surveillance streams show that the proposed technique improved segmentation accuracy for extracting moving objects of interest compared to other reference techniques.
Conference Paper
Multiple classiflers have shown capability to improve per- formance in pattern recognition. This process can improve the overall accuracy of the system by using an optimal decision criteria. In this pa- per we propose an approach using a weighted benevolent fusion strategy to combine two state of the art pixel based motion classiflers. Tests on outdoor and indoor sequences conflrm the e-cacy of this approach. The new algorithm can successfully identify and remove shadows and high- lights with improved moving-object segmentation. A process to optimise shadow removal is introduced to remove shadows and distinguish them from motion pixels. A particular advantage of our evaluation is that it is the flrst approach that compares foreground/background labelling with results obtained from ground truth labelling.
Conference Paper
Background estimation and removal based on the joint use of range and color data produces superior results than can be achieved with either data source alone. This is increasingly relevant as inexpensive, real-time, passive range systems become more accessible through novel hardware and increased CPU processing speeds. Range is a powerful signal for segmentation which is largely independent of color and hence not effected by the classic color segmentation problems of shadows and objects with color similar to the background. However range alone is also not sufficient for the good segmentation: depth measurements are rarely available at all pixels in the scene, and foreground objects may be indistinguishable in depth when they are close to the background. Color segmentation is complementary in these cases. Surprisingly, little work has been done to date on joint range and color segmentation. We describe and demonstrate a background estimation method based on a multidimensional (range and color) clustering at each image pixel. Segmentation of the foreground in a given frame is performed via comparison with background statistics in range and normalized color. Important implementation issues such as treatment of shadows and low confidence measurements are discussed in detail
Conference Paper
We present a new method to robustly and efficiently analyze foreground when we detect background for a fixed camera view by using mixture of Gaussians models and multiple cues. The background is modeled by three Gaussian mixtures as in the work of Stauffer and Grimson (11). Then the intensity and texture information are integrated to remove shadows and to enable the algorithm working for quick lighting changes. For foreground analysis, the same Gaussian mixture model is employed to detect the static foreground regions without using any tracking or motion information. Then the whole static regions are pushed back to the background model to avoid a common problem in background subtraction - fragmentation (one object becomes multiple parts). The method was tested on our real time video surveillance system. It is robust and run about 130 fps for color images and 150 fps for grayscale images at size 160x120 on a 2GB Pentium IV machine with MMX optimization.