Content uploaded by Zongyi Liu
Author content
All content in this area was uploaded by Zongyi Liu on Oct 07, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Gait-based human recognition at a distance: Performance, covariate impact and solutions
Abstract and Figures
It has been noticed for a long time that humans can identify others based on their biological movement from a distance. However, it is only recently that computer vision based gait biometrics has received much attention. In this dissertation, we perform a thorough study of gait recognition from a computer vision perspective. We first present a parameterless baseline recognition algorithm, which bases similarity on spatio-temporal correlation that emphasizes gait dynamics as well as gait shapes. Our experiments are performed with three popular gait databases: the USF/NIST HumanID Gait Challenge outdoor database with 122 subjects, the UMD outdoor database with 55 subjects, and the CMU Mobo indoor database with 25 subjects. Despite its simplicity, the baseline algorithm shows strong recognition power. On the other hand, the outcome suggests that changes in surface and time have strong impact on recognition with significant drop in performance.
To gain insight into the effects of image segmentation on recognition -- a possible cause for performance degradation, we propose a silhouette reconstruction method based on a Population Hidden Markov Model (pHMM), which models gait over one cycle, coupled with an Eigen-stance model utilizing the Principle Component Analysis (PCA) of the silhouette shapes. Both models are built from a set of manually created silhouettes of 71 subjects. Given a sequence of machine segmented silhouettes, each frame is matched into a stance by pHMM using the Viterbi algorithm, and then is projected into and reconstructed by the Eigen-stance model. We demonstrate that the system dramatically improves the silhouette quality. Nonetheless, it does little help for recognition, indicating that segmentation is not the key factor of the covariate impacts. To improve performance, we look into other aspects.
Toward this end, we propose three recognition algorithms: (i) an averaged silhouette based algorithm that deemphasizes gait dynamics, which substantially reduces computation time but achieves similar recognition power with the baseline algorithm; (ii) an algorithm that normalizes gait dynamics using pHMM and then uses Euclidean distance between corresponding selected stances -- this improves recognition over surface and time; and (iii) an algorithm that also performs gait dynamics normalization using pHMM, but instead of Euclidean distances, we consider distances in shape space based on the Linear Discriminant Analysis (LDA) and consider measures that are invariant to morphological deformation of silhouettes. This algorithm statistically improves the recognition over all covariates.
Compared with the best reported algorithm to date, it improves the top-rank identification rate (gallery size: 122 subjects) for comparison across hard covariates: briefcase, surface type and time, by 22%, 14%, and 12% respectively. In addition to better gait algorithms, we also study multi-biometrics combination to improve outdoor biometric performance, specifically, fusing with face data. We choose outdoor face recognition, a "known" hard problem in face biometrics, and test four combination schemes: score sum, Bayesian rule, confidence score sum, and rank sum. We find that the recognition power after combination is significantly stronger although individual biometrics are weak, suggesting another effective approach to improve biometric recognition.
The fundamental contributions of this work include (i) establishing the "hard" problems for gait recognition involving comparison across time, surface, and briefcase carrying conditions, (ii) revealing that their impacts cannot be explained by silhouette segmentation, (iii) demonstrating that gait shape is more important than gait dynamics in recognition, and (iv) proposing a novel gait algorithm that outperforms other gait algorithms to date.
Figures - uploaded by Zongyi Liu
Author content
All figure content in this area was uploaded by Zongyi Liu
Content may be subject to copyright.
... We designed a new gait recognition algorithm around this idea, exploiting just silhouette shape matching. The details of this algorithm is available in [35], here we present just the rough outline. ...
Gait as a behavioral biometric has been the subject of recent investigations. However, understanding the limits of gait-based recognition and the quantitative study of the factors effecting gait have been confounded by errors in the extracted silhouettes, upon which most recognition algorithms are based. To enable us to study this effect on a large population of subjects, we present a novel model based silhouette reconstruction strategy, based on a population based hidden Markov model (HMM), coupled with an eigen-stance model, to correct for common errors in silhouette detection arising from shadows and background subtraction. The model is trained and benchmarked using manually specified silhouettes for 71 subjects from the recently formulated HumanID Gait Challenge database. Unlike other essentially pixel-level silhouette cleaning methods, this method can remove shadows, especially between feet for the legs-apart stance, and remove parts due to any objects being carried, such as briefcase or a walking cane. After quantitatively establishing the improved quality of the silhouette over simple background subtraction, we show on the 122 subjects HumanID Gait Challenge Dataset and using two gait recognition algorithms that the observed poor performance of gait recognition for hard problems involving matching across factors such as surface, time, and shoe are not due to poor silhouette quality, beyond what is available from statistical background subtraction based methods.
Human recognition through walking styles is among the newest of biometric methods. By using this biometric, individuals can be identified, distantly, even at low visibility. Our aim is to provide such ability for a computer system. In other words, we intend to extract appropriate features through processing video images that can reflect individuals' identity. In order to set up such a system, we have used Fourier, Wavelet, and Multi-wavelet transforms. Using images from the USF dataset version 1.7, the results obtained indicate that SA4 Multi-wavelet transforms prove more efficient in extracting suitable features than Fourier and wavelet transforms, and combined with one-versus-one Support Vector Machine, they can provide a 85.7 % recognition accuracy rate. Our proposed method shows higher accuracy and precision compared to other frequency based methods.
We describe a new method for Automatic Gait Recognition based around the use of Fourier descriptors that model the periodic
deformation of human gait. Fourier descriptors have been used successfully in the past to model the boundary of static or
moving, rigid-bodied objects, but many objects actually deform in some way as they move. Here we use Fourier descriptors to
model not only the object’s boundary, but also the spatio-temporal deformations under which the object’s boundary is subjected.
We applied this new method to the Large Gait Database, compiled at the University of Southampton, and found that the Fourier
descriptors obtained for each person appear to be unique and can be used for recognition. Successful recognition rates of
over 85% were obtained from the Large Gait Database using only a small set of descriptors.
Human gait is an attractive modality for recognizing people at a distance. In this paper we adopt an appearance-based approach
to the problem of gait recognition. The width of the outer contour of the binarized silhouette of a walking person is chosen
as the basic image feature. Different gait features are extracted from the width vector such as the dowsampled, smoothed width
vectors, the velocity profile etc. and sequences of such temporally ordered feature vectors are used for representing a person’s
gait. We use the dynamic time-warping (DTW) approach for matching so that non-linear time normalization may be used to deal
with the naturally-occuring changes in walking speed. The performance of the proposed method is tested using different gait
databases.
A multi-view gait recognition method using recovered static body parameters of subjects is presented; we refer to these parameters as activity-specific biometrics. Our data consists of 18 subjects walking at both an angled and frontal-parallel view with respect to the camera. When only considering data from a single view, subjects are easily discriminated; however, discrimination decreases when data across views are considered. To compare between views, we use ground truth motion-capture data of a reference subject to find scale factors that can transform data from different viewsi nto a common frame ("walking-space"). Instead of reporting percent correct from a limited database, we report our results using an expected confusion metric that allows us to predict how our static body parameters filter identity in a large population: lower confusion yields higher expected discrimination power. We show that using motion-capture data to adjust vision data of different views to a common reference frame, we can get achieve expected confusions rates on the order of 6%.
In this paper, we systematically analyze different components of human gait, for the purpose of human identification. We investigate dynamic features such as the swing of the hands/legs, the sway of the upper body and static features like height in both frontal and side views. Both probabilistic and non-probabilistic techniques are used for matching the features. Various combination strategies may be used depending upon the gait features being combined. We discuss three simple rules: the sum, product and MIN rules that are relevant to our feature sets. Experiments using four different data sets demonstrate that fusion can be used as an effective strategy in recognition.
We propose a view based approach to recognize humans when engaged in some activity. The width of the outer contour of the binarized silhouette of a walking person is chosen as the image feature. A set of exemplars that occur during an activity cycle is chosen for each individual. Using these exemplars, a lower dimensional frame-to-exemplar distance (FED) vector is generated. A continuous HMM is trained using several such FED vector sequences. This methodology serves to capture compactly structural and dynamic features that are unique to an individual. The statistical nature of the HMM renders overall robustness to representation and recognition. Human identification performance of the proposed scheme is found to be quite good when tested on outdoor video sequences collected using surveillance cameras.
Gait is a spatio-temporal phenomenon that typifies the motion characteristics of an individual. In this paper, we propose a view-based approach to recognize humans through gait. The width of the outer contour of the binarized silhouette of a walking person is chosen as the image feature. A set of stances or key frames that occur during the walk cycle of an individual is chosen. Euclidean distances of a given image from this stance set are computed and a lower-dimensional observation vector is generated. A continuous hidden Markov model (HMM) is trained using several such lower-dimensional vector sequences extracted from the video. This methodology serves to compactly capture structural and transitional features that are unique to an individual. The statistical nature of the HMM renders overall robustness to gait representation and recognition. The human identification performance of the proposed scheme is found to be quite good when tested in natural walking conditions.
Current two-dimensional face recognition approaches can obtain a good performance only under constrained environments. However, in the real applications, face appearance changes significantly due to different illumination, pose, and expression. Face recognizers based on different representations of the input face images have different sensitivity to these variations. Therefore, a combination of different face classifiers which can integrate the complementary information should lead to improved classification accuracy. We use the sum rule and RBF-based integration strategies to combine three commonly used face classifiers based on PCA, ICA and LDA representations. Experiments conducted on a face database containing 206 subjects (2,060 face images) show that the proposed classifier combination approaches outperform individual classifiers.
Identification of people from gait captured on video has become a challenge problem in computer vision. However, there is not a baseline algorithm or standard dataset for measuring, or determining what factors affect performance. In fact, the conditions under which the problem is "solv- able" are not understood or characterized. This paper de- scribes a large set of video sequences (about 300 GB of data related to 452 sequences from 74 subjects) acquired to investigate important dimensions of this problem, such as variations due to viewpoint, footwear, and walking sur- face. We introduce the HumanID challenge problem. The challenge problem contains a set of experiments of increas- ing difficulty, a baseline algorithm, and its performance on the challenge problem. Our results suggest that differences in footwear or walking surface type between the gallery and probe video sequence are factors that affect perfor- mance. The data set, the source code for the baseline al- gorithm, and UNIX scripts to reproduce the basic results reported here are available to the research community at http://marathon.csee.usf.edu/GaitBaseline/
The intimate relationship between human walking and running lies within the skeleto-muscular structure. This is expressed as a mapping that can transform computer vision derived gait signatures from running to walking and vice versa, for purposes of deployment in gait as a biometric or for animation in computer graphics. The computer vision technique can extract leg motion by temporal template matching with a model defined by forced coupled oscillators as the basis. The (biometric) signature is derived from Fourier analysis of the variation in the motion of the thigh and lower leg. In fact, the mapping between these gait modes clusters better than the original signatures (of which running is the more potent) and can be used for recognition purposes alone, or to buttress both of the signatures. Moreover, the two signatures can be made invariant to gait mode by using the new mapping.
Face and iris identification have been employed in various biometric applications. Besides improving the verification performance, the fusion of both of the biometrics has several other advantages such as enlarging user population coverage and reducing enrollment failure. In this paper, we make a first attempt to combine face and iris biometrics. We use two different strategies for fusing iris and face classifiers. The first strategy is to compute either an unweighted or weighted sum of the two matching distances and compare the distances to a threshold. The second strategy is to treat the matching distances of face and iris classifiers as a two-dimensional feature vector and use a classifier such as the Fisher's discriminant analysis or a neural network with radial basis function (RBFNN) to classify the vector as being genuine or an impostor. We compare the results of the combined classifier with the results of the individual face and iris classifiers. Experimental results show the validity of the proposed fusion strategies.