PERFORMANCE ANALYSIS OF EIGENFACE RECOGNITION UNDER VARYING EXTERNALCONDITIONS

Full text

44
Scientific Bulletin of the „Petru Maior” University of Tîrgu Mureş
Vol. 11 (XXVIII) no. 2, 2014
ISSN-L 1841-9267 (Print), ISSN 2285-438X (Online), ISSN 2286-3184 (CD-ROM)
PERFORMANCE ANALYSIS OF EIGENFACE
RECOGNITION UNDER VARYING EXTERNAL
CONDITIONS
Szidónia LEFKOVITS1, László LEFKOVITS2
1Universitatea “Petru Maior”
Nicolae Iorga 1, Tîrgu-Mureş, Romania
1szidonia.lefkovits@science.upm.ro
2Universitatea Sapientia
Şos. Sighişoarei 1C, Tîrgu-Mureş/Corunca, Romania
2lefkolaci@ms.sapientia.ro
Abstract
In the field of image processing and computer vision face recognition is one of the most
studied research domain. It has large variety of applications in different areas like security
and surveillance systems, identification and authentication etc.
In this paper we propose to analyze the face recognition system based on the eigenface[22]
method under different conditions. The eigenface method is a statistical dimensionality
reduction method, which obtains the adequate face space, out of a given training
database. The idea of observing the performances i.e. the recognition rate in different
situations (like presence or absence of important facial features such as glasses or beard)
came from the diploma work [20]. The experiments described in this article study the
recognition performance of the algorithm, by varying the number of considered feature
vectors. Beside of these, we studied the behavior of such a system if the analyzed individual
is wearing glasses or beard. Finally, we concentrate on carrying out experiments for noisy
images by adding common types of noise like salt & pepper noise, Gaussian noise or
Poisson noise to every test image.
Key words: face recognition, Eigenfaces, Principal Component Analysis, performance evaluation
1. Introduction
Object recognition is an important research
domain because it can be applied in a wide variety of
systems. Especially human face recognition is the
most widespread research area, owing to the fact that
face is usually in handy it can be recorded easily by
different types of visible or hidden cameras, the
subject doesn’t even notice that he/she is recorded.
Other methods that are more reliable and are used
in biometric identification, for example fingerprint
recognition, iris recognition, need special devices
such as high resolution cameras.
On the other hand, face recognition is the most
used identification method wherewith people
recognize each other.
Automated facial recognition is a biometric
application from a single image or video sequence,
comparing the detected face to a large database of
faces, from individuals already known.
Several application areas of human face
recognition are known, such as: biometrical
identification, access control, video surveillance,
passport control, security systems, banc verification
for ATM, image categorization in films and videos,
identification of thefts in open-air cameras.
Face recognition systems can be divided in two
classes global aspect based methods and local aspect
based methods. The first one is called appearance
based method and the second one is also called part-
based or feature based method.
This article concentrates only on the eigenface
method, the most common face recognition system,
which can be included in the global aspect based
category. Besides implementing the eigenface
method, developed by Turk and Pentland [22], it
studies the influence of several external conditions on
the recognition performance, such as noise,
blurriness, illumination; change of facial features:
wearing glasses or presence of facial hear.
The paper is organized as follows: in the first
section a short introduction is presented, it follows a
summary about the related most important works in
the domain. The third section presents the eigenface

45
method and finally some comparative experimental
results and detection performances are exposed.
2. Related Work
Many approaches have been developed for face
recognition. In this section the most used such
systems are summed-up.
Several types of Artificial Neural Networks have
been used for face recognition: single layer adaptive
network, multilayer perceptron, convolutional
networks and probabilistic neural networks for
handling partial occlusion or distortion [7].
Elastic Bunch Graph Matching is also used for
face recognition. Here a dynamic graph is constructed
where vertexes are the features and edges the
distances between given features [14].
Face representation in 3D is one of the
geometrical representation techniques that are
developed as well, they are based on Hidden Markov
Models [2].
Despite of the large variety of face recognition
systems the most common approach with extremely
good results is still the eigenface method. This
method is the outcome of the simplest global aspect
based method, which takes in account the intensity of
pixels. Here, the two dimensional unknown face
image is compared to all the other faces from the
training database. Comparing faces pixel by pixel
works only in limited conditions, under given
circumstances. Its major bottleneck is the comparison
and classification in a very high dimensional space.
Thus, appears the need of dimensionality reduction.
One of the most common dimensionality reduction
methods is the extraction of Principal Components.
Kirby and Sirovich [8] exploit the PCA (Principal
Component Analysis) method for face recognition by
using the Karhunen-Loève conditions, in order to
define the geometry of faces. Turk and Pentland [22]
developed a recognition system which tracks the
subjects head and recognizes it by comparing the
characteristics of it to those of known individuals.
The system projects face images onto a feature space
named “face space” that spans the significant features
from known images. The significant projections are
called eigenfaces, because they are eigenvectors of
the face space. Su et al. [21] combines PCA and
Linear discriminant Analysis (LDA) for extracting
multi features and makes the final decision with
radial basis function network.
Recently, different types of PCA-based algorithms
have been developed by researchers of this domain:
weighted modular PCA [9], Kernel PCA [12],
diagonal PCA [23], adaptive PCA [3], two-
dimensional PCA [11]
Instead of PCA, Bartlett et al. [1] uses
Independent Component Analysis, because this
method is a generalization of PCA. Liu et al. [10]
combined Gabor wavelet transform with Fisher linear
discriminant and kernel PCA or Gabor features with
fractional polynomial models or Gabor features ICA
and PCA.
Nicholl et al. [13] created a face recognition
system which automatically selects the coefficient for
DWT and PCA.
Poon et al. [15, 16] analyze the performance of
PCA recognition for different datasets, varying the
image size, alignment, training set, blurriness,
illumination condition and noise. Shermina [19]
presents a method based on multi linear principal
component analysis.
Rady [17] compares different distance classifiers
with the same eigenface method. Dabhade [4]
combines Haar Detection, Gabor feature extraction
and Eigenface recognition in the achieved system.
For more details about other techniques and recent
advances in face recognition consult the survey
articles [6, 18]
3. The Eigenface method
The idea of retrieving relevant features from a set
of training images can be obtained by the extraction
of principal components. These features are not
necessarily evident, perceptible features such as facial
feature parts, but they characterize the common part
of a given set.
If we consider every gray-scale pixel of
nm
image, a feature space of this is obtained. Nowadays,
the number of pixels of an image is more hundred
thousand, even million. These dimensions can hardly
be applied in any classification algorithm, therefore
comes the necessity of dimensionality reduction.
The Principal Component Analysis is a statistical
method for dimensionality reduction, while
minimizing the mean square reconstruction
error [16, 5].
The PCA is able to extract only the relevant
information of a given space and transform every
element of it in a considerably lower dimensional
space.
Let us consider
L
images each of them having
the same dimension
nm
.
12
{, },
training L
S I I I
(1)
Each image is vectorized and we obtain an
N
dimensional column vector, where
N n m
.
The PCA is a linear algebra concept based on the
eigenvectors and eigenvalues of a matrix.
The eigenvector is a vector which is scaled by
a linear transformation. When a matrix acts on it, the
direction of the vector does not change, only the
magnitude.
Av v


(2)
In equation (2) the A is the analyzed matrix,
v
the
eigenvector and the scalar

is the corresponding
eigenvalue of the eigenvector
v
.
Equation (2) can be rewritten as the characteristic
equation of the matrix
A
.
 
0A I v


(3)
Nontrivial solution of the characteristic equation
exists if and only if
det
 
0AI


(4)

46
The characteristic equation is an
NN
linear
homogeneous equation system, with
N
equations
and
N
unknowns. The solutions of the
N
degree
characteristic polynomial (4) are the eigenvalues of
the
A
matrix. Because the degree characteristic
polynomial is
N
, we obtain
N
roots. These
N
roots
are not necessarily distinct. If all the eigenvalues are
distinct, the corresponding eigenvectors are linear
independent and form an
N
dimensional basis.
The matrix for which the eigenvalues are
computed is the covariance matrix of the input space
denoted by
training
S
in equation (1).
The covariance matrix of two random variables
e.g. images is
( , ) [(I )(I )]
i j i i jj
cov I I E

    
(5)
where
E
is the expected value.
The matrix form of the covariance matrix of the
whole set of images
training
S
can be rewritten as
( ) [(I [ ])(I [])]
T
cov I E E EII    
(6)
here
I
is the training set of images and
()EI


is
the mean image. The mean image is a column vector
of
N
pixels, as well.
1
1
() L
i
i
E I I
L




(7)
If the eigenvalues and eigenvectors of the covariance
matrix

are computed the “eigenfaces” are
obtained. This name comes from the parents of the
Eigenface method Turk and Pentland [22], they call it
so because the obtained eigenvectors are similar to
faces in appearance.
The covariance matrix is an
22
NN
matrix, which
can be simply formulated as
  
1
1LTT
ii
i
I I AA
L


    

(8)
where
A
is an
2
NL
matrix column-vise formed
of the differences of each image and the mean image,
the
i
th column is the
i
th difference.
21 [ ]
L
A I I I
  
  
(9)
The problem is the dimension of the covariance
matrix

which became
22
NN
. Fortunately, we
can obtain the same eigenvalues by computing them
for another covariance matrix. This covariance matrix
is
'T
AA
(10)
The dimension of this
'
matrix is
LL
, where
L
is the number of images. The order of magnitude of
2
N
is million, but the order of
L
is thousand
(
2
LN
). Thus, instead of computing eigenvalues
of a huge matrix (8) we compute the eigenvalues of a
such smaller matrix (10).
Consider the eigenvectors of
'
such that
'i i i
vv


(11)
Ti i i
A Av v


(12)
Left multiplying both sides with
A
we obtain
Ti i i
AA Av Av


(13)
We denoted
T
AA
Equation (13) becomes
i i i
Av Av


(14)
Let us denote
ii
u Av
 
1
L
i il i
l
u v I




(15)
then we obtain the eigenvalues
i

and eigenvectors
i
u
of the

covariance matrix.
i i i
uu


(16)
Thus, if we compute the eigenvalues of
'
, the same
are the eigenvalues of

and in the same way, if we
compute the eigenvectors
i
v
of corresponding
eigenvalues for
'
, we obtain the eigenvectors
i
u
of

using the equation (15).
After computing all the
L
eigenvalues, we have to
select the most representative
P
eigenvalues. This
value of
P
is less the hundred.
From the largest P eigenvalues the corresponding P
eigenvectors are computed, and a P dimensional span
is obtained, which is called by the authors of [22] the
“facespace”.
The training step is the computation of the facespace,
based on a set of input images (see eq.(1)).
In the test phase the input image has to be
projected into the facespace. Each of the selected
P
eigenvectors will have a corresponding weight.
These weights are, in fact, the eigenface components
of the input image, and are computed by a simple dot
product.
 
T
p p in
w u I


(17)
From these vectors putting column-wise we can form
a matrix W.
12
[ , ,..., ]
p
W w w w
(18)
If the eigenvectors form a basis then the input image
can be reconstructed using the following linear
combination:
rec p p
I w u
(19)
In order to determine which person resembles the
input image best, so an error has to be computed. This
error is the means square error of two
W
matrixes [22].
in
W
is the weight matrix of the input
image and
pers
W
is the average weight matrix of more
images for a given person.
2
in pers
MSE W W
(20)
The minimum of this mean square error will
determine the most alike person.
If the error is greater than a threshold we can say that
the input image is an unknown person.
In order to find out that input image is not a face
image another distance has to be computed, the
distance between the average reconstructed image
and the difference between the input image and the
mean image (
in in
AI


).
2
face in Irec
MSE A Avg
(21)

47
If this value is less than a threshold it means that the
input image is a face, otherwise it is an unknown
object the projection of which in the facespace is
useless.
4. Results and Experiments
In our experiments we have used the Yale
database [24] specially developed for face recognition
and facial expression recognition. This database has
165 grayscale images of 15 individuals and 11
different photos about each individual. These 11
poses to a facial expression: normal, happy, sad,
sleepy, surprised and wink. Different configuration
refers to center-light, left-light, right-light. Besides,
each subject appears also with and without glasses.
At the same time we generated another condition and
put beard to each subject.
We have to underline that these images are not
aligned and are in different illumination conditions.
In order to form an acceptable training set we have
normalized the images (see fig 1 original image and
normalized image). As we can observe the
normalization brings uniform illumination conditions.
It seems that this procedure messes up uniformly
lighted “normal” image, but repairs the left-lighted
and right-lighted image in some way. Further we will
see that different illumination conditions have a great
impact on recognition with this eigenface method.
Fig. 1: a).Yale database sample image [24]
b). Normalized sample image
c). Left light d). Normalized left light
The second step after normalization is obtaining the
mean image of the training data set. Because the
images are not properly aligned the mean image
shows only a blurred head shape formed of several
contours. Regarded to the covariance and
resemblance this image is the common part of each
image, that is why it has to be subtracted from each
image in the training set (see fig. 2 a). and b).)
Fig. 2: a). Mean image denoted by µ
b). Difference image
After obtaining the difference images, comes the
computation of covariance matrix and out of it we
obtain the eigenvalues and eigenvectors.
Fig. 3: First four eigenfaces
In our experiments we studied the influence of
number of used eigenvectors. As shown in fig. 3 the
eigenvectors corresponding to the most representative
(first four) eigenvalues are the strong contours of the
head, the first eigenvector is the shape of an average
head.
Fig. 4: 47th, 48th, 49th, 50th
As we compute more and more eigenvectors
corresponding to smaller values we can observe that
these represent the fine contours. Overall, we have to
use several eigenvectors corresponding to strong
contours. These give the basis of the shape. But at the
same time we have to consider some fine contours as
well.
We noticed that if the number of eigenvectors is
increased, the recognition accuracy becomes better,
until a certain amount, where it saturates.
In Table 1 we measured the recognition rate using 25,
50, 100, 112 eigenvectors.
The best detection rate is 99.38% for the considered
training data set. Table 1 shows that after 100
eigenvectors the increase of the number of such
significant vectors taken in account is useless.
Table 1: Number of most representative eigenvalues
Number of
Eigenface features
Recognition Rate
25
92.72%
50
96.36%
75
98.18%
100
99.39%
112
99.39%

48
In the testing phase each image is compared to the
existing classes in the training set. Each class
corresponds to a certain individual. For each image in
the training set we compute its weight. This weight is
the dot product between difference image and the
eigenvectors (equation (17)). After computing the
weight-vector for the training set we compute the
average of it for each individual. This average weight
is compared to the weight obtained for the tested
image (equation (20)). The similarity measure in this
case is the means square error between these two
vectors measured by the Euclidean distance.
As more eigenvectors we have considered as higher is
the dimensionality of the obtained weight-vector
(equation (18)). This means that we have more
components, so the vectors contain more information
based on which the error is more precise. This
statement can be also verified visually by computing
the reconstructed image. Comparing the reconstructed
images out of 10, 50, 100 and 112 eigenvectors we
can confirm that the more eigenvectors are used the
accurate the reconstruction is.
Fig. 5: Reconstructed image 10, 50, 100, 112 eigenvectors
Our second experiment studied the recognition rate of
the person if he/she wears glasses or he is bearded. In
this case the training set does not contain images with
these disturbing factors. As we have observed from
the measurements the reconstruction rate of people
with glasses is 86.66% and those with beard was
92.85% (table 2).
Table 2: Effect of different facial features
Noise type
Detection Rate
Glasses
86.66%
Beard
92.85%
Our final experiment observed the effect of noise in
the recognition process. We have tested three types of
different noises. The salt & pepper noise, the
Gaussian noise and the Poisson noise and other
illumination changes.
Considering our experiments we draw the following
conclusions: illumination changes have a great impact
on the recognition rate, mainly if the illumination is
not uniform and the source comes from different
directions. In the Yale database we compared the
center-light, left-light and right-light poses(figure 1c).
Gaussian noise is white noise with mean 0 and
covariance 1 (figure 6a).
Salt & pepper noise is a noise with white black pixels
with density 5% of the total pixels. (figure 6b)
The Poisson noise is separately computed for each
pixel with the Poisson mean which is equal to the
value of the pixel (figure 6c). The recognition rates
for these types of images are presented in table 3.
Table 3: Effect of noise
Noise type
Detection Rate
Salt & Pepper
98.36%
Gaussian
98.21%
Poisson
98.76%
Fig. 6: a).Gaussian b). Salt & Pepper c).Poisson
5. Conclusion and future work
Eigenface recognition is one of the most used face
recognition system, but it has to respect certain
conditions like geometric aligning of faces, uniform
illumination conditions. In the presented article we
analyzed these types of external conditions, putting
an accent on varying characteristic facial features
such as glasses or beard. We have observed that the
number of considered descriptive features increases
the recognition rate. The increase of image noise
slightly decreases the recognition accuracy.
Occlusion, facial features have little effect on
recognition, while illumination or light sources from
different directions have a great effect on the
recognition performance. Overall, these observations
can be helpful in designing future recognition
systems.
Acknowledgement
Financial support of the work of Szidónia
Lefkovits was provided from programs co-financed
by The Sectorial Operational Program of Human
Resources Development, Investing in people! Key
Area of Intervention 1.5 “Doctoral and Postdoctoral
scholarship in support of research”, Project title:
“Integrated system for quality improvement of the
doctoral and post-doctoral research in Romania and
promoting the role of science in society”, Contract:
POSDRU/159/1.5/S/133652.
The work of László Lefkovits in this paper is
supported by The Sectorial Operational Programme
Human Resources Development
POSDRU/159/1.5/S/137516 financed by the
European Social Found and by the Romanian
Government.

49
References
[1] Bartlett, M.S, Movellan, J.R. and Sejnowski T.J.
(2002) Face recognition by independent
component analysis IEEE Transaction on Neural
Networks vol. 13, no. 6, pp. 1450–1464.
[2] Castellani, U., M. Cristani, X. Lu, V. Murino
and Jain A.K. (2008), HMM-based geometric
signatures for compact 3D facerepresentation
and matching. IEEE Computer Society
Conference on Computer Visionand Pattern
Recognition Workshops, (CVPRW), pp. 1-6.
[3] Chen S, Shan T, Lovell BC (2007) Robust face
recognition in rotated eigenspaces. Proceedings
of Image and Vision Computing, New Zealand,
pp 1–6
[4] Dabhade, S. A., and Bewoor, M. S. (2012)
Rapid Face Recognition Technology using
Eigenfaces and Gabor Filter. International
Journal of Science and Applied Information
Technology, 1(4).
[5] Hiremath,V. and Mayakar, A. (2009), Face
recognition using Eigenface approach. IDT
workshop on interesting results in computer
science and engineering, Sweden.
[6] Jafri, R. and Arabnia, H. R. (2009) A Survey of
Face Recognition Techniques. Journal of
Information Processing Systems (JIPS), vol.5,
no. 2, pp. 41-68.
[7] Jie, L., M. Ji and D. Crookes (2008), A
probabilistic union approach to robust face
recognition with partialdistortion and occlusion.
IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP), pp.
993-996.
[8] Kumar, A. P., Das, S. and Kamakoti V. (2004),
Face recognition using weighted modular
principle component analysis, in Neural
Information Processing, vol.3316, Lecture Notes
In Computer Science: Springer
Berlin/Heidelberg, 2004, pp. 362-367.
[9] Kirby, M. and Sirovich L. (1990), Application of
the Karhunen-Loève procedure for the
characterisation of human faces. IEEE
Transaction on Pattern Analysis, vol. 12, pp.
831-835.
[10] Liu, C. (2004) Gabor-based kernel pca with
fractional power polynomial models for face
recognition. IEEE Transaction on Pattern
Analysis and Machine Intelligence vol. 26, pp.
572–581.
[11] Meng, J. and Zhang, W. (2007), Volume
measure in 2DPCA-based face recognition,
Pattern Recognition Letters, Vol.28, pp.1203-
1208, 2007.
[12] Nhat, V. D. M. and Lee, S. (2005) An
Improvement on PCA Algorithm for Face
Recognition, Advances in Neural Networks -
ISNN 2005, Vol.3498, Lecture Notes in
Computer Science. Chongqing: Springer, 2005,
pp.1016-1021.
[13] Nicholl, P. and Amira, A. (2008), DWT/PCA
face recognition using automatic coefficient
selection. 4th IEEE International Symposium on
Electronic Design, Test and Applications,
DELTA 2008, 23-25 Jan., Belfast, pp. 390-393.
[14] Pervaiz, A.Z. (2010), Real time face recognition
system based on EBGM framework. Computer
Modellingand Simulation (UKSim), pp: 262-
266.
[15] Poon, B., Amin, M. A. and Yan, H. (2009). PCA
based face recognition and testing criteria.
IEEE International Conference on Machine
Learning and Cybernetics, vol. 5, pp. 2945-
2949.
[16] Poon, B., Amin, M. A. and Yan, H. (2011),
Performance evaluation and comparison of
PCA based human face recognition methods for
distorted images. International Journal of
Machine Learning and Cybernetics, 2(4), 245-
259.
[17] Rady, H. (2011), Face Recognition using
Principle Component Analysis with different
Distance Classifiers. International Journal of
Computer Science and Network Security, vol.11
no.10, pp. 134-144
[18] Sharif, M, Sajjad M. and Javed, M.Y. (2012), A
Survey: Face Recognition Techniques. Research
Journal of Applied Sciences, Engineering and
Technology vol. 4, no. 23, pp. 4979-4990.
[19] Shermina, J., (2011), Face recognition system
using multi linear principal component analysis
and locality preserving projection. IEEE GCC
Conference and Exhibition, 19-22 Feb., Stirling,
UK, pp. 283-286.
[20] Ştefan Răzvan Cristian (2014) Lucrare de
licenţă, Recunoaştere facială: Metoda
Eigenface. Universitatea „Petru Maior” Tîrgu-
Mureş, coordonator ştiinţific Sz. Lefkovits
[Bachelor Thesis, Facial Recognition: The
Eigenface Method].
[21] Su, H., Feng, D.D., Zhao, R. (2002), Face
recognition using multi-feature and radial basis
function Network. Proceeding of the Pan-
Sydney Area Workshop on Visual Information
Processing, Sydney.
[22] Turk, M. and A. Pentland, A. (1991), Eigenfaces
For Recognition, Journal Of Cognitive
Neuroscience, Vol.3, pp.71-86.
[23] Zhang, D.Q.A., Zhou, Z.H. and Chen S.C.
(2006), Diagonal principal component analysis
for face recognition. Pattern Recognition vol.
39, no. 1, pp. 140–142.
[24] Yale Database
http://www.vision.ucsd.edu/content/yale-face-
database