Content uploaded by Kartik Meena
Author content
All content in this area was uploaded by Kartik Meena on Oct 15, 2020
Content may be subject to copyright.
LOCAL BINARY PATTERNS AND ITS
VARIANTS FOR FACE RECOGNITION
K.Meena#1, Dr.A.Suruliandi*2
#Assistant Professor, Department of Computer Science and Engineering, Sardar Raja college Of Engineering, Tirunelveli,
India.
* Associate Professor, Department of computer Science and Engineering, Manonmaniam Sundaranar University, Tirunelveli,
India.
1meen_nandhu@yahoo.co.in
2suruliandi@yahoo.com
Abstract—Face recognition is one of the most important tasks in
computer vision and Biometrics. Texture is an important spatial
feature useful for identifying objects or regions of interest in an
image. Texture based face recognition is widely used in many
applications. LBP method is most successful for face recognition.
It is based on characterizing the local image texture by local
texture patterns. In this paper performance evaluation of Local
Binary Pattern (LBP) and its modified models Multivariate
Local Binary Pattern (MLBP), Center Symmetric Local Binary
Pattern (CS-LBP) and Local Binary Pattern Variance (LBPV)
are investigated. Facial features are extracted and compared
using K nearest neighbour classification algorithm. G-statistics
distance measure is used for classification. Experiments were
conducted on JAFFE female, CMU-PIE and FRGC version2
databases. The results shows that CS-LBP consistently performs
much better than the remaining other models.
Keywords—Face recognition, local binary pattern (LBP),
Multivariate Local Binary Pattern (MLBP), Center Symmetric
Local Binary Pattern (CS-LBP), Local Binary Pattern Variance
(LBPV).
I. INTRODUCTION
Facial recognition plays a vital rule in human computer
interaction [4]. A Face recognition system can be either
verification or an identification system depending on the
context of an application. The verification system
authenticates a person’s identity by comparing the captured
image with his/her own templates stored in the system. It
performs a one to one comparison to determine whether the
person presenting himself/herself to the system is the person
he/she claims to be. An identification system recognizes a
person by checking the entire template database for a match. It
involves a one to many searches. The system will either make
a match or subsequently identify the person or it will fail to
make a match.
The human ability to recognize face is remarkable. We
can recognize the thousands of faces learned throughout our
lifetime and identify familiar faces at a glance even after years
of separation. This skill quite robust, despite large changes in
the visual stimulus due to viewing
conditions ,expressions ,aging and distractions such as glasses
or changes in hairstyle or facial hair. Existing biometric
systems are developed for corporate user applications like
access control, Computer logon, Surveillance camera,
Criminal identification and ATM.
Face recognition system can be grouped as 1.structure
based 2.appearance based. In structure based method [12] a
set of geometric face features, such as eyes, nose, mouth
corners, is extracted, the position of the different facial
features form a feature vector as the input to a structural
classifier to identify the subject. In the second method [2], the
appearance of face as input to decision making and they can e
further categorized as holistic and component based. The
holistic appearance methods operate on the global properties
of face image. Nowadays ,appearance based methods not only
operate on the raw image space ,but also other spaces ,such as
wavelet ,local binary pattern and ordinal pattern spaces.
The Local Binary Pattern is originally proposed by
Ojala [7] for the aim of texture classification, and then
extended for various fields, including face recognition [9],
face detection [3], facial expression recognition [13].The
Local Binary Pattern is a non parametric operator which is
used for describing a local spatial structure of an image. The
Local Binary Patter method is computationally simple and
rotation invariant method for face recognition [9].Adaptive
smoothing for face image normalization under variation of
illumination is presented by Y.K.Park [8]. The illumination is
estimated by iteratively convolving the input image with a 3-
by-3 averaging kernel weighted by a simple measure of the
illumination discontinuity at each pixel. In particular, weights
of a kernel are encoded into a local binary pattern (LBP) to
achieve fast and memory efficient processing.
Face image is divided into several regions and LBP
is applied and features are extracted over the region. These
features are concatenated to form face descriptor [10].
Although face recognition with local binary pattern has been
proven to be a robust algorithm, it suffers from heavy
computational load due to the very high dimensional feature
vectors that are extracted by concatenating the LBP
histograms from each local region. A new multichannel filter
based Gabor wavelet is designed based on theory and
practicality. Its center frequency is the range from low
frequency to high frequency, its orientation is 6 and scale is 6.
It can extract the feature of low quality facial expression
image target, and have well robust for automatic facial
expression recognition [5].
IEEE-International Conference on Recent Trends in Information Technology, ICRTIT 2011
978-1-4577-0590-8/11/$26.00 ©2011 IEEE
MIT, Anna University, Chennai. June 3-5, 2011
782
MLBP is proposed by Arco Lucifer [1] for texture
segmentation. Most of the images are multi band in nature. So
this method is widely used for image classification and
segmentation. CS- LBP method was introduced by Marko
Heikkila [6]. This new descriptor has several advantages such
as tolerance to illumination changes, robustness on flat image
areas and computational efficiency. LBP variance (LBPV) is
proposed by Zhenhua Guo [14] to characterize the local
contrast information into one dimensional LBP histogram. In
this paper LBP and its variants methods are evaluated in
JAFFE female database for face recognition. Among these
methods, the best method will be tested by CMU PIE, FRGC
version2 databases.
The rest of the paper is organized as follows. Section II
reviews about LBP and its derivatives MLBP, CS-LBP and
LBPV. Section III explains about classification principle.
Section IV reports the experimental data and the results on
JAFFE female, CMU-PIE and FRGC version2 databases.
Section V gives the conclusion of this paper.
II. TEXTURE MODELS
A. Local binary pattern(LBP)
Local Binary Pattern was introduced by Timo ojala [11].
The standard version of the LBP of a pixel is formed by
thresholding the 3X3 neighborhood of each pixel value with
the center pixel’s value. Let gc be the center pixel gray level
and gi (i=0,1,..7) be the gray level of each surrounding pixel.
Fig.1 illustrate the basic LBP operation. If gi is smaller than
gc , the binary result of the pixel is set to 0 otherwise set to 1.
All the results are combined to get 8 bit value. The decimal
value of the binary is the LBP feature.
Fig. 1 Illustration of Basic LBP operator
Fig.2. The LBP operator of a pixel’s circular neighborhoods with r=1 ,p=8
Bilinear interpolation method is used for a sampling point
does not fall in the center of the pixel. Let LBPp,r denote the
LBP feature of a pixel ‘s circularly neighborhoods, where r is
the radius and p is the number of neighborhood points on the
circle .From Fig.2 we can write,
(1)
otherwise 0
0 xif 1
S(x) , 2 )(
1
0
i
,¦
−
=¯
®
≥
=−=
p
p
rp ci ggsLBP
The concept of uniform patterns is introduced to reduce
the number of possible bins. Any LBP pattern is called as
uniform if the binary pattern consists of at most two bitwise
transitions from 0 to 1 or vice versa. For example if the bit
pattern 11111111(no transition) or 00110000 (two transitions)
are uniform where as 10101011 (six transition) are not
uniform. The uniform pattern constraint reduces the number
of LBP pattern from 256 to 58 and it is very useful for face
detection [10].
B. Multivariate Local binary pattern(MLBP)
The Multivariate Local Binary Pattern operator, MLBP
c was developed by Arco Lucifer [1] which describes local
pixel relations in three bands. In addition to the spatial
interactions of pixels within one band, interactions between
bands are considered. Thus, the neighborhood set for a pixel
consist the local neighbours in all three bands (Fig 3).
Fig. 3.MLBP texture measure describes spatial relations within a band and
between bands
(2)
)-(
)-()-(
)-()-(
)-()-(
)-()-(
1
0
33
3231
2322
2113
121b1
c i
ci c
c i c i
c i c
ci ci
¦
−
=
++
++
++
++
=
p
i
bb
bbbb
i
bbbb
bbbb
i
bbb
ggsign
ggsign ggsign
ggsign ggsign
ggsign ggsign
ggsignggsign
MLBP
From Eqn.2 the local threshold is taken from these bands,
which makes up a total of nine different combinations. This
results in the following operator for a local color texture
description. The color texture measure is the histogram of
MLBP c occurrence, computed over an image or a region of an
IEEE-ICRTIT 2011
783
image. This single distribution contains P ×32bins (e.g. P =8
results in 72 bins).
C. Center Symmetric Local binary pattern(CS-LBP)
The CS-LBP is another modified version of LBP. It model
was developed by Marko Heikkila [6] for the recognition of
object in PASCAL database. The original LBP was very long
its feature is not robust on flat images. In this method, instead
of comparing the gray level value of each pixel with the center
pixel, the center symmetric pairs of pixels are compared
(Fig.4). CS-LBP is closely related to gradient operator. It
considers the grey level differences between pairs of opposite
pixels in a neighborhood. So CS-LBP take advantage of both
LBP and gradient based features. It also captures the edges
and the salient textures.
Fig. 4 CS-LBP feature for a neighbourhood of 8 pixel
The CS-LBP features can be computed by
(3
)
otherwise 0
xif 1
s(x) , 2 ))2/((
12/
0
i
,, ¦
−
=¯
®
≥
=−=−
N
i
trp
t
NggsLBPCS ii
Where gi and gi+n/2 correspond to the gray level of center
symmetric pairs of pixels (N in total) equally spaced on a
circle of radius r. It also reduces the computational complexity
when compared with basic LBP [6].
D. Local binary pattern variance(LBPV)
The LBPV descriptor proposed by Zhenhua [14] offers a
better result than LBP. Local invariant features have the
drawback of losing global spatial information, while global
features preserve little local texture information. LBPV
proposes an alternative hybrid scheme; globally rotation
invariant matching with locally variant LBP texture features.
It is a simplified but efficient joint LBP and contrast
distribution method. LBPp,r /VARp,r is powerful because it
exploits the complementary information of spatial pattern
and local contrast. Threshold values are used to quantize the
VAR of the test images computed to partition the total
distribution into N bins with an equal number of entries.
(5)
otherwise ,
kjiLBP ji
kjiLBPW
where
(4) Kk kjiLBPWkLBPV
RPRP
RP
RP
M
j
N
i
RP
¿
¾
½
¯
®
=
=
∈= ¦¦
==
0
),(,),(var
)),,((
],0[)),,(()(
,,
,
,
11
,
These threshold values are used to quantize the variance of
test images.
III. CLASSIFICATION PRINCIPLE
A. Training
In the training phase, the texture features are extracted from
the samples selected randomly belonging to each face class,
using the proposed feature extraction algorithm. The average
of these features for each face class is stored in the feature
library, which is further used for classification.
B. Texture Similarity
To find out the similarity between training models and
testing sample G-statistic distance measure is used. Similarity
between the textures is evaluated by comparing their pattern
spectrum. The spectrums histograms are compared as a test of
goodness-of-fit using a non-parametric statistics, also known
as the G-statistics [7].The G statistic compares the two bins of
two histogram and is defined as
C. Classification
In the texture classification phase, the texture features are
extracted from the test sample x using the proposed feature
extraction algorithm, and then compared with model feature
using K-Nearest Neighbor classification algorithm .In
experiment 1, K=1 is used. (ie) minimum distance classifier is
used. Minimum distance between the model feature value and
the sample feature value is calculated.
IV. EXPERIMENTS
A. Experimental Data
The discrimination capability of any method is done by
experimental tests using the bench mark data base.
Fig 5 Sample Images from JAFFE Female database
Fig. 5 shows the images from JAFFE female database. It
contains 56 images. (8 images X 7 poses=56 images).All the
)6(
,1
log
,1
1,
log
.
1
log
,1
,1log
2
»
»
»
»
»
»
»
»
»
»
»
»
¼
º
«
«
«
«
«
«
«
«
«
«
«
«
¬
ª
»
»
¼
º
«
«
¬
ª
¸
¸
¹
·
¨
¨
©
§¦¦
=
¸
¸
¹
·
¨
¨
©
§¦¦
=
+
»
»
¼
º
«
«
¬
ª¦
=¸
¸
¹
·
¨
¨
©
§¦
¸
¹
·
¨
©
§¦
−
»
¼
º
«
¬
ª¸
¹
·
¨
©
§¦
=
¦¸
¹
·
¨
©
§¦
=
−
»
¼
º
«
¬
ª¦¦
=
=
ms
n
ii
f
ms
n
ii
f
n
ims i
f
ms i
f
n
ii
f
ms
n
ii
f
ms
n
ii
f
i
f
G
Local Binary Patterns and its Variants for Face Recognition
784
training and testing images are pre-processed to the size of
120X120.
Fig 6. Samples from the CMU-PIE face database. The first image from the
left is a sample training image and the others are the sample testing images.
Fig.6 and Fig.7 represents the input images for Fig.6
represents the images from CMU-PIE database. Among these
five images only one is used for training and the remaining
four images are used for testing purpose.
Fig 7. Samples from the FRGC Versi on2 face database. The first image from
the left is a sample training image and the others are the sample testing
images.
Similarly Fig 7 shows the images from FRGC version2
database. Here first image from the left side is used for
training and the remaining images are used for testing phase.
B. Experimental comparisons on JAFFE Female database
TABLE 1 RECOGNITION RATE FOR DIFFERENT WINDOW SIZE
No of testing Samples=30
SL
NO
Window
Size
Recognition Rate (%)
LBP MLBP CS-LBP LBPV
1 10x10 15 25 24 20
2 20x20 34 35 43 40.5
3 30x30 81 83 87 84
Experiments are conducted on JAFFFE database by varying
the window size and also varying the number of input samples
from each image. During experiment#1 the number of training
sample is fixed as 10 and the window size is varied from
10X10. During experiment#2, the window size is fixed as
30X30 and the number of testing sample is increased from 10.
Table 1 and Table 2 shows that recognition rate increases with
the increase in window size as well as the increase in the
number of samples taken for classification. Our experimental
results show that CS-LBP provides better results than the
remaining other methods.
TABLE 2 RECOGNITION RATE FOR DIFFERENT NUMBER OF
SAMPLES
Window size=30X30
C. Experimental comparisons on CMU-PIE and FRGC
Version2 databases for illumination variations
Experiment#3 is conducted on CMU-PIE and FRGC
version2 database which is shown in Fig.6. The first image
from the left side is taken as training image and the remaining
four images are used as testing images. In training phase,
facial features are extracted by CS-LBP method and stored in
the database.
SL
NO
No of
Samples
Recognition Rate (%)
LBP MLBP CS-
LBP LBPV
1 10 15 35 20 21.6
2 20 39 42 50 47
3 30 82 84 86 82
0
10
20
30
40
50
60
70
80
10x10 20x20 30x30
CMU-PIE
IEEE-ICRTIT 2011
785
During testing phase, facial features are extracted by using
the above method and the difference between two facial
features is evaluated by G-statistic distance measure with
k=1(nearest neighbour classification) algorithm. This
experimental results show that face recognition is mainly
depends on illumination changes.
TABLE 4 RECOGNITIO N RATE OF FRGC VERSION2 DATABASE BY
CS-LBP METHOD
TABLE 3 and 4 shows the recognition rate vs window
size of CS-LBP method on CMU-PIE and FRGC Version2
databases. It shows that recognition rate increases with
increase in window size. CMU-PIE database gives better
results than FRGC Version2 database under different lighting
conditions.
V. CONCLUSIONS
LBP is grey scale invariant and rotational invariant. This
property is well suitable for many applications. The facial
recognition based on Local Binary Patterns is extremely
simple. In this paper LBP and its modified models CS-LBP,
MLBP and LBPV were analysed. CS-LBP performs very well
and gives the recognition rate of 87% with the JAFFE female
database. CMU-PIE and FRGC Version2 databases are
experimented by the same model under different illuminations.
The model gives the recognition rate of 80% for CMU-PIE
and 82% for FRGC Version2 database. CS-LBP provide good
recognition rate than other methods and also it consumes less
computational time.
REFERENCES
[1] Arko Lucieer, Alfred Stein and Peter Fisher ,” Multivariate Texture-
based Segmentation of Remotely Sensed Imagery for Extraction of
Objects and Their Uncertainty”.
[2] K.Etemad and R.Chellappa,” Discriminant Analysis for face
recognition of Human Face images,” I.Optical Soc.
Am.,Vol.14,PP.1724-1733,1997.
[3] A.Hadid, ,M.Pietikainan, Timo Ahonen,”A Discriminative Facial
feature for detecting and recognizing faces,In CPVR(2),pages 784-
804,2004.
[4] Handbook of Face Recognition,S.Z Li and A.K.Jain,eds.Springer,2005.
[5] M.Lyons,S.Akamatsu,etc.,”Coding Facial Expressions with Gabor
wavelets “,Proceeding of the third IEEE International conference on
Automatic face and Gensture Recognition,Nero Japan,(1998)200-205.
[6] Marko Heikkila,Matti Pietikainen, Cordelia Schmid,”Description of
interest regions with Local BinaryPattern”, June 2008.
[7] T.Ojala,M.Pietikainan,and D.Hawood,” A Comparative study of
texture images with classification based on featured
distributions’,pattern recognition,29(1):51-59,1996.
[8] Y.K.Park and J.K.Kim ,”Fast adaptive smoothing based on LBP for
face recognition”,ELECTRON LETTERS, 22nd November 2007,Vol
43,No 24.
[9] Timo ojala , Matti Pietikäinen, Topi Mäenpää,” Multiresolution gray
scale and rotation invariant Texture Classification with Local Binary
Patterns” IEEE Transactions on pattern analysis and Machine
Intelligence Volume 24 Issue 7, July 2002.
[10] Timo Ahonen,Matti Pietikainen,” Face Description with Local Binary
Patterns: Application to Face Recognition”, IEEE Transactions on
pattern analysis and Machine Intelligence Volume 28 ,No
12,DECEMBER 2006.
[11] Timo Ahonen,Matti Pietikainen and T.Maenpaa,” Multiresulation gray
scale and rotation invariant texture classification with local binary
pattern”, IEEE Transactions on pattern analysis and Machine
Intelligence Volume 24 ,No 7,pp 971-987,2002.
[12] M.Turk and A.Pentland ,”Eigen faces for face recognition”,J.Cognitive
Neuroscience,Vol 3,no 1,pp.71-86,1991.
[13] Zhao and M.Pietikainan,Dynamic,” texture recognition using Local
Binary Patterns with an application to facial expressions”.IEEE
Transaction.Pattern Anal.Mach.Intell.,29(6):915-928,2007.
[14] Zhenhua Guo,Lei Zhang,David Zhang,”Rotation Invariant texture
classification using LBP variance (LBPV) with global matching”.
Pattern Recognition 43 (2010) 706–719.
0
10
20
30
40
50
60
70
80
90
10x 10 20x 20 30x 30
FRGC
Version2
Local Binary Patterns and its Variants for Face Recognition
786
