Content uploaded by Ke-Chen Song
Author content
All content in this area was uploaded by Ke-Chen Song on Jan 05, 2018
Content may be subject to copyright.
Research and Perspective on Local Binary Pattern
SONG Ke-Chen1YAN Yun-Hui1CHEN Wen-Hui1ZHANG Xu1
Abstract In view of the theoretical and practical value of local binary pattern (LBP), the various LBP methods in texture analysis
and classification, face analysis and recognition, and other detection applications are reviewed. Firstly, the principle of LBP method
is briefly discussed, which mainly analyses the threshold operation, the uniform pattern and rotation invariant pattern in LBP
method. Secondly, the texture analysis and classification of the LBP method, face analysis and recognition of the LBP method and
other detection applications of the LBP method are particular combed and commented. Finally, the existing important problems of
the LBP method are analyzed and the future for the LBP method is pointed out.
Key words Local binary pattern, feature extraction, texture analysis, face analysis, object detection
Citation Song Ke-Chen, Yan Yun-Hui, Chen Wen-Gui, Zhang Xu. Research and perspective on local binary pattern. Acta
Automatica Sinica, 2013, 39(6): 730−744
Representation of the image feature is an important ba-
sic work in computer vision and digital image processing.
Among the commonly used color feature, texture feature,
shape feature and spatial relationship feature, texture fea-
ture plays a very important role on the application analysis
of natural images. Therefore, how to effectively obtain tex-
ture feature information is an active research topic in image
feature extraction.
In recent years, local binary pattern (LBP)[1−4] feature
extraction method has made a remarkable progress in tex-
ture analysis and face recognition applications, and has
sprung up many improvement methods. LBP method is not
only relatively simple and with low computation complex-
ity, but also has the properties of rotational invariance, gray
scale invariance and other significant advantages. There-
fore, LBP is widely used in image matching, pedestrian
and car target detection and tracking, biological and med-
ical image analysis[5].
Despite the great success of LBP in early applications,
its practical results are not satisfactory in different fields.
Hence, many researchers have improved the LBP in the
specific domains, and achieved lots of significant results.
Obviously, it is necessary to summarize variants of LBP
methods, especially in texture analysis and face recogni-
tion applications, and there exists a need to describe the
remaining significant issues of LBP and research directions
of the future.
The rest of the paper is organized as follows. Section 1
briefly reviews local binary pattern. Section 2 presents LBP
methods for texture analysis and classification. Section 3
presents LBP methods for face analysis and recognition.
Section 4 reports LBP methods for other detection appli-
cations. Section 5 discusses the remaining issues of LBP
and Section 6 concludes the paper.
Manuscript received July 4, 2012; accepted November 7, 2012
Supported by Fundamental Research Funds for the Central Univer-
sities (N120603003)
Recommended by Associate Editor HU Zhan-Yi
1. School of Mechanical Engineering and Automation, Northeast-
ern University, Shenyang 110819
1 Brief review of local binary pattern
LBP is a kind of gray level within the scope of the tex-
ture measure, which is initially proposed by Ojala et al.[1]
to support the local contrast measure of image. LBP is
initially defined in a neighborhood of eight pixels, and the
gray value of center pixel is set as a threshold. All neigh-
bors that have values higher than or equal to the value of
the central pixel are given a value of 1, otherwise they are
set as 0. The values after the thresholding (namely 0 or
l) will respectively multiply with the corresponding pixel
weight, and their additive result is the LBP value. The
calculation principle of original LBP is shown in Fig. 1.
Fig. 1 The calculation principle of original LBP
Since the original LBP is unable to extract texture fea-
tures in large size and structure, Ojala[4] has modified the
LBP and formulated the completed theory to improve the
limitations. In a gray image, a local neighborhood is de-
fined as a set of sampling points evenly spaced on a circle,
which is centered at the pixel to be labeled. Fig. 2 shows
some examples of the LBP operator, where the notation (P,
R) denotes a neighborhood of P sampling points on a circle
of radius R. T is the local texture feature of neighborhood,
namely:
T=t(gc,g
0,···,g
p−1)(1)
where gcand gi(i=0,···,P−1) are, respectively, gray-
level values of the central pixel and Psurrounding pixels
in the circular neighborhood with radius R.tis a function
for texture.
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 731
Fig. 2 The circular neighborhood for different Pand R
With the increase of the radius, the correlation among
the pixels gradually reduces, therefore, most of the texture
information can be obtained in the small neighborhood.
Without losing information, the gray value of the center
pixel can be subtracted from the neighborhood. Hence,
the local texture feature is shown:
T=t(gc,g
0−gc,···,g
p−1−gc)(2)
Assuming that the differences are independent of gc,
which allows us to factorize (2):
T≈t(gc)t(g0−gc,···,g
p−1−gc)(3)
Since the distribution t(gc) in (3) describes the overall
luminance of the image (i.e., it is unrelated to local image
texture), it does not provide useful information for local
texture analysis. Hence, much of the information in the
original joint gray level distribution (1) about the textural
characteristics is conveyed by the joint difference distribu-
tion:
T≈t(g0−gc,···,g
p−1−gc)(4)
Although texture in (4) is not affected by changes in the
gray value, the texture feature will be changed when the
pixel values enlarge or shrink the same multiples. In order
to make the texture definition avoid the influence of the
changes, signed differences are only considered:
T≈t(s(g0−gc),···,s(gp−1−gc,)) ,
s(x)=1,x≥0
0,x<0
(5)
Function s(gi−gc) multiplied by the factor 2i, then the
only LBP value of local texture feature will be obtained:
LBPP,R =
P−1
i=0
s(gi−gc)2
i(6)
In the experimental process, different initial position of
unsigned binary number will produce 2Ppatterns for cor-
responding LBPP,R . Obviously, as the neighborhood sam-
pling point number increases, the binary pattern species
rapidly rise. So many patterns are adverse for texture
extraction and texture classification. To solve this issue,
Ojala[4] proposed “Uniform patterns”. In a cyclic operation
of the binary number, the patterns that at most only have
two bitwise changes are denoted “Uniform patterns”. For
example: 00000000 (zero bitwise change), 01110000 (two
bitwise changes), and 11001111 (two bitwise changes) are
uniform patterns, while 11001001 (four bitwise changes)
and 01010010 (six bitwise changes) are not uniform pat-
terns. To check some patterns are uniform patterns or not,
a simple method is shown as follows:
U(LBPP,R)=|s(gp−1−gc)−s(g0−gc)|+
P−1
p=1 |s(gp−gc)−s(gp−1−gc)|(7)
When U(LBPP,R) is less than or equal to 2, the pattern
is called uniform pattern, denoted as LBP u2
P,R . While other
patterns are called hybrid patterns. Hence the species of
improved binary pattern will greatly decrease, for exam-
ple: binary pattern of eight sampling points is reduced to
58 from the original 256.
To achieve rotation invariance, a locally rotation invari-
ant pattern is defined as follows:
LBP ri
P,R = min ROR LBP ri
P,R,i
i=0,1,··· ,P −1
(8)
where ROR(x,i) is rotating function. By introducing the
definition of rotation invariance, LBP not only has promi-
nent performance for image rotation, but also has less pat-
terns. In addition, the rotation invariance of LBP can also
be combined with uniform patterns:
LBP riu2
P,R =⎧
⎪
⎨
⎪
⎩
P−1
p=0
s(gp−gc),if U(LBPP,R)≤2
P+1,otherwise
(9)
where U(LBPP,R) is calculated by (7), superscript riu2
means rotation invariant “uniform” patterns.
Despite the great success of LBP in early application ex-
periments, its practical results are not satisfactory in dif-
ferent fields. Hence, many researchers have improved the
LBP for the specific application (especially in texture anal-
ysis and face recognition), and achieved lots of significant
results. The reviews for texture analysis and classification,
face analysis and recognition and other detection applica-
tions of LBP method are respectively introduced in detail
below.
2 LBP methods in texture analysis and
classification
Because LBP method is mainly used to describe tex-
ture feature information, the applications of the early stud-
ies have focused on the texture analysis and classification
processing. Several kinds of influential LBP improvement
methods are discussed and analyzed as follows.
2.1 Median binary pattern (MBP)
Hafiane et al.[6] proposed the median binary pattern
(MBP) method for texture classification, which seeks to
derive the localized binary pattern by thresholding the pix-
els against their median value over a 3 ×3 neighborhood.
The central pixel is included in this filtering process, there-
732 ACTA AUTOMATICA SINICA Vol.39
fore 29possible structures are obtained. MBP is defined as
follows:
MBP =
L
i=1
f(gi)×2if(gi)=1,if gi≥Med
0,otherwise
(10)
where Lis the number of neighbors and giis the intensity
value.
MBP captures the contrast between two intensity ranges
which also impacts the local structure. These patterns form
the basic element of texture. However, the scale changes
can influence local structures, thus impact the MBP de-
scriptor. To reduce this affect, the image is decomposed
into several frequency ranges by sub sampling method.
These subimages can capture relationship among pixels,
which are not immediate neighbors. Although a median
is invariant under rotation, the coding used to label the
pattern is not invariant to rotation.
2.2 Adaptive LBP (ALBP)
Guo et al.[7] exploited the adaptive LBP (ALBP) method
for texture classification, which introduces the directional
statistical information. Specifically, the mean and standard
deviation of the local absolute difference are extracted to
improve the classification efficiency of LBP. Three differ-
ent directional statistical features are calculated: the mean
μpand standard deviation σpof local difference, and the
weight of minimizing the directional difference wp. Adap-
tive LBP is formulated as follows:
ALBP =
P−1
p=0
s(gpwp−gc)2
p(11)
where wp=g
g
gT
pg
g
gc/(g
g
gT
pg
g
gp),g
g
gc=[gc(1,1) ; gc(1,2) ; ···;
gc(N,M)] is a column vector containing all the possible
gc(i, j)pixels. g
g
gp=[gp(1,1) ; gp(1,2) ; ···;gp(N, M)] is
the corresponding vector for all gp(i, j)pixels. g
g
gT
pis the
transposition of g
g
gp.
The mean μpand standard deviation σpof local differ-
ence are as follows:
μp=
N
i=1
M
j=1 |gc(i, j)−gp(i, j )wp|
M×N(12)
σp=
N
i=1
M
j=1
(|gc(i, j)−gp(i, j )wp|−μp)
M×N
2
(13)
Experiments on CUReT[8] texture database show that
the texture feature extraction and classification scheme of
ALBP can significantly improve the classification accuracy
of LBP. However, the robustness is not inspected in match-
ing as it contained noisy texture image.
2.3 Completed LBP (CLBP)
Guo et al.[9] proposed the completed LBP (CLBP)
method for texture classification, which accounts for the
reason that the simple LBP code could convey so much dis-
criminant information of the local structure. Further more,
CLBP designs a scheme to effectively represent the missing
information in the LBP style so that better texture classi-
fication can be achieved. A local region is represented by
its center pixel and a local difference sign-magnitude trans-
form (LDSMT), and three operators, namely CLBP-Center
(CLBP C), CLBP-Sign (CLBP S) and CLBP-Magnitude
(CLBP M), are proposed to code them. The difference be-
tween central pixel and spaced neighbours can be calculated
by dp:dp=gp−gc,dpcan be further decomposed into
two components:
dp=sp∗mp,s
p=sign(dp),m
p=|dp|(14)
where sp=1,d
p≥0
−1,d
p<0,spis the sign of dpand mp
is the magnitude of dp. It is clearly seen that the original
LBP uses only the sign vector to code the local pattern
(“-1” is coded as “0”). CLBP M operator is defined as
CLBP MP,R =
P−1
p=0
t(mp,c)2p,
t(x, c)=1,x≥c
0,x<c
(15)
where cis a threshold to be determined adaptively, it is set
as the mean value of the whole image in practice. CLBP C
operator is defined as
CLBP CP,R =t(gc,c
I) (16)
where tis defined in CLBP M and the threshold cIis set
as the average gray level of the whole image.
CLBP demonstrated that the sign component is more
important than the magnitude component in preserving
the local difference information, which can explain why the
CLBP S (i.e., conventional LBP) features are more effec-
tive than the CLBP M features. However, CLBP has not
given a scheme to solve the issue that LBP is sensitive to
gaussian noise.
2.4 LBP variance (LBPV)
In LBP method, VAR is the supplement for LBP. How-
ever VAR is a series of continuous values, and needed to
carry out quantitative treatment, and this process has great
influence on the result of the experiment. To overcome
the drawback, Guo et al.[10] exploited the LBP variance
(LBPV) method for texture classification, which treated
the variance of each point as weight of code value, and
then accumulated the histogram. The definition of LBPV
is as follows:
LBP V =
N
i=1
M
j=1
w(LBPP,R (i, j ),k,),k∈[0,K] (17)
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 733
w(LBPP,R (i, j ),k)=⎧
⎪
⎨
⎪
⎩
VAR
P,R (i, j),
LBPP,R (i, j )=k
0,otherwise
(18)
The value of VAR represents the regional change, so
larger VAR value means larger contribution for distinction
of the area and bigger corresponding coding weight. LBPV
does not need any quantization and it is totally training-
free. However, LBPV does not give a scheme to solve the is-
sue that LBP is sensitive to gaussian noise and affine trans-
formation.
2.5 Bayesian LBP (BLBP)
The stochastic nature of image formation is usually dis-
regarded by LBP methods leading to inaccuracy and sensi-
tivity to the illumination changes and noise. Therefore, He
et al.[11] proposed a novel Bayesian LBP (BLBP) operator
to deal with above issue. BLBP models the label acquired
from filter responses as a stochastic process, and embeds a
Markov random field (MRF) in the label space. The label-
ing procedure is then treated as a joint optimization process
under a criterion of maximum a posteriori (MAP). Finally,
a histogram estimating the probability density of the la-
bels is used as a descriptor. Labeling process of BLBP is
shown in Fig. 3. To analyze and compare the performance
in a unified way, a filtering, labeling and statistics (FLS)
framework is developed, whose calculation principle is as
showninFig.4.
Although BLBP method achieved excellent experimental
effect on Brodatz[12] database, alternative smoothing terms
and learning parameter values require further study.
2.6 Dominant LBP (DLBP)
Liao et al.[13] exploited the dominant LBP (DLBP)
method of texture classification, which uses the most fre-
quently occurred patterns to capture descriptive textural
information. The uniform LBPs effectively capture the
fundamental information of textures, while the uniform
LBPs are not the dominating patterns in some textures
with irregular edges and shapes. DLBP considers the most
frequently occurred patterns in a texture image, which is
demonstrated that a minimum set of pattern labels that
represents around 80 % of the total pattern occurrences in
an image. Hence, DLBP effectively captures the image tex-
tural information for classification tasks.
Although the DLBP features encapsulate more textu-
ral information than the conventional LBP features, they
lack the consideration of distant pixel interactions. To re-
plenish the missing information in the DLBP features, an
additional features set, i.e., features based on the Gabor fil-
ter responses are utilized as the supplement to the DLBP
features. DLBP has also been compared with six published
texture features (DBWP, RDBWP, TGF, CGF, ACGMRF,
and LBP) on Outex[14], Brodatz[12] and CUReT[8] texture
image databases.
To compare the experimental results accuracy of above
several methods, Table 1 respectively gives the classifica-
tion accuracy on Brodatz database[12], CUReT database[8]
and Outex database[14]. We can not comprehensively eval-
uate the performance of these methods, since only DLBP
obtained the corresponding data from three databases while
other methods obtained the data from one or two of the
databases. However, in available data: classification preci-
sion of ALBP is the lowest in the same CUReT database,
while LBPV’s results are better than CLBP and DLBP; on
the same Outex database, MBP’s classification accuracy in
“inca” is far lower than other methods, but it has won the
best classification accuracy in “Tl84” and “Horizon”. In
addition, Nanni[15] reviewed the improved LBP methods
in texture classification.
2.7 Other methods
As early as in 2000, M¨aenp¨a¨aet al.[16] used beam search
method and uniform patterns method to search the most
significant patterns, which can be carried out in a number
of ways. Moreover, these two methods demonstrated that
a small subset of local patterns can perform better than
the whole LBP histogram in problems involving geometric
transformation (Tilt) between training and testing images.
Table 1 The classification accuracies for various methods on three databases (“−”indicates no data.) (%)
Method Parameters Brodatz database CUReT database Outex database Note
Inca Tl84 Horizon
MBP[6] −− −47.90 97.30 96.10 KNN = 3
P=8,R=1 −57.40 −− −
ALBP[7] P=16,R =3 −67.30 −− −
P=24,R =5 −65.50 −− −
P=8,R=1 −86.67 96.56 90.30 92.29 Average value in CUReT
CLBP[9] P=16,R =3 −87.79 98.72 93.54 93.91 Outex: R=2
P=24,R =5 −85.84 98.93 95.32 94.53 Outex: R=3
P=8,R=1 −88.23 91.56 76.62 77.01
LBPV[10] P=16,R =2 −89.77 92.16 87.22 84.86
P=24,R =3 −91.09 95.26 91.31 85.04
BLBP[11] −90.77 −−− −Results of texture retrieval
DLBP[13] R= 2 98.49 86.84 97.70 92.10 88.70
R= 3 98.26 83.62 98.10 91.60 87.40
734 ACTA AUTOMATICA SINICA Vol.39
Fig. 3 The labeling procedure of BLBP
Fig. 4 The FLS framework of image descriptors
Despite the multi-resolution LBP has been shown to be
a powerful measure of image texture, its main limitations
have been sparse sampling and inability to cope with a large
number of different local neighborhoods. M¨aenp¨a¨aetal.
[17]
presented two techiques (Gaussian low-pass filters and cel-
lular automata) to solve these problems, and achieved ex-
cellent classification performance on Outex database[14].In
addition, Raja et al.[18] exploited the multiscale selected
local binary features (MSLBF) to deal with the issue of
multi-resolution LBP. However, there are two main draw-
backs to the pairwise-coupled approach. Firstly, stable re-
sults required the same number of features to be used for all
classes despite the varying numbers of features required for
a given error per class. Secondly, the complete separation
between the training of individual binary classifiers does
not preclude the possibility of histograms for two classes
being similar despite being constructed from completely
different features. He et al.[19] proposed the multi-structure
local binary pattern (MSLBP) method for texture classifi-
cation, which extracts three different kinds of structures:
isotropic micro structures, isotropic macro structures, and
anisotropic macro structures. The performance of MSLBP
method is limited by the size of images, since small images
are not enough to supply large macro structures.
Ahonen et al.[20] proposed using soft histograms for the
LBP operator, which makes the operator more robust to
noise and makes its output continuous with respect to in-
put. To increase the robustness of the operator, the thresh-
olding function of LBP is replaced by the two fuzzy mem-
bership functions. Possible drawbacks of the proposed op-
erator in comparison to original LBP are increased sensi-
tivity to grayscale changes and computational complexity.
Furthermore, Iakovidis et al.[21] extended the LBP method
by incorporating fuzzy logic in the representation of local
patterns of texture in ultrasound images.
Zhang et al.[22] proposed a novel texture feature extrac-
tor, namely Monogenic-LBP (M-LBP). M-LBP integrates
the traditional LBP operator with the other two rotation
invariant measures: the local phase and the local surface
type computed by the 1st-order and 2nd -order Riesz trans-
forms, respectively. M-LBP has the advantage of smaller
feature size and faster classification speed, which makes it
a more suitable candidate in real applications.
To solve the LBPs shortcomings (i.e., LBP discards
some important texture information and is sensitive to
noise), Zhou et al.[23] exploited the extended LBP (ELBP)
method for texture analysis, which classifies and combines
the “nonuniform” local patterns based on analyzing their
structure and occurrence probability. Although three ex-
periments on the Brodatz texture database show the per-
formance improvement of ELBP and its robustness against
noise, ELBP could not provide much improvement in a
smaller neighborhood. Recently, Liu et al.[24] presents a
novel extended LBP (ELBP), in which two different and
complementary types of features (i.e., pixel intensities and
differences) are extracted from local patches and four de-
scriptors (i.e., CI-LBP, NI-LBP, RD-LBP and AD-LBP)
are developed. However, AD-LBP descriptor cannot ef-
fectively provide a reliable and meaningful description of
texture. Moreover, to sufficiently utilize “nonuniform” lo-
cal patterns, Khellah et al.[25] proposed dominant neigh-
borhood structure (DNS) and Guo et al.[26] developed dis-
criminative features for texture description.
Guo et al.[27] designed a learning framework of image
descriptor based on the Fisher separation criteria (FSC)
to learn most reliable and robust dominant pattern types
considering intra-class similarity and inter-class distance.
Furthermore, Guo et al. developed a new FSC-based learn-
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 735
ing (FBL-LBP) descriptor. FBL-LBP differs from previous
LBP approaches since FBL framework learns robust domi-
nant types of each class instead of using fixed pattern types.
In addition, this learning framework is easy to generalize for
other purposes by introducing different histogram descrip-
tors. Zhao et al.[28] presented the local binary count (LBC)
method for texture classification. LBC extracts the local
binary gray-scale difference information and totally aban-
dons the local binary structural information. Furthermore,
LBC demonstrated that the local gray-scale difference in-
formation plays a main role in the LBP for rotation invari-
ant texture classification. Besides, Fathi et al.[29] proposed
a noise tolerant extension of LBP operators to extract sta-
tistical and structural image features for efficient texture
analysis.
The methods to color texture analysis can be roughly
divided into two categories: methods that process color
and texture information separately, and those that con-
sider color and texture a joint phenomenon. M¨aenp¨a¨aet
al.[30] argued that adding color information to texture mea-
sures indeed increases accuracy, while obtained with a three
times longer feature vector. His study suggested that using
color and texture in parallel is not the most powerful way
of utilizing this complementary information. All joint color
texture descriptors and all methods of combining color and
texture on a higher level are outperformed by either color
or gray-scale texture alone.
M¨aenp¨a¨aet al.[31] proposed an method based on sep-
arate processing of complementary color and pattern in-
formation, while this method needs to calculate nine LBP
images. To solve above issue, Porebski et al.[32] presented
a new method for color texture classification by use of Har-
alick features extracted from co-occurrence matrices com-
puted from LBP images. Recently, Zhang et al.[33] pro-
posed the local energy pattern (LEP) method for dynamic
texture classification, similarly, Zhao et al.[34] exploited a
novel approach to compute rotation-invariant features from
histograms of local noninvariant patterns, which can ef-
fectively deal with rotation variations of dynamic textures
(DTs).
In addition to the above improvements in LBP methods,
some texture analysis methods are also motivated by LBP.
For instance: Ojansivu et al.[35] exploited the local phase
quantization (LPQ) method for texture classification; Late-
gahn et al.[36] introduced Gaussian mixture models (GMM)
into joint probability density functions (JPDF) for texture
classification; Chen et al.[37] developed weber local descrip-
tor (WLD), and Liu et al.[38] proposed sorted random pro-
jections (SRP) method.
3 LBP methods in face analysis and
recognition
LBP method not only shows superior performance in
the texture analysis and classification of the application,
but also gets the same good effect in face recognition ap-
plication. Several kinds of influential LBP improvement
methods in face recognition application are discussed and
analyzed as follows.
3.1 Elongated LBP (ELBP)
Liao et al.[39] proposed the elongated LBP (ELBP)
method for face recognition, and developed a new feature
which is called average maximum distance gradient magni-
tude (AMDGM). AMDGM embeds the gray level difference
information between the reference pixel and neighboring
pixels in each ELBP pattern. There are three parame-
ters related to the ELBP approach: the long axis of the
ellipse, denoted by A=2; the short axis of the ellipse, de-
noted by B=3; the number of neighboring pixels, denoted
by m. Fig. 5 shows examples of the ELBP patterns with
different values of A,B,andm.
Fig. 5 Examples of ELBP with different values of A,B,andm
In fact, the ELBP features are more general than the
conventional LBP, more precisely, the conventional LBP
can be viewed as a special case of ELBP when setting the
values of Aand Bequal to each other. The ELBP is able
to capture anisotropic information from the facial images,
which are important features as there are many important
parts in the face such as eyes, mouth are all elongated struc-
tures. Therefore, it is expected that ELBP can have more
discriminative power than the conventional LBP.
3.2 Improved local binary pattern (ILBP)
Jin et al.[40] presented a novel face detection approach
using improved local binary patterns (ILBP) as facial rep-
resentation. In most cases, the central point provides more
information than its neighborhood. To get all the repre-
sentations of LBP, ILBP considers the effect of the central
pixel and gives it the largest weight. Fig. 6 shows the dis-
tribution of ILBP weight.
ILBP is defined as follows:
LBPP,R =
P−1
i=0
s(gi−m)2i+s(gc−m)2P,
s(x)=1,x>0
0,x≤0
(19)
where m=1
P+1 (
P−1
i=0
gi−gc).
736 ACTA AUTOMATICA SINICA Vol.39
Fig. 6 Mapping weights for ILBP8,1and ILBP4,1(“X”
indicates arbitrary pixel value)
The ILBP features are insensitive to the variation of illu-
mination, there is no need to do image enhancement such as
illumination equalization to remove the influence of light.
But there is a need to improve the computing efficiency and
performance by using other new classifiers and more ILBP
features at different neighborhood.
3.3 Local line binary patterns (LLBP)
Petpon et al.[41] introduced a novel face representation
method for face recognition, called local line binary pat-
tern (LLBP), which summarizes the local spacial structure
of an image by thresholding the local window with binary
weight and introducing the decimal number as a texture
presentation. Moreover it requires less computational cost.
Coding way of LLBP is shown in Fig.7.
Fig. 7 LLBP operator with line length of 9 pixels
The basic idea of LLBP is to first obtain the line binary
code along with horizontal and vertical direction seper-
ately and then compute its magnitude, which characterizes
the change in image intensity such as edges and corners.
Consequently, LLBP is more discriminative than other
methods even in extreme illumination condition. However,
LLBP needs to increase the discriminative power in macro-
structure of the image.
3.4 Local ternary patterns (LTP)
Tan et a l . [42] introduced the local ternary patterns
(LTP), which is less sensitive to noise in uniform regions.
LTP extends LBP to 3-valued codes, in which gray-levels
in a zone of width ±taround are quantized to zero, and
the ones above this are quantized to +1 and ones below it
to −1. Coding way of LTP is shown in Fig.8.
Fig. 8 Illustration of the basic LTP
The calculation of the three values is shown as follows:
s(u, ic,t)=⎧
⎪
⎨
⎪
⎩
1,u≥ic+t
0,|u−ic|<t
−1,u≤ic−t
(20)
where tis a user-specified threshold, which is set as 5 in
Fig. 8, hence its tolerance interval is [50, 60].
For simplicity, the experiments of LTP use a coding
scheme that splits each ternary pattern into its positive
and negative halves as illustrated in Fig.9. Subsequently,
LTP treats these components as two separate channels of
LBP descriptors for which separate histograms and simi-
larity metrics are computed, by combining the results only
at the end of the computation.
Fig. 9 Splitting an LTP code into positive and negative LBP
codes
LTP has a stronger discrimination ability against the
changes of noise and illumination than LBP in the uniform
region. But the issues of image multiscale variation and
partial occlusion are needed to solve.
3.5 Multi-scale block local binary patterns (MB-
LBP)
Liao et al.[43] proposed a novel representation, called
multiscale block local binary pattern (MB-LBP), and ap-
plied it to face recognition. In MB-LBP, the computation
is done based on average values of block subregions, in-
stead of individual pixels, its principle is shown in Fig. 10.
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 737
In each sub-region, average sum of image intensity is com-
puted. These average sums are then thresholded by that of
the center block. MB-LBP is then obtained.
Fig. 10 The basic LBP((a)) and the 9×9 MB-LBP((b))
MB-LBP encodes not only microstructures but also
macrostructures of image patterns, hence provides a more
completed image representation than the basic LBP. The
question of how to make the MB-LBP block size has better
representation of the image information sill needs further
study. In addition, Zhang et al.[44] also proposed a similar
MB-LBP.
3.6 Three-Patch LBP (TP-LBP)
Wolf e t a l . [45] exploited the three-patch LBP (TPLBP)
method, which is produced by comparing the values of three
patches to produce a single bit value in the code assigned
to each pixel. For each pixel in the image, TPLBP consid-
ers a w×wpatch centered on the pixel and Sadditional
patches distributed uniformly in a ring of radius raround it
(Fig. 11). For αparameter, TPLBP takes pairs of patches
(α-patches apart along the circle), and compares their val-
ues with those of the central patch. The value of a single
bit is set according to which of the two patches is more
similar to the central patch. The resulting code has Sbits
per pixel.
Fig. 11 The coding principle of TP-LBP with S=8,W=3,
α= 2 ((a)) and the computing method of TP-LBP code with
S=8,W=3,α= 2 ((b))
However, some of the experiments of TPLBP are par-
tial due to the computational complexity of the similarity
based method. Moreover, TP-LBP is unable to conduct
experiments on more than 100 classes.
To compare the experimental results accuracy of above
several methods, Table 2 respectively gives the classifica-
tion accuracy on ORL database[46], Yale B database[47],
FERET database[48], FRGC database[49], and LFW
database [50]. We cannot comprehensively evaluate the
performance of these methods, since only the experimen-
tal results of above methods are given in one or two of
the databases. However, in the available data: the clas-
sification precision of LTP is the best in the same Yale B
database; LLBP and MB-LBP obtained the same average
recognition rate; in the same FRGC database, the classi-
fication accuracy of LTP is lower than that of MB-LBP.
In addition, Huang et al.[51] reviewed the improved LBP
methods for face recognition.
3.7 Other methods
Chan et al.[52] proposed a novel discriminative face repre-
sentation derived by the linear discriminant analysis (LDA)
of multispectral local binary pattern histograms for color
face recognition. In this method, the color face image is
first photometrically normalized and partitioned into sev-
eral non-overlapping regions, and then multispectral local
binary pattern histograms are extracted and concatenated
into a regional feature. The feature is then projected into
a LDA space to be used as a regional discriminative fa-
cial descriptor. Furthermore, Chan et al.[53] extended the
method to multi-scale local binary pattern histograms for
face recognition.
Zhang et al.[54] exploited a non-statistics based face rep-
resentation method, namely local gabor binary pattern his-
togram sequence (LGBPHS), in which training procedure
is unnecessary to construct the face model, and LGBPHS
avoids the generalizability issue. This method modeled a
face image as a “histogram sequence” by concatenating
the histograms of all the local regions for the local Ga-
bor magnitude binary pattern maps. Although this method
achieved commendable result on FERET face database, the
effective and efficient match of two LGBPHSes, especially
for pose and occlusion variations are still needed to study.
Tan et a l . [55] combined two of the most successful local face
representations, Gabor wavelets and LBP. And they argued
that robust recognition requires several different kinds of
appearance information, and suggested the use of hetero-
geneous feature sets. In addition, Shan et al.[56] developed a
statistical extension for local gabor binary pattern (LGBP)
similarity computation by introducing Fisher discriminant
analysis (FDA) of the LGBP spatial histogram “features”.
Maturana et al.[57] introduced and analyzed a generaliza-
tion of LBP, decision tree local binary patterns (DT-LBP),
which learns the most discriminative LBP-like features for
each facial region in a supervised manner. In this method,
the tree has Slevels, where all the nodes at a generic level
lcompare the center pixel with a given neighbor. Lahde-
noja et al.[58] proposed a method for reducing the length
of the feature vectors in the LBP based face recognition,
which defines a discrimination concept of the uniform lo-
cal binary patterns called symmetry. Moreover, Zhang et
al.[59] exploited a high-order local pattern descriptor, local
derivative pattern (LDP), which is a general framework to
encode directional pattern features based on local deriva-
738 ACTA AUTOMATICA SINICA Vol.39
Table 2 The average recognition rates of various methods on different databases (“−”indicates no data.) (%)
Method ORL database Yale B database FERET database FRGC database LFW database Note
ELBP[39] 97.0 −86.7 −−
ILBP[40] −84.4 −−−
LLBP[41] −89.7 −−−
LTP [42] −98.7 −86.3 −
MB-LBP[43] −89.7 −98.3 −
TP-LBP[45] −− − −76.5
tive variations.
Shan et al.[60] developed the learn discriminative LBP-
histogram (LBPH) bins for the task of facial expression
recognition. This method adopted the Adaboost to learn
the most discriminative sub-regions (in term of LBP his-
togram) from a large pool of sub-regions generated by shift-
ing and scaling a sub-window over face images. Further-
more, An et al.[61] introduced a architecture of the future
interactive TV and proposed a real-time face analysis sys-
tem that can detect and recognize human faces and even
their expressions, and therefore understand their internal
emotional states.
Yan et a l . [62] described the locally assembled binary
(LAB) Haar feature for fast and accurate face detection.
LAB modified the Haar features to keep only the ordinal
relationship (named by binary Haar feature) rather than
the difference between the accumulated intensities. More-
over, Roy et al.[63] also proposed a face detection system
based on a new type of feature called the Haar local binary
pattern (HLBP) feature which combines the advantages of
both Haar and LBP.
Yang et al.[64] introduced the widely used Hamming dis-
tance to decrease the error rate caused by some noise dis-
turbances (assuming that the illumination, pose or expres-
sion changes of a face image are some kinds of “noise”).
Fu et al. [65] proposed the centralized binary pattern (CBP)
operator, which reduces significantly the histogramsdi-
mensionality by comparing pairs of neighbors in the opera-
tor. CBP considers the center pixel points effect and gives
it the largest weight, thus improving discrimination. Fur-
thermore, CBP also decreases the white noises influence
on face images. In addition, Zhang et al.[66] presented a
method for face recognition which used boosted statistical
local feature based classifiers.
4 LBP methods in other detection ap-
plications
Not only does LBP method get remarkable achievements
in the field of texture analysis and classification, face anal-
ysis and recognition, but also it is applied in pedestrian
detection, car detection, image matching and facial expres-
sion recognition etc. The mentioned methods are discussed
and analyzed as follows.
4.1 Applications in pedestrian detection
Mu et al.[67] applied LBP in pedestrian detection and
proposed two descriptors: semantic-LBP (S-LBP) and
Fourier-LBP (F-LBP). Since the feature vector of LBP re-
quires huge storage and cannot represent the semantically
similar features, the S-LBP descriptor is proposed to deal
with these issues. The definition of S-LBP is like this: sev-
eral continued “1” bits form an arch on the sampling circle,
which can be compactly represented with its principle direc-
tion and arch length. 2D histogram descriptor for any im-
age region can be obtained by collecting information from
the features. Finally S-LBP concatenates each column of
the 2D histogram to get a 1D vector. Fig. 12 shows the
computing principle of S-LBP.
Fig. 12 The computing principle of S-LBP
F-LBP descriptor is a “soft” LBP, i.e., skipping the bi-
narization step when calculating LBP. It avoids the po-
tential errors caused by improper local thresholding and
thus controllable compression is possible. F-LBP uses
s(k)(k=0,···,P−1) to denote raw feature vector, where
s(k) is real-valued color distance between the kth sam-
ples and central pixel. F-LBP transforms the feature vec-
tor into frequency domain. Coefficients for low frequen-
cies are more useful since they capture salient local struc-
tures around current pixel, and lossy compression can be
obtained via dropping some highest frequency coefficients.
One-dimensional DFT used in feature vector transforma-
tion is like this:
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 739
a(u)=1
P
P−1
k=0
s(k)e
−j2πuk
P(21)
Mu et al. used these descriptors to conduct a series of
experiments on the INRIA pedestrian dataset[68] , and got
excellent results. However, the method lost partial contour
information. Wang et al.[69] solved this problem and got a
better detection result by combining LBP and HOG.
4.2 Applications in car detection
Trefny et al.[70] proposed two descriptors: transition
LBP (tLBP) and direction LBP (dLBP), and applied the
two descriptors in car detection. The LBP encoding rule
thresholds the neighbor gray values by its center pixel
value. This gives rough knowledge of pixel with respect
to the center one, while relations between pixels with the
same binary value are lost. However, binary value of tLBP
is composed of neighbor pixel comparisons in clockwise di-
rection for all pixels except the central. Thus this rule
encodes relation between neighbor pixels. More precisely
let gpcorrespond to gray value pth neighbor of center pixel,
then
tLBPP,R=s(g0−gP−1)+
P−1
p=1
s(gp−gp−1)2p(22)
The dLBP descriptor provides better information of lo-
cal pattern in sense of direction functions. There are four
base directions through the center pixel in LBP, see Fig. 13.
The dLBP encodes intensity variation along these direc-
tions into two bits: the first bit encodes, whether the cen-
ter pixel is an extrema and the second bit encodes, whether
the difference of border pixels due to the center one grows
or declines. Let P=2P,sodLBP
P,R can be written as
follows:
dLBPP,R=
P−1
p=1
(s((gp−gc)(gp+P−gc))22p+
s(|gp−gc|−|gp+P−gc|)22p+1)
(23)
Fig. 13 Four basic directions of coding LBP
Trefny et al.[70] used these two descriptors to conduct
experiments on car dataset of UIUC[71], and got high de-
tection precision. But it doesnt work well when the object
is in different scales or in partial occlusion.
4.3 Applications in image matching
Heikkil¨aet al.[72] described the Center Symmetric LBP
(CS-LBP) descriptor, which is used in normalized inter-
est region. The problems in LBP method i.e. having a
rather long histogram and no robustness in flat image re-
gions are solved by CS-LBP. CS-LBP encodes the patterns
by two pixels center-symmetric, and produces only 16 dif-
ferent binary patterns while LBP produces 256. It can be
formulated as follows:
CS −LBPR,P,T =
P/2−1
i=0
s(gi−gi+P/2)2i,
s(x)=1,x>T
0,otherwise
(24)
Heikkil¨aetal
[72]. used many images with six different
kinds of situations: viewpoint change, scale change, im-
age rotation, image blur, illumination change and JPEG
compression. By combining the SIFT descriptor, using
a SIFT-like grid and LBP texture operator, CS-LBP ob-
tained higher matching accuracy rate.
4.4 Applications in facial expression recognition
Zhao et al.[73] proposed volume LBP (VLBP) descriptor
and LBP Three Orthogonal Planes (LBP-TOP) descriptor
for facial expression recognition. VLBP describes dynamic
texture in a local neighborhood, and extends LBP to DT
(dynamic texture) analysis. The descriptor gets local bi-
nary pattern of three images with time interval L(tc−L,
tc,tc+L) respectively. Each VLBP
L,P,R descriptor has five
parts at five different moments: local neighborhood center
pixel pattern at time tc−Land tc+L, binary pattern com-
posed of local neighborhood P-pixels at time tc−L,tcand
tc+L.
In the proposed VLBP, the parameter Pdetermines the
number of features. A large Pproduces a long histogram,
while a small Pmakes the feature vector shorter and means
more loss of information. To address the problem, the au-
thors simplified the descriptor by concatenating local bi-
nary pattern on three orthogonal planes: XY, XT, and
YT, considering only the co-occurrence statistics in these
three directions. The LBP code is extracted from the three
orthogonal planes respectively. And DT is encoded by the
XY-LBP, XT-LBP and YT-LBP while the appearance and
motion in three directions of DT are considered. The cal-
culation procedures of incorporation spatial domain infor-
mation (XY-LBP) and two spatial temporal co-occurrence
statistics (XT-LBP and YT-LBP) are shown in Fig. 14.
Zhao et al.[73] conducted a series of experiments on the
facial expression dataset of Cohn-Kanade[74], and achieved
an accuracy rate of 94.38 % when only using LBP-TOP
descriptor, while the accuracy rate rose up to 95.19 % by
combining the two descriptors: LBP-TOP and VLBP.
740 ACTA AUTOMATICA SINICA Vol.39
Fig. 14 Three orthogonal planes ((a)), LBP histogram from
each plane ((b)), concatenated feature histogram ((c))
4.5 Applications in other fields
Many researchers applied LBP in other fields except for
pedestrian detection, car detection, image matching and
facial expression recognition. Heikkil¨aetal.
[75−76] applied
LBP in background extraction and background modeling.
They utilized the original LBP to solve background model-
ing problem with object partial occlusion. And the method
is more efficient than that in [77], when used in foreground
extraction. Based on the research of Heikkil¨a et al., Takala
et al.[78], Yao et al.[79], and Liao et al.[80] applied LBP
in multi-object extracting, multi-hierarchy background ex-
tracting and 3D background modeling respectively, and
they got satisfactory results. Moreover, LBP was used into
gesture recognition and gait recognition by Kellokumpu et
al.[81−84], and a high recognition rate was obtained. Costa
et al.[85] applied LBP in music genre classification.
5 The issues and research direction
Despite the fruitful and significant progress made in the
current studies of local binary pattern method, more inno-
vative research in the perspectives of theory and algorithm
is necessary with regard to the problems concerning natural
image texture analysis and dynamic target detection and
recognition, which are becoming increasingly complex in
the practical engineering. The research will better indicate
the target and texture features and carry forward wider
application of LBP method in practical engineering while
optimizing theoretical analyses. The remaining significant
problems include:
1) Lacking of an LBP texture analysis method that
is more robust to illumination change, affine transforma-
tion and noise interference. Since the basic processing of
LBP method is the operation on threshold values, it is
sensitive to noise and illumination changes. Though re-
searchers have put forwarded various solutions, e.g., He
et al.s Bayesian LBP, Ahonen et al.[20]s soft histograms,
andZhouetal.
[23]s extended LBP have ameliorated the
method to avoid such problems, the solutions can only im-
prove the robustness of LBP to the interferences rather
than radically eliminating the instability brought about by
the operation on threshold values. Therefore it is necessary
to study the factors of the instability and solve the prob-
lem in order to realize a more robust LBP texture analysis
coping with the illumination change, affine transformation
and noise interference.
2) A texture analysis taking full advantage of the non-
uniform mode of LBP is needed. Ojala et al.[4] put for-
ward a unified model with the purpose of reducing the
variety of local binary patterns and putting all the non-
uniform modes into one category. However, this process
left out much texture information in non-uniform modes,
which becomes especially evident when large neighborhood
is applied. Though Zhou et al[23] . proposed to divide non-
uniform modes into two subsets, i.e., the ones indicating
structural information and ones indicating probability of
occurrence, and obtained desirable results using Brodatz
texture database[12], it was not so effective in smaller neigh-
borhood. Therefore, a further study on how to make the
best of the non-uniform mode to better serve texture anal-
ysis is necessary.
3) Deficiency in LBP texture analysis methods applied
on complex natural images. The current methods of LBP
texture analysis are applied in experiments based on Bro-
datz database[13] and Outex database, which are not very
complex, while ameliorated texture analysis methods that
are applied in more challenging CUReT database[8] and
KTHTIPS2b database[86−87] are still rare. Therefore fur-
ther study on texture analysis methods of more complex
natural images is necessary.
4) An in-depth study on LBP face analysis methods that
are more robust to the changes of gesture and illumination
is necessary. Much practice has shown that face recogni-
tion of computer is confronted with two challenges: changes
of gesture and illumination. The inevitability of the aging
of face, partial occlusion, disparities of imaging devices,
make-up and ornaments under non-controllable and non-
cooperative circumstances (e.g., video monitoring), and the
frequency of concurrence of the factors mentioned above all
add to difficulty of face recognition. Despite the various
LBP improvements brought forward by researchers, e.g.,
Jin et al.[40]s improved LBP, Tan et al.[42] s local ternary
patterns, Liao et al.[43]s multi-scale block LBP, currently
there is no comprehensive solution to the influences of un-
stable factors. Consequently, an intensive study on the res-
olution of the affecting factors is necessary to make LBP
face analysis more robust to the changes of gesture and
illumination.
5) Lack of an intensive study on LBP facial expres-
sion analysis of color images and physiological classifica-
tion method. Currently, LBP face analysis mainly focuses
on face detection and localization as well as classification
and identification, while study on facial expression analysis
and physiological classification is still in the primary stage.
Consequently, more and more researchers have turned their
attention to the analysis of facial expression. Shan et al.[60]
detected facial expression applying histogram with LBP
features; An et al.[61] designed a face recognizing system
using facial expression. In addition, the existing image data
of current study mainly focuses on gray image, while other
color channel information of color image is still in need of
further study. Therefore an intensive study on LBP facial
expression analysis of color images and physiological clas-
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 741
sification is worthy of attention.
6) The need of extension of LBP application fusing other
features. Currently multi-feature fusing method is the hot
spot of the study of feature extraction. Therefore a wider
application of LBP method needs further study on the fu-
sion of LBP and other features.
6 Conclusion
LBP method is not only relatively simple and of low
computation complexity, but also has a rotational invari-
ance, gray scale invariance and other significant advantages.
Therefore, LBP has obtained fruitful results and is widely
used in image matching, pedestrian and car target detec-
tion and tracking, as well as biological and medical image
analysis. The principle of LBP method is briefly discussed;
the texture analysis and classification, face analysis and
recognition, and other detection applications of the LBP
method are combed and commented in detail respectively;
and the remaining significant issues of the LBP method are
analyzed and the future directions are pointed out.
References
1 Ojala T, Pietik¨ainen M, Harwood D. A comparative study
of texture measures with classification based on featured dis-
tributions. Pattern Recognition, 1996, 29(1): 51−59
2 Pietik¨ainen M, Ojala T, Xu Z. Rotation-invariant texture
classification using feature distributions. Pattern Recogni-
tion, 2000, 33(1): 43−52
3 Ojala T, Valkealahti K, Oja E, Pietik¨ainen M. Texture dis-
crimination with multidimensional distributions of signed
gray-level differences. Pattern Recognition, 2001, 34(3):
727−739
4 Ojala T, Pietik¨ainen M, M¨aenp¨a¨a T. Multiresolution gray-
scale and rotation invariant texture classification with local
binary patterns. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 2002, 24(7): 971−987
5 Pietik¨ainen M, Hadid A, Zhao G Y, Ahonen T. Computer
Vision Using Local Binary Patterns. Berlin: Springer-Verlag,
2011. 193−202
6 Hafiane A, Seetharaman G, Zavidovique B. Median binary
pattern for textures classification. In: Proceedings of the
2007 International Conference on Image Analysis and Recog-
nition. Montreal, Canada: Springer, 2007. 387−398
7 Guo Z H, Zhang L, Zhang D, Zhang S. Rotation invariant
texture classification using adaptive LBP with directional
statistical features. In: Proceedings of the 17th IEEE In-
ternational Conference on Image Processing. Hong Kong,
China: IEEE, 2010. 285−288
8 Varma M, Zisserman A. A statistical approach to material
classification using image patch exemplars. IEEE Transac-
tions on Pattern Analysis and Machine Intelligence, 2009,
31(11): 2032−2047
9 Guo Z H, Zhang L, Zhang D. A completed modeling of lo-
cal binary pattern operator for texture classification. IEEE
Transactions on Image Processing, 2010, 19(6): 1657−1663
10 Guo Z H, Zhang L, Zhang D. Rotation invariant texture clas-
sification using LBP variance (LBPV) with global matching.
Pattern Recognition, 2010, 43(3): 706−719
11 He C, Ahonen T, Pietik¨ainen M. A Bayesian local binary
pattern texture descriptor. In: Proceedings of the 19th Inter-
national Conference on Pattern Recognition. Tampa, USA:
IEEE, 2008. 1−4
12 Brodatz P. Textures: A Photographic Album for Artists and
Designers. New York: Dover Publications, 1966. 1−112
13 Liao S, Law M W K, Chung A C S. Dominant local binary
patterns for texture classification. IEEE Transactions on Im-
age Processing, 2009, 18(5): 1107−1118
14 Ojala T, M¨aenp¨a¨a T, Pietik¨ainen M, Viertola J, Kyll¨onen J,
Huovinen S. Outex-new framework for empirical evaluation
of texture analysis algorithms. In: Proceedings of the 16th
International Conference on Pattern Recognition. Quebec,
Canada: IEEE, 2002. 701−706
15 Nanni L, Lumini A, Brahnam S. Survey on LBP based tex-
ture descriptors for image classification. Expert Systems with
Applications, 2012, 39(3): 3634−3641
16 M¨aenp¨a¨a T, Ojala T, Pietik¨ainen M, Soriano M. Robust tex-
ture classification by subsets of local binary patterns. In:
Proceedings of the 2000 International Conference on Pattern
Recognition. Barcelona, Spain: IEEE, 2000. 947−950
17 M¨aenp¨a¨a T, Pietik¨ainen M. Multi-scale binary patterns for
texture analysis. In: Proceedings of the 13th Scandinavian
Conference on Image Analysis. Lecture Notes in Computer
Science. Berlin, Germany: Springer, 2003. 885−892
18 Raja Y, Gong S G. Sparse multiscale local binary patterns.
In: Proceedings of the 2006 British Machine Vision Confer-
ence. Edinburgh, UK: BMVA, 2006. 1−10
19 He Y G, Sang N, Gao C X. Pyramid-based multi-structure
local binary pattern for texture classification. In: Proceed-
ings of the 10th Asian Conference on Computer Vision.
Berlin, Heidelberg: Springer, 2011. 133−144
20 Ahonen T, Pietik¨ainen M. Soft histograms for local binary
patterns. In: Proceedings of the 2007 Finnish Signal Pro-
cessing Symposium. Oulu, Finland: FINSIG, 2007. 1−4
21 Iakovidis D K, Keramidas E, Maroulis D. Fuzzy local binary
patterns for ultrasound texture characterization. In: Pro-
ceedings of the 5th International Conference on Image Anal-
ysis and Recognition. Berlin, Heidelberg: Springer, 2008.
750−759
22 Zhang L, Zhang L, Guo Z H, Zhang D. Monogenic-LBP: a
new approach for rotation invariant texture classification. In:
Proceedings of the 17th International Conference on Image
Processing. Hong Kong, China: IEEE, 2010. 2677−2680
23 Zhou H, Wang R S, Wang C. A novel extended local-binary-
pattern operator for texture analysis. Information Sciences,
2008, 178(22): 4314−4325
742 ACTA AUTOMATICA SINICA Vol.39
24 Liu L, Zhao L J, Long Y L, Kuang G Y, Fieguth P. Ex-
tended local binary patterns for texture classification. Image
and Vision Computing, 2012, 30(2): 86−99
25 Khellah F M. Texture classification using dominant neigh-
borhood structure. IEEE Transactions on Image Processing,
2011, 20(11): 3270−3279
26 Guo Y M, Zhao G Y, Pietik¨ainen M. Discriminative features
for texture description. Pattern Recognition, 2012, 45(10):
3834−3843
27 Guo Y, Zhao G, Pietik¨ainen M, Xu Z. Descriptor learning
based on Fisher separation criterion for texture classification.
In: Proceedings of the 2011 Asian Conference on Computer
Vision. Queenstown, New Zealand: Springer, 2011. 185−198
28 Zhao Y, Huang D S, Jia W. Completed local binary count for
rotation invariant texture classification. IEEE Transactions
on Image Processing, 2012, 21(10): 4492−4497
29 Fathi A, Naghsh-Nilchi A R. Noise tolerant local binary pat-
tern operator for efficient texture analysis. Pattern Recogni-
tion Letters, 2012, 33(9): 1093−1100
30 M¨aenp¨a¨a T, Pietik¨ainen M. Classification with color and
texture: jointly or separately? Pattern Recognition, 2004,
37(8): 1629−1640
31 Pietik¨ainen M, M¨aenp¨a¨a T, Viertola J. Color texture clas-
sification with color histograms and local binary patterns.
In: Proceedings of the 2008 International Workshop on Tex-
ture Analysis and Synthesis. Copenhagen, Denmark: IW-
TAS, 2002. 109−112
32 Porebski A, Vandenbroucke N, Macaire L. Haralick feature
extraction from LBP images for color texture classification.
In: Proceedings of the 2008 International Workshops on
Image Processing Theory, Tools and Applications. Sousse,
Tunisia: IEEE, 2008. 1−8
33 Zhang J, Liang J M, Zhao H. Local energy pattern for tex-
ture classification using self-adaptive quantization thresh-
olds. IEEE Transactions on Image Processing, 2013, 22(1):
31−42
34 Zhao G Y, Ahonen T, Matas J, Pietik¨ainen M. Rotation-
invariant image and video description with local binary pat-
tern features. IEEE Transactions on Image Processing, 2012,
21(4): 1465−1477
35 Ojansivu V, Heikkil¨aJ. Blur insensitive texture classification
using local phase quantization. In: Proceedings of the 2008
International Conference on Image and Signal Processing.
Cherbourg-Octeville, France: Springer, 2008. 236−243
36 Lategahn H, Gross S, Stehle T, Aach T. Texture classifi-
cation by modeling joint distributions of local patterns with
Gaussian mixtures. IEEE Transactions on Image Processing,
2010, 19(6): 1548−1557
37 Chen J, Shan S G, He C, Zhao G Y, Pietik¨ainen M, Chen
X L, Gao W. WLD: a robust local image descriptor. IEEE
Transactions on Pattern Analysis and Machine Intelligence,
2010, 32(9): 1705−1720
38 Liu L, Fieguth P, Clausi D, Kuang G Y. Sorted random pro-
jections for robust rotation-invariant texture classification.
Pattern Recognition, 2012, 45(6): 2405−2418
39 Liao S, Chung A C S. Face recognition by using elongated lo-
cal binary patterns with average maximum distance gradient
magnitude. In: Proceedings of the 8th Asian Conference on
Computer Vision. Tokyo, Japan: Springer, 2007. 672−679
40 Jin H L, Liu Q S, Lu H Q, Tong X F. Face detection using
improved LBP under Bayesian framework. In: Proceedings
of the 3rd International Conference on Image and Graphics.
Hong Kong, China: IEEE, 2004. 306−309
41 Petpon A, Srisuk S. Face recognition with local line binary
pattern. In: Proceedings of the 5th International Conference
on Image and Graphics. Xian, China: IEEE, 2010. 533−539
42 Tan X Y, Triggs B. Enhanced local texture feature sets for
face recognition under difficult lighting conditions. In: Pro-
ceedings of the 3rd International Conference on Analysis
and Modeling of Faces and Gestures. Rio de Janeiro, Brazil:
Springer, 2007. 168−182
43 Liao S C, Zhu X X, Lei Z, Zhang L, Li S Z. Learning multi-
scale block local binary patterns for face recognition. In: Pro-
ceedings of the 2007 International Conference on Biometrics.
Seoul, South Korea: Springer, 2007. 828−837
44 Zhang L, Chu R F, Xiang S M, Liao S C, Li S Z. Face de-
tection based on multi-block LBP representation. In: Pro-
ceedings of the 2007 International Conference on Advances
in Biometrics. Seoul, South Korea: Springer, 2007. 11−18
45 Wolf L, Hassner T, Taigman Y. Descriptor based methods in
the wild. In: Proceedings of the 2008 European Conference
on Computer Vision Workshop on Faces in Real-Life Images.
Marseille, France: Springer, 2008. 1−14
46 Samaria F S, Harter A C. Parameterisation of a stochastic
model for human face identification. In: Proceedings of the
2nd IEEE Workshop on Applications of Computer Vision.
Sarasota, USA: IEEE, 1994. 138−142
47 Belhumeur P N, Hespanha J P, Kriegman D J. Eigenfaces
vs. Fisherfaces: recognition using class specific linear projec-
tion. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 1997, 19(7): 711−720
48 Phillips P J, Wechsler H, Huang J, Rauss P J. The FERET
database and evaluation procedure for face-recognition algo-
rithms. Image and Vision Computing, 1998, 16(5): 295−306
49 Phillips P J, Flynn P J, Scruggs T, Bowyer K W, Chang
J, Hoffman K, Marques J, Jaesik M, Worek W. Overview
of the face recognition grand challenge. In: Proceedings of
the 2005 IEEE Conference on Computer Vision and Pattern
Recognition. San Diego, USA: IEEE, 2005. 947−954
50 Huang G B, Ramesh M, Berg T, Learned-Miller E. Labeled
Faces in the Wild: a Database for Studying Face Recogni-
tion in Unconstrained Environments, Technical Report, De-
partment of Computer Science, University of Massachusetts,
USA, 2007
No. 6 SONG Ke-Chen et al./ Research and Perspective on Local Binary Pattern 743
51 Huang D, Shan C F, Ardabilian M, Wang Y H, Chen L
M. Local binary patterns and its application to facial image
analysis: a survey. IEEE Transactions on Systems, Man, and
Cybernetics, Part C: Applications and Reviews, 2011, 41(6):
765−781
52 Chan C H, Kittler J V, Messer K. Multispectral local bi-
nary pattern histogram for component-based color face ver-
ification. In: Proceedings of the 2007 IEEE Conference on
Biometrics: Theory, Applications and Systems. Crystal City,
USA: IEEE, 2007. 1−7
53 Chan C H, Kittler J, Messer K. Multi-scale local binary pat-
tern histograms for face recognition. In: Proceedings of the
2007 International Conference on Biometrics. Seoul, South
Korea: Springer, 2007. 809−818
54 Zhang W C, Shan S G, Gao W, Chen X L, Zhang H M. Lo-
cal gabor binary pattern histogram sequence (LGBPHS): a
novel non-statistical model for face representation and recog-
nition. In: Proceedings of the 10th International Conference
on Computer Vision. Beijing, China: IEEE, 2005. 786−791
55 Tan X Y, Triggs B. Fusing Gabor and LBP feature sets for
kernel-based face recognition. In: Proceedings of the 2007
Corference on Analysis and Modeling of Faces and Gestures.
Rio de Janeiro, Brazil: Springer, 2007. 235−249
56 Shan S G, Zhang W C, Su Y, Chen X L, Gao W. Ensemble
of piecewise FDA based on spatial histograms of local (Ga-
bor) binary patterns for face recognition. In: Proceedings of
the 18th International Conference on Pattern Recognition.
Hong Kong, China: IEEE, 2006. 606−609
57 Maturana D, Soto A, Mery D. Face recognition with deci-
sion tree-based local binary patterns. In: Proceedings of the
2011 Asian Conference on Computer Vision. Queenstown,
New Zealand: Springer, 2011. 618−629
58 Lahdenoja O, Laiho M, Paasio A. Reducing the feature vec-
tor length in local binary pattern based face recognition. In:
Proceedings of the 2005 International Conference on Image
Processing. Genoa, Italy: IEEE, 2005. 914−917
59 Zhang B C, Gao Y S, Zhao S Q, Liu J Z. Local derivative
pattern versus local binary pattern: face recognition with
high-order local pattern descriptor. IEEE Transactions on
Image Processing, 2010, 19(2): 533−544
60 Shan C F, Gritti T. Learning discriminative LBP-histogram
bins for facial expression recognition. In: Proceedings of the
2008 British Machine Vision Conference. Leeds, UK: BMVC,
2008. 1−10
61 Ho An K, Chung M. Cognitive face analysis system for future
interactive TV. IEEE Transactions on Consumer Electron-
ics, 2009, 55(4): 2271−2279
62 Yan S Y, Shan S G, Chen X, Gao W. Locally assembled bi-
nary (LAB) feature with feature-centric cascade for fast and
accurate face detection. In: Proceedings of the 2008 IEEE
Conference on Computer Vision and Pattern Recognition.
Anchorage, USA: IEEE, 2008. 1−7
63 Roy A, Marcel S. Haar local binary pattern feature for
fast illumination invariant face detection. In: Proceedings of
the 2009 British Machine Vision Conference. London, UK:
BMVC, 2009
64 Yang H, Wang Y D. A LBP-based face recognition method
with Hamming distance constraint. In: Proceedings of
the 4th International Conference on Image and Graphics.
Sichuan, China: IEEE, 2007. 645−649
65 Fu X F, Wei W. Centralized binary patterns embedded with
image Euclidean distance for facial expression recognition.
In: Proceedings of the 4th International Conference on Nat-
ural Computation. Jinan, China: IEEE, 2008. 115−119
66 Zhang G C, Huang X S, Li S Z, Wang Y S, Wu X H. Boost-
ing local binary pattern (LBP)-based face recognition. In:
Proceedings of the 2005 Advances in Biometric Person Au-
thentication. Beijing, China: Springer, 2005. 179−186
67 Mu Y D, Yan S C, Liu Y, Huang T, Zhou B F. Discrim-
inative local binary patterns for human detection in per-
sonal album. In: Proceedings of the 2005 IEEE Conference
on Computer Vision and Pattern Recognition. Anchorage,
USA: IEEE, 2008. 1−8
68 Dalal N, Triggs B. Histograms of oriented gradients for hu-
man detection. In: Proceedings of the 2005 IEEE Conference
on Computer Vision and Pattern Recognition. San Diego,
USA: IEEE, 2005. 886−893
69 Wang X Y, Han T X, Yan S C. An hog-lbp human detec-
tor with partial occlusion handling. In: Proceedings of the
2009 International Conference on Computer Vision. Kyoto,
Japan: IEEE, 2009. 32−39
70 Trefny J, Matas J. Extended set of local binary patterns for
rapid object detection. In: Proceedings of the 2010 Com-
puter Vision Winter Workshop. Nove Hrady, Czech Repub-
lic: CVWW, 2010. 1−7
71 Agarwal S, Awan A, Roth D. Learning to detect objects in
images via a sparse, part-based representation. IEEE Trans-
actions on Pattern Analysis and Machine Intelligence, 2004,
26(11): 1475−1490
72 Heikkil¨a M, Pietik¨ainen M, Schmid C. Description of inter-
est regions with local binary patterns. Pattern Recognition,
2009, 42(3): 425−436
73 Zhao G Y, Pietik¨ainen M. Dynamic texture recognition us-
ing local binary patterns with an application to facial expres-
sions. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 2007, 29(6): 915−928
74 Kanade T, Cohn J F, Tian Y L. Comprehensive database for
facial expression analysis. In: Proceedings of the 4th Inter-
national Conference on Automatic Face and Gesture Recog-
nition. Grenoble, France: IEEE, 2000. 46−53
75 Heikkil¨a M, Pietik¨ainen M. A texture-based method for mod-
eling the background and detecting moving objects. IEEE
Transactions on Pattern Analysis and Machine Intelligence,
2006, 28(4): 657−662
744 ACTA AUTOMATICA SINICA Vol.39
76 Heikkil¨a M, Pietik¨ainen M, Heikkil¨a J. A texture-based
method for detecting moving objects. In: Proceedings of
the 2004 British Machine Vision Conference. London, UK:
BMVC, 2004. 187−196
77 Toyama K, Krumm J, Brumitt B, Meyers B. Wallflower:
principles and practice of background maintenance. In: Pro-
ceedings of the 1999 IEEE International Conference on Com-
puter Vision. Kerkyra, Greece: IEEE, 1999. 255−261
78 Takala V, Pietik¨ainen, M. Multi-object tracking using color,
texture and motion. In: Proceedings of the 2007 IEEE In-
ternational Workshop on Visual Surveillance. Minneapolis,
USA: IEEE, 2007. 1−7
79 Yao J, Odobez J M. Multi-layer background subtraction
based on color and texture. In: Proceedings of the 2007 IEEE
Conference on Computer Vision and Pattern Recognition.
Minneapolis, USA: IEEE, 2007. 3658−3665
80 Liao S C, Zhao G Y, Kellokumpu V, Pietik¨ainenM,LiS
Z. Modeling pixel process with scale invariant local pat-
terns for background subtraction in complex scenes. In: Pro-
ceedings of the 2010 IEEE Conference on Computer Vision
and Pattern Recognition. San Francisco, USA: IEEE, 2010.
1301−1306
81 Kellokumpu V, Zhao G Y, Pietik¨ainen M. Human activity
recognition using a dynamic texture based method. In: Pro-
ceedings of the 2008 British Machine Vision Conference. Lon-
don, UK: BMVC, 2008. 1−10
82 Kellokumpu V, Zhao G Y, Pietik¨ainen M. Texture based
description of movements for activity analysis. In: Proceed-
ings of the 2008 International Conference on Computer Vi-
sion Theory and Applications. Funchal, Portugal: VISAPP,
2008. 206−213
83 Kellokumpu V, Zhao G Y, Li S Z, Pietik¨ainen M. Dynamic
texture based gait recognition. In: Proceedings of the 3rd In-
ternational Conference on Advances in Biometrics. Alghero,
Italy: Springer, 2009. 1000−1009
84 Kellokumpu V, Zhao G Y, Pietik¨ainen M. Recognition of hu-
man actions using texture descriptors. Machine Vision and
Applications, 2011, 22(5): 767−780
85 Costa Y M G, Oliveira L S, Koerich A L, Gouyon F, Martins
J G. Music genre classification using LBP textural features.
Signal Processing, 2012, 92(11): 2723−2737
86 Caputo B, Hayman E, Mallikarjuna P. Class-specific mate-
rial categorisation. In: Proceedings of the 2005 IEEE In-
ternational Conference on Computer Vision. Beijing, China:
IEEE, 2005. 1597−1604
87 Caputo B, Hayman E, Fritz M, Eklundh J O. Classifying
materials in the real world. Image and Vision Computing,
2010, 28(1): 150−163
SONG Ke-Chen Ph.D. candidate at
the School of Mechanical Engineering and
Automation, Northeastern University. He
received his master degree from Northeast-
ern University in 2011. His research in-
terest covers computer vision and pattern
recognition. Corresponding author of this
paper.
E-mail: unkechen@gmail.com
YAN Yun-Hui Professor at the School
of Mechanical Engineering and Automa-
tion, Northeastern University. He received
his Ph. D. degree from Northeastern Uni-
versity in 1997. His research interest covers
intelligent detection and quality control.
E-mail: yanyh@mail.neu.edu.cn
CHEN Wen-Hui Master student at
the School of Mechanical Engineering and
Automation, Northeastern University. He
received his bachelor degree from Jilin Uni-
versity in 2011. His research interest covers
intelligent detection and quality control.
E-mail: wenhui chen@126.com
ZHANG Xu Master student at the
School of Mechanical Engineering and Au-
tomation, Northeastern University. He re-
ceived his bachelor degree from Northeast-
ern University at Qinhuangdao in 2011.
His research interest covers object detec-
tion and pattern recognition.
E-mail: zhangxufirstlove@163.com
