Content uploaded by Shrinivas Pundlik
Author content
All content in this area was uploaded by Shrinivas Pundlik on Jul 06, 2015
Content may be subject to copyright.
Periocular Region Appearance Cues for Biometric Identification
Damon L. Woodard Shrinivas J. Pundlik Jamie R. Lyle Philip E. Miller
Biometrics and Pattern Recognition Lab, School of Computing
Clemson University, Clemson, SC 29634 USA
{woodard, spundli, jlyle, pemille}@clemson.edu
Abstract
We evaluate the utility of the periocular region appear-
ance cues for biometric identification. Even though peri-
ocular region is considered to be a highly discriminative
part of a face, its utility as an independent modality or
as a soft biometric is still an open ended question. It is
our goal to establish a performance metric for the peri-
ocular region features so that their potential use in con-
junction with iris or face can be evaluated. In this ap-
proach, we employ the local appearance based feature rep-
resentation, where the image is divided into spatially salient
patches, and histograms of texture and color are computed
for each patch. The images are matched by computing the
distance between the corresponding feature representations
using various distance metrics. We report recognition re-
sults on images captured in the visible and near-infrared
(NIR) spectrum. For the color periocular region data con-
sisting of about 410 subjects and the NIR images of 85 sub-
jects, we obtain the Rank-1 recognition rate of 91% and
87% respectively. Furthermore, we also demonstrate that
recognition performance of the periocular region images is
comparable to that of face.
1. Introduction
In recent years, face and iris have established them-
selves as widely popular modalities for human identification
[4, 23]. Despite the recent advances, the performance of iris
and face recognition algorithms declines when dealing with
non-ideal scenarios such as non-uniform illumination, pose
variations, occlusions, expression changes, and radical ap-
pearance changes in the input images. These challenges can
be tackled either by: i) improving the existing algorithms
that handle various aspects of the recognition problem, ii)
combining multiple complementary modalities, or iii) ex-
ploring newer traits that can perform or aid the process of
recognition. Periocular region has potential to be such a
trait, as it is considered to be one of the highly discrimi-
native regions of a face [19]. In this paper, our goal is to
Figure 1. Some key periocular region features. LEFT: Level-one
features that are mostly geometric in nature. RIGHT:Level-two
features, like skin texture and color can be extracted from the area
surrounding the eye.
explore various appearance based cues that can be extracted
from the region surrounding the human eye and evaluate
their utility to perform identification, independent of other
modalities. The eventual goal of this empirical study is to
establish techniques that use periocular region appearance
features to help iris and face recognition algorithms.
The periocular region is a small region in the neighbor-
hood of the eye and may include eyebrows. Its use for iden-
tification or verification predominantly figures in the case
of approaches that are based on iris [6], vein patterns of the
eye[5], or partial facial features [9, 8, 7, 18, 22]. A recent
approach performs identification solely based on the perioc-
ular region texture features [15]. While iris or retinal pattern
based identification relies only on the specific information
found within the eye, approaches dealing with face recog-
nition using partial facial features consider the entire eye
region as one of the many discriminative features. Though
they differ in terms of how a periocular region is precisely
defined, the above mentioned approaches dealing with par-
tial facial features conclude that the periocular region is su-
perior as compared to other salient facial regions such as
nose or mouth, for identification. Apart from this, our mo-
tivation for using the information present in the periocular
region also stems from the fact that periocular region is the
common link between face and iris based recognition, and
can potentially aid both these modalities.
Traditional iris recognition algorithms are based on en-
coding and matching the iris texture pattern [6], but are sen-
sitive to the quality of the iris images. This is especially
true in the case of images captured in the visible spectrum.
1
Combining the discriminative information gathered around
the eye with iris features has the potential to improve the
overall recognition performance. Similarly, in the context
of facial recognition, the use of periocular region may be
beneficial in situations where the face is partially occluded,
or the subjects have facial hair, etc. Also, it has been pro-
posed in some recent approaches that part-based face repre-
sentation [1, 8, 9], as opposed to the traditional holistic face
representation [2, 21], may lead to improved face recogni-
tion performance. Of the various facial components, eyes
and the region surrounding them is known to be highly dis-
criminative [18]. In this regard, it would be noteworthy to
study the effect of features extracted from the periocular re-
gion on face recognition in greater detail.
Some of the prominent periocular region features are
shown in Figure 1. Excluding the iris, these features can
be classified as level one-features that include both upper
and lower eyelids, eye folds, and eye corners; or level-two
features such as detailed skin texture, fine wrinkles, color,
or skin pores. Level-one features tend to be more dominant
in nature as compared to the level-two features. Alterna-
tively, the features can also be classified based on geometry
(shape of the eye, eye lids, eye folds and various lines), tex-
ture, or color. Again, most of the level-one features fall into
the category of geometric features (shape dependent), while
level-two features can be classified as texture or color fea-
tures.
In this paper we demonstrate that low-level features ex-
tracted from the periocular region can be effectively used
for identification. As compared to the previous approaches
dealing with periocular region such as [15], the chief nov-
elty in this work lies in our use of only the level-two perioc-
ular features based on skin texture and color information (if
available) to perform identification. To this effect, we mask
the eye in the periocular region (see Figure 4) thus removing
the iris and various level-one features. The reason for such
an exclusion is that it eliminates the effect of iris on the
computed texture and color features, thus facilitating a way
to compute unbiased recognition performance metric for the
periocular region features. Although removal of the eye
from a periocular region image may seem like a heavy loss
of discriminating information, it could be potentially advan-
tageous as the level-one features are highly sensitive to the
opening and closing of the eyes and may end up influenc-
ing the texture features adversely (as they are very dominant
due to relatively high gradients). We show recognition re-
sults using color periocular images extracted from the Facial
Recognition Grand Challenge (FRGC) II [17] dataset. Fur-
thermore, we demonstrate the recognition results using the
near-infrared (NIR) periocular images extracted from the
Multiple Biometrics Grand Challenge (MBGC) [16] NIR
face videos. We are not aware of any other approach that
has used the NIR periocular region images. For further val-
Figure 2. Different ways of extracting periocular regions from face
images. For this work, non-overlapping regions for the left and the
right eyes are extracted.
idation, we show that the recognition results obtained from
the color periocular region compare favorably with those
obtained using full face images. The subsequent sections
describe the details of our approach and the experimental
results.
2. Periocular Region Data
One of the ways to obtain periocular region images is
to extract them from face images. Different schemes of
extracting periocular region images from face images are
shown in Figure 2 that include cropping the entire strip con-
taining both the eyes, or extracting two overlapping regions
corresponding to both. In this work, we extract two distinct
non-overlapping images belonging to each left and right
eye. Since one of our eventual goals is to combine informa-
tion obtained from the periocular region with the iris tex-
ture, and since iris images are typically captured separately
for each eye, such an independent handling of both the peri-
ocular regions would facilitate efficient information fusion
between the two modalities. The size of the extracted peri-
ocular region images is another consideration. Ideally, the
periocular region image should be such that it captures the
region around the eye in sufficient detail, while not compro-
mising on the size (or quality) of the iris region. Increasing
the image size can lead to increased discriminative ability
only up to a certain extent because as the periocular image
becomes larger in size, a larger number of pixels belonging
to the forehead or cheek region would form part of it. Oper-
ating under these constrains, we extract the color and NIR
periocular region images from the face images as described
below.
2.1. Color Periocular Region Images
The color periocular region data used in the experi-
ments described in this paper are extracted from the frontal
face images present in the FRGC database [17]. The
FRGC database consists of high resolution color images (≈
1200×1400, 72 dpi) of a large number of subjects mostly
between ages 18 and 22, collected over a two year period
from multiple recording sessions involving controlled and
uncontrolled lighting conditions, and with and without ex-
pressions (neutral expression). In controlled conditions, the
frames from MBGC NIR face videos right eye left eye
Figure 3. TOP ROW: Steps for obtaining NIR periocular region im-
ages from the MBGC NIR face video data. BOTTOM ROW: Some
examples of face image frames extracted from a MBGC NIR face
video, along with the periocular region images corresponding to
the first example frame. Notice that not all the frames in the video
are usable.
distance between the subjects and the camera is approxi-
mately the same. The FRGC dataset was chosen for this
work because of the availability of high resolution face im-
ages which would lead to relatively larger periocular region
images. This would enable us to perform iris recognition
in the visible spectrum in the future. Moreover, periocular
color and texture information can be captured reliably from
the high resolution images.
The ground truth eye centers for the faces are provided
in the FRGC dataset which act as the centers of the perioc-
ular images to be cropped out of the original face images.
The size of the cropping region is calculated by a ratio of
the distance between the eye centers. The periocular region
images are then scaled down to a uniform size of 100x160
pixels. These extracted images, as compared to the orig-
inal facial image, are shown in Figure 2. For computing
texture, the images are converted to grayscale and prepro-
cessed by histogram equalization. For color image prepro-
cessing, the RGB image is converted to the CIE L*a*b*
color space, the luminance channel histogram is equalized,
and then converted back to the RGB color space. These
steps are followed in order to preserve the color informa-
tion. To eliminate the effect of texture and color in the iris
and the surrounding sclera area, an elliptical mask of neutral
color is placed over the center of the periocular region im-
age. The dimensions of the ellipse are predefined based on
the dimensions of the input periocular image rather than the
dimensions of the subjects’ eye. The underlying assump-
tion (based on empirical study of the data that we are using)
is that the change in size of the eye is mostly on account
of its varying amount of opening and not so much due to
changes in scale. This, coupled with the fact that the im-
ages are aligned and scaled to a fixed size, allows placing of
a fixed size ellipse on the eye such that a significant amount
of periocular skin is still visible.
2.2. NIR Periocular Region Images
The NIR periocular region data used in this paper are
extracted from frontal face videos that form a part of the
Figure 4. Overview of the steps for computing texture and color
representation of a periocular region image.
MBGC portal challenge, where the subjects walk through
a portal that captures videos of the face (both the visible
and NIR spectrum) and iris (NIR). The goal of the MBGC
dataset is to perform recognition using multiple modalities
of face and iris (still images and videos). The NIR iris video
and still images are not suitable for this work as very little
periocular region is visible. Hence, we use the NIR face
videos and extract periocular regions to construct our test
dataset.
The videos range from 15 to 30 frames in length and
consist of 2048 ×2048 images of varying quality (see Fig-
ure 3). The variation in the image quality is on account
of the NIR lights shining only for a brief duration of the
time as the subject walks through the portal. This causes
some frames to be too dark or too bright. Other effects such
as blinking, partial occlusion of the face, motion blur, etc.
are also present in a large number of frames. In order to
process these videos, we employ the steps shown in Fig-
ure 3. The images extracted from the video are thresholded
based on their average grayscale value so as to reject those
frames in which the face is not visible. Then the eye cen-
ters are marked and periocular regions of size 601 ×601
are extracted. Even though there is some scale change in
the face as the subject walks toward the camera, most of
the usable frames (higher quality) are in the earlier part of
the video, where the scale change is not substantial. For
this reason, the effects of scale change are ignored and a
fixed sized periocular region image centered at the eye is
extracted. To remove blurred images, high frequency con-
tent is measured from the 2D Fourier spectrum obtained by
convolving the image with a 8×8 kernel. A threshold is set
for each subject based on the maximum and the minimum
energy obtained from processing all the frames belonging
to the subject. Rejecting the blurred images results in a set
of usable left and right periocular region images used for
the experiments presented in section 4.
3. Periocular Region Features
As described earlier, in this work we match two peri-
ocular region images using level-two periocular region fea-
tures. In case of the images from visible spectrum, we use
both skin texture and color, while for the NIR periocular
region images, only texture is used. Use of skin texture
for recognition has been investigated from various perspec-
tives. While some approaches explicitly use the so called
skin micro-features such as moles, scars, or freckles [10],
others adopt a more general representation for the overall
texture in the periocular region using popular texture mea-
sures such as discrete cosine transform (DCT) [7], gradi-
ent orientation histogram (GOH, or alternatively known as
Histograms of Oriented Gradients), or local binary patterns
(LBP) [13, 15]. Usually, these popular texture measures are
used for computing the texture locally, and the entire image
is represented as a combination of these local texture fea-
tures, thus leading to the idea of local appearance features.
To this effect, we divide the periocular region image into
blocks and locally compute the texture and color features
(see Figure 4). A texture or color feature vector describing
the entire image is then computed by concatenating the vec-
tors corresponding to the each of the blocks. Such a local
appearance based representation is well suited for this work
because it preserves the spatial relationship of the features.
As our input images are aligned and normalized (for size),
it is easier to partition the images into blocks. This leads to
a fixed length feature vector for each image that can be di-
rectly used for matching without any further normalization
of the vector.
Let Ibe the preprocessed input periocular region image
(color, Icor grayscale, Itdepending on the data) divided
into Nblocks of Mpixels each with I(i)representing the ith
image block, then the texture feature representation of the
image is given by an ordered set T(It)=T(1)
,...,T(N),
where T(1),..., T(N)are the texture histograms correspond-
ing to the Nblocks. Similarly, the color feature representa-
tion of the image is given by C(Ic)=C(1)
,...,C(N), with
C(1), ..., C(N)being the color histograms of the Nblocks.
Details of our color and texture feature implementations are
described below.
3.1. Skin Texture
We use the local binary patterns (LBPs) as the periocular
texture measure. Local Binary Patterns (LBP) [14] quantify
intensity patterns found in local pixel neighborhood patches
and are useful for identifying spots, line ends, edges, cor-
ners, and other distinct texture patterns. Their use is popular
in various biometric applications such as face recognition
[1, 11], facial expression recognition [12], and iris recogni-
tion [20].
We now briefly describe the computation of LBP his-
togram for an image. For a more detailed description of
LBP, see [14]. As the name suggests, computing a LBP
score for a pixel involves counting the binary changes of in-
tensity patterns in a ppixel neighborhood along a circle of
radius raround that pixel. Subtracting the intensity of the
center pixel from its neighbors and using the sign of the dif-
ference instead of the actual difference leads to illumination
and scale invariance in the local neighborhood. Each pixel
intensity change is assigned a binomial weight to obtain a
binary code for the local neighborhood. If x(i)
jrepresent the
coordinates of the jth pixel of the ith block, then the LBP
vector at this pixel location is given by
φ
x(i)
j=
p−1
∑
k=0
sIt(x(i)
k)−It(x(i)
j)2p
,(1)
where s(.)is the sign operator, pis the number of pixels in
the local neighborhood of x(i)
j, and It(x(i)
k),It(x(i)
j)represent
intensities at locations x(i)
k, and x(i)
jrespectively. The above
equation results in 2pdimensional vector for each pixel cor-
responding to the 2pbinary patterns found in the ppixel
neighborhood of a pixel. The number of bitwise changes
in the pattern obtained in a pixel neighborhood is called the
uniformity measure and is denoted by U
φ
x(i)
j.Inthis
work, we use the uniformity value of 2 leading to a LBP
vector
φ
x(i)
j=∑p−1
k=0sIt(x(i)
k)−It(x(i)
j),if U
φ
x(i)
j≤2
p+1,otherwise
where
U
φ
x(i)
j =sIt(x(i)
p−1)−It(x(i)
j)−sIt(x(i)
k−1)−It(x(i)
j)
+
p+1
∑
k=1sIt(x(i)
k)−It(x(i)
j)−sIt(x(i)
k−1)−It(x(i)
j).
An LBP vector is computed for each pixel in an image
patch I(i), which is in turn encoded into a histogram of bins
bt=p(p−1)+3 given by:
T(i)=hist
φ
x(i)
1,...,
φ
x(i)
M,bt.(2)
The dimensionality of this patch histogram results from the
fact that there are p−1 uniform patterns for each of the p
bit changes and an addition bit change. The 3 remaining
bins store the 2 uniform patterns where uniformity value is
0, and all non-uniform patterns. Our implementation sets
r=1 and p=8, resulting in a 59 bin histogram for a 3 ×3
neighborhood.
3.2. Color Features
Color histogram based representation of the periocular
region images can be computed in various color spaces. We
experiment with the RGB and HSV color spaces and their
sub-spaces. In case of RGB images, it is common to con-
sider all three color channels for histogramming (3D his-
togram). But we observed that using only two channels
instead of three for histogram construction (2D histogram)
gave a comparable performance while being more efficient.
Let Icbe the preprocessed color periocular image. Like tex-
ture computation, color histograms are computed for each
image block I(i)
cand are given by C(i)=hist
ψ
(I(i)
c),bc,
where
ψ
(I(i)
c)is the transformation of a RGB image into
either a different color space or a sub-space, and bcis the
number of bins. We experimented with RB, RG, GB and
HSV color spaces and found that RG color space outper-
forms the others. Of the various bin configurations that we
tried, the 4 ×4 histogram (with bc=16) performed better
than both coarser and finer binning configurations.
3.3. Matching Periocular Region Features
Matching of two images is done by computing the dis-
tance (or similarity) metric using the corresponding fea-
ture representations. For the FRGC dataset, we perform
score level fusion for the texture and color representations
(T(It),C(Ic)respectively) and hence, both can be sepa-
rately matched for a pair of images. For matching, T(It)
and C(Ic)are converted to their vectorized forms T(It)
and C(Ic)of N×btand N×bcdimension respectively. We
set N=28 for the color periocular region images, while
for NIR images, N=36. A function DT(It,pr ),T(It,gl)
compares the texture features for the probe and gallery im-
ages (It,pr and It,gl ). Similar convention is followed in the
case of color images. We experiment with various com-
monly used histogram comparison functions such as L1, L2
norm, correlation, covariance, cityblock distance, Jefferey
divergence, chi-squared distance, Bhattacharya coefficient,
and histogram intersection. In the case of LBP texture fea-
tures, cityblock performs the best while in the case of color
histograms Bhattacharya coefficient outperforms the others.
The color and texture are combined at the match score level
(distance or similarity values) using summing with min-
max normalization. For all the experiments, we weigh both
color and texture equally, although texture could be given a
higher weight as it gives better recognition on its own.
4. Experimental Results
The experimental set of color periocular region images
comes from the face images captured under controlled light-
ing that constitute the original FRGC Experiment 1. Two
images per subject are used as a gallery set that come from
the neutral expression face images captured in the same ses-
sion. The three experiments, named FRGC-1,FRGC-2, and
FRGC-3 are differentiated by their probe images. FRGC-
1contains one neutral expression image from each subject
from a different recording session, FRGC-2 is composed of
alternate expression images from the same recording ses-
sion, and FRGC-3 has alternate expression images from a
different recording session. Each of these experiments have
sub-experiments using only left eyes, only right eyes, and a
4000 6000 8000 10000 12000 14000
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
distance
prob. of occurance
impostor
genuine
0.06 0.08 0.1 0.12 0.14 0.16
0
0.05
0.1
0.15
0.2
0.25
distance
prob. of occurance
impostor
genuine
texture color
00.2 0.4 0.6 0.8 1
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
distance
prob. of occurance
impostor
genuine
combined
Figure 5. Genuine and impostor score distribution for the left pe-
riocular region images for FRGC-1 experiment.
fusion of both eyes (score-level fusion with min-max nor-
malization). Additionally, we use the full face images in
each of these experiments for the purpose of comparing the
recognition rates obtained using the periocular region. The
idea of local appearance based recognition that was applied
while computing the periocular features is also used in the
case of face recognition, i.e., the face image is divided into
blocks, and features are computed and matched for each
block. The preprocessing of the face images is done using
the approach detailed in [3] resulting in 195 ×225 facial
images that are divided in 49 blocks. Of the 466 test sub-
jects in the original FRGC dataset, there are 410 subjects
that have images taken in multiple recording sessions, re-
sulting in a total of 4100 (2050 for each region, 3 probe, 2
gallery) periocular region images for all three experiments
combined.
Figure 5 shows the match score distribution for the tex-
ture and color features for the left periocular region of
FRGC-1, emphasizing the relatively higher discriminative
ability of the texture based representation as compared to
the color based representation. Similar distribution is ob-
served for the right periocular region. Figure 6 shows the
Cumulative Match Characteristics (CMC) for the left, right,
and combined left and right periocular regions for FRGC-1
and FRGC-3 experiments using texture, color, and a fusion
of both. The joint Rank-1 recognition rates obtained using
both right and left periocular regions are consistently higher
for the texture and color as compared to those using only
either left or right. The combined texture and color recog-
nition is higher as compared to their individual recognition,
except in the case of joint left and right periocular region for
FRGC-1 experiment, where the relatively lower recognition
rates of color features as compared to texture bring down
2 4 6 8 10 12 14 16 18 20
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
2 4 6 8 10 12 14 16 18 20
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
2 4 6 8 10 12 14 16 18 20
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
2 4 6 8 10 12 14 16 18 20
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
2 4 6 8 10 12 14 16 18 20
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
2 4 6 8 10 12 14 16 18 20
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
FRGC-1 FRGC-3
Figure 6. Cumulative Match Characteristics for the experiments
FRGC-1 (left) and FRGC-3(right) on the left, right and left + right
(top to bottom) using texture, color, and combination of both.
2 4 6 8 10 12 14 16 18 20
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
periocular region
2 4 6 8 10 12 14 16 18 20
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
periocular region
FRGC-1 FRGC-2
2 4 6 8 10 12 14 16 18 20
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
texture
color
texture + color
periocular region
FRGC-3
Figure 7. Cumulative Match Characteristics for the experiments
FRGC-1,FRGC-2,andFRGC-3 using full face image.
the overall recognition rates. A similar trend is observed in
the case of the recognition using full face images (see Fig-
ure 7). The main reason that the color cannot perform well
in the case of the face images is that there is more intra-
class variability in the case of the face images due to slight
illumination changes or presence of facial hair, as compared
to the periocular images. When comparing neutral to alter-
nate expression faces, in addition to the above problems,
expression change can cause the mouth (and teeth) to span
different blocks, thus degrading the color histograms per-
formance for face recognition in the FRGC-3 experiment.
For FRGC-2 experiment, since the images were captured
in the same session, the similarity of the color information
outweighs the differences caused by expression in the face,
resulting in higher recognition rates for the face and peri-
ocular regions. Table 1 summarizes the Rank-1 recognition
rates for all experimental configurations.
It can be observed from Table 1 that the recognition rate
drops off across the board when using probe images from
different sessions and this factor is more prominent than the
expression change in affecting the recognition rates. It can
also be seen that recognition obtained using only the peri-
ocular region is comparable to that obtained using the en-
tire face. In the current experimental setting, the texture
and color representation of periocular regions are less influ-
enced by expression change as there is only a slight drop in
their recognition performance (FRGC-1 vs. FRGC-3). In
some cases, the combined texture and color representation
for the periocular regions outperform the face. Our goal is
not to claim that periocular region is superior to the face,
but to merely point out that in certain situations, periocular
regions alone is sufficient for accurate recognition. There
may be certain situations, such as illumination changes, that
affect the periocular region features drastically and in such
cases, using the entire face would lead to a better perfor-
mance. We cannot directly compare the results obtained in
this work to some of the previous works due to the differ-
ences in the datasets used or due to the differences in the
experimental setup. But as compared to [7] that uses FRGC
Experiment 1 images (P2 partitioning, F1 experimental set,
left eye ≈90 %, right eye ≈88%), we obtain similar recog-
nition results. But it should be noted that in their case, the
experimental setup is not clearly defined and the number of
subjects involved in testing is much less than ours. Also,
we are using the information strictly around the eye and no
information inside the eye is utilized. This indicates the
highly discriminative nature of the periocular region texture
and color features.
The NIR periocular region dataset consists of videos of
115 subjects with 2-20 usable face images per subject lead-
ing to over 1700 face images. Of these, not all have both
right and left eyes visible, thus leading to 85 subjects, 689
right periocular region images, and 113 subjects, 911 im-
ages for the right and left periocular region images respec-
tively. Many of these images are of very low quality as de-
scribed in Section 2.2. We retain the first two images of the
video as gallery images, while choosing a probe image at
random from the remaining useful frames for that subject.
The left and the right periocular region are compared sepa-
1 2 3 4 5 6 7 8 9
x 104
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
distance
prob. of occurance
genuine
impostor
1 2 3 4 5 6 7 8 9
x 104
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
distance
prob. of occurance
genuine
impostor
510 15 20
0.7
0.75
0.8
0.85
0.9
0.95
1
Rank
Recognition Rate
left periocular region
right periocular region
Figure 8. Genuine and impostor score distribution for the (a) left,
(b) right periocular region images for MBGC NIR experiment
along with the CMC for both (c).
rately, giving the Rank-1 recognition rates of 81% and 87%
respecively. Figure 8 shows the score distribution and the
CMC statistics for the NIR periocular images. The separa-
bility of the genuine and the impostor scores as well as the
Rank-1 recognition rate is lower in the case of NIR images
as compared to the color images due to the poor quality of
the NIR data. Still, these statistics indicate that periocular
region when used with iris may provide some boost to the
overall recognition rate.
Relative discriminative ability of each image block for
FRGC-1 left eye using texture is shown in Figure 9. The
center of the eye receives lowest weight due to the presence
of the mask while the blocks on the periphery (especially
the top ones as they represent the eyebrows) are highly dis-
criminative. Figure 10 shows some examples where the
probe image failed to match with the gallery images. In case
of the FRGC images, the main reasons for failure to match
are occlusion of the periocular region (Figure 10a), pres-
ence of glasses (Figure 10d), or radical appearance changes
(Figure 10e). Another cause is the misalignment of the im-
ages Figure 10b). Since the ground truth eye centers are lo-
cated on the center of the pupil, movement of the eye shifts
the center and consequently affects the cropped periocular
region. In the case of the NIR images, blurring and scale
change is the most likely cause of a non-match. Another
reason is the illumination changes caused due to the NIR
lighting leading to washout of the images.
5. Conclusion
This paper investigates the utility of various appearance
cues such as periocular skin texture and color for biomet-
ric identification. The experiments presented in this paper
measure the performance of this new biometric on color and
010 20 30
0
0.5
1
block number
weight
2 4 6
1
2
3
4
0
0.5
1
Figure 9. The discriminative ability of each local block in the peri-
ocular region. The plot (left) and the weight map (right) is shown
for FRGC-1 right region using texture.
abb
def
Figure 10. Some examples pairs of gallery (left) and probe (right)
images of the same subjects from both the FRGC and MBGC NIR
datasets that failed to match.
near-infrared periocular region image data. The recognition
rates obtained by the periocular region are comparable to
those obtained by using the full face using similar features
based on local appearance. The experiments presented in
the paper demonstrate that at its best, the periocular region
holds a lot of promise as a novel modality for identifying
humans with a potential of influencing other established
modalities based on iris and face. At the very least, the
results suggest a potential for using periocular region as a
soft biometric. Future work includes evaluation of more
periocular features, comparison of periocular based recog-
nition performance to a commercial face recognition algo-
rithm, exploration of how the capture conditions and the im-
age quality such as uncontrolled lighting, or subjects wear-
ing cosmetics affect the periocular skin texture and color,
among others
References
[1] T. Ahonen, A. Hadid, and M. Pietikainen. Face description
with local binary patterns: application to face recognition.
IEEE Transactions on Pattern Analysis and Machine Intelli-
gence, 28(12):2037–2041, 2006.
[2] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigen-
faces vs. fisherfaces: recognition using class specific linear
projection. In Proceedings of the European Conference on
Computer Vision, pages 45–58, 1996.
[3] J. Beveridge, D. Bolme, B. Draper, and M. Teixeira. The csu
face identification evaluation system: Its purpose, features,
and structure. Machine Vision and Applications, 14(2):128–
138, 2006.
[4] K. Bowyer, K. Hollingsworth, and P. Flynn. Image under-
standing for iris biometrics: a survey. Journal of Computer
Vision and Image Understanding, 110(2):281–307, 2007.
FRGC-1: neutral expression FRGC-2 : alternate expression FRGC-3: alternate expression
(different session) (same session) (different session)
Texture Color Texture + Color Texture Color Texture + Color Texture Color Texture + Color
left periocular 84.6 72.9 87.1 95.6 95.4 96.8 81.2 72.4 87.1
right periocular 87.1 76.3 88.3 95.6 96.1 96.8 83.4 74.2 87.1
left + right 91.5 82.9 91.0 97.6 97.6 98.3 88.8 82.0 91.2
face 94.6 63.2 91.7 98.8 92.2 98.8 90.7 52.4 85.1
Table 1. Rank - 1 recognition rate for the three FRGC experiments described in this paper.
[5] S. Crihalmeanu, A. Ross, and R. Derakhshani. Enhancement
and registration schemes for matching conjunctival vascula-
ture. Proc. of the 3rd IAPR/IEEE International Conference
on Biometrics (ICB), pages 1240–1249, 2009.
[6] J. Daugman. How iris recognition works. IEEE Trans. on
Circuits and Systems for Video Technology, 16:21–30, 2004.
[7] H. K. Ekenel and R. Stiefelhagen. Generic versus salient
region based partitioning for local appearance face recogni-
tion. In IAPR/IEEE International Conference on Biometrics
(ICB), 2009.
[8] B. Heisele, T. Serre, and T. Poggio. A component-based
framework for face detection and identification. Interna-
tional Journal of Computer Vision, 74(2):167–181, 2007.
[9] P. Heisele, B.and Ho, J. Wu, and T. Poggio. Face recogni-
tion: component-based versus global approaches. Computer
Vision and Image Understanding, 91((1-2)):6–21, 2003.
[10] A. K. Jain and U. Park. Facial marks: soft biometric for
face recognition. In Proceedings of the IEEE International
Conference on Image Processing, 2009.
[11] S. Z. Li, R. Chu, M. Ao, L. Zhang, and R. He. Highly
accurate and fast face recognition using near infrared im-
ages. Proc. Advances in Biometrics, International Confer-
ence, Lecture Notes in Computer Science 3832, Springer,
pages 151–158, 2006.
[12] S. Liao, W. Fan, A. Chung, and D. Yeung. Facial expres-
sion recognition using advanced local binary patterns, tsallis
entropies and global appearance features. Proc. of the IEEE
International Conference on Image Processing (ICIP), pages
665–668, 2006.
[13] P. Miller, A. Rawls, S. Pundlik, and D. Woodard. Personal
identification using periocular skin texture. In ACM Sympo-
sium on Applied Computing, 2009.
[14] T. Ojala, M. Pietik ¨ainen, and T. M ¨aenp ¨a¨a. Multiresolution
gray-scale and rotation invariant texture classification with
local binary patterns. IEEE Transactions on Pattern Analysis
and Machine Intelligence (PAMI), 24(7):971–987, 2002.
[15] U. Park, A. Ross, and A. K. Jain. Periocular biometrics in the
visible spectrum: a feasibility study. In Biometrics: Theory,
Applications and Systems, 2009.
[16] J. Philips. Multiple Biometrics Grand Challenge,
http://face.nist.gov/mbgc/.
[17] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang,
K. Hoffman, J. Marques, J. Min, and W. Worek. Overview
of face recognition grand challenge. IEEE Conference on
Computer Vision and Pattern Recognition, 2005.
[18] M. Savvides, R. Abiantun, J. Heo, C. Xie, and B. K. Vi-
jayakumar. Partial and holistic face recognition on frgc ii
using support vector machines. Proc. of IEEE Computer Vi-
sion Workshop (CVPW), page 48, 2006.
[19] P. Sinha, B. Balas, Y. Ostrovsky, and R. Russell. Face recog-
nition by humans: Nineteen results all computer vision re-
searchers should know about. Proc. IEEE, 94(11):1948–
1962, November 2006.
[20] Z. Sun, T. Tan, and X. Qiu. Graph matching iris image blocks
with local binary pattern. Proc. Advances in Biometrics, In-
ternational Conference (ICB), Lecture Notes in Computer
Science 3832, Springer, pages 366–373, 2006.
[21] M. A. Turk and A. P. Pentland. Face recognition using eigen-
faces. In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, pages 586–591, 1991.
[22] J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma.
Robust face recognition via sparse representation. IEEE
Transactions on Pattern Analysis and Machine Intelligence,
31(2):210–227, Feb. 2009.
[23] W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld.
Face recognition: A literature survey. ACM Comput. Surv.,
35(4):399–458, 2003.
