Content uploaded by Paulo Lobato Correia
Author content
All content in this area was uploaded by Paulo Lobato Correia on Jan 13, 2015
Content may be subject to copyright.
A SINGLE SENSOR HAND BIOMETRIC MULTIMODAL SYSTEM
Tiago Sanches, João Antunes, Paulo Lobato Correia
Instituto de Telecomunicações, Instituto Superior Técnico, UTL, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal
e-mail: {tmss, jmfa}@mega.ist.utl.pt, plc@lx.it.pt
ABSTRACT
Nowadays the question of identifying a person assumes a
major role in many applications. To circumvent the limita-
tions of traditional identity recognition mechanisms (e.g.,
passwords or ID cards), modern security control procedures
often exploit people biometrics.
This paper proposes a multimodal biometric system for
personal recognition, based on three different biometrics
computed from the same hand image. Features extracted
from each of the five finger surface areas are fused at score
level into a single biometric mode. Hand geometry, palmprint
and finger surface biometric features are finally fused at de-
cision level to come to a recognition decision.
The achieved recognition results of FAR=0.31%,
FRR=0.80% and a maximum recognition rate of 98.28%
indicate that this work should be continued and might be
considered for high security applications.
1. INTRODUCTION
Reliable and secure access control systems are often re-
quired for many applications, ranging from border control
security checks to the access to restricted areas, or even to
control the presence of employees at the workplace, among
others. The need for improved security systems has been
accompanied by a growing research interest in biometric
technologies. Biometric recognition systems target the
automatic recognition of a person’s identity based on physi-
cal, physiological or behavioural characteristics (something
a person is or produces).
A major advantage of biometric features is that they
cannot be easily stolen or lost, and typically are unique for
each person. Fingerprints are among the most used biomet-
ric features, but many others have been considered, such as
face, hand geometry, palmprints, iris, voice, signature, or
gait, among others. Recent systems often combine multiple
biometrics to increase recognition accuracy and reliability.
Biometric systems need to capture an individual’s
unique biometric features, which are converted into a digital
format, called template. This template is then enrolled into a
database or some other secure storage location (e.g. a smart
card) and later used for comparison with new samples, to
determine whether there is a match for recognition purposes.
Biometric systems’ performance is usually measured
by the type and frequency of errors, namely: acceptance of
impostors as true users – false acceptance rate (FAR) – and
rejection of legitimate users – false rejection rate (FRR).
Also an equal error rate (EER) is often considered, cor-
responding to the operation point for which the FRR and
FAR have equal values. Another relevant measure is the
failure to enroll (FTE), indicating the portion of the popula-
tion for whom the system fails to complete the enrolment
process, according to the conditions specified by the pre-
processing block.
Several types of biometric features can be extracted
from hand images: (i) hand geometry features, such as hand
shape, palm area, width and length of fingers and other
measurements; (ii) palmprint characteristics, like principal
lines, wrinkles, feature points, and skin texture; (iii) finger-
print or finger-strip features, composed of the ridges, furrows
and texture on the surface of the finger.
In this paper the biometric features to be exploited for
recognition are the hand geometry, the texture of the palm-
print and the texture of finger surfaces – see Figure 1. A spe-
cial focus is put on the surface of fingers, as this feature has
only recently started being investigated as biometric for rec-
ognition purposes [1][2][3].
Figure 1 – Hand features to be used as biometric identifiers.
Individual hand features, like finger width and length or palm
area are usually considered for hand geometry [4][5]. Other
times shape-based hand recognition algorithms [6] are con-
sidered. In this paper a selection of finger lengths, widths,
perimeters and palm based measurements are used.
For palmprint biometrics, several techniques have been
actively researched in the past, like: algebraic approaches
analysing statistical data [1][7] examination of the palm line
features [7]; texture-based approaches [7]. This paper uses an
algebraic approach to extract palm features.
For finger surface analysis two main approaches have
been considered in the literature: one analyzing the texture of
the inner surface [1][2][3], the other looking at the curvature
©2007 EURASIP 30
15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3-7, 2007, copyright by EURASIP
of knuckle surface of the fingers [8]. This paper takes the
first approach.
In the remainder of this paper, the three proposed bio-
metrics and the way to combine their partial results (fusion)
are described in Section 2, recognition results are presented
and discussed in Section 3, and conclusions are drawn in
Section 4.
2. SYSTEM DESCRIPTION
The architecture of the proposed multimodal biometric rec-
ognition system is shown in Figure 2. The system is used for
both user enrolment and recognition purposes. Enrolment
consists in the acquisition of a set of hand images from each
user. These images are pre-processed and a feature template
is generated for each biometric modality. The templates are
then stored in the template database.
At recognition time, a hand image is sensed, pre-
processed and templates for each of the three biometrics are
generated. The acquired templates are tested by the corre-
sponding matching modules, being compared with those
stored in the database. The final step is the fusion block,
which combines the information obtained from the three
different modalities to produce a recognition decision.
Figure 2 – Proposed system architecture.
No sophisticated hardware is needed for image acquisition of
hand images, either for enrolment or recognition purposes. A
medium resolution digital camera, a tripod for image stability
and a well defined environment (i.e., image background), or
in alternative a digital scanner, can be used. A computer is
then needed to run the recognition algorithms.
2.1 Pre-processing
To simplify the segmentation of hand images a constant
background that contrasts with skin colour is selected.
After hand image capture, it is pre-processed to seg-
ment the hand region, leading to a black and white silhouette
used as a mask in subsequent processing steps.
The hand binary mask is used to detect a set of relevant
hand points that will serve as reference points for the three
biometric modalities analysed in this paper. Notably, the fin-
gertips and finger-webs, illustrated in Figure 3, are taken as
hand reference points. To find the hand feature points loca-
tions, a combination of two commonly used techniques is
employed: radial distance to a reference point and contour
curvegram [6]. Since the first is sensitive to rotation and the
second produces a noisy data plot, the combination of both
techniques allows a more robust reference point localization.
Figure 3 – Fingertip and finger-web locations on a hand image.
2.2 Hand Geometry
For recognition based on the hand geometry biometric, a
subset of the features discussed in the literature are used
[4][5]: five finger lengths, twenty finger widths (four for each
finger), five palm based measurements, and five finger pe-
rimeters – see Figure 4.
Figure 4 – Hand geometry template features.
Those 35 features are then statistically analysed for
discriminability, to select only the best performing ones, in
terms of the ratio between interclass and intraclass variabil-
ity of each feature. The most discriminant features present
the highest ratio values.
Interclass variability evaluates how much a specific fea-
ture varies between different users’ hands, based on the stan-
dard deviation. This value is desirably high, indicating that
the specific feature is different for most users.
Intraclass variability evaluates the variation of a specific
feature regarding each user’s set of hand images. A good
feature should not vary much for different images of the
same hand, meaning the feature will always be extracted with
a similar value.
After a statistical analysis of the test database, the 25
features with highest ratio (i.e., the most discriminant) are
selected as the default hand geometry feature set for usage in
©2007 EURASIP 31
15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3-7, 2007, copyright by EURASIP
the multimodal biometric system being developed.
As the values of the selected measurements have differ-
ent value ranges, the comparison of two different feature
measurements will also assume significant differences: for
instance a finger perimeter is significantly greater than a fin-
ger width. As a consequence, the feature values need to be
normalized, in order to guarantee that fair distance measure-
ments are used in the subsequent matching phase. The final
set of biometric feature measurements is arranged into a fea-
ture vector.
2.3 Palmprint
For palmprint analysis, a region-of-interest (ROI) of the hand
is first extracted. The ROI for palmprint recognition purposes
is usually a square region in the central part of the palm.
To obtain the palm ROI, the previously identified hand
feature points are used as reference. The middle points of the
line segments that define the beginning of the index and
pinky fingers are used as vertices of a square region of the
palm [7], from where features will be extracted.
Since for different hand images the ROIs will be of di-
verse sizes and orientations, normalization is required. The
ROI image is converted to grayscale and resized to a fixed
size using bicubic interpolation, so that features can be accu-
rately extracted and compared with other samples.
Due to performance considerations, regarding the proc-
essing speed of the palmprint recognition algorithms used,
the ROI is resized to 16x16 pixels. This size, smaller than the
ones usually considered in the literature, that range from
64x64 [1] to 300x300 [5], nevertheless allows achieving a
reasonably good recognition performance, which is a useful
input to the multimodal recognition system being proposed,
via the fusion with the other extracted biometrics.
As a final step, the ROI image is converted into a tem-
plate vector consisting of luminance values. This template
vector is then linearly transformed into a more discriminating
feature vector by means of statistical analysis algorithms.
The entire process described above is illustrated in Figure 5.
Figure 5 – Palmprint’s ROI processing procedure.
An optimal technique, in view of class separability pur-
poses, is Linear Discriminant Analysis [9], which is used in
this project for both palmprint and finger surface analysis.
2.4 Finger Surface
To analyze the finger surfaces, a region of interest (ROI) for
each finger needs to be extracted. This is done by finding the
largest rectangle area lying inside the contour of the finger in
a region bounded at about 1/8 and 7/8 of the finger length.
An example of the final set of finger surface ROIs, formed
by rectangular areas for the thumb, index, middle, ring and
pinky finger, is shown in Figure 6.
Figure 6 – Extracted ROIs for the five fingers.
The image of each finger’s ROI is converted to gray scale.
Then, its size is normalized by resizing the ROI to a standard
size, again using bicubic interpolation. To guarantee a fast
processing while maintaining the recognition ability, the ROI
is resized to 32x8 pixels. This size is smaller than those typi-
cally used in the available literature, which ranges from
64x16 [1][3] to 128x32 [2]. In spite of using of a smaller ROI
size, the recognition rates of the proposed algorithm,
achieved from the fusion of the five fingers’ results, are good.
Finally, the ROI image is vectorized into a template con-
sisting of luminance values. While in [1][2][3] the Principal
Component Analysis (PCA) algorithm is used to extract fea-
tures from this type of template, this paper proposes the us-
age of the Linear Discriminant Analysis (LDA) algorithm
[9], due to its higher discriminability characteristics.
2.5 Fusion
A multimodal biometric system requires an integration of
the various individual biometrics, to allow making a deci-
sion on the user’s identity. This is the step of biometric data
fusion. Recently the interest in multimodal biometric sys-
tems has increased, with results showing this is a worthwhile
investment and promising research area. The fusion methods
adopted in the literature include weighted combination of
scores, support vector machines, decision templates, and
behaviour knowledge space methods [10].
Two different levels of fusion are applied in this paper:
score level fusion is used for the five finger surface features,
by computing their mean score; and decision level fusion is
applied for data fusion of the various modalities, based on the
majority vote rule. For three modalities, as is the case, a
minimum of two accept votes is needed for a final accep-
tance decision.
3. EXPERIMENTAL RESULTS
The test of the proposed biometric recognition system con-
sists in the evaluation of the matching modules and the fu-
sion block represented in Figure 2.
The matching algorithms generate a score for each tem-
plate comparison based on the distance between the tested
and stored feature vectors. The Euclidean distance metric is
used, as it achieves good results at a low computation cost
[4]. The lowest distance score value indicates the best match.
A flag, set by the pre-processing stage, indicating if the
template belongs to a right of left hand is used to eliminate
unnecessary template matching comparisons. Database tem-
©2007 EURASIP 32
15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3-7, 2007, copyright by EURASIP
plates belonging to users that enrolled using a different hand
than that of the query template are not considered for com-
parison. The matching procedure is illustrated in Figure 7.
Figure 7 – The matching procedure.
Whenever the best matching score exceeds a prede-
fined threshold the recognition attempt is considered as an
impostor access, otherwise the recognition attempt is con-
sidered a client access and the system assumes the user has
been correctly identified. When several database templates
scores are below the threshold, the one with lowest score
should correspond to the correct user identity.
Different thresholds can be chosen in order to achieve
the desired FAR or FRR levels of operation, depending on
the application considered for the biometric system. For
instance, high-security applications require a FAR close or
equal to zero.
The results presented in the following were obtained
considering the UST Hand Image Database [11].
The test database enrolment produced a FTE value of
8.2%. Most of the failed registrations, approximately 95%,
are due to poor image acquisitions: the hand crosses two
image borders, e.g. a finger is not completely captured by
the sensing device. This type of error should be corrected at
the image capture stage, by requiring a correct placement of
the hand, always within the camera view.
The results for the finger surface biometric recogni-
tion, after the fusion of individual finger features, are illus-
trated in Figure 8. This biometric generates a good separa-
tion of clients and impostors in the score distribution, as can
also be seen in Figure 9. Also, only one test image of the
564 users1 of the UST database is scored outside the top ten
matching scores.
Figure 8 – Finger surface performance measures.
1 Left and right hands of the same person are considered as different users.
Figure 9 – Finger surface client/impostor score distribution.
The score level fusion of the individual finger surface
scores into a single biometric greatly improves recognition
rates and the EER, compared to the usage of individual finger
results, as shown in Figure 10.
Figure 10 – Finger surface recognition performance rates.
Using threshold values that maximize the correct rec-
ognition rates for each individual biometric, after fusion a
FAR of 0.31% was obtained, as illustrated in Table 1.
Table 1 – Results for thresholds equivalent to maximum correct
authentications.
Hand
Geometry Palmprint Finger
Surface
Multimodal
Fusion
Recognition
Rate 91.65% 86.19% 97.25% 96.80%
FAR 3.55% 4.12% 0.46% 0.31%
FRR 4.80% 9.69% 2.29% 2.90%
As the table shows, by applying decision level fusion,
the majority vote method leads to a reduced overall FAR.
By adequately adjusting the thresholds of each biometric
mode to achieve reduced individual FRR values, the overall
FRR is also reduced, while the recognition rate is increased
when compared to each individual biometric modality. It is
for instance possible to set these thresholds to achieve a
correct recognition rate (after fusion) of 98.28%, with a
FAR of 0.92% and a FRR of 0.80%, as illustrated in Figure
11.
Figure 11 – Error rates for thresholds equivalent to a low FRR.
©2007 EURASIP 33
15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3-7, 2007, copyright by EURASIP
The GUI of the developed biometric recognition appli-
cation, which when run against an entire database also pro-
vides the statistical data and graphs needed to assess the
system’s performance, is illustrated in Figure 12.
Figure 12 – The GUI of the biometric recognition application.
4. CONCLUSIONS
This paper proposes a multimodal biometric recognition sys-
tem that exploits several modalities present in hand images.
Image acquisition is based on a simple setup, using fairly
inexpensive equipment. From a single acquired image, sev-
eral biometric features are computed: hand geometry, palm-
print, and finger surface. Different sensors for each biometric
mode are not required, nor does it need specific hand place-
ment as in pegged image acquisition devices. These charac-
teristics make the system practical and easy to use.
The proposed multimodal biometric system has shown
that the usage of multiple biometrics improves performance
in comparison to systems using a single biometric. The com-
bined results are better than the best of the individual biomet-
ric recognition results.
Compared to the literature, the proposed system is able
to achieve a performance similar to the other hand recogni-
tion multimodal systems [1][3][5]. In reference [1], a maxi-
mum recognition rate of 99.28% and an EER of 0.58% were
achieved by fusion of palmprint and finger surface features.
Another multimodal system [5], using bimodal fusion of
palmprint and hand geometry features, was able to achieve a
maximum recognition rate of 98.59% and a 0% FAR. Using
finger surface and hand geometry fusion [3], performance
results with a maximum recognition rate of 97.97% and an
EER of 1.71% were also reported.
From the individual biometrics considered, hand geome-
try and finger surface biometrics achieved the expected per-
formance values – similar to the results described in [3][5].
The palmprint modality did not obtain the performance
shown in other work [5], mainly due to the small size of the
normalized ROIs considered here, which was nevertheless
considered sufficient for integration in the multimodal recog-
nition platform, while keeping the computational cost, both
for feature extraction and for feature matching, lower than
those of the alternative solutions. The selected option could
make sense for large databases.
Future work will focus on the comparison of different
fusion algorithms, for example considering a weighted score
level fusion for each finger, so that individual finger per-
formance is also taken into consideration. Other matching
classifiers shall be investigated and compared, such as
Hamming or Mahalanobis distance, and Gaussian Mixture
Models. Also the usage of the so-called soft-biometrics, such
as the size of the hand, can be used to speed up the recogni-
tion procedure.
REFERENCES
[1] S. Ribaric and I. Fratric, “A Biometric Identification Sys-
tem Based on Eigenpalm and Eigenfinger Features”, IEEE
Transactions on Pattern Analysis and Machine, Vol. 27, No.
11, pp. 1698-1709, Nov. 2005.
[2] Pavesik, N., Zibert, J., Ribaric, S., Fratric, I. and Krapac,
J., “A Multimodal Authentication Based on Hand Features
Extracted from Multispectral Images”, in BioSecure Re-
search Project Workshop, Vigo, Spain, Sep. 2006.
[3] Ribaric, S. and Fratric, I., “An Online Biometric Au-
thentication System Based on Eigenfingers and Finger-
Geometry”, in 13th European Signal Processing Conference,
Antalya, Turkey, Sep. 2005.
[4] Sanchez-Reillo, R., Sanchez-Avila, C. and Gonzalez-
Marcos, A., “Biometric Identification Through Hand Geome-
try Measurements”, IEEE Transactions on Pattern Analysis
and Machine Intelligence, Vol. 22, No. 10, pp. 1168-1171,
Oct. 2000.
[5] Kumar, A., Wong, D., Shen, H. and Jain, A., “Personal
Verification Using Palmprint and Hand Geometry Biomet-
ric”, in Proc. 4th International Conference on Audio- and
Video-based Biometric Person Authentication, Guildford, U.
K., June 2003, pp. 668-678.
[6] Konukoğlu, E., Yörük, E., Darbon, J. and Sankur, B.,
“Shape-Based Hand Recognition”, IEEE Transactions on
Image Processing, Vol. 15, No. 7, pp. 1803- 1815, July 2006.
[7] Kumar, A. and Zhang, D., “Personal Authentication using
Multiple Palmprint Representation”, Pattern Recognition,
Vol. 38, No. 10, pp. 1695-1704, Oct. 2005.
[8] Woodward, D. and Flynn, P., “Personal Identification
Utilizing Finger Surface Features”, in Proc. 2005 IEEE
Computer Society Conference on Computer Vision and Pat-
tern Recognition, San Diego, USA, June 2006, Vol. 2, pp.
1030-1036.
[9] Beuchat, A., Sorzano, C., Sage, D., Senn, A., Urner, F.
and Unser, M., “Multivariate Statistical Analysis and Classi-
fication of Zygotes”, in Proc. 2005 Annual Meeting of the
Swiss Society for Biomedical Engineering, Lausanne VD,
Switzerland, Sep. 2005.
[10] Ross, A. and Jain, A., “Information Fusion in Biomet-
rics”, Pattern Recognition Letters, Vol. 24, No. 3, pp. 2215-
2125, Sep. 2003.
[11] UST Hand Image Database, Department of Computer
Science, The Hong Kong University of Science and Technol-
ogy. (Kindly provided by Dr. Helen Shen).
©2007 EURASIP 34
15th European Signal Processing Conference (EUSIPCO 2007), Poznan, Poland, September 3-7, 2007, copyright by EURASIP
