Likelihood ratio based features for a trained biometric score fusion

Abstract

In this work, we present a novel trained method for combining biometric matchers at the score level. The new method is based on a combination of machine learning classifiers trained using the match scores from different biometric approaches as features. The parameters of a finite Gaussian mixture model are used for modelling the genuine and impostor score densities during the fusion step.Several tests on different biometric verification systems (related to fingerprints, palms, fingers, hand geometry and faces) show that the new method outperforms other trained and non-trained approaches for combining biometric matchers.We have tested some different classifiers, support vector machines, AdaBoost of neural networks, and their random subspace versions, demonstrating that the choice for the proposed method is the Random Subspace of AdaBoost.

Full text

1
Likelihood Ratio based features for a trained biometric
score fusion
Loris Nannia 1, Alessandra Luminia, Sheryl Brahnamb
a Department of Electronic, Informatics and Systems (DEIS), Università di Bologna,
Via Venezia 52, 47023 Cesena, Italy.
{loris.nanni, alessandra.lumini}@unibo.it
b Computer Information Systems, Missouri State University,
901 S. National, Springfield, MO 65804, USA
sbrahnam@missouristate.edu
Abstract
In this work, we present a novel trained method for combining biometric matchers at the score
level. The new method is based on a combination of machine learning classifiers trained using the
match scores from different biometric approaches as features. The parameters of a finite Gaussian
mixture model are used for modelling the genuine and impostor score densities during the fusion
step.
Several tests on different biometric verification systems (related to fingerprints, palms,
fingers, hand geometry and faces) show that the new method outperforms other trained and non-
trained approaches for combining biometric matchers.
We have tested some different classifiers, Support Vector Machines, AdaBoost of neural
networks, and their random subspace versions, demonstrating that the choice for the proposed
method is the Random Subspace of AdaBoost.
Keywords: Likelihood Ratio; Mixture of Gaussians; Support Vector Machine; Biometric score
fusion.
1 corresponding author: Tel.: +39 0547 339121; fax: +39 0547 338890.

2
1. Introduction
One recent focus of interest in biometrics research is the successful combination of different
sources of information resulting in the so-called multi-biometric. Unibiometric systems, which are
based on a single source of information may suffer from limitations such as the lack of uniqueness
and non-universality of the chosen biometric trait, noisy data and spoof attacks [3]. Multibiometric
systems fuse information from multiple biometric sources in order to achieve better recognition
performance and to overcome other limitations of unibiometric systems [4][5][6]. A sound
theoretical framework for combining classifiers with application to biometric verification is
described in [21], where an algorithm functioning as a supervisor in a multi expert decision making
machine is proposed which uses the Bayes theory in order to estimate the biases of individual expert
opinions (the scores of each unibiometric system). Machine learning approaches have also been
applied for combining biometric classifiers [23].
A first study on the combination of different fingerprint systems submitted to FVC2004 is
carried out in [24][26] where the benefits and limits of the resulting multiple classifier approaches
have been analysed. In these works it is shown that combining systems that are based on
heterogeneous matching strategies permits a reduction of the Equal Error Rate with respect to the
best unibiometric system. In [25] a very effective multi-biometric system based of the combination
of different fingerprint systems and an Iris matcher is proposed.
In [32], starting from the similarity scores obtained by two biometric matchers (Face and Iris),
a set of 8 “original” features are extracted to discriminate between genuine and impostor classes.
Moreover several new “artificial” features are generated by combining one or more original ones,
by means of some mathematical operators. The resulting system, based on the original and a
selection of the artificial features has experimentally demonstrated to give a very good verification
performance.

3
In multi-biometric systems fusion performed at the score level is generally preferred [5] to
fusion at the feature and decision levels; the score fusion techniques proposed in the literature can
be divided into three categories (following the taxonomy used in [20]):
 Transformation-based score fusion: The match scores are first normalized
(transformed) to a common domain and then combined. The main drawback is that
these methods are data-dependent and require extensive empirical evaluation [6][8][9].
 Classifier-based score fusion: Scores from multiple matchers are used to train a
classifier that discriminates between genuine and impostor [4][10][11] features.
 Density-based score fusion: This approach is based on the likelihood ratio test, and it
requires the densities estimation of genuine and impostor match scores [12]. A
comparison of eight biometric fusion techniques conducted by NIST [13] with data
from 187.000 subjects concluded that Likelihood Ratio was the most accurate method,
but it was complex to implement (their density estimation was based on the use of
kernel density estimator (KDE) [14]).
In [20] it is shown that a mixture of Gaussians (MoG) is quite effective in modelling the
genuine and impostor score densities, and it is easier to implement than KDE. Their results based on
NIST fusion data [7] show that MoG outperforms both the standard Sum Rule [19] and the Support
Vector machine [30] based trained fusion.
In this work, we propose a supervised fusion where the classifiers are trained using as features
the match scores and the parameters of the finite Gaussian mixture model that are used for
modelling the genuine and impostor score densities of the training data.
Experimental results are reported for two different state-of-the-art classifiers: the Support
Vector Machine (SVM) and the AdaBoost of neural network (ADA), and for each classifier their
random subspace version has also been tested.

4
Several tests using different biometric characteristics (fingerprints, the palm, fingers, hand
geometry, and the face) show that our method (mainly the one based on the Random Subspace of
ADA) outperforms other trained and non-trained approaches for combining biometric matchers.
This paper is organized as follows. In section 2 the details of the new feature extraction
approach is presented. In section 3 some experimental results are presented and discussed. Finally,
we draw conclusions in section 4.
2. System Overview
According to the Neyman-Pearson theorem [14], the optimal test for assigning a score vector
x to the class genuine or impostor is the likelihood ratio test given by fgen(x)/fimp(x), where fgen(x)
and fimp(x) are the densities of the genuine training data and of the impostor training data.
It is well known that the Gaussian density is not appropriate for modelling biometric match
scores; to obtain a more reliable density method, the normal distribution can be extended to a
mixture of Gaussians (MoG)2 [18] (i.e., the linear combination of normal distributions). The main
drawback of MoG is that it requires far more data for training [16][17]. In this paper, the mixture is
estimates using the EM algorithm [29] and a number of Gaussians. K is automatically calculated by
means of the minimum message length criterion.
The estimates of fgen(x) and fimp(x) are obtained as a mixture of Gaussians; the probability
distribution for a d-dimensional object x is given by:
T1
1/2
d/2
11
f( ) exp ( ) ( )
2
(2 )



    



xxμ x μ
,
where μ is the mean and  is the covariance matrix of the training set. The estimates of fgen(x) is:
fgen(x)=∑i pgen,if(x,μi,i),
2 The MATLAB code for this algorithm is available at http://www.lx.it.pt/mtf/mixturecode.zip
[bestk,bestpp,bestmu,bestcov,dl,countf] = mixtures4(DATA,1,15,1e-5,1e-4,0);

5
where pgen,i is the weight assigned to the ith mixture component (in a similar way we estimate
fimp(x)).
Given a set of K Gaussians for modelling the genuine training data and other K Gaussians for
modelling the impostor training data we extract a set of K2 feature for describing each pattern.
Each feature is given by pgen,if(x,μi,i), i=1,...K then for each pattern and for each component of
the mixture of Gaussians that model the impostor training data another set of features is created.
In Figure 1 our system is detailed.
Figure 1. Biometric fusion system proposed in this work (in the case of the fusion of two
matchers).
3. Experiments and discussion
As classifier we have tested the Support Vector Machine (SVM) and the AdaBoost of neural
networks (ADA). Moreover, for each classifier, we have tested also their random subspace (RS)
version [31]. We use an AdaBoost.M1 with 50 iterations of a feed-forward back-propagation
Mixture of Gaussians
Scores (s1) of the 1-st
Biometric Matcher
Scores (s2) of the 2-nd
Biometric Matcher
pimp,2f(x,μ2,2),
pgen,1f(x,μ1,1),
s1,s2,
classifier

6
network. As SVM we report results on a Linear SVM and a radial basis function SVM. The
Random Subspace Method modifies the training data set (generating NK new training sets
containing only NFe of the original features; in this paper NK=25 and NFe=50%). It builds
classifiers on these modified training sets, and then combines them into a final decision rule (in this
paper the Sum Rule is used [19]).
Experiments have been conducted on several datasets:
 The four fingerprint databases from FVC 2004 DBs [26], each containing 800 images
from 100 individuals (DB1-DB3 are obtained using different sensors, while DB4 is
obtained using an artificial generator [22]);
 A Palm database that contains 1000 inkless right-hand images from a digital Camera, 7
samples from each user, for 100 users. From this dataset several biometric
characteristics are extracted (Palm, Hand Geometry, Middle Finger, and Ring Finger).
The palm is extracted using a method similar to that proposed in [27]. The images of
the Palm and of the Finger have been resized to the same dimension of 100100
before processing.
 A Face database, the Notre-Dame Dataset3 collection D [28], that contains a total of
275 different persons who participated in one or more sessions. Two four-week
sessions were conducted for data collection with approximately a six weeks time lapse
between the two.
In Figure 2 we show some samples from the datasets. According to the very difficult
FVC2002 testing protocol, the following matching attempts are calculated:
3 http://www.nd.edu/~cvrl/

7
 Genuine recognition attempts: The template of each impression is matched against the
remaining impressions of the same individual, while avoiding symmetric matches;
 Impostor recognition attempts: The template of the first impression is matched against the
first impressions of the remaining individuals while avoiding symmetric matches.
The performance have been measured by means of the Equal Error Rate (EER) [1]. Moreover,
in order to confirm the benefit of the our method, the DET curve has been also considered. The
DET curve [2] is a two-dimensional measure of classification performance that plots the probability
of false acceptation against the rate of false rejection.
Figure 2. Some samples from the dataset used in this work, (a) fingerprint; (b) face; (c) palm; (d)
Hand Geometry features (the length of the green lines that link two green balls are the extracted
features); (e) finger.
(a)
(b)
(c)
(d)
(e)

8
Now we report the matchers involved in the fusions tested in this paper:
 In the FVC2004 DBs, we use the winner of the competition and the third best matcher (the
second best matcher has as scores mainly the values 0 or 1, and hence it is not well suited
for the fusion);
 The Palm matcher, the Finger matcher and the first Face matcher is the Euclidean distance
on the 100 Discrete Cosine Coefficients with higher variance. The pre-processing stage used
in [27] is performed to normalize the images in order to smoothen the noise and lighting
effect;
 The second Face matcher is the Euclidean distance on the Locally Binary Patterns features
(the histograms of 10 bins, 18 bins and 26 bins are concatenated as in [33]);
 The Hand Geometry matcher is the Euclidean distance where the features are the length of
the lines that link two datum points (see Figure 2).
For each fusion (see Tables 2-3 and Figures 3-4), we report which matchers are involved:
 FVC2004 DB1-4: the two FVC2004 matchers are combined;
 PALMFINGER: the Palm matcher and the middle finger matchers are combined;
 HAND: the Palm matcher, the middle finger matcher, the ring finger and the hand
geometry matchers are combined;
 FACE: the two Face Matchers are combined.
In Table 1 we report the number of mixture found for the genuine data and for the impostor
data in the seven datasets used in this work. In Figure 2 we show some real examples of MoG.

9
FVC2004
PALMFINGER
HAND
FACE
DB1
DB2
DB3
DB4
GENUINE
8
9
9
6
6
3
7
IMPOSTOR
7
5
2
6
6
5
9
Table 1. Number of mixtures found for the genuine data and for the impostor data.
Figure 3. Real examples of MoG: (a) PALMFINGER-Genuine; (b) PALMFINGER-Impostor; (c)
DB3-Genuine; (d) DB3-Impostor.
(a)
(b)
(c)
(d)

10
In Tables 2-3 we compare several methods on the tested databasess varying the dimension of
the training set. In Table 2 the training set contains 80% of the users. In Table 3 the training set
contains 50% of the users. For the testing set we consider only the matches where the users that
belong to the training set are not present. We randomly divide the users in the training and testing
sets ten times, and we report the average EER.
We compare the following state-of-the-art methods:
 ADA, the trained fusion where ADA is trained considering only the match scores;
 SVM, the trained fusion where the Linear SVM is trained considering only the match scores;
 LR, the method proposed in [20];
 ADA-LR, ADA trained using the features described in Section 2;
 SVM-LR, SVM trained using the features described in Section 2;
 RS-ADA, RS of ADA trained using the features described in Section 2;
 RS-SVM, RS of Radial Basis Function SVM (Gamma=1, Cost of the constrain violation=100)
trained using the features described in Section 2.

11
Table 2. EER obtained when the training set contains 80% of the users.
We want to stress that in Table 2 the best performance is always obtained using the classifiers
trained using the features described in Section 2. Among the seven tests, the best results are
obtained by RS-ADA; it always outperforms the standard methods. Moreover, only in the FACE
fusion test does LR works better than SVM. In our opinion this is due to the fact that LR needs a
very large dataset (as the datasets used in [20]) for training.
Finally, when SVM is trained considering only the match scores, we obtain the best
performance using the Linear SVM. When we use the features proposed in this paper, we use the
Radial Basis Function SVM. Notice that we report the performance obtained by Radial Basis
Function SVM with the same parameters in all the seven fusion tests.
Method
FVC2004
PALMFINGER
HAND
FACE
DB1
DB2
DB3
DB4
1-ST MATCHER
1.89
2.74
0.58
0.56
8.9
8.9
14.39
2-nd MATCHER
3.91
2.3
1.38
0.58
9.4
9.4
24.25
3- th MATCHER
-
-
-
-
-
11.3
-
4-th MATCHER
-
-
-
-
-
13.2
-
SUM RULE
1.75
1.2
0.7
0.46
7.4
7.6
13.99
ADA
1.66
1.11
0.73
0.35
25.3
7.5
14.17
SVM
1.64
1.16
0.53
0.46
7.1
7.4
14.12
LR
3.62
2.69
1.21
0.73
10.5
19
11.50
ADA-LR
2.08
0.88
0.53
0.54
6.2
7.3
12.71
SVM-LR
1.64
0.93
0.44
0.39
5.4
5.7
15.78
RS-ADA
1.61
1.09
0.58
0.33
5.4
6.6
11.43
RS-SVM
1.43
1
0.58
0.42
5.3
5.2
16.43

12
In Figure 4 the DET-Curve of a single run of RS-ADA (green line), SVM (black line) and
SUM (red line) are reported.
Figure 4. DET-curves: (a) FVC2004-DB2; (b) PALMFINGER; (c) HAND; (d) FACE.
(a)
(b)
(c)
(d)

13
Table 3. EER obtained when the training set contains the 50% of the users.
In Table 3, where we use a reduced training set, our methods obtain the best performance. In
Figure 5 the DET-curves obtained using the reduced training set are reported.
Method
FVC2004
PALMFINGER
HAND
FACE
DB1
DB2
DB3
DB4
1-ST MATCHER
2.60
3.35
1.04
0.79
8.24
8.24
14.30
2-nd MATCHER
4.47
2.53
1.56
0.55
10.62
10.62
25.93
3-TH MATCHER
-
-
-
-
-
7.09
-
4-th MATCHER
-
-
-
-
-
9.86
-
SUM RULE
2.23
1.49
0.83
0.50
6.53
4.99
13.74
ADA
2.11
1.41
0.61
0.47
26.53
5.44
16.20
SVM
2.15
1.56
0.69
0.52
6.57
4.79
13.53
LR
3.72
2.95
1.26
1.11
12.42
13.39
11.45
ADA-LR
2.30
1.31
0.66
0.50
7.00
4.71
13.35
SVM-LR
2.20
0.99
0.63
0.49
5.42
4.16
12.23
RS-ADA
2.11
1.01
0.54
0.45
5.73
3.98
11.70
RS-SVM
2.11
0.99
0.58
0.43
5.16
3.86
11.80

14
Figure 5. DET-curves: (a) FVC2004-DB2; (b) PALMFINGER; (c) HAND; (d) FACE.
4. Conclusions
In this work we have presented a feature extraction approach for the fusion of match scores in
a multibiometric system based on the likelihood ratio test. We show that densities estimated using a
mixture of Gaussian models can be used to train a machine learning classifier.
(d)
(a)
(b)
(c)

15
Based on these experiments, our conclusions are the following:
 The likelihood ratio based feature coupled with a Random Subspace of AdaBoost of
neural networks achieves a low Equal Error Rate in several tests without parameter
tuning for each dataset;
 Both SVM and LR work well in some datasets and and not so well in other datasets;
however, our best proposed method works well in all the tested datasets.
As future work we want to study whether the incorporation of the Biometric sample quality
information (as in [20]) within the likelihood ratio based fusion framework, improves performance
in the proposed systems.
References
[1] D. Maio, D. Maltoni, A.K. Jain, S. Prabhakar, Handbook of Fingerprint Recognition, Springer,
New York, 2003.
[2] Martin, A., et al.: The DET curve in assessment of decision task performance. In: Proc. of
EuroSpeech. (1997) 1895–1898.
[3] A. Ross, K. Nandakumar, and A. K. Jain, Handbook of Multibiometrics. Springer-Verlag, 2006.
[4] R. Brunelli and D. Falavigna, “Person Identification Using Multiple Cues,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, vol. 17, no. 10, pp. 955–966, October 1995.
[5] S. Prabhakar and A. K. Jain, “Decision-level Fusion in Fingerprint Verification,” Pattern
Recognition, vol. 35, no. 4, pp. 861–874, April 2002.
[6] K.-A. Toh, X. Jiang, and W.-Y. Yau, “Exploiting Global and Local Decisions for Multimodal
Biometrics Verification,” IEEE Transactions on Signal Processing, (Supplement on Secure
Media), vol. 52, no. 10, pp. 3059–3072, October 2004.

16
[7] National Institute of Standards and Technology, “NIST Biometric Scores Set - release 1,” 2004,
Available at http://www.itl.nist.gov/iad/894.03/biometricscores.
[8] R. Snelick, U. Uludag, A. Mink, M. Indovina, and A. K. Jain, “Large Scale Evaluation of
Multimodal Biometric Authentication Using State-of-the-Art Systems,” IEEE Transactions on
PAMI, vol. 27, no. 3, pp. 450–455, March 2005.
[9] A. K. Jain, K. Nandakumar, and A. Ross, “Score Normalization in Multimodal Biometric
Systems,” Pattern Recognition, vol. 38, no. 12, pp. 2270–2285, December 2005.
[10] J. Fierrez-Aguilar, J. Ortega-Garcia, J. Gonzalez-Rodriguez, and J. Bigun, “Discriminative
Multimodal Biometric Authentication based on Quality Measures,” Pattern Recognition, vol.
38, no. 5, pp. 777–779, May 2005.
[11] Y. Ma, B. Cukic, and H. Singh, “A Classification Approach to Multi-biometric Score
Fusion,” in Proceedings of Fifth International Conference on AVBPA, Rye Brook, USA, July
2005, pp. 484–493.
[12] P. Griffin, “Optimal Biometric Fusion for Identity Verification,” Identix Research, Tech.
Rep. RDNJ-03-0064, 2004.
[13] B. Ulery, A. R. Hicklin, C. Watson, W. Fellner, and P. Hallinan, “Studies of Biometric
Fusion,” NIST, Tech. Rep. IR 7346, September 2006.
[14] E. L. Lehmann and J. P. Romano, Testing Statistical Hypotheses. Springer, 2005.
[15] B. W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman & Hall,
1986.
[16] J. Q. Li and A. Barron, “Mixture Density Estimation,” in Advances in Neural Information
Processings Systems 12, S. A. Solla, T. K. Leen, and K.-R. Muller, Eds. San Mateo, USA:
Morgan Kaufmann Publishers, 1999.
[17] A. Rakhlin, D. Panchenko, and S. Mukherjee, “Risk Bounds for Mixture Density
Estimation,” ESAIM: Probability and Statistics, vol. 9, pp. 220–229, June 2005.

17
[18] M. Figueiredo and A. K. Jain, “Unsupervised Learning of Finite Mixture Models,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 3, pp. 381–396, March
2002.
[19] J. Kittler, M. Hatef, R. P. Duin, and J. G. Matas, “On Combining Classifiers,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 226–239, March
1998.
[20] K. Nandakumar, Y. Chen, S.C. Dass, A.K. Jain, Likelihood Ratio Based Biometric Score
Fusion, to appear IEEE Transactions on Pattern Analysis and Machine Intelligence 2007.
[21] Bigun E.S., Bigun J., Duc B., Fischer S. (1997), “Expert conciliation for multi modal person
authentication systems by bayesian statistics”. Proc. of AVBPA, pp. 291-300.
[22] Cappelli R. (2004), “SFinGe: an Approach to Synthetic Fingerprint Generation”,
proceedings International Workshop on Biometric Technologies (BT2004), pp.147-154.
[23] Fierrez-Aguilar J., Garcia-Romero D., Ortega-Garcia J., Gonzalez-Rodriguez J. (2004),
“Exploiting general knowledge in user-dependent fusion strategies for multimodal biometric
verification”. In: Proc. of ICASSP, pp. 617-620.
[24] Fierrez-Aguilar J., Nanni L., Ortega-Garcia J., Cappelli R., Maltoni D., Combining Multiple
Matchers for Fingerprint Verification: A Case Study in FVC2004, in proceedings 13th
International Conference on Image Analysis and Processing (ICIAP2005), Cagliari, September
2005.
[25] A. Lumini and L. Nanni, "When Fingerprints Are Combined with Iris - A Case Study:
FVC2004 and CASIA", International Journal of Network Security, vol.3, no.2, pp.317-324,
February 2006.
[26] D. Maio and L. Nanni. "Combination of different fingerprint systems: a case study
FVC2004", Sensor Review, Vol. 26, No. 1. (2006), pp. 51-57.
[27] T. Connie, A.T.B. Jin, M.G.K. Ong, D.N.C. Ling, An automated palmprint recognition
system Image and Vision Computing 23 (2005) 501–515.

18
[28] http://www.nd.edu/~cvrl/
[29] L. Nanni, "Experimental comparison of one-class classifiers for on-line signature
verification, Neurocomputing, 69 (7-9) March 2006, 869-873.
[30] R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, Wiley, New York, 2nd edition,
2000.
[31] T.K. Ho, The Random Subspace Method for Constructing Decision Forests, IEEE
Transactions on Pattern Analysis and Machine Intelligence, 20 (8) (August-1998) 832-844.
[32] L. Nanni, A. Lumini, Over-complete feature generation and feature selection for biometry,
Expert Systems With Applications, vol.35, no.4, pp.2049-2055, November 2008.
[33] L. Nanni, A. Lumini, "RegionBoost Learning for 2D+3D based Face Recognition", Pattern
Recognition Letters, vol.28, no.15, pp.2063-2070, November 2007.
[34] T. Connie, A.T.B. Jin, M.G.K. Ong, D.N.C. Ling An automated palmprint recognition
system Image and Vision Computing 23 (2005) 501–515.