Fusion of deep-learned and hand-crafted features for cancelable recognition systems

Abstract

The recent years have witnessed a dramatic shift in the way of biometric identification, authentication, and security processes. Among the essential challenges that face these processes are the online verification and authentication. These challenges lie in the complexity of such processes, the necessity of the personal real-time identifiable information, and the methodology to capture temporal information. In this paper, we present an integrated biometric recognition method to jointly recognize face, iris, palm print, fingerprint and ear biometrics. The proposed method is based on the integration of the extracted deep-learned features together with the hand-crafted ones by using a fusion network. Also, we propose a novel convolutional neural network (CNN)-based model for deep feature extraction. In addition, several techniques are exploited to extract the hand-crafted features such as histogram of oriented gradients (HOG), oriented rotated brief (ORB), local binary patterns (LBPs), scale-invariant feature transform (SIFT), and speeded-up robust features (SURF). Furthermore , for dimensional consistency between the combined features, the dimensions of the hand-crafted features are reduced using independent component analysis (ICA) or principal component analysis (PCA). The core of this paper is the template protection via a cancelable biometric scheme without significantly affecting the recognition performance. Specifically, we have used the bio-convolving approach to enhance the user's privacy and ensure the robustness against spoof attacks. Additionally, various CNN hyper-parameters with their impact on the proposed model performance are studied. Our experiments on various datasets revealed that the proposed method achieves 96.69%, 95.59%, 97.34%, 96.11% and 99.22% recognition accuracies for face, iris, fingerprint, palm print and ear recognition, respectively.

Full text

METHODOLOGIES AND APPLICATION
Fusion of deep-learned and hand-crafted features for cancelable
recognition systems
Essam Abdellatef
1
•Eman M. Omran
2
•Randa F. Soliman
3
•Nabil A. Ismail
4
•Salah Eldin S. E. Abd Elrahman
4
•
Khalid N. Ismail
5,6
•Mohamed Rihan
7
•Fathi E. Abd El-Samie
7,8
•Ayman A. Eisa
2
Springer-Verlag GmbH Germany, part of Springer Nature 2020
Abstract
The recent years have witnessed a dramatic shift in the way of biometric identification, authentication, and security
processes. Among the essential challenges that face these processes are the online verification and authentication. These
challenges lie in the complexity of such processes, the necessity of the personal real-time identifiable information, and the
methodology to capture temporal information. In this paper, we present an integrated biometric recognition method to
jointly recognize face, iris, palm print, fingerprint and ear biometrics. The proposed method is based on the integration of
the extracted deep-learned features together with the hand-crafted ones by using a fusion network. Also, we propose a
novel convolutional neural network (CNN)-based model for deep feature extraction. In addition, several techniques are
exploited to extract the hand-crafted features such as histogram of oriented gradients (HOG), oriented rotated brief (ORB),
local binary patterns (LBPs), scale-invariant feature transform (SIFT), and speeded-up robust features (SURF). Further-
more, for dimensional consistency between the combined features, the dimensions of the hand-crafted features are reduced
using independent component analysis (ICA) or principal component analysis (PCA). The core of this paper is the template
protection via a cancelable biometric scheme without significantly affecting the recognition performance. Specifically, we
have used the bio-convolving approach to enhance the user’s privacy and ensure the robustness against spoof attacks.
Additionally, various CNN hyper-parameters with their impact on the proposed model performance are studied. Our
experiments on various datasets revealed that the proposed method achieves 96.69%, 95.59%, 97.34%, 96.11% and
99.22% recognition accuracies for face, iris, fingerprint, palm print and ear recognition, respectively.
Keywords Deep learning Feature fusion Cancelable biometrics
1 Introduction
Biometrics achieve a promising performance for the per-
sonal verification and authentication in many applications,
which include law enforcement, forensics, immigration,
border, and access control. Unlike passwords or tokens that
Communicated by V. Loia.
&Essam Abdellatef
essam_abdellatef@yahoo.com
Eman M. Omran
omran_eman91@yahoo.com
Randa F. Soliman
randafouad2010@yahoo.com
Nabil A. Ismail
nabil.Ismail@el-eng.menofia.edu.eg
Salah Eldin S. E. Abd Elrahman
salaheldeen@el-eng.menofia.edu.eg
Khalid N. Ismail
khalid.n.ismail@gmail.com
Mohamed Rihan
mohamed.elmelegy@el-eng.menofia.edu.eg
Fathi E. Abd El-Samie
fathi_sayed@yahoo.com
Ayman A. Eisa
Ayman_Eisa@yahoo.com
Extended author information available on the last page of the article
123
Soft Computing
https://doi.org/10.1007/s00500-020-04856-1(0123456789().,-volV)(0123456789().,-volV)

could be forgotten, stolen or lost, the biometric recognition
systems (Jain 2004,2006) do not suffer from such prob-
lems and may achieve a better security performance. The
biometric attributes are categorized into stable and/or be-
havioral characteristics. The stable characteristics contain
face, iris, fingerprint, palm print, and ear shape. The
behavioral traits deal with the behavioral characteristics
like the key stroke pattern and signature.
Recently, trends targeted the automation of biometric
identification, while maintaining the authentication secu-
rity. Since CNNs represent one of the key mechanisms for
resolving computer vision issues, the CNN-based systems
can perform all recognition methods. Therefore, the
robustness and distinctive characteristics introduced by
CNNs have been utilized for biometric recognition, image
classification, and object detection. Face recognition (FR)
is commonly used for individuals’ recognition. It is based
on some spatial metrics like size, shape and face structure
of a person. A human face is considered as one of the most
effective biometric traits compared to other biometrics due
to the low cost, contactless nature, and high acceptability
during acquisition (Pichao et al. 2016; Tomas et al. 2016;
Xiaolin and Yicong 2018; Mariana-Iuliana et al. 2019).
The automation of the individuals’ recognition is based
on their iris traits in a framework called iris recognition
(IR). Moreover, the IR techniques revealed high matching
rates with large datasets. This distinctive success is
attributed to the sophisticated iris stroma texture which
differs upon persons, the permanent perception of iris
special features, and the genetic limitation penetration of
the iris (Daugman 2016). Superior recognition rates of the
applicable techniques for IR have been achieved by the
National Institute of Science and Technology (NIST)
(2012).
Fingerprint recognition is used as a biometric solution
for authentication on computerized systems due to the ease
of fingerprint acquisition. In addition, for everyone, there
are ten sources of biometrics. In fingerprint recognition
systems, a person may be recognized using the features of
minutiae points and ridges (Bolle et al. 2002). Palm print
recognition (Ratha et al. 2007) depends on unique patterns
of various characteristics in palms of people’s hands for the
recognition operation. The palm print and fingerprint
recognition systems include similar details, and hence they
are used together to improve the personal identification
accuracy. Ear recognition (Omara et al. 2016) could be
used for personal identification and authentication due to
universality, distinctiveness, permanence and collectability
of ear patterns. Additionally, the structure of an ear does
not change radically over time.
Nowadays, the protection of biometric data got more
attention. The cancelable biometric recognition techniques
could be used for the protection of biometric data
depending on template transformation schemes. This is
based on intentional repeated distortions to achieve secu-
rity for biometric templates. The distortions could be per-
formed either at the feature level or at the image level.
Ratha et al. (2006) firstly introduced the notion of bio-
metrics cancelability. They rearranged the fingerprint
minutiae in polar and cartesian domains to obtain the
cancelable templates. Although their work renders the
satisfactory accuracy performance, the non-invertibility
was seen weak (Harjoko et al. 2009). Meanwhile, the work
presented in Ratha et al. (2006) inspired IrisCode protec-
tion schemes later (Ignat et al. 2013). Instead of using the
whole iris template as reported in Chin et al. (2006), Pillai
et al. (2010) used sectored random projections for gener-
ating the cancelable iris templates. Rathgeb et al. (2010)
and Zuo et al. (2008) suggested the generation of the
cancelable iris templates by applying the row permutation
on IrisCodes. In Tarek et al. (2017), a random key was used
to convert an online signature data into discrete sequences
that are convolved together to create the cancelable tem-
plate. A one more cancelable biometric method has been
presented by Teoh et al. (2004) for increasing recognition
rates of cancelable templates. Rathgeb et al. (2014,2015)
used bloom filters to construct cancelable templates from
iris codes.
In this research paper, a cancelable biometric recogni-
tion method is proposed. This method incorporates the use
of a fusion network to combine deep features (DFs) with
hand-crafted features. The DFs are extracted using a CNN
and the hand-crafted features are extracted using traditional
feature extraction algorithms. The bio-convolving method
is applied to provide protection of the biometric data. The
main contributions of this work can be listed as follows:
1. Proposal of a CNN model for deep-learned feature
extraction. The proposed CNN incorporates a depth
concatenation and residual learning block (ReSBL).
Also, the proposed model consists of ‘‘22’’ convolu-
tional, ‘‘8’’ max-pooling, ‘‘1’’ batch normalization, ‘‘1’’
fully-connected, ‘‘1’’ feature normalization and ‘‘1’’
softmax layers.
2. Proposal of cancelable face, iris, fingerprint, palm print
and ear recognition systems based on generating a new
template from fusing the extracted deep-learned fea-
tures together with the hand-crafted ones by using a
fusion network, and then applying the bio-convolving
approach on the fused result to provide a cancelable
feature descriptor.
3. Study of the effect of different hand-crafted feature
extraction techniques such as SURF, ORB, LBPs,
SIFT, and HOG on the recognition performance.
4. For dimensional consistency between the fused fea-
tures, we perform reduction of the dimensions of the
E. Abdellatef et al.
123

hand-crafted features. Furthermore, the influence of
two different dimensionality reduction techniques,
which are ICA and PCA, on the recognition perfor-
mance, is studied.
5. Tuning of different CNN hyper-parameters and study
of their impact on the performance.
The remaining parts of this article are organized as
follows. Section 2presents the main ideas that have been
manipulated in the literature. Section 3presents the pro-
posed method. Section 4illustrates the experimental
results, and Sect. 5gives the concluding remarks of the
presented work.
2 Related work
Biometric recognition systems, which are widely used for
individuals’ recognition, have a set of advantages over
traditional password/token-based authentication schemes.
So, research works aimed to tackle the related issues and
sorted out many challenges for biometric recognition sys-
tems. The CNNs are used in several computer vision
applications such as recognition of objects and image
segmentation (LeCun et al. 2015; Bengio et al. 2013). The
CNNs belong to a certain deep learning category that aims
to processing of videos and images. Moreover, CNNs are
capable not only of automating learning of image features,
but also of overcoming the disadvantages of a lot of con-
ventional hand-crafted feature extraction schemes (Deng
et al. 2017a,b; Zhao et al. 2017,2018). The earliest
DeepFace (Taigman et al. 2014) trained a CNN on nearly
4.4E6 face images, and it adopted a CNN for feature
extraction in the face verification tasks. Furthermore, this
technique reached an accuracy of 97.35% when applied on
Labeled Faces in the Wild (LFW) dataset with 4096-D
feature vectors. In extending DeepFace, a semantic boot-
strapping has been applied to choose more efficient training
sets from the large databases (Taigman et al. 2015). In Sun
et al. (2015), the intra-class distance is decreased, and the
verification losses are further integrated.
Few deep networks have been introduced for enhancing
the iris recognition performance. DeepIris network was
presented by Liu et al. (2016). The deep network proposed
in Liu et al. (2016) attained a superior recognition rate on
various databases. DeepIrisNet-A (Gangwar and Joshi
2016) achieved a superior performance on various data-
bases (Phillips et al. 2010). Zhu et al. (2005) trained the
neural network to estimate the correct ridge orientation of
the fingerprint. Liu et al. (2010) used the neural networks
based on backward propagation for detecting unique points
from the gray-scale fingerprint images. Yang et al. (2005)
used the neural networks to extract the minutiae points
from the gray-scale images. Sarbadhikari et al. (1998) used
two-stage classifiers for fingerprint image classification. In
Xu et al. (2016), a method has been presented relying on a
multi-class projection extreme learning machine (MPELM)
dataset to enhance the recognition performance of multi-
spectral palm prints. This method achieved an accuracy of
97.33%. In Ekinci and Aykut (2007), a Gabor wavelet-
based kernel has been proposed for the palm print recog-
nition, and it realized an accuracy of 95.17%. In Connie
et al. (2005), the palm print images were aligned and Fisher
discriminant analysis (FDA) was applied. Emersic et al.
(2017) adopted the data augmentation approach for the ear
recognition to overcome the problem of insufficient labeled
data. In addition, the selective learning technique was
implemented to decrease the over-fitting problem. Squee-
zeNet (Iandola et al. 2016) was also evaluated on an
unconstrained ear database. In Emersic et al. (2017) and
Pedro et al. (2016), computer vision laboratory (CVL) and
annotated web ears (AWE) datasets were combined to
obtain enough data.
The increasing demand for providing security and pri-
vacy of biometric templates raises more challenges for
recognition systems. Thanks to cancelable biometric
recognition techniques (Polash et al. 2014), biometric data
protection could be provided with a slight degradation in
the system performance. Cancelable biometric techniques
adopt the transformation of the original biometric tem-
plates using a one-way function. This strategy provides
irreversibility, which means that no information about the
original biometric templates could be obtained from the
transformed ones.
3 The proposed recognition method
For the biometric recognition improvement, we think of
performing the recognition operation based on a hybrid
feature descriptor. The hand-crafted and DFs are fused
together using a fusion network to form a single feature
vector for classification. The proposed framework is dis-
played in Fig. 1. Firstly, the input images are split into
training and testing images. The training images are fed to
the proposed CNN model to generate the DFs using (deep
feature extraction layers such as convolutional, pooling,
ReSBL, and depth concatenation layers). Additionally,
various techniques are applied on the training images to
extract the hand-crafted features. Several traditional
methods are examined, such as HOG (Ali et al. 2017),
ORB (Vinay et al. 2015), LBP (Pei et al. 2017), SIFT
(Sylvia and Kamalaharidharini 2017), and SURF (Cheng
et al. 2017). On the other hand, the dimension of the
generated hand-crafted feature vector is larger than the
dimension of the DF vector. So, there will be a need to use
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

dimensionality reduction techniques before the fusion step.
The ICA (Swathi et al. 2018) or PCA (Xiaolin and Yicong
2018) is used for dimensional consistency between the
generated features. A fusion network is used to generate the
final feature descriptor. The bio-convolving method is
executed on the final feature descriptor to improve the
system performance against spoof attacks. Finally, the
latest layer in the CNN (softmax layer) is used to give the
recognition output.
3.1 Convolutional neural network
A CNN is a deep learning algorithm that is responsible for
taking an input image and assigning learnable weights and
biases to various objects in that image. The architecture of
a CNN is inspired from the organization of the visual
cortex. The CNN architecture is similar to the connectivity
pattern of neurons in the human brain. Individual neurons
respond to stimuli only in a restricted region of the visual
field known as the receptive field. The collection of such
fields overlaps to cover the entire visual area. Additionally,
an efficient CNN architecture is proposed. The CNNs
contain several layers, and each layer performs a specific
operation on the input images. Table 1shows a description
of the essential rules of a number of CNN layers (Yuan
et al. 2017; Ioffe and Szegedy 2015; Zhang et al. 2016;
Yakopcic et al. 2017; Theodoridis and Koutroumbas 2008;
Hasnat et al. 2017). Table 2provides an explanation of the
proposed CNN model with the number and the size of
filters for each layer.
3.2 Feature extraction
Key-points/features are valuable points that are extracted
from an image to give the best definition for an object.
Feature extraction can be used in several applications such
as object detection, object tracking, and object recognition.
Feature extraction is the process of computing the
abstraction of the image information and making a local
decision at every image point to see if there is a feature in
that point or not. In the proposed method, several tradi-
tional techniques are examined for hand-crafted feature
extraction: HOG (Ali et al. 2017), ORB (Vinay et al. 2015),
LBP (Pei et al. 2017), SIFT (Sylvia and Kamalaharidharini
2017), and SURF (Cheng et al. 2017).
3.3 Dimensionality reduction
Dimensionality reduction is the process of obtaining a set
of principal variables to reduce the number of random
variables under consideration. In the proposed method,
dimensionality reduction using PCA or ICA is applied to
reduce the dimensions of the hand-crafted features to be
consistent with the dimensions of the DFs.
3.3.1 Independent component analysis
Let us represent the random observed vector X¼
X1;X2;...:; Xm
½
Twhose melements are mixtures of m
independent elements of a random vector S¼
S1;S2;...:; Sm
½
Tas (Swathi et al. 2018):
Dimensionality Reducon
Training
Images
Feature Extracon
Deep-Learned
Features
Hand-Craed
Features
Fusion Network
Final Facial Descriptor
Convoluonal
Pooling
ReSBL
Depth Concatenaon
Bio-convolving
Somax Layer
Fig. 1 The proposed system
architecture
E. Abdellatef et al.
123

X¼AS ð1Þ
where Adenotes an mmmixing and j=1,2,…,m. The
main target of the ICA is to find the non-mixing matrix W
(i.e., the inverse of A) that will give Y. The computation of
Sis illustrated in Eq. (2) (Swathi et al. 2018):
Y¼WX ffiSð2Þ
Finally, the number of independent components is
determined to be equal to the dimension of the deep feature
vector, such that the fusion process can be performed
correctly.
3.3.2 Principal component analysis
The PCA is a popular technique that is used for expressing
and representing data in such a way to show similarities
and differences. It is a powerful tool that can deal with data
of high dimensions. It is applied in image processing
applications to compress data without loss of important
information. As described in Fig. 2, the PCA starts with
computing the covariance matrix as illustrated in Eq. (3)
(Xiaolin and Yicong 2018):
cov X;YðÞ¼A¼Pn
i¼1XiX0
ðÞYiY0
ðÞ
n1ðÞ ð3Þ
where X0and Y0are the mean values of Xand Y, respec-
tively. From the covariance matrix, the eigenvalues are
computed according to Eq. (4) (Xiaolin and Yicong 2018):
AkI
jj
¼0ð4Þ
where ‘‘A’’ , ‘‘ I’’ and ‘‘k‘‘ are the covariance matrix, the
identity matrix and the eigenvalues, respectively. Further-
more, the eigenvector ‘‘E’’ is computed as (Xiaolin and
Yicong 2018):
AkI½E½¼0ð5Þ
The principal components are formed from the calculated
eigenvectors as shown in Eq. (6) (Xiaolin and Yicong
2018):
Principal components ¼e1;e2;e3;...:; en
½ð6Þ
Table 1 Essential rules of CNN layers
CNN layers Essential rule
Convolutional Learnable filters are used to compute dot products between the entries of both the filter and the input image. The
output feature map fC;l
x;y;kfor a particular layer land an input fOp;l1
x;y, can be computed as:
fC;l
x;y;k¼wlT
kfOp;l1
x;yþbl
k
where wl
kis the shared weights, bl
kis the bias and Cdenotes convolution. O
p
represents the input image, for l=1,
while it represents convolution, pooling or activation, for l[1
Max pooling Max pooling is performed by computing the maximum value in a local spatial neighborhood and then reducing the
spatial resolution:
fP;l
x;y;k¼max
m;nðÞ2Nx;y
fOp;l1
m;n;k
where the pooling operation is denoted by P, and the local spatial neighborhood of (x, y) coordinate is denoted by
Nx;y.
Batch normalization It is used for normalizing the activations of the previous layer, training the network faster, making weights easier to
be initialized and simplifying the creation of deeper networks.
Residual learning block
‘‘ResBL’’
It is used to optimize the loss of CNNs in an easy way. The output of a residual block Rcan be expressed as:
fR;l
x;y;k¼fOp;lq
x;yþFfOp;lq
x;y;wk
fg

where fOp;lq
x;yis the input feature map, F(.) is the residual mapping to be learned and qis the total number of
stacked layers
Depth concatenation It is used to increase the depth of the feature map by concatenating the output filter banks of a number of layers into
a single output vector
Feature normalization It is used to ensure that all features have equal contributions to the cost function. Normalized features fNr
ito the
softmax loss will be provided as fNr¼fOpl
ffiffiffiffi
r2
p, where land r
2
represent the mean and variance, respectively.
Softmax loss It is used for computing the loss. The form of computing softmax loss is:
Lsoftmax ¼P
N
i¼1
log ewT
yifiþbyi
PK
j¼1ewT
jfiþbj
where f
i
denotes features and y
i
is the true class label of the image. w
j
and b
j
are the weights and bias of the jth
class, respectively. Nis the number of training samples and Kis the number of classes.
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

Table 2 The proposed CNN model
Layer name No. of filters Filter size Stride size Padding size
Conv1 64 7 9793292393
ReLU n/a n/a n/a n/a
Max pooling 1 3 93292191
Batch normalization Batch normalization
Conv2 64 1 91964 1 910
ReLU n/a n/a n/a n/a
Conv3 128 3 93964 1 91191
ReLU n/a n/a n/a n/a
Max pooling 1 3 93292191
ResBL 1 3 93964 1 91191
Conv4 192 3 93964 1 91191
ReLU n/a n/a n/a n/a
Max pooling 1 3 93191191
ResBL 1 3 93964 1 91191
Conv5 64 1 919192 1 910
ReLU n/a n/a n/a n/a
Conv6 96 1 919192 1 910
ReLU n/a n/a n/a n/a
Conv7 128 3 93996 1 91191
ReLU n/a n/a n/a n/a
Conv8 16 1 919192 1 910
ReLU n/a n/a n/a n/a
Conv9 32 5 95916 1 91292
ReLU n/a n/a n/a n/a
Max pooling 1 3 93191191
Conv10 32 1 919192 1 910
ReLU n/a n/a n/a n/a
Depth concatenation Depth concatenation of 4 inputs
Conv11 128 1 919256 1 910
ReLU n/a n/a n/a n/a
Conv12 128 1 919256 1 910
ReLU n/a n/a n/a n/a
Conv13 192 3 939128 1 91191
ReLU n/a n/a n/a n/a
Conv14 32 1 919256 1 910
ReLU n/a n/a n/a n/a
Conv15 96 5 995932 1 91292
ReLU n/a n/a n/a n/a
Max pooling 1 3 93191191
Conv16 64 1 919256 1 910
ReLU n/a n/a n/a n/a
Depth concatenation Depth concatenation of 4 inputs
Conv17 192 1 919480 1 910
ReLU n/a n/a n/a n/a
Conv18 96 1 919480 1 910
ReLU n/a n/a n/a n/a
Conv19 208 3 93996 1 91191
ReLU n/a n/a n/a n/a
E. Abdellatef et al.
123

Finally, the number of principal components is deter-
mined to be equal to the dimension of the deep feature
vector, such that the fusion process can be performed
correctly.
3.4 The fusion network
We adopt a fusion network to combine the extracted fea-
tures into a more representative, reliable, useful and
detailed facial descriptor. This network consists of two
layers: local and fusion layers. The local layer is composed
of two parallel CNNs. If we consider that F
(i)
(.) represents
the DF vector, which is extracted from a CNN i, then the
output of the fusion layer could be computed as illustrated
in Eq. (7):
Final facial descriptor ¼X
n
i¼1
WiðÞ
f:FiðÞ :ðÞþbf
!
ð7Þ
where bfand WiðÞ
fare the fusion layer bias and weights,
and nrepresents the number of CNNs (in this case n¼2).
3.5 The bio-convolving method
This method (Patel et al. 2015) adopts a convolution
approach that leads to generating cancelable biometric
templates. A transformed sequence f(i), i=1, …,F,is
obtained using an original sequence r(i), i=1,…,F,
through a convolution with a random kernel h(i).
fiðÞ¼riðÞhiðÞ ð8Þ
4 Experimental results
This section reveals the effectiveness of the proposed
method using various biometric traits that include face, iris,
fingerprint, palm print, and ear. We performed experiments
on various datasets, which are Point and Shoot Face
Recognition Challenge (PaSC) (Beveridge et al. 2013) for
FR, the Institute of Automation, Chinese Academy of
Sciences (CASIA)-IrisV3 (2018) for IR, CASIA Finger-
print (2018) for fingerprint recognition, College of Engi-
neering—Pune (COEP) Palm Print (2018) for palm print
recognition, and Mathematical Analysis of Images (AMI)
Ear (2018) for ear recognition. Furthermore, comparisons
with the state-of-the-art methods are provided in terms of
Hand-craed Features
Calculaon of Covariance Matrix
Calculaon of Eigenvalues
Calculaon of Eigenvectors
Principal Components
Fig. 2 PCA algorithm
Table 2 (continued)
Layer name No. of filters Filter size Stride size Padding size
Conv20 16 1 919480 1 910
ReLU n/a n/a n/a n/a
Conv21 48 5 95916 1 91292
ReLU n/a n/a n/a n/a
Max pooling 1 3 93191191
Conv22 64 1 919480 1 910
ReLU n/a n/a n/a n/a
Depth concatenation Depth concatenation of 4 inputs
Max pooling 1 3 932920
Dropout 40% Drop-out
Fully-connected layer 1000 Fully-connected layer
Feature normalization Feature normalization
Softmax n/a n/a n/a n/a
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

the performance metrics presented in Table 3. Table 4
shows the specifications of the experiments platform. We
exploited the stochastic gradient descent with momentum
(SGDM) method to train the novel CNN model and the
momentum is set to 0.9. Moreover, we apply L
2
regular-
ization with the weight decay set to 5 910
-4
. We begin
the CNN training with a learning rate of 0.1 and stop the
training after 5 epochs. An epoch is a full training cycle on
the entire training dataset. The mini-batch size is adjusted
to 64. The experimental results are obtained using 5-fold
cross validation. The network accuracy is monitored during
training by specifying the validation data and the validation
frequency. The data is shuffled every epoch. The software
trains the network on the training data and calculates the
accuracy on the validation data at regular intervals during
training. The validation data is not used to update the
network weights. Finally, the network is trained using the
architecture that is defined by the layers, the training data,
and the training options. Tables 5,6, and 7describe the
effect of different hyper-parameters on the recognition
performance using various optimization algorithms:
SGDM, root mean square propagation (RMS prop), and
adaptive moment estimation (Adam), respectively.
The experimental results of the proposed method are
organized as FR results, IR results, fingerprint recognition
results, palm print recognition results, and ear recognition
results. The performance of the proposed method is studied
using various algorithms for hand-crafted feature
extraction. The PCA or the ICA is applied to reduce the
dimensions of the extracted hand-crafted features to be
consistent with the dimensions of the DFs. The proposed
CNN model is used for DF extraction. In addition, com-
parisons with the state-of-the-art CNNs are presented for
more validation of the effectiveness of the proposed CNN.
4.1 Face recognition results
Table 8presents the performance of the proposed FR
method. From the results, the utilization of HOG algorithm
for feature extraction and ICA for dimensionality reduction
achieves remarkable results.
Table 9provides a comparison between the proposed
CNN and the state-of-the-art CNNs. The proposed CNN
achieves a promising performance compared to the other
CNNs. Figure 3shows the ROC plot for the proposed and
CoCo loss CNN models. A graphical comparison between
various CNNs in terms of recognition accuracy for FR is
given in Fig. 4.
4.2 Iris recognition results
Table 10 presents the experimental results of IR. The
results show that the utilization of the LBPs algorithm for
feature extraction and the ICA algorithm for dimensional-
ity reduction achieves promising results on CASIA-IrisV3
dataset.
Table 11 shows a comparison between the proposed
method and the state-of-the-art IR methods. The proposed
method achieves a superior recognition performance. Fig-
ure 5displays the ROC plot for the proposed and the CoCo
loss CNN models. Figure 6illustrates a graphical com-
parison between different CNN architectures for IR.
4.3 Fingerprint recognition results
Table 12 summarizes the experimental results of the fin-
gerprint recognition using different methods for feature
extraction and dimensionality reduction. The experimental
results reveal that the utilization of the SIFT for feature
extraction and the ICA for dimensionality reduction gives
superior recognition results.
Table 13 presents a comparison between the perfor-
mance of the proposed method and the state-of-the-art
methods for fingerprint recognition. The results indicate
that the proposed method has a superior performance.
Figure 7demonstrates the ROC plots for the proposed and
the CoCo loss CNN models. In addition, the recognition
accuracy levels obtained with different CNNs for finger-
print recognition are illustrated graphically in Fig. 8.
Table 3 Equations of the performance metrics
Performance metric Equation
Accuracy TPþTN
TPþFPþFNþTN
Specificity TN
FPþTN
Precision TP
TPþFP
Recall TP
TPþFN
F
score
2RecallPrecision
RecallþPrecision
TP true positive, FN false negative, FP false positive, and TN true
negative
Table 4 The platform specifications
System Specifications
Type 64-bit Win 10
Processor Intel Xeon 5670, 12 cores
Graphics card NVIDIA GeForce GTX 1070
Installed memory (RAM) 48G memory
E. Abdellatef et al.
123

Table 5 The effect of hyper-parameters on the recognition performance using SGDM optimization algorithm
Hyper-parameters Performance metrics
Mini-batch size Learning rate Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
32 0.1 94.58 94.68 92.26 93.57 92.91
0.01 93.89 93.96 91.48 92.73 92.1
0.001 89.24 89.35 86.82 88.17 87.48
64 0.1 97.14 97.23 94.93 96.23 95.57
0.01 96.92 97.05 94.51 95.85 95.17
0.001 95.66 95.74 93.34 94.58 93.95
128 0.1 95.27 95.39 92.94 94.24 93.58
0.01 92.74 92.86 91.06 91.38 91.21
0.001 92.51 92.63 90.83 91.17 90.99
Best reults are shown in bold
Table 6 The effect of hyper-parameters on the recognition performance using RMS prop optimization algorithm
Hyper-parameters Performance metrics
Mini-batch size Learning rate Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
32 0.1 94.84 94.97 92.35 93.89 93.11
0.01 91.96 92.08 89.81 90.89 90.35
0.001 88.85 88.94 86.31 87.86 87.07
64 0.1 96.77 96.86 94.23 95.74 94.97
0.01 96.53 96.62 94.03 95.49 94.75
0.001 93.68 93.76 91.5 92.27 91.88
128 0.1 95.26 95.35 92.71 94.28 93.48
0.01 94.19 94.27 91.64 93.12 92.37
0.001 93.42 93.49 90.83 92.34 91.57
Best reults are shown in bold
Table 7 The effect of hyper-parameters on the recognition performance using Adam optimization algorithm
Hyper-parameters Performance metrics
Mini-batch size Learning rate Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
32 0.1 94.25 94.37 91.86 93.46 92.65
0.01 90.59 90.67 88.37 89.61 88.99
0.001 88.89 88.99 86.49 88.07 87.27
64 0.1 96.84 96.96 94.48 95.91 95.18
0.01 96.58 96.68 94.15 95.78 94.95
0.001 94.82 94.93 92.63 93.71 93.17
128 0.1 94.95 95.06 92.56 94.13 93.33
0.01 95.32 95.45 92.95 94.48 93.7
0.001 93.59 93.67 91.15 92.74 91.93
Best reults are shown in bold
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

4.4 Palm print recognition results
Table 14 displays the palm print recognition results using
LBPs as a feature extraction method with PCA for
dimensionality reduction. This scheme gives the best
performance.
Table 15 summarizes the experimental results for palm
print recognition based on the proposed and the state-of-
the-art methods. It is shown that the proposed method has a
better performance. Figure 9displays the ROC plots for the
proposed and DeepVisage CNN models. Also, for palm
print recognition, Fig. 10 shows the recognition accuracy
using various CNN architectures.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False positive rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
True positive rate
ROC Plot for Face Recognition
Proposed CNN
CoCo loss CNN
Fig. 3 The ROC plot of the proposed and the CoCo loss models for
FR
94.80%
95.00%
95.20%
95.40%
95.60%
95.80%
96.00%
96.20%
96.40%
96.60%
96.80%
Recognion accuracy
CNN architectures
Fig. 4 Graphical comparison between various CNNs for FR
Table 8 The experimental
results of the proposed FR
method
FE algorithm DR method Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
SURF PCA 93.14 93.23 91.34 92.34 91.83
SIFT 94.23 94.36 92.45 93.53 92.98
LBP 94.76 94.85 92.97 93.83 93.39
ORB 94.57 94.66 92.75 93.81 93.27
HOG 95.07 95.15 93.22 94.26 93.73
SURF ICA 95.04 95.12 93.15 94.22 93.68
SIFT 95.3 95.41 93.43 94.51 93.96
LBP 96 96.11 94.16 95.26 94.7
ORB 95.83 95.92 93.95 95.06 94.5
HOG 96.69 96.77 94.88 95.94 95.4
Best reults are shown in bold
Table 9 Comparison between the proposed and the state-of-the-art CNNs for FR
CNN Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
Ring loss (Zheng et al. 2018) 95.56 95.64 93.77 94.83 94.29
CoCo loss (Liu et al. 2017) 96.47 96.55 94.64 95.78 95.2
DeepVisage (Hasnat et al. 2017) 95.87 95.97 94.09 95.14 94.61
DeepID3 (Sun et al. 2015) 95.71 95.8 93.84 94.92 94.37
TBE-CNN (Ding and Tao 2017) 96.25 96.34 94.47 95.56 95.01
KinGAP (Sh et al. 2016) 96.13 96.28 94.43 95.48 94.95
Proposed CNN 96.69 96.77 94.88 95.94 95.4
Best reults are shown in bold
E. Abdellatef et al.
123

4.5 Ear recognition results
Table 16 collects the results for ear recognition based on
different methods for feature extraction and dimensionality
reduction. The results ensure that the utilization of SIFT
and ICA gives superior performance.
Table 17 provides a comparison between the proposed
methods and different state-of-the-art methods for ear
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False positive rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
True positive rate
ROC Plot for IRIS Recognition
Proposed CNN
CoCo loss CNN
Fig. 5 The ROC plots of the proposed and the CoCo loss CNNs for
IR
89.00%
90.00%
91.00%
92.00%
93.00%
94.00%
95.00%
96.00%
Recognion accuracy
Various CNN architectures
Fig. 6 Graphical comparison between various CNNs for IR
Table 10 The experimental results of the proposed IR method
Feature extraction algorithm Dimensionality reduction method Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
SURF PCA 90.73 90.78 87.94 89.98 88.94
SIFT 92.82 92.94 90.16 92.14 91.13
LBP 94.35 94.47 91.68 93.66 92.66
ORB 93.16 93.22 90.47 92.39 91.41
HOG 93.66 93.79 90.93 92.99 91.94
SURF ICA 93.63 93.71 90.96 92.96 91.94
SIFT 93.89 93.95 91.15 93.17 92.14
LBP 95.59 95.68 92.82 94.87 93.83
ORB 94.42 94.51 91.75 93.73% 92.72
HOG 95.28 95.39 92.69 94.65 93.65
Best reults are shown in bold
Table 11 Comparison between the proposed and the state-of-the-art IR CNNs
CNN Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
Ring loss (Zheng et al. 2018) 91.84 91.94 89.85 90.65 90.24
CoCo loss (Liu et al. 2017) 95.23 95.36 93.34 94.03 93.68
DeepVisage (Hasnat et al. 2017) 94.36 94.46 92.33 93.18 92.75
DeepID3 (Sun et al. 2015) 93.64 93.68 91.54 92.37 91.95
TBE-CNN (Ding and Tao 2017) 94.55 94.64 92.48 93.26 92.87
KinGAP (Sh et al. 2016) 92.5 92.57 90.45 91.26 90.85
Proposed CNN 95.59 95.68 92.82 94.87 93.83
Best reults are shown in bold
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

recognition. The results indicate that the proposed method
achieves a more promising performance than those of the
other CNN models. Figure 11 displays the ROC plots for
the proposed and TBE-CNN models. A graphical
comparison between various CNNs in terms of recognition
accuracy is presented in Fig. 12.
4.6 Evaluation of cancelable biometric methods
The proposed method uses the bio-convolving encryption
to provide protection of biometric data with a slight
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False positive rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
True positive rate
ROC Plot for Fingerprint Recognition
Proposed CNN
TBE-CNN
Fig. 7 The ROC plots for the proposed and the TBE-CNN Finger-
print recognition models
94.00%
94.50%
95.00%
95.50%
96.00%
96.50%
97.00%
97.50%
98.00%
Recognion accuracy
Various CNN architectures
Fig. 8 Graphical comparison between various CNNs for Fingerprint
recognition
Table 12 The experimental results of the proposed fingerprint recognition method
Feature extraction algorithm Dimensionality reduction method Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
SURF PCA 92.72 92.85 90.84 91.64 91.23
SIFT 95.06 95.17 93.25 94.05 93.64
LBP 94.09 94.22 92.34 93.08 92.7
ORB 93.45 93.59 91.76 92.47 92.11
HOG 94.74 94.88 93.08 93.77 93.42
SURF ICA 94.93 95.08 93.11 93.95 93.52
SIFT 97.34 97.47 95.57 96.32 95.94
LBP 96.56 96.68 94.75 95.55 95.14
ORB 96.25 96.37 94.5 95.26 94.87
HOG 95.63 95.76 93.83 94.64 94.23
Best reults are shown in bold
Table 13 Comparison between the proposed and the state-of-the-art CNNs for fingerprint recognition
CNN Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
Ring loss (Zheng et al. 2018) 95.61 95.73 93.63 94.42 94.02
CoCo loss (Liu et al. 2017) 96.77 96.86 94.84 95.66 95.24
DeepVisage (Hasnat et al. 2017) 96.49 96.57 94.57 95.34 94.95
DeepID3 (Sun et al. 2015) 96.82 96.95 94.88 95.67 95.27
TBE-CNN (Ding and Tao 2017) 97.13 97.26 95.34 96.13 95.73
KinGAP (Sh et al. 2016) 95.12 95.24 93.14 93.93 93.53
Proposed CNN 97.34 97.47 95.57 96.32 95.94
Best reults are shown in bold
E. Abdellatef et al.
123

degradation in the system accuracy. Table 18 illustrates the
change in the recognition accuracy of the proposed method
after applying the bio-convolving and Bloom filter (Rath-
geb et al. 2015) methods. Table 6shows that the recogni-
tion accuracy is slightly affected after applying the bio-
convolving method. In addition, Fig. 13 shows a graphical
comparison between different cancelable recognition
techniques and their influence on the recognition accuracy.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False positive rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
True positive rate
ROC Plot for Palm print Recognition
Proposed CNN
DeepVisage
Fig. 9 The ROC plots of the proposed and the DeepVisage models
for palm print recognition
93.00%
93.50%
94.00%
94.50%
95.00%
95.50%
96.00%
96.50%
Recognion accuracy
Various CNN architectures
Fig. 10 Graphical comparison between various CNNs for Palm print
recognition method
Table 14 The experimental results of the proposed palm print method
Feature extraction algorithm Dimensionality reduction method Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
SURF PCA 92.71 92.83 90.84 91.63 91.23
SIFT 96.11 96.25 94.26 95.06 94.65
LBP 94.13 94.24 92.24 93.05 92.64
ORB 95.82 95.97 94.4 94.77 94.58
HOG 94.47 94.55 92.54 93.34 92.93
SURF ICA 94.45 94.58 92.53 93.36 92.94
SIFT 95.63 95.72 93.65 94.45 94.04
LBP 93.37 93.49 91.51 92.28 91.89
ORB 95.31 95.43 93.47 94.24 93.85
HOG 93.55 93.68 91.63 92.46 92.04
Best reults are shown in bold
Table 15 Comparison between the proposed and the state-of-the-art CNNs for palm print recognition
CNN Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
Ring loss (Zheng et al. 2018) 94.69 94.78 92.67 93.42 93.04
CoCo loss (Liu et al. 2017) 95.44 95.57 93.43 94.28 93.85
DeepVisage (Hasnat et al. 2017) 95.72 95.81 93.66 94.43 94.04
DeepID3 (Sun et al. 2015) 94.92 95.07 92.98 93.74 93.35
TBE-CNN (Ding and Tao 2017) 95.63 95.75 93.62 94.45 94.03
KinGAP (Sh et al. 2016) 94.36 94.44 92.36 93.12 92.73
Proposed CNN 96.11 96.25 94.26 95.06 94.65
Best reults are shown in bold
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

It is shown that both techniques have high recognition
accuracy for ear recognition but lower values for iris
recognition. Furthermore, bio-convolving has higher
recognition accuracy for all traits compared to the Bloom
filter technique. The difference in recognition accuracy
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False positive rate
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
True positive rate
ROC Plot for Ear Recognition
Proposed CNN
TBE-CNN
Fig. 11 The ROC plots for the proposed and the TBE-CNN ear
recognition models
98.00%
98.20%
98.40%
98.60%
98.80%
99.00%
99.20%
99.40%
Recognion accuracy
Various CNN architectures
Fig. 12 Graphical comparison between various CNNs for ear
recognition
Table 16 The experimental results of the proposed ear recognition method
Feature extraction algorithm Dimensionality reduction method Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
SURF PCA 95.98 96.11 94.25 94.95 94.59
SIFT 97.74 97.86 95.93 96.64 96.28
LBP 97.35 97.46 95.55 96.25 95.89
ORB 97.02 97.16 95.13 95.92 95.52
HOG 97.62 97.74 95.85 96.54 96.19
SURF ICA 97.64 97.77 95.83 96.58 96.2
SIFT 99.22 99.31 97.48 98.16 97.81
LBP 98.54 98.67 96.76 97.43 97.09
ORB 98.41 98.52 96.62 97.35 96.98
HOG 97.93 98.14 96.25 96.95 96.59
Best reults are shown in bold
Table 17 Comparison between the proposed and the state-of-the-art ear recognition CNNs
CNN Accuracy (%) Specificity (%) Precision (%) Recall (%) F
score
(%)
Ring loss (Zheng et al. 2018) 98.66 98.78 96.82 97.56 97.18
CoCo loss (Liu et al. 2017) 98.93 99.04 96.97 97.73 97.34
DeepVisage (Hasnat et al. 2017) 98.82 98.94 96.98 97.68 97.32
DeepID3 (Sun et al. 2015) 98.78 98.85 96.86 97.55 97.2
TBE-CNN (Ding and Tao 2017) 98.97 99.11 97.04 97.82 97.42
KinGAP (Sh et al. 2016) 98.51 98.62 96.65 97.34 96.99
Proposed CNN 99.22 99.31 97.48 98.16 97.81
Best reults are shown in bold
Table 18 Recognition accuracies of cancelable biometric recognition
methods
Recognition method Bio-convolving (%) Bloom filter (%)
Face 96.69 93.17
Iris 95.59 92.79
Fingerprint 97.34 93.73
Palm print 96.11 93.09
Ear 99.22 95.86
E. Abdellatef et al.
123

between the two cancelable recognition methods reaches
3.52% for FR, 2.8% for IR, 3.61% for fingerprint recog-
nition, 3.02% for palm print recognition, and 3.36% for ear
recognition.
We can verify the ability of the bio-convolving method
to provide security and privacy of users’ data through
performing encryption and decryption operations on a
number of images that represent various biometric traits.
Figure 14 illustrates the original images, the encrypted
images after applying the bio-convolving method, the
decrypted images with the same de-convolution mask, and
the decrypted images with a slightly different de-convo-
lution mask.
The probability density function (PDF) of the mean
square error (MSE) and the normalized absolute error
(NAE) for the face, iris, fingerprint, palm print, and ear
images are illustrated in Figs. 15,16,17,18, and 19,
respectively.
From the Figs. 15:19, it can be noticed that the utiliza-
tion of the same de-convolution mask in the decryption
operation results in low values of MSE and NAE. On the
other hand, the utilization of a slightly different mask in the
decryption operation results in high values of MSE and
88.00%
90.00%
92.00%
94.00%
96.00%
98.00%
100.00%
Face Iris Fingerprint Palm print Ear
Recognion accuracy
Recognion method
Bio-convolving
Bloom filter
Fig. 13 Recognition accuracies of cancelable biometric recognition
systems
Original
images
Encrypted images
with bio-
convolving
method
Decrypted images
with the same
mask
Decrypted images
with a slightly different
mask
Fig. 14 Encryption and decryption operations on a number of face, iris, fingerprint, palm print, and ear images using different de-convolution
masks
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

NAE. So, we can say that the bio-convolving method
succeeds in providing security and privacy of users’ data.
Hence, it is clear that the proposed method achieves a
promising performance by taking the advantage of
extracting various types of features and combining them
into a more efficient descriptor. In addition, the bio-con-
volving has achieved the required data protection with a
slight degradation in the recognition accuracy.
5 Conclusion
In this paper, we proposed a new cancelable biometric-
based recognition method with a superior performance on
various biometric datasets. We have employed depth con-
catenation and residual layers to construct a novel CNN
structure that is used to extract the DFs from biometric
images. Consequently, the DFs are combined with another
MSE
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
PDF
Slightly different de-convolution mask
Correct de-convolution mask
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
NAE
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
PDF
Slightly different de-convolution mask
Correct de-convolution mask
Fig. 15 The PDFs of MSE and NAE on the face images
0
0.05
0.1
0.15
0.2
0.25
PDF
Slightly different de-convolution mask
Correct de-convolution mask
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
PDF
Slightly different de-convolution mask
Correct de-convolution mask
MSE
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
NAE
Fig. 16 The PDFs of MSE and NAE on the iris images
E. Abdellatef et al.
123

type of features; hand-crafted features, using a fusion
network. Finally, the bio-convolving technique is applied
on the final biometric descriptors to maintain the template
protection and ensure the users’ privacy. In addition, we
have studied the effect of varying the mini-batch size and
the learning rate on the recognition performance using
various optimization algorithms. The experimental results
on various datasets demonstrated that the proposed CNN
model extracts better DFs than those of the other state-of-
the-art CNN models. The HOG and the ICA are suitable for
FR, the LBPs and the ICA are suitable for IR, the SIFT and
the ICA are suitable for fingerprint recognition, the SIFT
and the PCA are suitable for palm print recognition, and
the SIFT and the ICA are suitable for ear recognition. The
bio-convolving technique performs better than the Bloom
filter in providing cancelability.
MSE
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
PDF
Slightly different de-convolution mask
Correct de-convolution mask
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
NAE
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
PDF
Slightly different de-convolution mask
Correct de-convolution mask
Fig. 17 The PDFs of MSE and NAE on the fingerprint images
MSE
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
NAE
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
PDF
Slightly different de-convolution mask
Correct de-convolution mask
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
PDF
Slightly different de-convolution mask
Correct de-convolution mask
Fig. 18 The PDFs of MSE and NAE on the palm print images
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of
interest.
Ethical approval All procedures performed in studies involving
human participants were in accordance with the ethical standards of
the institutional and/or national research committee and with the 1964
Helsinki Declaration and its later amendments or comparable ethical
standards.
Informed consent Informed consent was obtained from all individual
participants included in the study.
References
Ali H, Jahangir U, Yousuf B, Noor A (2017) Human action
recognition using SIFT and HOG method. In: Proceedings of
the international conference on information and communication
technologies (ICICT), pp 6–10
AMI ear database (2018), http://ctim.ulpgc.es/research_works/ami_
ear_database/ Accessed Nov 2018
Bengio Y, Courville A, Vincent P (2013) Representation learning: a
review and new perspectives. In: IEEE transactions on pattern
analysis and machine intelligence, pp 1798–1828
Beveridge J, Phillips J, Bolme D, Draper B, Givens G, Lui Y, Teli M,
Zhang H, Scruggs W, Bowyer K (2013) The challenge of face
recognition from digital point-and-shoot cameras. In: Proceed-
ings of IEEE international conference biometrics, theory,
application system, pp 1–8
Bolle RM, Connel JH, Ratha NK (2002) Biometrics perils and
patches. In: Pattern Recognation, pp 2727–2738
CASIA Fingerprint Image Database Version 5.0, http://biometrics.
idealtest.org/dbDetailForUser.do?id=7. Accessed Nov 2018
CASIA-IrisV3 Database (2018), http://www.cbsr.ia.ac.cn/english/
IrisDatabase.asp. Accessed May 2018
COEP Palm Print Database (2018) http://www.coep.org.in/resources/
coeppalmprintdatabase.m Accessed Dec 2018
Cheng C, Wang X, Li X (2017) UAV image matching based on surf
feature and harris corner algorithm. In: Proceedings of the
international conference on smart and sustainable city (ICSSC),
pp 1–6
Chin C, Jin A, Ling D (2006) High security iris verification system
based on random secret integration. In: Elsevier computer vision
and image understanding, pp 169–177
Connie T, Jin T, Ong M, Ling D (2005) Ling D (2005) An automated
palmprint recognition system. Image Vis Comput 23:501–515
Daugman J (2016) Information theory and the iris code. In: IEEE
transactions on information forensics and security, pp 400–409
Deng W, Yao R, Zhao H, Yang X, Li G (2017a) A novel intelligent
diagnosis method using optimal LS-SVM with improved PSO
algorithm. Soft Comput 23:2445
Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017b) A novel
collaborative optimization algorithm in solving complex opti-
mization problems. Soft Comput 21:4387–4398. https://doi.org/
10.1007/s00500-016-2071-8
Ding C, Tao D (2017) Trunk-branch ensemble convolutional neural
networks for video-based face recognition. IEEE Trans Pattern
Anal Mach Intell 40(4):1002–1014
Ekinci M, Aykut M (2007) Gabor-based kernel PCA for palmprint
recognition. Electr Lett 43:1077–1079
Emersic Z, Stepec D, Struc V (2017a) Training convolutional neural
networks with limited training data for ear recognition in the
wild. In: Automatic face gesture recognition, pp 987–994
Emersic Z, Struc V, Peer P (2017b) Ear recognition: more than a
survey. Neurocomputing 255:26–39
Gangwar A, Joshi A (2016) Deepirisnet: deep iris representation with
applications in iris recognition and cross-sensor iris recognition.
In: IEEE international conference on image processing (ICIP),
pp 2301–2305
MSE
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
NAE
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
PDF
Slightly different de-convolution mask
Correct de-convolution mask
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
PDF
Slightly different de-convolution mask
Correct de-convolution mask
Fig. 19 The PDFs of MSE and NAE on the ear images
E. Abdellatef et al.
123

Harjoko A, Hartati S, Dwiyasa H (2009) A method for iris recognition
based on 1d coiflet wavelet. In: World academy of science
engineering and technology, pp 126–129
Hasnat A, Bohn
´e J, Milgram J, Gentric S, Chen L (2017) DeepVisage:
making face recognition simple yet with powerful generalization
Skills. In: Proceedings of the IEEE conference on computer
vision and pattern recognition (CVPR), pp 1–12
Iandola F, Han S, Moskewicz M (2016) Squeezenet: AlexNet-level
accuracy with 50x fewer parameters and \0.5 MB model size.
In: arXiv preprint arXiv:1602.07360
Ignat A, Luca M, Ciobanu A (2013) Iris features using dual tree
complex wavelet transform in texture evaluation for biometrical
identification. In: IEEE E-health and bioengineering conference,
pp 1–4
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep
network training by reducing internal covariate shift. In:
Proceedings of the international conference on machine learning
(ICML). arXiv:1502.03167v3
Jain AK (2004) An introduction to biometric recognition. In: IEEE
transactions on circuits and systems for video technology,
pp 4–20
Jain AK (2006) A tool for information Security. In: IEEE transactions
on information forensics and security, pp 125–143
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature
521(7553):436–444
Liu Y, Li H, Wang X (2017) Rethinking feature discrimination and
polymerization for large-scale recognition. arXiv preprint arXiv:
1710.00870
Liu N, Zhang M, Li H, Sun Z, Tan T (2016) Deepiris: learning
pairwise filter bank for heterogeneous iris verification. Pattern
Recognit Lett 82:154–161
Mariana-Iuliana G, Radu T, Marius P (2019) Local learning with deep
and handcrafted features for facial expression recognition. arXiv:
1804.10892v6
NIST (2012) IREX III—performance of iris identification algorithms.
In: National Institute of Science and Technology, USA, Tech-
nical Report NIST Interagency Report 7836
Omara I, Li F, Zhang H, Zuo W (2016) A novel geometric feature
extraction method for ear recognition. Expert Syst Appl
65:127–135
Patel V, Ratha N, Chellappa R (2015) Cancelable biometrics: a
review. In: IEEE signal processing magazine, pp 54–65
Pedro L, Galdamez P, Raveane W (2016) A brief review of the ear
recognition process using deep neural networks. J. Appl. Logic
24:62–70
Pei S, Chen M, Yu Y, Tang S, Zhong C (2017) Compact LBP and
WLBP descriptor with magnitude and direction difference for
face recognition. In: Proceedings of the IEEE international
conference on image processing (ICIP), pp. 1067–1071
Phillips P, Scruggs W, O’Toole A, Flynn P, Bowyer K, Schott C,
Sharpe M (2010) Frvt 2006 and ice 2006 large scale experi-
mental results. In: IEEE transactions on pattern analysis and
machine intelligence, pp 831–846
Pichao W, Zhaoyang L, Yonghong H, Wanqing L (2016) Combining
convnets with hand-crafted features for action recognition based
on an HMM-SVM Classifier. In: Proceedings of computer vision
and pattern recognition (CVPR). arXiv:1602.00749
Pillai JK, Patel VM, Chellappa R, Ratha N (2010) Sectored random
projections for cancelable iris biometrics. In: IEEE international
conference on acoustics speech and signal processing,
pp 1838–1841
Polash P, Gavrilova M, Klimenko S (2014) Situation awareness of
cancelable biometric system. J Vis Comput 30:1059–1067
Ratha N, Connell J, Bolle R, Chikkerur S (2006) Cancelable
biometrics: a case study in fingerprints. In: International
conference of pattern recognition, pp 370–373
Ratha N, Chikkerur S, Connell J, Bolle R (2007) Generating
cancelable fingerprint templates. IEEE Trans Pattern Anal Mach
Intell 29:561–572
Rathgeb C (2010) A secure iris recognition based on local intensity
variations. In: Proceedings of the 7th international conference on
image analysis and recognition, pp 266–275
Rathgeb Ch, Barrero M, Busch Ch, Galbally J, Fierrez J (2015)
Towards cancelable multi-biometrics based on bloom filters: a
case study on feature level fusion of face and iris. In: 3
rd
International workshop on biometrics and forensics (IWBF)
Rathgeb C, Breitinger F, Busch C, Baier H (2014) On the application
of bloom filters to iris biometrics. IET J Biom 3(4):207–218
Rathgeb C, Breitinger F, Baier H, Busch C (2015) Towards bloom
filter-based indexing of iris biometric data. In: 15th IEEE
international conference on biometrics, pp 422–429
Sarbadhikari S, Basak J, Pal S, Kundu M (1998) Noisy fingerprints
classification with directional based features using MLP. Neural
Comput Appl 7:180–191
Sh X, Jinhui T, Hanjiang L, Zhiheng N, Shuicheng Y (2016) Kinship
Guided Age Progression. Pattern Recognit 59:156–157
Sun Y, Wang X, Tang X (2015) Deeply learned face representations
are sparse, selective, and robust. In: IEEE conference on
computer vision and pattern recognition
Swathi K, Kalyana V, Quek C (2018) Evolutionary based ICA with
reference for EEGlRhythm Extraction. In: IEEE Access,
pp 19702–19713
Sylvia W, Kamalaharidharini T (2017) Robust face recognition and
classification system based on SIFT and DCP techniques in
image processing. In: Proceedings of the international confer-
ence on intelligent computing and control (I2C2), pp 1–8
Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing
the gap to human-level performance in face verification. In:
Proceedings of IEEE CVPR, pp 1701–1708
Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training
for face identification. In: IEEE conference on computer vision
and pattern recognition
Sun Y, Liang D, Wang X, Tang X (2015) DeepID3: Face recognition
with very deep neural networks. arXiv:1502.00873
Tarek M, Ouda O, Hamza T (2017) Pre-image resistant cancelable
biometrics scheme using bidirectional memory model. Int J
Netw Secur 19(4):498–506
Teoh ABJ, Ngo DCL, Goh A (2004) Biohashing: two factor
authentication featuring fingerprint data and tokenised random
number. Pattern Recognit 37(11):2245–2255
Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn.
Academic Press, Cambridge
Tomas M, Sule Y, Jon Y (2016) Combining deep learning and hand-
crafted features for skin lesion classification. In: Sixth interna-
tional conference on image processing theory, tools and
applications (IPTA)
Vinay A, Kumar C, Shenoy R (2015) ORB-PCA based feature
extraction technique for face recognition. In: Proceedings of the
second international symposium on computer vision and the
internet, pp 614–621
Xiaolin X, Yicong Z (2019) Two-dimensional quaternion PCA and
sparse PCA. IEEE Trans Neural Netw Learn Syst 30(7):1–15.
https://doi.org/10.1109/TNNLS.2018.2872541
Xu X, Lu L, Zhang X, Lu H, Deng W (2016) Multispectral palmprint
recognition using multiclass projectionextreme learning machine
and digital shearlet transform. Neural Comput Appl 27:143–153
Yakopcic C, Alom M, Taha T (2017) Extremely parallel memristor
crossbar architecture for convolutionalneural network imple-
mentation. In: Proceedings of the international joint conference
on neural networks (IJCNN), pp 1696–1703
Yang G, Shi D, Quek C (2005) Fingerprint minutia recognition with
fuzzy neural network. In: Wang J, Liao X-F, Yi Z (eds) ISNN
Fusion of deep-learned and hand-crafted features for cancelable recognition systems
123

2005, Part II. LNCS, vol 3497. Springer, Heidelberg,
pp 165–170 (2005)
Yong-Xia L, Jin Q, Rui X (2010) A new detection method of singular
points of fingerprints based on neural network. In: The 3rd IEEE
international conference on computer science and information
technology, ICCSIT
Yuan Z, Liu Y, Yue J, Li J, Yang H (2017) CORAL: coarse-grained
reconfigurable architecture for convolutional neural networks.
In: Proceedings of the IEEE/ACM international symposium on
low power electronics and design (ISLPED), pp 1–6
Zhang X, Ren S, Sun J (2016) Deep residual learning for image
recognition. In: Proceedings of the IEEE conference on
computer vision and pattern recognition, pp 770–778
Zhao H, Sun M, Deng W, Yang X (2017) A new feature extraction
method based on EEMD and multi-scale fuzzy entropy for motor
bearing. Entropy 19:14
Zhao H, Yao R, Xu L, Yuan Y, Li G, Deng W (2018) Study on a
novel fault damage degree identification method using high-
order differential mathematical morphology gradient spectrum
entropy. Entropy 20:682
Zheng Y, Pal D, Savvides M (2018) Ring loss: convex feature
normalization for face recognition. In: CVPR
Zhu E, Yin J, Hu C, Zhang G (2005) Quality estimation of fingerprint
image based on neural network. In: Wang L, Chen KS, Ong Y
(eds.) ICNC 2005, Part II. LNCS, vol. 3611, pp. 65–70. Springer,
Heidelberg
Zuo J, Ratha NK, Connel JH (2008) Cancelable iris biometric. In:
Proceedings of the 19th international conference on pattern
recognition, pp 1–4
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Affiliations
Essam Abdellatef
1
•Eman M. Omran
2
•Randa F. Soliman
3
•Nabil A. Ismail
4
•Salah Eldin S. E. Abd Elrahman
4
•
Khalid N. Ismail
5,6
•Mohamed Rihan
7
•Fathi E. Abd El-Samie
7,8
•Ayman A. Eisa
2
1
Electronics and Communication Department, Delta Higher
Institute for Engineering and Technology (DHIET),
Mansoura, Egypt
2
Department of Nuclear Safety and Radiological Emergencies,
NCRRT, Egyptian Atomic Energy Authority (EAEA), Cairo,
Egypt
3
Mathematics and Computer Science Department, Faculty of
Science, Menoufia University, Menofia Governorate 32511,
Egypt
4
Department of Computer Science and Engineering, Faculty
of Electronic Engineering, Menoufia University,
Menouf 32952, Egypt
5
Department of Computer Science, Durham University,
Durham DH1 3LE, UK
6
Information Technology Department, Faculty of Computers
and Information, Menoufia University, Al Minufiyah, Egypt
7
Department of Electronics and Electrical Communications
Engineering, Faculty of Electronic Engineering, Menoufia
University, Menouf 32952, Egypt
8
Department of Information Technology, College of
Computer and Information sciences, Princess Nourah Bint
Abdulrahman University, Riyadh, Saudi Arabia
E. Abdellatef et al.
123