ArticlePDF Available

DeepMasterPrint: Generating Fingerprints for Presentation Attacks

Authors:

Abstract and Figures

We present two related methods for creating MasterPrints, synthetic fingerprints that a fingerprint verification system identifies as many different people. Both methods start with training a Generative Adversarial Network (GAN) on a set of real fingerprint images. The generator network is then used to search for images that can be recognized as multiple individuals. The first method uses evolutionary optimization in the space of latent variables, and the second uses gradient-based search. Our method is able to design a MasterPrint that a commercial fingerprint system matches to 22% of all users in a strict security setting, and 75% of all users at a looser security setting.
Content may be subject to copyright.
DeepMasterPrint: Generating Fingerprints for
Presentation Attacks
Philip Bontrager
philipjb@nyu.edu
Julian Togelius
julian@togelius.com
Nasir Memon
memon@nyu.edu
Abstract
We present two related methods for creating MasterPrints, synthetic fingerprints
that a fingerprint verification system identifies as many different people. Both
methods start with training a Generative Adversarial Network (GAN) on a set of
real fingerprint images. The generator network is then used to search for images
that can be recognized as multiple individuals. The first method uses evolutionary
optimization in the space of latent variables, and the second uses gradient-based
search. Our method is able to design a MasterPrint that a commercial fingerprint
system matches to 22% of all users in a strict security setting, and 75% of all users
at a looser security setting.
1 Introduction
Fingerprints are commonly, and increasingly, used for authentication in a large variety of systems,
from doors to workstations and smartphones. Like all biometric authentication procedures, finger-
prints are potentially vulnerable to presentation attacks—in other words, vulnerable to someone using
a synthetic fingerprint to match as someone else. Due to the demand for quick matches, low cost, and
the decreasing space left for fingerprint sensors in mobile system, fingerprint presentation attacks are
potentially practical, if a method for generating useful synthetic fingerprints can be found.
Could neural networks be used to generate fingerprint images to be used for MasterPrints? In
particular, Generative Adversarial Networks (GAN) have shown great promise in generating images
that reproduce a particular style or domain. However, their standard design does not allow the
generator to target additional constraints and objectives beyond matching the data. For a MasterPrint,
we need to create a synthetic fingerprint that fools a fingerprint verification system. The verification
system not only has recognize that the image is a fingerprint, but also match the fingerprint to
many different identities. Therefore, a generator network would need to be combined with a way of
searching for fingerprints that are suitable MasterPrints.
In this paper, we present two related techniques for creating MasterPrints, i.e., partial fingerprints that
can fool fingerprint recognition software into matching a large number of people. Both methods start
with training a GAN to create partial fingerprint images based on a corpus of either capacitive or ink
rolled fingerprint images. The first of the methods for creating MasterPrints is based on evolutionary
optimization of latent variables: a Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is
used to search for the inputs to the the generator network that creates images that are recognized
as many separate fingerprints as possible by a commercial fingerprint recognition software. The
second method, which does not require access to an external fingerprint recognition software, trains a
neural network-based fingerprint classifier and then uses gradient descent to find latent variables that
maximize the number of activated outputs.
Previous work on finding MasterPrints has relied on stochastic search in the space of minutiae, a
relatively high-level representation of fingerprints, which cannot easily be translated to high-quality
fingerprint images. In contrast, our method produces complete images which are visually similar to
natural fingerprint images.
arXiv:1705.07386v1 [cs.CV] 21 May 2017
2 Background
2.1 Fingerprint presentation attacks
Recently, Roy et al. [
12
] studied the vulnerability of fingerprint-based biometric systems that have
small sensors for authentication and therefore only scan part of the fingerprint. They found these
systems are highly susceptible to a type of presentation attack that is known as a “wolf attack” [
14
].
A "wolf" is a biometric sample, real or synthesized, that impersonates multiple subjects’ biometrics.
This type of attack does not require an individual’s biometric sample but instead is be deployed
against unknown people with some probability of a successful attack.
Small fingerprint scanners are too small to read the entire fingerprint and therefore the system must
authenticate on just the cross-section of the finger that is read from the sensor. Since it would be
impractical to require the user to place their finger the exact same way every time, these systems
normally take multiple readings from the same finger. When an input is presented to the system, it
compares the input against all of the partial prints that it has for a subject. This means that if a subject
has
n
fingers in the system and
k
partials saved per fingerprint, then there are are
n×k
opportunities
for a match and the system only needs a match on one of them. These types of biometric systems are
common on consumer mobile devices.
Roy et al. [
12
] showed that there exists synthetic fingerprints that can match for many fingerprints in a
dataset. Their method represents fingerprints as minutiae templates. A minutia template represents all
the points of interest in a fingerprint. This includes where ridges end, where they split, and the angle
going away from that point. Many fingerprint identification systems will first identify the minutiae in
the fingerprint and then compare them to the minutia template saved in the system. Roy et al then
use a hill-climbing algorithm to search for a minutia template to use for a wolf attack; the objective
function is simply the number of people in the database that the fingerprint identifies positively as.
In our work we generate the images directly instead of working at the minutia level. This provides
actual synthetic fingerprints that can be used, whereas the previous work only generated minutia
templates. There is work to synthesize fingerprints from minutiae [
4
], but by working at the image
level we can also optimize for systems that don’t use minutiae, systems such as the iPhone [1].
2.2 Image generation
Recently there has been rapid advancements in image generation by way of neural networks. Some
of the most popular methods for image generation are Fully Visible Belief Networks (FVBN),
Variational Autoencoders (VAE), and Generative Adversarial Networks (GAN) [
6
]. FVBNs such as
PixelRNN produce one pixel at a time, similar to text generation and can have a bit of noise in their
output. VAEs on the other hand tend to produce very smooth outputs. Current GAN methods are
perceived to produce the best results with fewer artifacts than FVBNs and sharper images than VAEs
[6].
GANs learn to generate images in a semi-supervised fashion. There are two parts to GAN; a generator
and a discriminator. The generator is typically a neural network that takes random noise as an input
and outputs an image. The discriminator is also typically a neural network, it takes an image as an
input and classifies it as real or generated. To train the generator to produce images within the domain
of the sample images, training happens in three steps:
1. Provide real images to the discriminator. Train the discriminator to classify them as real.
2. Provide generated images to the discriminator. Train it to classify them as generated.
3.
Provide the generator with the discriminator’s gradients. Train the generator to produce
images that are classified as real.
This process is repeated until the network converges on the distribution of the real data.
A major difficulty during training is keeping the two networks balanced so one doesn’t get significantly
better than the other. A recent technique to stabilize training is the Wasserstein GAN (WGAN) [
2
].
In standard GAN training the discriminator classifies the input as one of two classes, “real” or
“generated”. The difference between the real data distribution and the generated data is then measured
using the Jensen-Shannon divergence (JS) [
2
]. This metric does not provide a gradient everywhere
2
for the generator and therefore requires that the discriminator and generator are closely matched.
This makes training unstable. WGAN, instead uses an approximation of the Wasserstein distance
function to measure the difference between the real and fake distributions [
2
]. Since it is differentiable
everywhere, it provides meaningful gradients for the generator. The two networks don’t have to be
well balanced so the discriminator can be better trained preventing mode collapse.
This is just one of a few GAN architectures that have recently had success with producing detailed,
higher resolution images. Other examples not used here are the Boundary Equilibrium GAN [
3
] and
Energy Based GAN [16].
3 Datasets
In this work we model two types of fingerprint images; those scanned from rolled impressions and
those obtained from a capacitive sensor. Rolled fingerprints are produced by applying ink to the
finger and rolling the finger on paper. Capacitive sensors record ridge information based on where
the finger contacts the sensor.
Rolled images
The rolled fingerprints come from the publicly available NIST Special Database
9 fingerprint dataset [
15
]. The dataset consists of all 10 fingerprints of 5400 unique subjects. Each
fingerprint is an 8-bit gray scale image. For each subject, the right thumbprint is selected since that is
a common finger used in authentication systems. The images are preprocessed by cropping out the
whitespace and then downscaling to
256 ×256
pixels. To get partial fingerprint samples, a random
128 ×128 region is selected every time an image is selected.
Capacitive images
The capacitive fingerprint images come from the FingerPass DB7 dataset [
10
].
This dataset has 12 partial fingerprints per each of its 720 subjects. Each partial print is
144 ×144
at
500 dpi. This is similar to what is scanned by an iPhone [
1
]. This is the same dataset used by Roy et
al. in their MasterPrint paper [12].
4 Methods
There are two parts to our approach to creating a MasterPrint; (1) learning the prior distribution
of fingerprint images and (2) searching the distribution for an image that satisfies the criteria of a
wolf attack. To learn the prior distribution, we use the WGAN method, described earlier, to train
a generator network. Our generators receive 100 random numbers as input and output a random
fingerprint image. We learn the priors of two different datasets to test how resilient the results are to
different types of fingerprint sensors. To search the prior distribution for a MasterPrint, we search the
generators’ 100 latent variables. We develop two separate techniques for searching this latent space.
4.1 Latent variable evolution
When there is access to the fingerprint verification system, then the MasterPrint can be directly
optimized for that system. This can be achieved by incorporating the fingerprint verification system
into the optimization process as a black-box fitness function. To search the latent variables of the
image generator for an input that will output a wolf attack, an evolutionary algorithm is a powerful
technique. An evolutionary algorithm does not require gradients and therefore are ideal for black-box
optimization. For an evolutionary algorithm, a potential optimal solution is represented as a vector.
In this case the vector represents the 100 inputs to the fingerprint generator. A fixed number of
these proposed solutions are tested and scored. The best ones are kept and mutated to form the next
generation [5].
In this paper we use a specific evolutionary strategy called Covariance Matrix Adaption (CMA-ES).
CMA-ES learns the covariance matrix of the variables being optimized. This knowledge allows the
algorithm to intelligently guide mutations when many variables are highly correlated [
8
]. Since the
variables in this work are the inputs to a neural network, there it is high likelihood that many variables
will be correlated and this strategy will be more effective than standard evolutionary strategies. In
this work we use Hansen’s Python implementation of CMA-ES [7].
3
For our work we use two different fingerprint verification systems. To be able to compare our results
to previous work, we use the widely deployed commercial fingerprint system, VeriFinger 9.0 SDK.
This is the system used by Roy et al. [
12
]. To be able to test how well optimization for one black-box
transfers to another, we also use the Bozorth3 system. This system is provided open source by NIST
as part of their NIST Biometric Image Software (NBIS).
4.2 Multi-label activation maximization
When access to the fingerprint verification system is not available, it is necessary to use a proxy
verification system. We use a convolutional neural network (CNN) as the proxy system. To use a
CNN as a fingerprint verification system, it has to be trained to classify fingerprints by subject. The
advantage of using a CNN as the verification system is that it provides a gradient that can be used for
optimization. Instead of using evolution to optimize the generator’s latent variables, backpropagation
can be used.
The CNN is trained on inputs of
128 ×128
partial fingerprint images. For every subject in the dataset,
it has a corresponding output class. For a wolf attack, there needs to be a single input that activates
all of the CNN’s outputs. During optimization, start with random latent variable values,
z
. Use
z
to
generate an image and calculate a CNN output. Calculate the error for a label corresponding to all
the outputs being activated. Backpropagate the error through the CNN and the generator back to
z
to
get z0. This repeats until the optimal zis found.
Activation maximization for one or two neurons in a CNN has recently been done in a similar manner
by Nguyen et al. [
11
]. They used this technique for understanding networks better and proposed
using maximizing two neurons for art. Here we are maximizing many neurons at once and each
neuron represents a functional objective for our generator.
Our CNN architectures are the Residual Network (ResNet) that has achieved superhuman results on
the ImageNet classification competition [
9
], the smaller, yet still powerful, VGG16 architecture [
13
],
and a simple convolutional design with 3 convolutional layers and 3 fully connected layers on top.
To allow these networks to make multi-label predictions, the top layer is replaced. Normally the top
layer is a softmax layer, this forces all the outputs to sum to 1 so that the network outputs a probability
distribution of its predictions. To make each prediction independent, each class is represented in the
top layer by one sigmoid activation function. To produce the training data, many partial fingerprints
are sampled from each full fingerprint that is used to identify a subject.
5 Results
5.1 WGAN fingerprints
The results of training the WGAN generator can be seen in Figure 1. In the right column are the
generated images, the left column contains samples from the actual datasets. The image generator
seems to have captured the basic structures in both instances.
Figure 1a shows partial fingerprints for rolled fingerprints from the NIST dataset. Looking at the right
batch, it is clear that the generator has learned the general ridge structure of a fingerprint. Looking
closer, there are certain areas that look smudged. Most likely due to the fact that the data is generated
from random sections of the fingerprint, the generator had a difficult time learning the more global
shapes of a fingerprint. From visual inspection it appears to have learned the texture. Some of the
outputs do look very similar to the input data.
Figure 1b displays the results for the capacitive fingerprints. The results seem to be a little better
for this dataset. There are less smudges on the images and the ridges are better connected. Looking
at larger batches, the generated capacitive images are more consistently good the the rolled images.
This is likely because the capacitive data is cleaner and there is far less detail to learn.
For the generator in 1a it took 120,000 generator updates before the outputs stopped improving. After
120,000 the generator started to produce blocky images that eventually just became a grid. To avoid
this checkered artifact, we replaced all the deconvolutional layers in the generator with upsampling
and regular convolutions for the capacitive generator. This removed the artifacts and combined with
the cleaner data resulted in a much faster training time for the generator in 1a.
4
(a) Real (left) and generated (right) samples for the NIST dataset.
(b) Real (left) and generated (right) samples for the FingerPass capacitive dataset.
Figure 1
Figure 2: Optimally evolved MasterPrints for rolled fingerprints (left three) and for capacitive
fingerprints (right three). Each set of three is optimized for an FMR at 0.01%, 0.1%, and 1%
respectively.
5.2 Latent variable evolution
To test the results of the Latent Variable Evolution (LVE) method, we devised a number of different
tests. Both generators are optimized for the VeriFinger verification system, as seen in Figure 2.
Similar to the work of Roy et al., the system is tested at three different security levels. The three
levels are a 1%, 0.1%, and 0.01% False Match Rate (FMR). The FMR is the probability that an
impostor fingerprint will be marked as a match. If the FMR is set too high, the system is not very
secure. If it is too low, it will reject too many legitimate fingerprint readings.
The results for the VeriFinger system are achieved by optimizing the generator on half of the dataset
and testing on the other half. Our results were comparable to previous, non-image-based methods. To
further understand how well these results generalize to other systems, the fingerprints optimized for
VeriFinger are also scored on the Bozorth3 system.
In Table 1 the percentage of false subject matches are displayed. The number of false subject matches
are the number of subjects in the dataset that the MasterPrint matches. Each subject in the dataset
is represented by 12 partial fingerprints, to get a subject match only 1 in 12 have to match. The
percentage represents the percentage of subjects in the database that the MasterPrint matches. The
second row in the table shows the results on the VeriFinger system when run on the test data. The
third row are the percent of subject matches when the fingerprints are matched with the Bozorth3
system. Since the images aren’t evolved for this system, this represents how well the match rate
transfers.
Rolled MasterPrints
The three rolled MasterPrints make up the left half of Figure 2. Each
MasterPrint has around 20 minutiae points. At the lowest security level, a single fingerprint is able to
5
Table 1: Percentage of subjects matched for the fingerprints in Figure 2
Rolled MasterPrint Matches Capacitive MasterPrint Matches
0.01% FMR 0.1% FMR 1% FMR 0.01% FMR 0.1% FMR 1% FMR
Training 5.00% 13.89% 67.50% 6.94% 29.44% 89.44%
VeriFinger 0.28% 8.61% 78.06% 1.11% 22.50% 76.67%
Bozorth3 0.14% 16.25% 55.83% 0.00% 5.28% 6.67%
match with 78% of the people in the dataset. This is a large number of subjects but it is unlikely that
many systems are running with this low of a security setting. At the 0.1% FMR level, the MasterPrint
matches 8.61% of the dataset. This represents a much more realistic security expectation and is a
much higher number of matches than the FMR would lead one to expect. At the highest security level
the results aren’t very good, but this is also an unlikely security level as it would be inconvenient to
the user.
A very interesting part of the results is how well these MasterPrints do on the Bozorth3 verification
system. These fingerprints are not optimized for it and yet the results are still good. At the middle
security setting, the MasterPrint actually does better on the Bozorth3 system. This suggests that the
fingerprint features in the MasterPrint that allow it to match so many subjects have general properties
and it is not just targeting quirks in the VeriFinger system. Since both systems are minutiae based, it
does not suggest generality beyond minutiae systems but it is an important fist step in showing that
these results are applicable.
Capacitive MasterPrints
The three capacitive MasterPrints make up the right half of Figure 2.
The generator used to create these images was trained on the same data that the fingerprint verification
systems are matching against. This means that these MasterPrints are much more visually similar to
the real data than the other MasterPrints. This should allow these MasterPrints to do better than the
rolled MasterPrints. Looking at Table 1, the results appear to be a little bit improved. At the 0.01%
FMR level, the results are much better. A one in five subject match rate at a realistic security level
very high.
The results on the Bozorth3 system, on the other hand, are not nearly as good as what is seen with the
rolled MasterPrint. One explanation for this is that the Bozorth3 system is designed and tested for
the NIST data which consists of rolled fingerprint impressions. It may struggle with the capacitive
MasterPrints. Another possibility is that the capacitive generator has less general ways to target the
VeriFinger system. The images produced from the capacitive generator have much thicker ridges
which means the model might be less flexible in targeting the system. To compensate images were
evolved that have lots of noise in the center. Noise can easily target an individual system, but it does
not usually transfer. Figure 2 supports this. The rolled MasterPrints are for the most part coherent
while the capacitive MasterPrints have a lot of noise at the center of the image.
MasterPrint comparisons
The results obtained here are comparable to the results previously
reported by Roy et al. [
12
]. They do not report on the results they obtained for individual synthetic
minutiae points, but they graphically present the subject match rate of a set of 5 minutiae points. Our
results at the 1% FMR level are as good as their result using five different fingerprints on a system that
allows five attempts. The capacitive MasterPrint’s scores are not much worse than their 5 fingerprint
match rate at the higher security levels either. The results cannot be compared directly, but it is clear
that these results are on par with the state of the art. They have the added benefit of being full images
which can be fabricated into deployable attacks.
5.3 Multi-label activation maximization
The Multi-Label Activation Maximization (MLAM) technique uses a CNN as a proxy for the
fingerprint verification system. Yet, to analyze the quality of its results, we have to compare it to a
benchmark. For this we once again use the VeriFInger verification system on the FingerPass dataset.
This allows us to analyze how well the the CNN acts as a proxy to an unknown verification system.
In analyzing the MLAM MasterPrints on the VeriFinger system, we found that the results are very
noisy. With MLAM, the results are highly dependent on the starting random latent variable values.
6
Figure 3: Proxy optimized MasterPrint samples. Rolled fingerprints (left) and capacitive fingerprints
(right) are optimized for a ResNet CNN.
These values are incremented to maximize the sum of all the label outputs in the proxy classifier
but often get stuck in local maxima. To reduce the dependence on the starting latent values, we
try 50 random starts sets and after training all 50, take the best result. This helps stabilize and
reduce the noise in the scores assigned by the proxy network. Despite the proxy network scoring the
MasterPrints similarly, the scores from the VeriFinger system are not very consistent. This implies
that which features get optimized are highly dependent on the initial random seed as the beginning of
training. To be able analyze the effects of MLAM despite the noise, we produce 30 samples for every
test and use the average value.
Despite their high variability in score, all of the optimized samples look very similar. Part of this is
perception is probably related to people’s poor ability to distinguish fingerprints. Yet the fingerprints
in Figure 3 are much more self similar than they are similar to other fingerprints presented in this
paper.
Table 2: Average percentage of subjects matched for MasterPrints optimized for different proxy
architectures at 1% FMR.
Rolled Data Capacitive Data
N Mean Std. Dev N Mean Std. Dev
Random Data Samples 30 5.58% 6.53% 30 3.31% 5.37%
Random Generated Samples 30 10.35% 9.08% 30 6.06% 5.82%
ResNet Optimized Samples 30 20.75% 10.18% 30 10.02% 7.54%
VGG Optimized Samples 30 10.13% 7.76%
Basic Optimized Samples 30 9.69% 6.72%
Rolled MasterPrint
The images on the left half of Figure 3 are created using the rolled fingerprint
generator and optimized with a CNN using the ResNet architecture. The false subject probabilities of
these MasterPrints is in Table 2. The Rolled data section shows; the average match rate for MLAM
Masterprints, images generated at random, and random rolled fingerprint data samples at the 1%
FMR level. The data shows that the rolled MasterPrints designed using this method match 20% of
the subjects in the dataset. This can be compared to the MasterPrints optimized through evolution
which has a match rate of 78%. The difference here is the design system has no knowledge of the
verification system or even of minutiae. Verification systems that use minutiae will have an emphasis
on endpoints and parts of the fingerprint deemed important by the people that designed the concept
of minutiae. The score on this proxy system shows the general feature optimization that is happening
irrespective of the verification design.
Due to the high variance in the results, it is important to prove that the system is optimizing the
fingerprints. To do this, we run a 1 way ANOVA on the three groups to test for statistical significance
between the groups. This results in F = 22.88 with a p value much lower than 0.01. Running Tukey’s
HSD post-hoc test shows more specifically that the difference between the optimized fingerprint and
both random fingerprints are significant at p < .01. This means that we can be over 99% certain
that the optimized results are better than the unoptimized results and that both results are better than
random data. We tried to test against random noise, but the verification system could not detect any
minutiae points.
7
While we expected the optimized results to be better than the unoptimized results, we did not expect
the the generated results to have a higher match rate in general than the data that the generator is
trained on. This suggests that the adversarial training used to train the image generator is already
capitalizing on general features found in the dataset. The images being generated are already
prototypical fingerprints which make them natural MasterPrints. For this particular context, loss of
variability in the generated images has its advantages.
Optimizing for the proxy CNN verification system doubles the match rate from the randomly
generated results. Looking at these images in Figure 3, they do not really look like fingerprints. Of
all the outputs we looked at from both generators, these images looked the least like real fingerprints.
They look like fingerprints with parts whited out. One possibility is that the bottom and right edge of
the fingerprints in our data have a lot of similarities while the rest of the image is where most of the
unique aspects are. By having nothing in the unique regions, the fingerprints are less likely to get
rejected for individual matches.
Capacitive MasterPrint
The images on right half of Figure 3 are created using the capacitive
fingerprint generator and optimized with a CNN using the ResNet architecture. The results for the
capacitive generator are on the right side of Table 2. For the capacitive data we experimented with
the proxy architectures to test how the CNN architecture effects the effectiveness of the fingerprint
designs.
As was done with the rolled MasterPrint results, the five groups of “MasterPrints” are analyzed with
a 1 way ANOVA. This results in F = 6.18 with a p value much lower than 0.01. This is a lower F than
the rolled data results but still significant. Running Tukey’s HSD post-hoc test shows that the three
optimized groups are significantly different than the random data at p < .01. The effect size here
is not large but the results follow what we have already seen. The randomly generated results have
roughly twice the number of matches as the real data, and the optimized results are roughly twice as
good as the random results. The main difference is the capacitive data gets fewer matches than the
rolled data. This may again be related to the much lower number of ridges in the capacitive images.
The different CNN architectures did not make a difference. It is interesting that a 6, 16, and 50 layer
network all have the same effect in terms of optimizing a MasterPrint. This demonstrates a robustness
in the results but also makes us question what features these networks are all optimizing. Looking at
the MasterPrint images for all three architectures, they all appear similar. If we had optimized for
hidden layers instead of the output of the CNN, then the internal representations of these networks
might have been more important. This analysis indicates that the underlying data is one of the most
important aspects in generating a MasterPrint.
6 Conclusion
This paper presents two related methods for generating MasterPrints; partial fingerprints which
can be used in presentation attacks on fingerprint authorization systems. Both methods start with
training a Wasserstein GAN on a fingerprint dataset. They then search for latent variable values
(“inputs”), for the GAN generator, that maximize the number of people the output of the generator
can match. The first method, which requires access to an external fingerprint recognition system,
uses evolutionary computation to find the variable values. The second method instead trains a neural
network to do multi-label classification of the fingerprints, and uses gradient descent to find the
latent variable values. The results of our methods are state of the art, and, unlike previous work,
our method generates complete fingerprint images. Testing with three different CNN architectures
and two different external fingerprint recognition systems show that the method is robust and not
dependent on the artifacts of any particular fingerprint recognition system.
Future testing should verify the effectiveness of this system in the wild, against e.g. smartphone
fingerprint recognition systems. It is also plausible that the method could be extended to many other
types authentication modalities that are vulnerable to presentation attacks.
8
References
[1] Apple. ios security - white paper, 2017.
[2]
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint
arXiv:1701.07875, 2017.
[3]
David Berthelot, Tom Schumm, and Luke Metz. Began: Boundary equilibrium generative
adversarial networks. arXiv preprint arXiv:1703.10717, 2017.
[4]
Kai Cao and Anil K Jain. Learning fingerprint reconstruction: From minutiae to image. IEEE
Transactions on information forensics and security, 10(1):104–117, 2015.
[5]
Agoston E Eiben, James E Smith, et al. Introduction to evolutionary computing, volume 53.
Springer, 2003.
[6]
Ian Goodfellow. Nips 2016 tutorial: Generative adversarial networks. arXiv preprint
arXiv:1701.00160, 2016.
[7]
Nikolaus Hansen. The cma evolution strategy: a comparing review. Towards a new evolutionary
computation, pages 75–102, 2006.
[8]
Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolu-
tion strategies. Evolutionary computation, 9(2):159–195, 2001.
[9]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im-
age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pages 770–778, 2016.
[10]
Xiaofei Jia, Xin Yang, Yali Zang, Ning Zhang, and Jie Tian. A cross-device matching fingerprint
database from multi-type sensors. In Pattern Recognition (ICPR), 2012 21st International
Conference on, pages 3001–3004. IEEE, 2012.
[11]
Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune. Synthesizing
the preferred inputs for neurons in neural networks via deep generator networks. In Advances
in Neural Information Processing Systems, pages 3387–3395, 2016.
[12]
Aditi Roy, Nasir Memon, and Arun Ross. Masterprint: Exploring the vulnerability of partial
fingerprint-based authentication systems. IEEE Transactions on Information Forensics and
Security, 2017.
[13]
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale
image recognition. 2015.
[14]
Masashi Une, Akira Otsuka, and Hideki Imai. Wolf attack probability: A new security measure
in biometric authentication systems. In International Conference on Biometrics, pages 396–406.
Springer, 2007.
[15]
Craig I Watson. Nist special database 9, mated fingerprint card pairs. National Institute of
Standard and Technology (February 1993), 1993.
[16]
Junbo Zhao, Michael Mathieu, and Yann LeCun. Energy-based generative adversarial network.
arXiv preprint arXiv:1609.03126, 2016.
9
... They focused on optimizing toward playable levels with a scalable difficulty. The idea of using latent variable evolution (LVE) to explore the generator's latent space was first introduced by Bontrager et al., in their works to match generated fingerprints to as many real fingerprints as possible (Bontrager, Togelius, and Memon 2017). Evolving the latent space to gain control over the output stands in contrast to Conditional GANs (CGANs) (Mirza and Osindero 2014), which utilize a condition vector com- ...
Article
Full-text available
This paper investigates the suitability of using Generative Adversarial Networks (GANs) to generate stable structures for the physics-based puzzle game Angry Birds. While previous applications of GANs for level generation have been mostly limited to tile-based representations, this paper explores their suitability for creating stable structures made from multiple smaller blocks. This includes a detailed encoding/decoding process for converting between Angry Birds level descriptions and a suitable grid-based representation, as well as utilizing state-of-the-art GAN architectures and training methods to produce new structure designs. Our results show that GANs can be successfully applied to generate a varied range of complex and stable Angry Birds structures.
... They focused on optimizing toward playable levels with a scalable difficulty. The idea of using latent variable evolution (LVE) to explore the generator's latent space was first introduced by Bontrager et al., in their works to match generated fingerprints to as many real fingerprints as possible (Bontrager, Togelius, and Memon 2017). Evolving the latent space to gain control over the output stands in contrast to Conditional GANs (CGANs) (Mirza and Osindero 2014), which utilize a condition vector com- ...
Preprint
Full-text available
This paper investigates the suitability of using Generative Adversarial Networks (GANs) to generate stable structures for the physics-based puzzle game Angry Birds. While previous applications of GANs for level generation have been mostly limited to tile-based representations, this paper explores their suitability for creating stable structures made from multiple smaller blocks. This includes a detailed encoding/decoding process for converting between Angry Birds level descriptions and a suitable grid-based representation, as well as utilizing state-of-the-art GAN architectures and training methods to produce new structure designs. Our results show that GANs can be successfully applied to generate a varied range of complex and stable Angry Birds structures.
... This work applies the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen and Ostermeier 2001;Hansen 2016). First, Mario levels are generated using a Deep Convolutional GAN (DCGAN) based on a single training sample, and then CMA-ES is used to search through the space of those levels for samples satisfying various fitness requirements This approach builds on latent variable evolution (LVE) (Bontrager, Togelius, and Memon 2017), which trains a neural network to generate images and searches over the latent variables of the network, or the space of input vectors, for input that will produce the right output. LVE was originally applied to fingerprints, and then used to generate levels for Super Mario Bros (Volz et al. 2018). ...
Article
Generative Adversarial Networks (GANs) have been used with great success to generate images. They have also been applied to the task of Procedural Content Generation (PCG) in games, particularly for level generation, with various approaches taken to solving the problem of training data. One of those approaches, TOAD-GAN (Token-Based One-Shot Arbitrary Dimension Generative Adversarial Network) (Awiszus, Schubert, and Rosenhahn 2020), can generate levels based on a single training example and has been able to closely reproduce token patterns found in the training sample. While TOAD-GAN is an impressive achievement, questions remain about what exactly it has learned. Can the generator be made to produce levels that are substantially different from the level it has been trained on? Can it reproduce specific level segments? How different are the generated levels? We investigate these questions and others by using the CMA-ES algorithm for Latent Space Evolution. To make the search space feasible, we use a random projection in latent space. We propose the investigation undertaken here as a paradigm for studies into what machine-learned generators have actually learned, and also as a test of a new method for projecting from a smaller search space to a larger latent space.
... Another study uses a GAN trained on real biometric fingerprint data to generate fingerprint minutiae features that were then used to create "masterprints". The results show that on a commercial fingerprint system, the masterprint matches to 22% of all users in strict security settings, and 75% of all users in looser security settings [11]. Adversarial systems like PassGAN can be used to spoof authentication systems. ...
Chapter
Full-text available
The availability of biometric data for cybersecurity studies is very important and the Generative Adversarial Network (GAN) provides a means of generating realistic fake biometric data that may mitigate some of the privacy concerns with using real biometric data. This paper trains a Deep Convolutional Generative Adversarial Network (DCGAN) to generate fake biometric fingerprint images. The quality of the generated images is measured using the NFIQ2 algorithm and although the qualitative analysis of the fake fingerprints indicate that the images are of good quality, the NFIQ2 scores obtained for the fake fingerprints are of low utility value. We also explore fingerprint matching by first extracting the minutiae feature points from both the fake generated fingerprint samples and the real fingerprint images using the MINDTCT technique. The minutiae feature points from the fake fingerprint images are then matched (one-to-many) against feature points extracted from real fingerprint images using the BOZORTH3 algorithm. This is done in order to appraise the GANs ability to generate data points with features that generalize beyond the precise domain of the training datasets features. The results from the fingerprint matching show that the fake generated fingerprint samples contain unique fingerprint images.
Article
Generative adversarial networks (GANs) are a powerful indirect genotype-to-phenotype mapping for evolutionary search. Much previous work applying GANs to level generation focuses on fixed-size segments combined into a whole level, but individual segments may not fit together cohesively. In contrast, segments in human designed levels are often repeated, directly or with variation, and organized into patterns (the symmetric eagle in Level 1 of The Legend of Zelda , or repeated pipe motifs in Super Mario Bros. ). Such patterns can be produced with compositional pattern producing networks (CPPNs). CPPNs define latent vector GAN inputs as a function of geometry, organizing segments output by a GAN into complete levels. However, collections of latent vectors can also be evolved directly, producing more chaotic levels. We propose a hybrid approach that evolves CPPNs first, but allows latent vectors to evolve later, combining the benefits of both approaches. These approaches are evaluated in Super Mario Bros . and The Legend of Zelda . We previously demonstrated via a quality-diversity algorithm that CPPNs better cover the space of possible levels than directly evolved levels. Here, we show that the hybrid approach first, covers areas that neither of the other methods can, and second, achieves comparable or superior quality diversity (QD) scores.
Article
Full-text available
Deep neural networks (DNNs) have demonstrated state-of-the-art results on many pattern recognition tasks, especially vision classification problems. Understanding the inner workings of such computational brains is both fascinating basic science that is interesting in its own right - similar to why we study the human brain - and will enable researchers to further improve DNNs. One path to understanding how a neural network functions internally is to study what each of its neurons has learned to detect. One such method is called activation maximization (AM), which synthesizes an input (e.g. an image) that highly activates a neuron. Here we dramatically improve the qualitative state of the art of activation maximization by harnessing a powerful, learned prior: a deep generator network (DGN). The algorithm (1) generates qualitatively state-of-the-art synthetic images that look almost real, (2) reveals the features learned by each neuron in an interpretable way, (3) generalizes well to new datasets and somewhat well to different network architectures without requiring the prior to be relearned, and (4) can be considered as a high-quality generative method (in this case, by generating novel, creative, interesting, recognizable images).
Article
We propose a new equilibrium enforcing method paired with a loss derived from the Wasserstein distance for training auto-encoder based Generative Adversarial Networks. This method balances the generator and discriminator during training. Additionally, it provides a new approximate convergence measure, fast and stable training and high visual quality. We also derive a way of controlling the trade-off between image diversity and visual quality. We focus on the image generation task, setting a new milestone in visual quality, even at higher resolutions. This is achieved while using a relatively simple model architecture and a standard training procedure.
Article
This report summarizes the tutorial presented by the author at NIPS 2016 on generative adversarial networks (GANs). The tutorial describes: (1) Why generative modeling is a topic worth studying, (2) how generative models work, and how GANs compare to other generative models, (3) the details of how GANs work, (4) research frontiers in GANs, and (5) state-of-the-art image models that combine GANs with other methods. Finally, the tutorial contains three exercises for readers to complete, and the solutions to these exercises.
Article
We introduce the "Energy-based Generative Adversarial Network" (EBGAN) model which views the discriminator in GAN framework as an energy function that associates low energies with the regions near the data manifold and higher energies everywhere else. Similar to the probabilistic GANs, a generator is trained to produce contrastive samples with minimal energies, while the energy function is trained to assign high energies to those generated samples. Viewing the discriminator as an energy function allows to use a wide variety of architectures and loss functionals in addition to the usual binary discriminant network. Among them, an instantiation of EBGANs is to use an auto-encoder architecture alongside the energy being the reconstruction error. We show that this form of EBGAN exhibits more stable behavior than regular GANs during training. We also show that a single-scale architecture can be trained to generate high-resolution images.
Article
The set of minutia points is considered to be the most distinctive feature for fingerprint representation and is widely used in fingerprint matching. It was believed that the minutiae set does not contain sufficient information to reconstruct the original fingerprint image from which minutiae were extracted. However, recent studies have shown that it is indeed possible to reconstruct fingerprint images from their minutiae representations. Reconstruction techniques demonstrate the need for securing fingerprint templates, improving the template interoperability, and improving fingerprint synthesis. But, there is still a large gap between the matching performance obtained from original fingerprint images and their corresponding reconstructed fingerprint images. In this paper, the prior knowledge about fingerprint ridge structures is encoded in terms of orientation patch and continuous phase patch dictionaries to improve the fingerprint reconstruction. The orientation patch dictionary is used to reconstruct the orientation field from minutiae, while the continuous phase patch dictionary is used to reconstruct the ridge pattern. Experimental results on three public domain databases (FVC2002 DB1_A, FVC2002 DB2_A, and NIST SD4) demonstrate that the proposed reconstruction algorithm outperforms the state-of-the-art reconstruction algorithms in terms of both: 1) spurious minutiae and 2) matching performance with respect to type-I attack (matching the reconstructed fingerprint against the same impression from which minutiae set was extracted) and type-II attack (matching the reconstructed fingerprint against a different impression of the same finger).
Article
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively.
Conference Paper
Databases play an important role in evaluating the performance of fingerprint identification algorithms. But which can be used to test the interoperability? That is to say, few of databases can test the performance of an algorithm on images acquired by different sensors. In order to solve the problem, we create the FingerPass cross-device matching fingerprint database which consists of almost 80 thousand fingerprint images from 90 subjects on nine different fingerprint sensors. We take both technology type and interaction type into consideration when choosing the sensors, totally different from other databases. It can test the interoperability of an algorithm at both the sensor level and the sensor type level. So we can use the FingerPass to test the performance of a cross-device matching algorithm for sensors of a specific type or different types. We apply the Ver-iFinger fingerprint recognition algorithm on it, and the experimental results indicate that the FingerPass cross-device matching database is a challenge for fingerprint algorithms.