Hyperspectral Document Image Processing, Applications, Challenges and Future Prospects

Abstract

Automatic image analysis is a crucial component of many intelligent systems de
signed for high-level understanding of documents. Most document image under
standing systems are usually based on applying pattern recognition techniques
to conventional three channel RGB images. Airborne and satellite based macro
scale Hyperspectral Imaging (HSI) systems are well established for geosciences.
Recently, owing to advancements in imaging speed and reduced camera costs,
micro scale HSI systems are also gaining importance in ground based applica
tions such as hyperspectral document image analysis. HSI is non destructive and
offers new opportunities via measuring richer information along spectral dimen
sion by imaging the document in contiguous bands across the electromagnetic
spectrum. Hyperspectral document imaging has shown potential for solving
many challenging problems of document image analysis including signature ex
traction, ink or document aging, information retrieval from historical document
images, paintings and forensic analysis of documents. In this paper, we explore
the potential of HSI for document image analysis and present a comprehensive
review of the literature and future prospects. We highlight and discuss the
challenges involved in the acquisition and processing of hyperspectral document
images. A review of commercial HSI systems for document image analysis is
also presented.

Full text

Pattern Recognition 90 (2019) 12–22
Contents lists available at ScienceDirect
Pattern Recognition
journal homepage: www.elsevier.com/locate/patcog
Hyperspectral document image processing: Applications, challenges
and future prospects
Rizwan Qureshi
a , b
, Muhammad Uzair
c , ∗, Khurram Khurshid
b
, Hong Yan
a
a
Department of Electronic Engineering, City University of Hong Kong, Hong Kon g
b
Institute of Space Technology, Islamabad, Pakistan
c
Defense and Systems Institute, University of South Australia, Australia
a r t i c l e i n f o
Article history:
Received 16 November 2017
Revised 18 October 2018
Accepted 13 January 2019
Available online 15 January 2019
Keywo rds:
Hyperspectral document imaging
Signature extraction
Historical document image analysis
Cultural heritage
Ink mismatch detection
a b s t r a c t
Automatic image analysis is a crucial component of many intelligent systems designed for high-level un-
derstanding of documents. Most document image understanding systems are usually based on applying
pattern recognition techniques to conventional three channel RGB images. Airborne and satellite based
macro scale Hyperspectral Imaging (HSI) systems are well established for geosciences. Recently, owing
to advancements in imaging speed and reduced camera costs, micro scale HSI systems are also gain-
ing importance in ground based applications such as hyperspectral document image analysis. HSI is non
destructive and offers new opportunities via measuring richer information along spectral dimension by
imaging the document in contiguous bands across the electromagnetic spectrum. Hyperspectral docu-
ment imaging has shown potential for solving many challenging problems of document image analysis
including signature extraction, ink or document aging, information retrieval from historical document
images, paintings and forensic analysis of documents. In this paper, we explore the potential of HSI for
document image analysis and present a comprehensive review of the literature and future prospects. We
highlight and discuss the challenges involved in the acquisition and processing of hyperspectral document
images. A review of commercial HSI systems for document image analysis is also presented.
©2019 Published by Elsevier Ltd.
1. Introduction
Automatic document image analysis involves designing and ap-
plying pattern recognition and machine learning methods to con-
vert document images to symbolic form for modification, storage,
information retrieval, forensic analysis and high-level understand-
ing of documents [1–4] . Although document image analysis has
been in commercial use for a long time [5,6] , the research in this
field is still growning rapidly due to advancement in computer
hardware and new more efficient pattern recognition algorithms.
Some common tasks in document image analysis are text analysis
and recognition [7–10] , optical character recognition [11,12] , layout
analysis [13,14] , authentication/verification [15–17] and informa-
tion retrevial from historical documents [18] etc. Automatic docu-
ment image analysis has been successfully applied for automation
in may areas such as offices, libraries, banks, retail, forensics and
investigation [19] .
∗Corresponding author.
E-mail addresses: muhammad.uzair@unisa.edu.au (M. Uzair),
khurram.khurshid@ist.edu.pk (K. Khurshid), h.yan@cityu.edu.hk (H. Yan ).
Pattern recognition is one of the key tools used in automatic
document image analysis. Methods based on various pattern recog-
nition techniques such as neural networks, support vector ma-
chines, fuzzy logic, hidden Markov models, evolutionary algo-
rithms, adaptive learning methods and deep learning have been
successfully applied to solving different document image analy-
sis problems [12,20–24] . The majority of state-of-the-art pattern
recognition techniques for document image analysis are designed
to work with three channel RGB images.
Conventional imaging devices such as digital cameras and scan-
ners are usually developed to match the tri-chromatic nature of
human visual system. Similarly, most computer vision applications
also directly employ the gray scale or RGB images for image under-
standing. Typically, RGB images constitute three wavelength mea-
surements per pixel in the visible region (40 0–70 0 nm). In con-
trast, images acquired with more than three wavelengths per pixel
covering a broad range of the electromagnetic spectrum are re-
ferred to as multispectral images ( Fig. 1 ). Multispectral images con-
tain much richer spectral information than RGB. Machine vision
systems can also be designed to take advantage of multispectral
imaging for more effective f eature extraction, classification, analy-
sis and recognition.
https://doi.org/10.1016/j.patcog.2019.01.026
0031-3203/© 2019 Published by Elsevier Ltd.

R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22 13
Tabl e 1
Types of images and their application areas.
Image Type Bands Applications
Binary 1 Quality control [47–49]
Morphologcial processing [50–52]
RGB 3 Computer vision [53–56]
Medical imaging analysis [57–59]
Multispectral 4 to 10 Food/Agriculture research [60–62]
Pharmaceutics [63,64]
Geology/Atmosphere [65,66]
Hyperspectral 11 to several hundred Materials characterization [67,68]
Environment [69] , Transportation [70]
Remote Sensing/Geo Science [71–73]
Document image analysis [16,74–77]
Biometrics [30,78–81] , [28]
Fig. 1. An example of a multiperspectral document image shown as a series of im-
ages (bands) along the spectral dimension [42] .
A hyperspectral image has finer spectral resolution or higher
number of bands compared to RGB and multispectral. Hyperspec-
tral image may have dozens to several hundreds of spectral bands
compared to the few spectral bands in multispectral image [25] .
Hyperspectral Imaging (HSI) systems for remote sensing applica-
tions are well established [26] . Examples include satellite image
processing for earth observation [27] and astrophysical image pro-
cessing [28] .
During the last few decades, hyperspectral imaging has found
its utility in ground based computer vision applications such as bi-
ological and medical image analysis [29] , face recognition [30,31] ,
chemistry [32,33] , agriculture [34] and archaeology [28,35] . Simi-
larly HSI has been applied to differentiate between visually similar
inks [16] , pen verification [36] , document or ink aging [28] , preser-
vation of documentation [37,38] , and material identification [39–
41] . Table 1 summarizes different categories of images based on
number of spectral bands and their applications.
Despite the success of HSI for solving various challenging prob-
lems of remote sensing and computer vision, analysis of docu-
ment images captured using HSI has not received much atten-
tion [16,43,44] . HSI has the potential of solving many challeng-
ing problems of document image analysis by exploiting the richer
spectral information [45] . For example, signature segmentation can
be performed by simply differentiating between the spectral re-
sponse of the signature and that of the background [46] . Similarly,
analysis of ink spectra, ink aging, document aging based on HSI
spectral signature can be a key factor in determining forgery, back-
dating and historical document analysis.
In this article, we explore the potential of HSI for document im-
age analysis. First, we discuss acquisition and pre-processing tech-
niques and challenges of hyperspectral document images. Next, we
present a detailed literature review of the pattern recognition tech-
niques used for hyperspectral document image analysis. We also
discuss the currently available commercial hyperspectral document
image processing systems. Finally, we present interesting new re-
search directions in the area of hyperspectral document image pro-
cessing.
2. Hyperspectral document imaging acquisition and challenges
In this section, we briefly discuss the acquisition hardware
and challenges of hyperspectral document imaging. Hyperspectral
imaging is used to collect spectral data in the form of images
(bands) over a wide and continuous wavelength range with a large
number of discrete wavelength bands. Ground based hyperspectral
image acquisition systems are usually built with the help of op-
tical devices such as bandpass spectral filters, image sensors and
lenses. The filters are electronically controlled and their pass band
can be tuned at a very high speed [82,83] . Such type of filters are
preferred to use in time constraint applications.
Barry et al. [84] presented a typical setup for hyperspectral doc-
ument imaging using hyperspectral sensors coupled with off the
shelf optical devices. Similarly, Khan et al. [16] used Liquid Crystal
Tunable Filter (LCTF), monochromatic CCD sensor, programmable
hardware and illumination source such as halogen lamps (see
Fig. 2 ). A control software is used to tune the filter to a desired
wavelength region and then allowing the incoming light to reach
the CCD. The bandwidth of LCTF is a function of its center wave-
length as shown in Fig. 3 .
There are several challenges in acquiring high quality hyper-
spectral document images. These are mainly related to the image
acquisition time, illumination direction and power as well as the
signal to noise ratio in certain wavelength bands. For example, the
transmittance response of a commonly used LCTF filter is shown
in Fig 3 . It can be observed that the amount of transmitted light
varies for different wavelengths λ. Light with small values of λ
[400–450 nm] capture insufficient amount of energy. To overcome
this issue, exposure time was increased by Luo et al. [16,75] as an
inverse function of wavelength λ, such that longer the wavelength,
the shorter the exposure time and vice-versa ( Fig. 3 ) according to
t (λ) = a (T
max
−T
λ) + t
E
, where T
max is the maximum filter trans-
mittance corresponding to 700 nm in this case. t
E is the exposure
time that ensures no image saturation and αis the balancing co-
efficient.

14 R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22
Fig. 2. A hyperspectral document imaging system [16] consisting of a monochrome
camera coupled with liquid crystal tuable filter (LCTF) to capture images at discrete
wavelengths. The document is illuminated by two halogen lamps for compensation
of energy at low wavelengths.
Illumination hardware and its power also affect the captured
image. Nonuniform illumination pattern can be observed in the
image if the document is not illuminated properly. For example,
Zohaib et al. [16] used high power halogen lamps and concentrated
most of the illumination on the center of the document. Therefore
a circular illumination pattern can be seen in the acquired bands
( Fig. 5 ).
3. Pre-processing of hyperspectral document images
Pre-processing of images is performed to reduce noise, correct
illumination effects and enhance relevant information [85] . Differ-
ent image processing and pattern recognition techniques can be
used for this purpose. In this section, we highlight some common
pre-processing challenges of HSI.
3.1. Spikes removal
Spikes are sudden rise followed by a sharp fall in the ob-
served energy in a local region of a band. Spikes are generated
due to abrupt behavior of instrument, environment or imperfec-
tion of electronic circuitry. They often mask the detail in an image
and lead to inacurate analysis. One of the choices to detect spikes
is manual supervision, however, this requires human care and is
time consuming for hyperspectral data. Interpolation or removal
of spikes based on the neighboring pixel [86,87] is a good choice
for removal of spikes. Other solutions include median and median
modified wiener filters [88] .
3.2. Dead pixel removal
Another pre-processing task involves the detection and removal
of dead pixels which are pixels replaced by zero or maximum
value. Dead pixels are caused by anomalies in the detector. The
location and size of dead pixels varies between specific regions,
group of pixels or a complete pixel line. Thresholding is a sim-
ple choice but involves tuning the threshold parameters manually.
Once dead pixels are located, the best choice is to interpolate them
with neighboring pixels. The non-informative background or out-
liers (not consistent with rest of the data set) further complicates
the information extraction [79] . These aberrant observations can be
generated due to three reasons; namely, the instrument, the geom-
etry of sample and the geometry of illumination.
To overcome these challenges, classical image pre-processing
techniques like histogram analysis to detect the sharp changes in
an image or manual selection of a threshold value can be used
to clean up and extract the relevant information from the hy-
perspectral cube. Spectral pre-processing is required for avoiding
the undesirable phenomena like scattering of light, particle size
affects or morphological differences affecting the spectral mea-
surement. Variety of pre-processing techniques such as de-noising,
Multiplicative Scatter Correction (MSC) and Standard Normal Vari-
ate (SNV) [89,90] can be used in spectral pre-processing.
3.3. Compression
A hyperspectral document image contains thousands or even
million of pixels, e.g, an 8-bit HSI having 200 bands contains more
than 13 million data points. This huge amount of information re-
quires much memory and transmission time [91] . In many appli-
cations storage is also still an important issue [92] . Therefore com-
pression of an image is often necessary to retain the desired infor-
mation [93] . Most common ways of compressing images are byte
encoding and data binning. Pattern recognition algorithms such
Principle Component Analysis (PCA) and multivariate curve anal-
ysis can be used to significantly reduce the spatio-spectral dimen-
sionality of hyperspectral data.
Fig. 3. Transmittance function of a Liquid Crystal Tunable Filter, (LCTF)(left diagram). At shorter wavelengths, insufficient energy is transmitted which requires some com-
pensation. In the right diagram, average exposure time (blue) is greater than constant exposure time (red) at shorter wavelengths for compensation of insufficient energy.
The diagram is adopted from Luo et al. [75] . (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.).

R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22 15
Fig. 4. Signature variations in the dataset of [46] . (a) No Overlap (b) Partially over-
lap with text (c) Complete overlap with text.
4. Pattern recognition techniques used in hyperspectral
document image analysis
A hyperspectral document image is in the form of a data cube
having two spatial and one spectral dimensions. Spectral dimen-
sion has series of images captured at different wavelengths. The
richer spectral information can be exploited through pattern recog-
nition algorithm to design intelligent machine vision based docu-
ment image analysis applications. In this section, we explore some
applications and pattern recognition algorithms for hyperspectral
document image analysis. We highlight the advantages of HSI for
solving some challenging problems of document image analysis
such as signature segmentation and extraction, ink mismatch de-
tection, historical document image analysis and cultural heritage
document analysis.
4.1. Signature segmentation
Signatures are one of the most common ways of authenticat-
ing a document [94,95] . Financial institutions use signatures for
verifying personal identities in financial and administrative trans-
actions [96] . The use of signature as an authenticating source in
daily life justifies the necessity of a fully automated signature veri-
fication system [97,98] . Handwritten signature verification and seg-
mentation is a challenging problem due to the free flow nature of
human writing and overlapping of signature with logos and back-
ground text.
In order to perform verification, it is first required to segment
the signature [43,99,100] . If segmentation is done accurately, the
accuracy of signature verification will increase. In real time appli-
cations, e.g, bank cheques, contracts, invoices, bill payments, sig-
natures are available with other diverse information, such as text,
tables and logos. Signature analysis begins with document bina-
rization as first processing step [101,102] . An evaluation of various
binarization methods is presented by Trier and Taxt [103] . Signa-
tures normally found at the end of the page often intersect with
other factors like rubber stamp, logos or background printing. In
this scenario, beginning and ending features of the letters, line di-
rections, dots and other fine detail may be lost due to the noise of
intersecting ink. Chaikovsky et al. [104] exploited the potential of
color separation using advanced photography techniques to facili-
tate the examination of questioned signature analysis. Hyperspec-
tral unmixing techniques were also epxlored for automatic extrac-
tion of signature from document images [105] .
Malik et al. [46] performed a detailed study of signature anal-
ysis using hyperspectral images. They presented a dataset consist-
ing of 300 invoices of handwritten signatures captured using a hy-
perspectral camera in the spectral range of 40 0–90 0nm with 240
spectral bands, 640 spatial bands and a high resolution of 2.1 nm.
Signatures were collected with different types of blue, black and
gel ink pens. To evaluate the performance, the dataset was divided
into 30 training and 270 testing patches with overlapping, no over
lapping and partial overlapping as shown in Fig. 4 . It was observed
that the spectral response of printed text and background was al-
most same, however the spectral response of the signature var-
ied significantly specially in the near infra red region. This pro-
vided a base for signature extraction. They applied speed up robust
features (SURF) on the extracted signatures for verification. How-
ever, their algorithm was not evaluated for signature verification
on other publicly available hyperspectral datasets such as Tobacco
800 [106] and Maryland Arabic [107] .
An efficient system was proposed by Butt et al. [77] by sepa-
rating the printed text and signature using part based features and
connected component analysis and then hyperspectral information
is used to extract the signature pixels. The system proposed by
Butt et al. [77] achieves precision and recall rates of 10 0 and 84%
respectively, which is higher than [46] . However, these algorithms
were evaluated on small scale databases. In order to assess the true
potential of hyperspectral signature segmentation integrable with
signature verification systems, these methods should be evaluated
on new real world large scale benchmark datasets. Once signatures
are extracted, any state of the art signature verification system can
be applied, as proposed by Leclerc and Plamondon [99] for authen-
tication of document and a fully automated signature verification
system can be realized.
4.2. Forgery detection for forensic analysis
Hyperspectral imaging is a relatively new analytical tool for
forensic sciences. For checking the authenticity of any document,
there can be two possible ways: destructive examination and non
destructive examination. Chemical analysis such as thin layer chro-
matography (TLC) is a destructive test, which separates the inks
into its constituents. This approach compromises the originality
of the sample for forensic analysis, moreover analysis of chro-
matograph consumes considerable time. The second approach is
to apply hyperspectral imaging to differentiate among visually
similar colors which is non-invasive and non-destructive. Chlebda
et al. [108] , used hyperspectral document imaging to distinguish
between modern black gel ink pens. The proposed scheme suc-
cessfully classified different gel inks into several groups in a non-
invasive manner. However, the experiments were performed on a
limited database in a controlled manner where the query pens
were assumed to be in the training data. Similarly, study in
[109] Raman imaging and chemometrics in used to solve forensic
document examination
Detecting forgery or outliers is a key factor in forensic docu-
ment image analysis [110,111 ] . Hyperspectral images coupled with
chemometric techniques have been previously explored for this
purpose. For example, Silva et al. [112] employed HSI to address
three challenges in forensic document image analysis for forgery
detection. The three forgeries include obliterating text, adding text
and approaching the cross line problem. Document samples were
imaged in the range of 928–2524 nm with spectral and spatial res-
olution of 6.3 nm and 10 μm respectively. Different pre-processing
techniques were evaluated and the best result was obtained using
Standard Normal Variate method [89] . The correct identification
rates were 43%, 82% and 85% for obliteration text, forgery and cross
line problems, respectively. Although not all samples were identi-
fied, HSI coupled with chemometrics showed potential for docu-
ment forgery identification. Pereira et al. [113] evaluated the per-
formance of hyperspectral imaging in Near Infrared (NIR and Mid-
dle Infrared (MIR) to detect the document falsification by means
of addition or alteration. Similarly, Tahouth et al. [114] used HSI in
the NIR region to detect forgery in documents. They employed fine
resolution spectroscopy for improving the localization and acquisi-
tion of fingerprints. Chen et al. [115] explored HSI for identification
of fingerprints on documents.
4.2.1. Ink mismatch detection
Analysis of ink can also be a key factor in detecting forgery,
since different inks exhibit different spectral response [116–120] .

16 R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22
Fig. 5. An RGB Document image with a flatbed scanner (left). Bands from a hyperspectral document image at specified wavelengths in the database of [16] .
Fig. 6. Hyperspectral image from Pereira et al. [123] showing different visibly sim-
ilar inks in different directions of a manuscript.
However the accuracy of any image processing approach is de-
pendent upon several factors [121] such as image quality, small
color differences and visually similar inks. A detailed study about
the methods used for identification and characterization of inks is
done by Chen et al. [118] . Spectral response of inks can be used to
detect forgery or fraudulent document, based on the assumption
that forgery is done with a different ink or pen. This leads to the
term of ink mismatch detection.
For ink mismatch detection, Braun et al. [27] proposed Fourier
transform based HSI system to detect forgery. They used fuzzy
clustering to group the similar ink spectra. The experiments show
that inks can be qualitatively segmented into two clusters. One
limitation of the system was the lack of quantitative information
and slow imaging process. A visual comparison of black inks done
by Hammond et al. [122] using multispectral document imaging.
George et al. [123] used HSI for visual enhancement of docu-
ments by separating the text written using two different inks in
two directions as shown in Fig. 6 . Khan et al. [16,124] proposed
a method based on quantitative analysis. Their work was solely
based on analyzing the spectral responses of different inks. They
used joint sparse band selection algorithm to find the subset of
bands that maximizes the mismatch accuracy [50,125] . They pre-
pared a database of 70 hyperspectral images of hand written notes
by seven different subjects. All subjects were directed to write the
sentence, “The quick brown fox jumps over the lazy dog” as shown
in Fig 5 . It was ensured that the pen came from different compa-
nies, while the inks look visibly similar. Moreover, all the samples
were collected at the same time to have the same aging effect. This
is the only publicly available data set on hyperspectral document
images [126] .
Furthermore, Khan et al. [16] proposed forward band selec-
tion algorithm for automatic mismatch detection in visually sim-
ilar inks based on ink spectral response. However, one drawback
of their work was the number of ink to be distinguished is to be
given in advance and their relative proportion in inspected note is
roughly equal. Similarly, Luo et al. [75] proposed a system for lo-
calized forgery detection using anomaly detection algorithm com-
bined with unsupervised learning to handle the cases where the
pixels belonging to different classes are highly unbalanced. In the
paper [127] , an efficient automatic ink mismatch detection tech-
nique is proposed which uses Fuzzy C-Means Clustering (FCM) to
divide the spectral responses of ink pixels in handwritten notes
into different clusters which relate to the unique inks used in the
document. The proposed method gives a clear discrimination be-
tween inks in questioned documents.
Aythami et al. [36] proposed a system to detect forgeries in
hand written documents particularly for bank cheques based on
ink discrimination. They proposed an approach for ink analysis
in handwritten documents and pen verification using hyperspec-
tral imaging and support vector machine classifier. Experiments for
this automated ink analysis system were performed using 25 dif-
ferent pens, including viscous pens, gel ink pens, liquid ink pens
and marker pens. They achieved 87.5 ink discrimination results on
10 0 0 samples.
In future, it would be interesting to see whether HSI can aid to
writer identification [128] . Since hand writings can be identified
using ink decomposition traces [74] and texture [129] . Ink mis-
match detection also finds its application in historical document
analysis [130] .
4.3. Historical document analysis
The majority of work for historical document image analysis is
based on gray scale, three channel RGB or fused images [27,131] .
HSI is relatively a new technique for processing of historical doc-
uments, libraries and archives [27,39,45,122,132,133] . The major
work in this direction was initially done by National Archives of
Netherlands (NAN). NAN in collaboration with Art Innovation BV
developed an instrument named SEPIA [134] ( Fig. 7 ). SEPIA is a
quantitative hyperspectral imaging tool that provides both spa-
tial and spectral information of archived documents such as pa-
per, leather, inks and pigments. The SEPIA operates between 365–
110 0 nm with a step size of 10, thus capturing 70 hyperspectral
images [74] . This system is used for various purposes including
monitoring assessment of archives [28] , ink aging of historical doc-
uments [71] . An interactive tool for the visualization of historical
documents is developed by Kim et al. [135] with the NAN col-
laboration. The visualization tool offers an assortment of analysis
methods like spectral selection, spectral similarity analysis, time
varying data analysis and selective spectral band fusion.
Ciortan et al. [136] exploited the potential of HSI for the leg-
ibility of historical documents. Padodan et al. [28] exploited the
potential of quantitative hyperspectral imaging for monitoring the
aging of documents, aging of inks, biological and physical damages
in old documents. Kim et al. [76] used invisible spectra for visual
enhancement of old documents. They used NIR bands for visual

R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22 17
Fig. 7. SEPIA Quantitative hyperspectral imaging device for historical document im-
age analysis [135] .
enhancement of text documents having low quality or corrupted
with noise. Their framework was based on detecting the low con-
trast regions in a document that lies near the NIR spectra.
Apart from HSI, a lot of research on multispectral imaging (MSI)
for processing of historical document images has been done. Fabian
Hellas et al. [137] employed multispectral imaging for binarization
of historical document images. They applied state of the art bina-
rization algorithm on a single image from a series of multispectral
images. The output is combined with the output of adaptive detec-
tion method. Experimental results showed that the combination of
algorithms lead to increase in performance.
Hedjam et al. [18] used MSI for restoration of ancient docu-
ment images. They proposed a mathematical model based on the
additional information provided by the invisible spectrum and as
well as the physical properties of the ink. Document images of-
ten suffer from degradation task [102] that makes the information
retrieval process difficult. The proposed model transformed the de-
graded image into a clean version, the degradation was isolated in
the infrared spectrum and eliminated in the visible spectrum. Their
model achieved promising results and can be extended with some
modification to build an intelligent system for ancient document
processing.
Due to the growing interest of research community in mul-
tispectral images for historical document processing, recently, a
competition for text extraction from historical document images
using multispectral imaging was held for the first time in ICDAR
2015 [138] . The goal of the competition was to evaluate the per-
formance of state of the art algorithm for text extraction from an-
cient document images based on multispectral imaging. Various
research group from across the world participated in the competi-
tion. The method submitted by Vienna University won the compe-
tition [138] . The proposed algorithm was the combination of three
steps. The first step provides a rough estimation of foreground by
thresholding a cleaned channel. In the second step, an adaptive co-
herence estimator is trained on the rough foreground images. In
the last step, the cleaned image is combined with the mean and
standard deviation images and a grabcut [139] is applied.
4.4. Cultural heritage/ Paintings analysis
The study of cultural heritage in paintings, museums and
libraries is an interdisciplinary problem. Development of non-
invasive tools have advanced the analysis of cultural heritage mate-
rial. In most situations, ink undergoes degradation, parchment and
scrolls are affected by missing or overwritten subscripts [140] . The
adaptation of hyperspectral imaging has been a great benefit in
this direction [110,141] . Spectral imaging enables the identification
and analysis of materials based on their unique spectral signature.
These spectral imaging systems offers the chemical identification
information about materials without sampling, which is a crucial
factor for cultural heritage material [142] . Hyperspectral is useful
tool for this challenging problem [143,144] .
Past projects like Archimedes Palimpsest, a thousand year old
manuscript, written on parchment and Dead sea scrolls proves
the utility of hyperspectral imaging to identify inks that enhances
the content of the ancient manuscript [140] . Spectral images cap-
tured from Archimedes Palimpsest enabled to see the underwrit-
ten text and better understanding of material properties. Recently,
multispectral imaging also found its application in the analysis of
Archimedes Palimpsest, Salerno et al. [145] reported promising re-
sults by applying statistical techniques to the multispectral im-
ages of Archimedes Palimpsest. They worked for the extraction of
faint and highly degraded text which constitutes the most ancient
source for several treaties by Archimedes. Statistical techniques
are a useful tool for extracting the information from hyperspec-
tral cube [146,147] . They [145] applied principal component analy-
sis and independent component analysis on 14 hyperspectral views
of the palimpsest. In most of the cases, they successfully extracted
the clean map of the Archimedes, the overwritten text and mold
pattern present in the pages, however in some cases the desired
result was not achieved because of the non perfect adherence of
the data model to reality.
A system was developed at National Archives of Netherlands
(NAN) by Padodan et al. [71] for analysis of historical docu-
ments. Giacometti et al. [153] proposed a model for conservation
of cultural heritage documents based on multispectral imaging,
multivariate analysis and statistical process control theory. They
achieved satisfactorily results and the proposed model accurately
detected the degradation process.
Goltz et al. [151] used HSI for enhancing the assessment of vi-
sual properties of stains on documents. These included old doc-
uments that were in frequent use and had signs of stains. They
employed the imaging software (ENVI) to quantitatively assess the
extent of staining in two different types of documents. Letner
et al. [154] proposed a model for segmenting characters of an-
cient document images. They combined the multispectral features
and contextual spatial information. They compared the results
with traditional segmentation methods, e.g, thresholding, cluster-
ing of spectral features. Their model detected low contrast features,
where others models failed.
Ink bleeding is another serious problem that effects the legi-
bility of old documents. Marango et al. [155] proposed a model
for deterioration of parchment. They degraded a manuscript us-
ing physical and chemical agents. Samples were collected before
and after degradation using a multispectral sensor to record the
effects on both writing and parchment [156] . Lu [157] addressed
this challenge using a directed assistance approach. According to
their method, initial mark up is to be provided by the user. After
the initial mark up, the image is classified and regions with low
confidence are grouped and displayed to the user for another iter-
ation of mark up. The key idea is to direct where the mark up is
needed.
4.5. Summary
In Table 2 , we have summarized the major pattern recogni-
tion techniques for hyperspectral document image analysis accord-
ing to the application area. A summary of the datasets available

18 R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22
Tabl e 2
Literature on Hyperspectral document image analysis
Ref Application Spectrum Bands Algorithm Database
[75] Localized forgery detection 400–720 nm 33 Unsupervised clustering UWA
a
[36] Ink/Pen verification 40 0–110 0 nm 1024 SVM Private
[43] Ink mismatch detection 400–720 nm 33 Band selection UWA
[16] Ink mismatch detection 400–720 nm 33 Forward band selection UWA
[46] Signature extraction 40 0–90 0 nm 240 SURF Features Private
[77] Signature segmentation 40 0–90 0 nm 70 Gradient map of NIR Private
[76] Old documents enhancement 365–1100 nm 70 Gradient map of NIR NA N
b
[148] Old documents enhancement 365–1100 nm 70 Gradient map of NIR NAN
[149] Gel pens forensic analysis 40 0–10 0 0 nm 121 Comparison of spectra Freeman
[78] Historical documents analysis 365–1100 nm 70 NAN
[71] Aging process monitoring 365–1100 nm 70 Quantitative analysis NAN
[27] Forensic documents analysis 400–850
nm −FFT/Clustering Private
[150] Forensic ink trace analysis 40 0–150 0 nm −Private
[113] Text alteration analysis 40 0 0–750cm −PCA/PP Private
[112] Text obliteration analysis 925–2524 nm - PCA/MCR/ALS Private
[135] Historical doc. visualization 365–1100 nm 70 + Image Fusion NAN
[123] Historical ink classification 40 0–10 0
0 nm 16 0 Spectral Angle Mapper NTNU
c
[151] Historical doc. stain analysis 420–720 nm 30 Imaging Software ENVI Private
[148] Art work authentication 900–3750 nm 121 Spectral analysis Freeman
[145] Text enhancement in Archimedes palimpsest 420–720 nm 30 PCA & ICA Private
a National Archives of Netherlands (NAN)
b University of Western Australia (UWA)
c Norwegian Technical National University (NTNU)
Tabl e 3
Hyperspectral document image databases.
Ref. Application Spectral Range Bands Database
[126] Ink mismatch detection 400–720 nm 33 UWA
[135] Historical document Analysis 365–1100 nm 70 + NAN
[136] Ink classification 40 0–110 0 nm 160 NTNU
[46] Signature extraction and analysis 2.1 nm 240 Private
[152] Forensic document analysis 930–2520 nm 159 Private
Tabl e 4
Commercial hyperspectral document imaging systems.
System Spectral range Spectral resolution Application Software tool
ChemImage [158] 40 0–110 0 nm Upto 1 nm Document Analysis HSI200QD
Ink Aging/Material Identification
Free & Foster [42] 40 0–10 0 0 nm Upto 1 nm Forensic Examination of Documents, VSC 80 0
ID, Bank notes
SEPIA [135] 365–1100 nm Upto 10 nm Netherlands Cultural Heritage Proprietary
Historical Document Analysis Proprietary
Headwall [159] 250–2500 nm Upto 1 nm Machine Vision, Document Forensics Hyperspec III
ForensicXP [160] 350–1100 nm Currency Notes, Cheques, Passport Analysis Proprietary
SurfaceOptics [161] 40 0–170 0 nm 2.31 nm Pigment Identification, Forgery Detection MATLAB, ENVI
for hyperspectral document image analysis research is provided in
Table 3 .
5. Commercial hyperspectral imaging systems
Some of the commercial hyperspectral imaging systems are
listed in Table 4 . A well known hyperspectral document image
analysis system was developed by Foster and Freeman [42] . Instru-
ment VSC 80 0 0 has been recognized as premier imaging system
for forensic examination of questioned documents. This system is
used for authentication of passports, bank notes, aging of inks and
examination of fraudulent document. The HSI Examiner [158] is a
hyperspectral imaging instrumentation and software for chemical
and biological applications, including forensic document analysis,
pharmaceutical testing, threat detection and biomedical imaging.
The Headwall System [159] provides commercial products for de-
fense and security, biotechnology, machine vision, document foren-
sics and remote sensing. Similarly, System developed by National
Archives of Netherlands (NAN) is used for analysis of historical
document.
6. Future research directions
Hyperspectral document image analysis is a relatively new area
of research. Interesting future directions include exploring new
more efficient hyperspectral document imaging systems, collecting
large scale hyperspectral document image databases, and develop-
ing new machine learning algorithms for this purpose.
Although different commercial hyperspectral document imaging
systems ( Table 4 ) are available, their cost is still relatively high.
Therefore, the development of low cost, small and portable hyper-
spectral document imaging systems is an interesting research di-
rection. This involves designing new hardware such as spectral fil-
ters, CCD sensors and illumination sources targeted to documents
only. For example, methods such as the ones in [162] involving re-
constructing of hyperspectral images by using multiple low cost
cameras can be explored.
New pattern recognition techniques can be explored for more
accurate understanding of the information encoded in high di-
mensional hyperspectral document cubes. Deep learning has re-
cently achieved remarkable results in many image processing

R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22 19
areas. It has shown promising results for detection and classifi-
cation of hyperspectral data [163] . However, deep learning based
algorithms require large training datasets but most existing hy-
perspectral document image databases ( Table 3 ) are small in size
and limited to a specific application. Therefore, the design of hard-
ware, software and acquisition protocols for the collection of large
scale hyperspectral document image databases is an important re-
search direction. Once sufficient data is available, the next question
would be the design of new convolutional neural network archi-
tectures and training methodologies specifically tailored for pro-
cessing the 100s of channels of hyperspectral document images.
Meanwhile, new data augmentation techniques can be explored
to enhance the existing hyperspectral document image datasets.
Moreover, new transfer learning [164] and domain adaptation al-
gorithms are needed for leveraging the current big RGB image
datasets for the purpose of hyperspectral document image analy-
sis. One such strategy is to simply use the pre-trained RGB models
as feature extractors for the hyperspectral document images.
In future it will be interesting to apply hyperspectral document
imaging to solve new problems. For example, it will be interest-
ing to see whether HSI can be useful for the tasks such as writer
identification using ink deposition traces [74] and texture anal-
ysis [129] . Moreover, hyperspectral imaging based ink aging and
document aging as well as document material classification are
also open areas of research.
7. Conclusion
Hyperspectral imaging has the potential of becoming a key
technology for document image processing. HSI has recently been
successfully applied to solve several challenging document analysis
problems such as signature extraction, ink or document aging, in-
formation retrieval from historical document images, paintings and
forensic analysis of documents. In this paper, we provided a de-
tailed discussion of hyperspectral document image processing. We
discussed the challenges in acquiring and pre-processing of hyper-
spectral document images and provided a literature review of the
pattern recognition techniques involved to solve these problems.
Moreover, we have highlighted some of the commercial hyperspec-
tral imaging systems with special focus on the analysis of docu-
ment image. We believe that this survey will provide useful infor-
mation to researchers and forensic scientists about HSI technology
for solving challenging problems of document image analysis.
References
[1] R. Kasturi , L. O Gorma , V. Govinadaraju , Document image analysis: a primer,
Sadhana 27 (1) (2002) 3–22 .
[2] A. Harley , A. Ufkes , K. Derpanis , Evaluation of deep convolutional nets for
document image classification and retrieval, in: Proceedings of the Interna-
tional Conference on Document Analysis and Recognition, 2015, pp. 991–995 .
[3] A.P. Giotis , G. Sfikas , B. Gatos , C. Nikou , A survey of document image word
spotting techniques, Pattern Recognit. 68 (2017) 310–332 .
[4] S. Eskenazi , P. Gomez-Krämer , J.-M. Ogier , A comprehensive survey of mostly
textual
document segmentation algorithms since 2008, Pattern Recognit. 64
(2017) 1–14 .
[5] L. O’Gorman , R. Kasturi , Document Image Analysis, 39, IEEE Computer Society
Press Los Alamitos, 1995 .
[6] G. Nagy , Twenty years of document image analysis in pami, IEEE Trans. Pat-
tern Anal. Mach. Intell. 22 (1) (20 0 0) 38–62 .
[7] A. Khan , B. Baharudin , L.H. Lee , K. Khan , A review of machine learning al-
gorithms for text-documents classification, J. Adv. Inf. Technol. 1 (1) (2010)
4–20 .
[8] A.D. Le , M. Nakagawa , A system for recognizing online handwritten
mathe-
matical expressions by using improved structural analysis, Int. J. Document
Anal. Recognit. (IJDAR) 19 (4) (2016) 305–319 .
[9] S. He , L. Schomaker , Beyond ocr: Multi-faceted understanding of handwritten
document characteristics, Pattern Recognit. 63 (2017) 321–333 .
[10] A. Vinciarelli , A survey on off-line cursive word recognition, Pattern Recognit.
35 (7) (2002) 1433–1446 .
[11] C.-L. Liu , F. Yin , D.-H. Wan g , Q.-F. Wang , Online and offline handwritten chi-
nese character recognition: benchmarking on new databases, Pattern Recog-
nit. 46 (1) (2013) 155–162 .
[12] Y. LeCun , L. Bottou , Y. Bengio , P. Haffner , Gradient-based learning applied to
document recognition, Proc. IEEE 86 (11) (1998) 2278–2324 .
[13] R. Cattoni , T. Coianiz , S. Messelodi , C.M. Modena , Geometric layout analysis
techniques for document image understanding: a review, ITC-Irst 9703 (09)
(1998) .
[14] L. O’Gorman , The document spectrum for page layout analysis, IEEE Trans.
Pattern Anal. Mach. Intell. 15 (11) (1993) 1162–1173 .
[15] F. Daraee , S. Mozaffari , Watermarking in binary document images using frac-
tal codes, Pattern Recognit. Lett. 35 (2014) 12 0–129 .
[16] Z. Khan
, F. Shafait , A. Mian ,Automatic ink mismatch detection for forensic
document analysis, Pattern Recognit. 48 (11) (2015) 3615–3626 .
[17] F. Ahmad , L.M. Cheng , Authenticity and copyright verification of printed im-
ages, Signal Process. 148 (C) (2018) 322–335 .
[18] R. Hedjam , M. Cheriet , Historical document image restoration using multi-
spectral imaging system, Pattern Recognit. 46 (8) (2013) 2297–2312 .
[19] D. Doermann , J. Liang , H. Li , Progress in camera-based document image anal-
ysis, in: Proceedings of the ICDAR, 2003, pp. 606–616 .
[20] I. Ahmad , S.A. Mahmoud , G.A. Fink , Open-vocabulary recognition of ma-
chine-printed arabic text using hidden markov models, Pattern Recognit. 51
(2016) 97 – 111 .
[21] S. Lu , T. Chen , S. Tian , J.-H. Lim , C.-L. Tan , Scene text extraction based on
edges and support vector regression, Int. J. Document Anal. Recognit. (IJDAR)
18 (2) (2015) 125–135 .
[22] A. Rehman , T. Saba , Neural networks for document image preprocessing: state
of the art, Artif. Intell. Rev. 42 (2) (2014) 253–273 .
[23] R. Moghaddam , M. Cheriet , A multi-scale framework for adaptive binariza-
tion of degraded document images, Pattern Recognit. 43 (6) (2010) 2186–
2198 .
[24] E.R. Hruschka , R.J. Campello , A .A . Freitas , et al. , A survey of evolutionary al-
gorithms for clustering, IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 39
(2) (20 09) 133–155 .
[25] J.M. Amigo , H. Babamoradi , S. Elcoroaristizabal , Hyperspectral
image analysis.
a tutorial, Anal. Chimica Acta 896 (2015) 34–51 .
[26] M. Khan , H. Khan , K. Khurshid , A. Abbas , Modern trends in hyperspectral im-
age analysis: a review, IEEE Access 6 (2018) 14118–14129 .
[27] E. Brauns , B. Dyer , Fourier transform hyperspectral visible imaging and the
nondestructive analysis of potentially fraudulent documents, Appl. Spectrosc.
60 (8) (2006) 833–840 .
[28] B. Aalderink , M. Klein , R. Padodan , Quantitative hyperspectral imaging tech-
nique for condition assessment and monitoring of historical documents, in:
Proceedings of the 38th AIC Annual Meeting, 2010, pp. 1–8 .
[29] G. Lu , B. Fei , Medical hyperspectral imaging: a review, J. Biomed. Opt. 19 (1)
(2014) . 010901–010901.
[30] M. Uzair , A . Mahmood , A . Mian , Hyperspectral face recognition with spa-
tiospectral information fusion and Pls regression, IEEE Trans. Image Process.
24 (3) (2015) 1127–1137 .
[31] M. Uzair , A. Mahmood , A. Mian , Hyperspectral face recognition using 3D-dct
and partial least squares, in: Proceedings of the British Machine Vision Con-
ference, 2013, pp. 1–9 .
[32] M. Kamruzzaman , G. ElMasry , D.-W. Sun , P. Allen , Non-destructive prediction
and visualization
of chemical composition in lamb meat using Nir hyperspec-
tral imaging and multivariate regression, Innovat. Food Sci. Emerg. Technol. 16
(2012) 218–226 .
[33] J. Qin , R. Lu , Measurement of the absorption and scattering properties of tur-
bid liquid foods using hyperspectral imaging, Appl. Spectrosc. 61 (4) (2007)
388–396 .
[34] S. Mahesh , D. Jayas , J. Paliwal , N. White , Hyperspectral imaging to classify and
monitor quality of agricultural materials, J. Stored Prod. Res. 61 (2015) 17–26 .
[35] A. Braz , M. López-López , C. García-Ruiz , Raman spectroscopy for forensic anal-
ysis of inks
in questioned documents, Forens. Sci. Int. 232 (1) (2013) 206–212 .
[36] A. Morales , M. Ferrer , M. Diaz-Cabrera , G. Thomas , The use of hyperspectral
analysis for ink identification in handwritten documents, in: Proceedings of
the International Carnahan Conference on Security Technology, 2014, pp. 1–5 .
[37] F. France , et al. , Spectral imaging for preservation documentation, in: Proceed-
ings of the Archiving Conference, 2016, pp. 2–5 .
[38] H.Z. Nafchi , S.M. Ayatollahi , R.F. Moghaddam , M. Cheriet , An efficient ground
truthing tool for binarization of historical manuscripts, in: Proceedings of the
12th International Conference on Document Analysis and Recognition (IC-
DAR), IEEE, 2013, pp. 807–811 .
[39] S. Baronti , A. Casini , F. Lotti , S. Porcinai , Principal component analysis of visi-
ble and near-infrared multispectral images of works of art, Chemomet. Intell.
Lab. Syst. 39 (1) (1997) 103–114 .
[40] R.A. Schultz , T. Nielsen , J.R. Zavaleta , R. Ruch , R. Wyatt , H.R. Garner , Hyper-
spectral imaging: a novel approach for microscopic analysis, Cytometry 43
(4) (20 01) 239–247 .
[41] R. Shiradkar , L. Shen , G. Landon , S. Heng Ong , P. Tan , A
new perspective on
material classification and ink identification, in: Proceedings of the IEEE Com-
puter Vision and Pattern Recognition, 2014, pp. 2267–2274 .
[42] Foster and Freeman, ( www.fosterfreeman.com ).
[43] Z. Khan , F. Shafait , A. Mian , Hyperspectral document imaging: challenges and
perspectives, in: Proceedings of the International Workshop on Camera-Based
Document Analysis and Recognition, 2013, pp. 150–16 3 .
[44] K. Makantasis , K. Karantzalos , A. Doulamis , N. Doulamis , Deep supervised
learning for hyperspectral data classification through convolutional neural
networks, in: Proceedings of the IEEE International Geoscience and Remote
Sensing Symposium (IGARSS), IEEE, 2015, pp.
4 959–4 962 .

20 R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22
[45] C. Tan , R. Cao , P. Shen , Restoration of archival documents using a wavelet
technique, IEEE Trans. Pattern Anal. Mach. Intell. 24 (10) (2002) 1399–
140 4 .
[46] I. Malik , S. Ahmed , F. Shafait , A. Mian , C. Nansen , A. Dengel , M. Liwicki
, Hy-
per-spectral analysis for automatic signature extraction, in: Proceedings of the
International Graphonomics Society Conference, 2015, pp. 1–10 .
[47] A. Brink , J. Smit , M. Bulacu , L. Schomaker , Writer identification using di-
rectional ink-trace width measurements, Pattern Recognit. 45 (1) (2012)
162–171 .
[48] R. Stojanovic , P. Mitropulos , C. Koulamas , Y. Karayiannis , S. Koubias , G. Pa-
padopoulos , Real-time vision-based system for textile fabric inspection, Real
Time Imaging 7 (6) (2001) 507–518 .
[49] X. Wang , S. Ghosh , S.-W. Guo , Quantitative quality control in microarray
image processing and data acquisition, Nucleic Acids Res. 29 (15) (2001) .
e75–e75
[50] L.C. Molina , L. Belanche , À. Nebot , Feature selection algorithms: a survey and
experimental evaluation, in: Proceedings of the International Conference on
Data Mining, 2002, pp. 306–313 .
[51] N. Bouaynaya , M. Charif-Chefchaouni , D. Schonfeld , Theoretical foundations of
spatially-variant mathematical morphology part I: Binary images, IEEE Trans.
Pattern Anal. Mach. Intell. 30 (5) (2008) 823–836 .
[52] P. Soille , Morphological Image Analysis: Principles and Applications, Springer
Science & Business Media, 2013 .
[53] D.G. Lowe , Distinctive image features from scale-invariant keypoints, Int. J.
Comput. Vis. 60 (2) (2004) 91–110 .
[54] D.G. Lowe , Object recognition from local scale-invariant features, in:
Proceedings of the International Conference on Computer Vision, 1999,
pp. 1150–1157 .
[55] R. Girshick , J. Donahue , T. Darrell , J. Malik , Rich feature hierarchies for ac-
curate object detection and semantic segmentation, in: Proceedings of the
Computer Vision and Pattern Recognit., 2014, pp. 580–587 .
[56] A. Sharif Razavian , H. Azizpour , J. Sullivan , S. Carlsson , Cnn features of-
f-the-shelf: an astounding baseline for recognition, in: Proceedings of
the Computer Vision and Pattern Recognition Workshops, 2014, pp. 806–
813 .
[57] J.A . Maintz , M.A . Viergever , A survey of medical image registration, Med. Im-
age Anal. 2 (1) (1998) 1–36 .
[58] T. Heimann , H.-P. Meinzer , Statistical shape models for 3D medical image seg-
mentation: a review, Med. Image Anal. 13 (4)
(2009) 543–563 .
[59] S. Bauer , R. Wiest , L.-P. Nolte , M. Reyes , A survey of Mri-based medical image
analysis for brain tumor studies, Phys. Med. Biol. 58 (13) (2013) R97 .
[60] A. Gowen , C. O’Donnell , P. Cullen , G. Downey , J. Frias , Hyperspectral imag-
ing–an emerging process analytical tool for food quality and safety control,
Trends Food Sci. Technol. 18 (12) (2007) 590–598 .
[61] T. Lillesand , R.W. Kiefer , J. Chipman , Remote Sensing and Image Interpreta-
tion, John Wiley & Sons, 2014 .
[62] C. Liu , W. Liu
, X. Lu , W. Chen , J. Yang , L. Zheng , Nondestructive determination
of transgenic bacillus thuringiensis rice seeds (oryza sativa l.) using multi-
spectral imaging and chemometric methods, Food Chem. 153 (2014) 87–93 .
[63] Y. Roggo , A. Edmond , P. Chalus , M. Ulmschneider , Infrared hyperspectral
imaging for qualitative analysis of pharmaceutical solid forms, Anal. Chim.
Acta 535 (1) (2005) 79–87 .
[64] T. Rajalahti , O.M. Kvalheim , Multivariate data analysis in pharmaceutics: a tu-
torial review, Int. J. Pharmaceut. 417 (1) (2011) 280–290 .
[65] J. Vila , J. Calpe , F. Pla , L. Gómez , J. Connell , J. Marchant , J. Calleja ,
M. Mulqueen , J. Muñoz , A. Klaren , et al. , Smartspectra: applying multispectral
imaging to industrial environments, Real-Time Imaging 11 (2) (2005) 85–98 .
[66] X. Chen , T.A . Wa rner , D.J. Campagna , Integrating visible, near-infrared and
short-wave infrared hyperspectral and multispectral thermal imagery for ge-
ological mapping at cuprite, nevada, Remote Sens. Environ. 110 (3) (2007)
344–356 .
[67] D. Manolakis , G. Shaw , Detection algorithms for hyperspectral imaging appli-
cations, IEEE Signal Process. Mag. 19 (1) (2002) 29–43 .
[68] L. Randeberg
, E. Larsen , L. Svaasand , Characterization of vascular structures
and skin bruises using hyperspectral imaging, image analysis and diffusion
theory, J. Biophoton. 3 (1-2) (2010) 53–65 .
[69] T. Stuffler , K. Förster , S. Hofer , M. Leipold , B. Sang , H. Kaufmann , B. Penné,
A. Mueller , C. Chlebek , Hyperspectral imaging–an advanced instrument con-
cept for the Enmap mission (environmental mapping and analysis pro-
gramme), Acta Astronaut. 65 (7) (2009) 1107 – 1112 .
[70] R. Gomez , Hyperspectral imaging: a useful technology for transportation
analysis, Opt. Eng. 41 (9) (2002) 2137–2143 .
[71] R. Padoan
, M.E. Klein , G. de Bruin , B.J. Aalderink , T.A.G. Steemers , Monitoring
aging processes of archival documents by means of quantitative hyperspectral
imaging: a part of the hyperspectral project at the nationaal archief (national
archives of the netherlands), The Book and Paper Group Annual 28 (2009)
63–72 .
[72] F. Melgani , L. Bruzzone , Classification of hyperspectral remote sensing images
with support vector machines, IEEE Trans. Geosci. Remote Sens. 42 (8) (2004)
1778–1790 .
[73] Y. Chen , Z. Lin , X. Zhao , G. Wang , Y. Gu , Deep learning-based classification of
hyperspectral data, IEEE
J. Selected Top. Appl. Earth Observat. Remote Sens. 7
(6) (2014) 2094–2107 .
[74] K. Franke , O. Bunnemeyer , T. Sy , Ink texture analysis for writer identification,
in: Proceedings of the IEEE Conference on Frontiers in Handwriting Recogni-
tion, 20 02, pp. 268–273 .
[75] Z. Luo , F. Shafait , A. Mian , Localized forgery detection in hyperspectral docu-
ment images, in: Proceedings of the 13th International Conference on Docu-
ment Analysis and Recognition (ICDAR), IEEE, 2015, pp. 496–500 .
[76] S.J. Kim , F. Deng , M.S. Brown , Visual enhancement of old documents with
hyperspectral imaging, Pattern Recognit. 44 (7) (2011) 1461–1469 .
[77] U. Butt , S. Ahmad , F. Shafait , C. Nansen , A. Mian , M.I. Malik , Automatic signa-
ture segmentation using hyper-spectral imaging, in: Proceedings of the 15th
International Conference on Frontiers in Handwriting Recognition (ICFHR),
IEEE, 2016, pp. 19–24 .
[78] R. Padoan , T.A . Steemers , M. Klein , B. Aalderink , G. De Bruin , Quantitative hy-
perspectral imaging of historical documents: technique and applications, Art
Proc. (2008) 25–30 .
[79] M. Uzair , A. Mahmood , F. Shafait , C. Nansen , A. Mian , Is spectral reflectance
of the face a reliable biometric? Optics Exp. 23 (12) (2015) 15160–15173 .
[80] M. Uzair , A. Mahmood , A. Mian , C. McDonald , Periocular region-based person
identification in the visible, infrared and hyperspectral imagery, Neurocom-
puting 14 9 (2015) 854–867 .
[81] M. Uzair , A. Mahmood , S.A. Al-Maadeed , Non-cooperative and occluded
person identification using periocular region with visible, infra-red, and
hyperspectral imaging, in: Biometric Security and Privacy, Springer, 2017,
pp. 223–251 .
[82] B. Thai , G. Healey , Invariant subpixel material detection in hyperspectral im-
agery, IEEE Trans. Geos. Rem. Sens. 40 (3) (20 02) 599–608 .
[83] N. Gat , Imaging spectroscopy using tunable filters: a review, in: Proceedings
of the AeroSense, 20 0 0, pp. 50–64 .
[84] W. Christens-Barry , K. Boydston , F. France , K. Knox , R. Easton , M. Toth , Cam-
era system for
multispectral imaging of documents, Sensors Came. Syst. Ind.
Scient. Appl. X 7249 (2009) . 724908–724908
[85] M. Brislawn , B. Clodius , R. Harvey , M.D. Quirk , J. Theiler , Multispectral and
hyperspectral image processing, part 3: transforms, classification, and coding,
Encycloped. Opt. Eng. 2 (2003) 1421–1441 .
[86] C.J. Behrend , C.P. Tarnowski , M.D. Morris , Identification of outliers in hyper-
spectral raman image data by nearest neighbor comparison, Appl. Spectrosc.
56 (11) (2002) 1458–1461 .
[87] L. Zhang , M.J. Henson , A practical algorithm to remove cosmic spikes in ra-
man imaging data for pharmaceutical applications, Appl. Spectrosc. 61 (9)
(2007) 1015–1020 .
[88] C. Cannistraci , F. Montevecchi , M. Alessio , Median-modified wiener filter pro-
vides efficient denoising, preserving spot edge and morphology in 2-de image
processing, Proteomics 9 (21) (2009) 4 908–4 919 .
[89] ˚
A. Rinnan , F. van den Berg , S.B. Engelsen
, Review of the most common
pre-processing techniques for near-infrared spectra, TrAC Trends Anal. Chem.
28 (10) (2009) 1201–1222 .
[90] J. Burger , P. Geladi , Hyperspectral Nir image regression part i: calibration and
correction, J. Chemomet. 19 (5-7) (20 05) 355–363 .
[91] S. El-Rahman , Hyperspectral imaging classification using Isodata algorithm:
Big data challenge, in: Proceedings of the e-Learning, 2015, pp. 247–250 .
[92] M. Vidal , J.M. Amigo , Pre-processing of hyperspectral images. essential steps
before image analysis, Chemomet. Intell. Lab. Syst. 117 (2012) 138–148 .
[93] J.M. Amigo , Practical issues of hyperspectral imaging analysis of solid
dosage
forms, Anal. Bioanal. Chem. 398 (1) (2010) 93–109 .
[94] S. Ooi , A.J. Teoh , Y. Pang , B. Hiew , Image-based handwritten signature veri-
fication using hybrid methods of discrete radon transform, principal compo-
nent analysis and probabilistic neural network, Appl. Soft Comput. 40 (2016)
274–282 .
[95] S. Ahmed , M.I. Malik , M. Liwicki , A. Dengel , Signature segmentation from doc-
ument images, in: Proceedings of the International Conference on Frontiers in
Handwriting Recognition (ICFHR), IEEE, 2012, pp. 425–429 .
[96] R. Plamondon , G. Lorette , Automatic signature verification and writer identi-
fication–the state
of the art, PR 22 (2) (1989) 107–131 .
[97] D. Impedovo , G. Pirlo , Automatic signature verification: The state of the art,
IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38 (5) (2008) 609–635 .
[98] M.E. DeCoster , A.H. Firpi , S.K. Jacobs , S.R. Cone , N.H. Tzeng , B.M. Rodriguez ,
Spectral signature verification using statistical analysis and text mining, in:
Proceedings of the SPIE Defense+ Security, International Society for Optics
and Photonics, 2016 . 984025–984025.
[99] F. Leclerc , R. Plamondon , Automatic signature verification: the state of the
art–1989–1993, Int. J. Pattern Recognit. Artif. Intell. 8 (03) (1994) 643–
660 .
[100] B. Rabil , R. Sabourin , E. Granger , Impact of watermarking on offline signa-
ture verification in intelligent bio-watermarking systems, in: Proceedings of
the IEEE Workshop on Computational Intelligence in Biometrics and Identity
Management (CIBIM), IEEE, 2011, pp. 13–20 .
[101] N.R. Howe , Document binarization with automatic parameter tuning, Int. J.
Document Anal. Recognit. (IJDAR) 16 (3) (2013) 247–258 .
[102] S. Lu , B. Su , C.L. Tan , Document image binarization using background esti-
mation and stroke edges, Int. J. Document Anal. Recognit. 13 (4) (2010) 303–
314 .
[103] O. Trier , T. Taxt , Evaluation of binarization methods for document images,
IEEE Trans. Pattern Anal. Mach. Intell. 17 (3) (1995) 312–315 .
[104] A. Chaikovsky , S. Brown , L.S. David , A. Balman , A. Barzovski , Color separation
of signature and stamp inks to facilitate handwriting examination, J. Forensic
Sci. 48 (6) (2003) 1396–1405 .
[105] K. Iqbal , K. Khurshid , Automatic signature extraction from document images
using hyperspectral unmixing, A. Phys. Comput. Sci. 54 (3) (2017) 269–276 .

R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22 21
[106] The legacy tobacco document library LTDL, ( https://www.
industrydocumentslibrary.ucsf.edu/tobacco/ ).
[107] J.K. Guo , D. Doermann , A. Rosenfeld , Forgery detection by local correspon-
dence, Int. J. Pattern Recognit. Artif. Intell. 15 (04) (2001) 579–641 .
[108] D.K. Chlebda , A. Majda , T. Łojewski , J. Łojewska , Hyperspectral imaging cou-
pled with chemometric analysis for non-invasive differentiation of black pens,
Appl. Phys. A 122 (11) (2016) 957 .
[109] F.d.S .L . Borba , T. Jawhari , R.S. Honorato , A. de Juan ,Confocal raman imaging
and chemometrics applied to solve forensic document examination involving
crossed lines and obliteration
cases by a depth profiling study, Analyst 14 2
(7) (2017) 11 0 6–1118 .
[110] C. Berger , Objective ink color comparison through image processing and ma-
chine learning, Sci. Just. 53 (1) (2013) 55–59 .
[111 ] O. Mayer , M.C. Stamm , Accurate and efficient image forgery detection using
lateral chromatic aberration, IEEE Trans. Inf. Forens. Secur. (2018) .
[112] C. Silva , M. Pimentel , R. Honorato , C. Pasquini , J.M. Prats-Montalbán , A. Ferrer ,
Near infrared hyperspectral imaging for forensic analysis of document forgery,
Analyst 139 (20) (2014) 5176–5184 .
[113] J. Pereira , C. Silva , A. Braz
, M. Pimentel , R. Honorato , C. Pasquini , P. Wentzell ,
Projection pursuit and Pca associated with near and middle infrared hyper-
spectral images to investigate forensic cases of fraudulent documents, Mi-
crochem. J. 130 (2017) 412–419 .
[114] M. Tahtouh , P. Despland , R. Shimmon , J.R. Kalman , B.J. Reedy , The application
of infrared chemical imaging to the detection and enhancement of latent fin-
gerprints: method optimization and further findings, J. Forensic Sci. 52 (5)
(2007) 1089–1096 .
[115] T. Chen , Z. Schultz , I.W. Levin , Infrared spectroscopic imaging of latent finger-
prints and associated forensic evidence, Analyst 134 (9) (2009) 1902–1904 .
[116] Z. Khan , A. Shafait , A. Mian , Towar ds automated hyperspectral document im-
age analysis., in: Proceedings of the AFHA, 2013, pp. 41–45 .
[117] V. Da Silva , M. Talhavini , I. Peixoto , J. Zacca , A. Maldaner , J. Braga , Non-de-
structive identification of different types and brands of blue pen inks in cur-
sive handwriting by visible spectroscopy and Pls-da for forensic analysis, Mi-
crochem. J. 116 (2014) 235–243 .
[118] H.-S. Chen , H.-H. Meng , K.-C. Cheng , A survey of methods
used for the iden-
tification and characterization of inks, Forensic Sci. J. 1 (1) (2002) 1–14 .
[119] D. Fadoua , F. Le Bourgeois , H. Emptoz , Restoring ink bleed-through degraded
document images using a recursive unsupervised classification technique,
Lecture Notes Comput. Sci. 3872 (2006) 38 .
[120] Y. Huangy , M.S. Brown , D. Xuy , A framework for reducing ink-bleed in old
documents, in: Proceedings of the IEEE CVPR, 2008, pp. 1–7 .
[121] C. Berger , J. Koeijer , W. Glas , H. Madhuizen , Color separation in forensic image
processing, J. Forensic Sci. 51 (1) (2006)
100 –10 2 .
[122] J. Havermans , A. Aziz , H. Scholten , Non destructive detection of iron-gall inks
by means of multispectral imaging part 2: application on original objects af-
fected with irongall ink, Restaurator 24 (2) (2003) 88–94 .
[123] S. George , J.Y. Hardeberg , Ink classification and visualisation of historical
manuscripts: application of hyperspectral imaging, in: Proceedings of the IC-
DAR, 2015, pp. 1131–1135 .
[124] Z. Khan , F. Shafait , A. Mian , Hyperspectral imaging for ink mismatch detec-
tion, in: Proceedings of the 12th International Conference on Document Anal-
ysis and Recognition (ICDAR), 2013, pp.
877–881 .
[125] A. Abbas , K. Khurshid , F. Shafait , Towards automated ink mismatch detection
in hyperspectral document images, in: Proceedings of the 14th IAPR Inter-
national Conference on Document Analysis and Recognition (ICDAR), 1, IEEE,
2017, pp. 1229–1236 .
[126] Z. Khan, Uwa WIHSI, 2013, ( https://docs.google.com/file/d/0B0M2zmshL10
IMGczWlhEOVpYaEk/edit ), [Online; accessed 19-Aug-2017].
[127] M. Khan , A. Yousaf , K. Khurshid , F. Shafait , A. Abbas , Automated forgery detec-
tion in multispectral document images using fuzzy clustering, in: Proceedings
of the International Conference on Document Analysis Systems (DAS), Austria,
2018, pp. 1–10 .
[128] Y. Hannad
, I. Siddiqi , M.E.Y. El Kettani , Writer identification using texture de-
scriptors of handwritten fragments, Expert Syst. Appl. 47 (2016) 14–22 .
[129] K. Franke , S. Rose , Ink-deposition model: The relation of writing and ink de-
position processes, in: Proceedings of the Ninth International Workshop on
Frontiers in Handwriting Recognition IWFHR-9, IEEE, 2004, pp. 173–178 .
[130] J. Valdiviezo-N , G. Urcid , E. Lechuga , Digital restoration of damaged color doc-
uments based on hyperspectral imaging and lattice associative memories, Sig-
nal Image Video Process. 11 (5) (2017) 937–944 .
[131] Z. Shi , V. Govindaraju
, Historical document image enhancement using back-
ground light intensity normalization, in: Proceedings of the International
Conference on Pattern Recognition, 1, 2004, pp. 473–476 .
[132] C. Balas , V. Papadakis , N. Papadakis , A. Papadakis , E. Vazgiouraki , G. Themelis ,
A novel hyper-spectral imaging apparatus for the non-destructive analysis of
objects of artistic and historic value, J. Cultural Herit. 4 (2003) 330–337 .
[133] R. Hedjam , M. Cheriet , M. Kalacska , Constrained energy maximization and
self-referencing method for invisible ink detection from multispectral his-
torical document images, in: Proceedings of the ICPR, 2014, pp. 3026–
3031 .
[134] M. Klein , B. Aalderink , R. Padoan , T. Steemers , Quantitative hyperspectral re-
flectance imaging, Sensors 8 (9) (2008) 5576–5618 .
[135] S.J. Kim , S. Zhuo , F. Deng , C.-W. Fu , M. Brown , Interactive visualization of
hyperspectral images of historical documents, IEEE Trans. Visual. Comput.
Graph. 16 (6) (2010) 1441–1448 .
[136] I. Ciortan , H. Deborah , S. George , J.Y. Hardeberg , Color and hyperspectral im-
age segmentation for historical documents, in: Proceedings of the Digital Her-
itage, 2015, 1, IEEE, 2015, pp. 199–206 .
[137] F. Hollaus , M. Diem , R. Sablatnig , Binarization of multispectral document im-
ages, in: Proceedings of the International Conference on Computer Analysis
of Images and Patterns, Springer, 2015, pp. 109–120 .
[138] R. Hedjam , H.Z. Nafchi , R.F. Moghaddam , M. Kalacska , M. Cheriet , Icdar 2015
contest on multispectral text extraction (ms-tex 2015), in: Proceedings of the
ICDAR, IEEE,
2015, pp. 1181–1185 .
[139] C. Rother , V. Kolmogorov , A. Blake , Grabcut: Interactive foreground extraction
using iterated graph cuts, in: Proceedings of the ACM Transactions on Graph-
ics (TOG), 2004, pp. 309–314 .
[140] R. Easton , W.A . Christens-Barry , K. Knox , Spectral image processing and analy-
sis of the archimedes palimpsest, in: Proceedings of the European Signal Pro-
cessing Conference, 2011, pp. 1440–1444 .
[141] F. France , Advanced spectral imaging for noninvasive microanalysis of cultural
heritage materials: review of application to documents in the us library of
congress, Applied Spectrosc. 65 (6) (2011) 565–574 .
[142] J.V. ASPEREN DE BOER , Reflectography of paintings using an infrared vidicon
television system, Stud. Conservat. 14 (3) (1969) 96–118 .
[143] C. Fischer , I. Kakoulli , Multispectral and hyperspectral imaging technologies
in conservation: current research and potential applications, Stud. Conservat.
51 (sup1) (2006) 3–16 .
[144] I. Giakoumis , N. Nikolaidis , I. Pitas , Digital image processing techniques for
the detection and removal of cracks in digitized paintings, IEEE Transact. Im-
age Process. 15 (1) (2006) 178–188 .
[145] E. Salerno , A. Tonazzini , L. Bedini , Digital image analysis to enhance under-
written text in
the archimedes palimpsest, Int. J. Document Anal. Recognit. 9
(2) (20 07) 79–87 .
[146] A. Tonazzini , L. Bedini , E. Salerno , A markov model for blind image separa-
tion by a mean-field em algorithm, IEEE Trans. Image Process. 15 (2) (2006)
473–482 .
[147] A. Tonazzini , L. Bedini , E. Salerno , Independent component analysis for docu-
ment restoration, Int. J. Document Anal. Recognit. 7 (1) (2004) 17–27 .
[148] A. Polak , T. Kelman , D.J. Stothard , N. Eastaugh , F. Eastaugh , Hyperspectral
imaging combined with data classification techniques as an aid for artwork
authentication,
J. Cultural Herit. (2017) .
[149] Gowen , Hyperspectral imaging- an emerging process analytical tool for food
quality and safety control, Trends Food Sci. Technol. 18 (12) (2007) 590–598 .
[150] G. Edelman , E. Gaston , T. Van Leeuwen , P. Cullen , M. Aalders , Hyperspectral
imaging for non-contact analysis of forensic traces, Forensic Sci. Int. 223 (1)
(2012) 28–39 .
[151] D. Goltz , M. Attas , G. Youn g , E. Cloutis , M. Bedynski , Assessing stains on his-
torical documents using hyperspectral imaging, J. Cultural Herit. 11 (1) (2010)
19–26 .
[152] F.d.S.L. Borba , T. Jawhari , R.S. Honorato , A. de Juan ,Confocal raman imaging
and chemometrics applied to solve forensic document examination involving
crossed lines and obliteration cases by a depth profiling study, Analyst 142
(7) (2017) 110 6–1118 .
[153] A . Giacometti , A . Campagnolo , L. MacDonald , S. Mahony , M. Terras , S. Robson ,
T. Weyrich , A. Gibson , Cultural heritage destruction: documenting parchment
degradation via multispectral imaging., in: Proceedings of the EVA, 2012,
pp. 1–10 .
[154] M. Lettner , R. Sablatnig , Multispectral imaging for analyzing ancient
manuscripts, in: Proceedings of the European Signal Processing Conference,
2009, pp. 1200–1204 .
[155] E. Marengo , M. Manfredi , O. Zerbinati , E. Robotti , E. Mazzucco , F. Gosetti ,
G. Bearman , F. France , P. Shor , Development of a technique based on multi-
-spectral imaging for monitoring the conservation of cultural heritage objects,
Anal. Chim. Acta 706 (2) (2011) 229–237 .
[156] H. Deborah , N. Richard , J. Hardeberg , Hyperspectral crack detection in paint-
ings, in: Proceedings of the Colour and vision Computer, 2015, pp. 1–6 .
[157] Z. Lu , Z. Wu , M. Brown , Directed assistance for ink-bleed reduction
in old
documents, in: Proceedings of the IEEE CVPR, 2009, pp. 88–95 .
[158] Chemimage, ( www.chemimage.com ).
[159] Headwall, ( www.headwall.com ).
[160] ForensicXP, ( http://bilgem.tubitak.gov.tr/en/icerik/forensic- xp- 4010d-
questioned- doc- exam- system ).
[161] Sufrace Optics Corporation, ( https://surfaceoptics.com/ ).
[162] S. Wug Oh , M. Brown , M. Pollefeys , S. Joo Kim , Do it yourself hyperspec-
tral imaging with everyday digital cameras, in: Proceedings of the IEEE CVPR,
2016, pp. 2461–2469 .
[163] H. Petersson , D. Gustafsson , D. Bergstrom , Hyperspectral image analysis using
deep learning–a review, in: Proceedings of the International Conference on
Image Processing
Theory Tools and Applications, 2016, pp. 1–6 .
[164] N. Vincent , J.-M. Ogier , Shall deep learning be the mandatory future of docu-
ment analysis problems? Pattern Recognit. 86 (2019) 281–289 .
Rizwan Qureshi received B.E degree in Electronics Engineering from Mehran Uni-
versity of Engineering and Technology, Jamshoro, Pakistan in 2010 and M.S degree
in Electrical Engineering from Institute of Space Technology, Islamabad, Pakistan in
2015. Currently, he is working towards his Ph.D. degree in Electronic Engineering
at City University of Hong Kong. Before joining City University, he was a lecturer
in Electrical Engineering department at COMSATS University Islamabad,
Wah Cam-
pus, Pakistan. His research interests include Bioinformatics, signal/image processing,
energy management and spectral imaging.

22 R. Qureshi, M. Uzair and K. Khurshid et al. / Pattern Recognition 90 (2019) 12–22
Muhammad Uzair received the B.Sc. degree in computer systems engineering from
the University of Engineering and Technology at Peshawar, Pakistan, in 2006, and
the M.S. degree in electronics and computer engineering from Hanyang University,
Seoul, Korea, in 2009. He received his Ph.D. degree in computer engineering from
the University of Wester n Australia, Crawley, WA , Australia, in 2015. He was Assis-
tant Professor at COMSATS University, Wah Campus, Pakistan. Currently, he is work-
ing as research associate at the Defense and Systems Institute, University of South
Australia. His research interests include computer vision, hyperspectral image pro-
cessing and machine learning.
Khurram Khurshid is the head of Electrical Engineering department and look-
ing after the Signal and Image Processing research group. He did Ph.D from Paris
Descartes University, France in 2009. Since then, he has been involved in different
research ventures in the field of Image Processing, Pattern Recognition and Com-
puter Vision. He was awarded the Best University Teacher Award in January 2016
by the Higher Education Commission of Pakistan.
Hong Ya n received his Ph.D. degree from Ya le University. He was professor of imag-
ing science at the University of Sydney and currently is chair professor of computer
engineering at City University of Hong Kong. He was elected an IAPR fellow for
contributions to document image analysis and an IEEE fellow for contributions to
image recognition techniques and applications. Professor Yan was a Distinguished
Lecturer of IEEE SMC Society
during 20 0 0 to 2015. He received the 2016 Norbert
Wiener Award from IEEE SMC Society for contributions to image and biomolecular
pattern recognition techniques.