No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Feature extraction of hyperspectral images based on preserving neighborhood discriminant embedding
Abstract
A novel manifold learning feature extraction approach-preserving neighborhood discriminant embedding (PNDE) of hyperspectral image is proposed in this paper. The local geometrical and discriminant structure of the data manifold can be accurately characterized by within-class neighboring graph and between-class neighboring graph. Unlike manifold learning, such as LLE, Isomap and LE, which cannot deal with new test samples and images larger than 70×70, the method here can process full scene hyperspectral images. Experiments results on hyperspectral datasets and real-word datasets show that the proposed method can efficiently reduce the dimensionality while maintaining high classification accuracy. In addition, only a small amount of training samples are needed.
... 高维遥感图像通常呈现高维非线性的特征 [19] , serving embedding, NPE) [26] , 最 大 折 叠 度 量 学 习 (maximally collapsing metric learning, MCML) [27] 和 信息论度量学习(information-theoretic metric learning, ITML) [28] ; 3 种多视角分类方法包括协作式多 视 图 度 量 学 习 (collaborative multi-view metric learning, CMML) [29] , 多视角核向量机(multi-view core vector machine, MvCVM) [16] 和多视角 TSK 模 糊 系 统 (multiview TSK fuzzy system, MV-TSK-FS) [17] . 类别 NPE [26] MCML [27] ITML [28] CMML [29] MvCVM [16] MV-TSK-FS [17] 非线性 SW-MVML [26] MCML [27] ITML [28] CMML [29] MvCVM [16] MV-TSK-FS [17] 非线性 SW-MVML 参考文献(References): ...
... 高维遥感图像通常呈现高维非线性的特征 [19] , serving embedding, NPE) [26] , 最 大 折 叠 度 量 学 习 (maximally collapsing metric learning, MCML) [27] 和 信息论度量学习(information-theoretic metric learning, ITML) [28] ; 3 种多视角分类方法包括协作式多 视 图 度 量 学 习 (collaborative multi-view metric learning, CMML) [29] , 多视角核向量机(multi-view core vector machine, MvCVM) [16] 和多视角 TSK 模 糊 系 统 (multiview TSK fuzzy system, MV-TSK-FS) [17] . 类别 NPE [26] MCML [27] ITML [28] CMML [29] MvCVM [16] MV-TSK-FS [17] 非线性 SW-MVML [26] MCML [27] ITML [28] CMML [29] MvCVM [16] MV-TSK-FS [17] 非线性 SW-MVML 参考文献(References): ...
... 高维遥感图像通常呈现高维非线性的特征 [19] , serving embedding, NPE) [26] , 最 大 折 叠 度 量 学 习 (maximally collapsing metric learning, MCML) [27] 和 信息论度量学习(information-theoretic metric learning, ITML) [28] ; 3 种多视角分类方法包括协作式多 视 图 度 量 学 习 (collaborative multi-view metric learning, CMML) [29] , 多视角核向量机(multi-view core vector machine, MvCVM) [16] 和多视角 TSK 模 糊 系 统 (multiview TSK fuzzy system, MV-TSK-FS) [17] . 类别 NPE [26] MCML [27] ITML [28] CMML [29] MvCVM [16] MV-TSK-FS [17] 非线性 SW-MVML [26] MCML [27] ITML [28] CMML [29] MvCVM [16] MV-TSK-FS [17] 非线性 SW-MVML 参考文献(References): ...
The remote sensing images are susceptible to interference factors such as illumination and meteorological conditions, and with the improvement of the resolution of remote sensing equipment, more surface details appear in the remote sensing image. In order to improve the accuracy of remote sensing image scene classification, a self-weighted multi-view metric learning (SW-MVML) method is proposed. Firstly, data features from multiple views are used to learn a discriminative metric space, which makes the similar images compact and dissimilar images as far away as possible. Then, the weight vector is introduced to adaptively adjust the weight relationship among different views. Finally, the kernel technique is used to extend the method to non-linear space, so that the correlation and complementary information hidden between views can be effectively exploited. The experimental results on Google and WHU-RS remote sensing image datasets show that the proposed method achieves good classification performance with the average classification accuracy of 90.26% and 92.62% respectively, which is significantly better than the comparative single-view and multi-view classification methods. © 2021, Beijing China Science Journal Publishing Co. Ltd. All right reserved.
... Dimension reduction techniques can be roughly categorized into linear approaches and nonlinear ones. The linear approaches include principal component analysis (PCA) [2], random projection (RP) [3], linear discriminant analysis (LDA) [4], and locality preserving projection (LPP), whereas the nonlinear approaches include isomap mapping (Isomaps) [5,6], diffusion maps (DMaps) [7], and locally linear embedding (LLE) [8]. ...
... , } ∈ R × , the objective dimension to be embedded , the nearest neighbor parameter (default: ≡ 7), and the test sample ∈ R Output: Transformation matrix ∈ R × Steps are as follows: (1) % Initialize matrices (2)̂← 0 × ; % within-class scatter (3)̂← 0 × ; % between-class scatter (4) (5) % Compute within-class affinity matrix (6) for = 1, 2, . . . , do % in a classwise manner (7) { } =1 ← { | = }; % the th class data samples (8) ← { 1 , 2 , . . . , } % sample matrix ...
The computational procedure of hyperspectral image (HSI) is extremely complex, not only due to the high dimensional information, but also due to the highly correlated data structure. The need of effective processing and analyzing of HSI has met many difficulties. It has been evidenced that dimensionality reduction has been found to be a powerful tool for high dimensional data analysis. Local Fisher’s liner discriminant analysis (LFDA) is an effective method to treat HSI processing. In this paper, a novel approach, called PD-LFDA, is proposed to overcome the weakness of LFDA. PD-LFDA emphasizes the probability distribution (PD) in LFDA, where the maximum distance is replaced with local variance for the construction of weight matrix and the class prior probability is applied to compute the affinity matrix. The proposed approach increases the discriminant ability of the transformed features in low dimensional space. Experimental results on Indian Pines 1992 data indicate that the proposed approach significantly outperforms the traditional alternatives.
... We design 10 classification tasks, and the basic information of tasks is as shown in Table 2. In order to show the performance of our method, we compare AMDML with four single-view classification methods [including DMSI (Xing et al., 2003), large margin nearest neighbor (LMNN) (Weinberger and Saul, 2009), neighborhood preserving embedding (NPE) (Wen et al., 2010), and RDML-CCPVL (Ni et al., 2018)] and three multi-view methods [including MvCVM (Huang et al., 2015), VMRML-LS (Quang et al., 2013), and DMML ]. In the LMNN method, the number of target neighbors k was set to k = 3, and the weighting parameter µ is selected within the grid {0, 0.2,..., 1}. ...
Metric learning is a class of efficient algorithms for EEG signal classification problem. Usually, metric learning method deals with EEG signals in the single view space. To exploit the diversity and complementariness of different feature representations, a new auto-weighted multi-view discriminative metric learning method with Fisher discriminative and global structure constraints for epilepsy EEG signal classification called AMDML is proposed to promote the performance of EEG signal classification. On the one hand, AMDML exploits the multiple features of different views in the scheme of the multi-view feature representation. On the other hand, considering both the Fisher discriminative constraint and global structure constraint, AMDML learns the discriminative metric space, in which the intraclass EEG signals are compact and the interclass EEG signals are separable as much as possible. For better adjusting the weights of constraints and views, instead of manually adjusting, a closed form solution is proposed, which obtain the best values when achieving the optimal model. Experimental results on Bonn EEG dataset show AMDML achieves the satisfactory results.
... The two classifiers were implemented to reduce the possible bias induced by a single classification algorithm. We compared the EDLML algorithm with three representative dimensionality reduction methods, i.e., LFDA, DLA, and neighborhood preserving embedding (NPE) [62], to thoroughly evaluate the performance of the proposed algorithm. We also compared the proposed algorithm with three metric learning methods, i.e., ITML, NCA, and maximally collapsing metric learning (MCML) [63], which can also be applied to dimensionality reduction. ...
Owing to the strong correlation between the spectral bands of hyperspectral images (HSIs), many feature extraction (FE) methods have been proposed to reduce the redundancy of hyperspectral data. However, Euclidean distance-based FE methods are sensitive to noise. To address this issue, this letter proposed a new unsupervised FE method called robust projection learning (RPL) by integrating the low-rank and sparse decomposition with projection learning. Specifically, in order to enhance the discrimination of traditional robust principal component analysis (RPCA), discriminative RPCA (DRPCA) is first proposed by decomposing the raw data into a low-rank part, a discriminative sparse part, and a structured noise. Moreover, for the purpose of redundancy reduction, projection learning is integrated into DRPCA to obtain a projection matrix with robustness and discrimination. To verify the validity of RPL, two real hyperspectral data sets are used for basic comparison and robust analysis. The corresponding experimental results demonstrate that RPL outperforms the comparative FE methods.
Much work in the study of hyperspectral imagery has focused on macroscopic mixtures and unmixing via the linear mixing model. A substantially different approach seeks to model hyperspectral data non-linearly in order to accurately describe intimate or microscopic relationships of materials within the image. In this paper we present and discuss a new model (MacMicDEM) that seeks to unify both approaches by representing a pixel as both linearly and non-linearly mixed, with the condition that the endmembers for both mixture types need not be related. Using this model, we develop a method to accurately and quickly unmix data which is both macroscopically and microscopically mixed. Subsequently, this method is then validated on synthetic and real datasets.
A supervised neighborhood preserving embedding (SNPE) linear manifold learning feature extraction method for hyperspectral image classification is presented in this paper. A point's k nearest neighbors is found by using new distance which is proposed according to prior class-label information. The new distance makes intra-class more tightly and inter-class more separately. SNPE overcomes the single manifold assumption of NPE. Data sets lay on (or near) multiple manifolds can be processed. Experimental results on AVIRIS hyperspectral data set demonstrate the effectiveness of our method.
Macroscopic and microscopic mixture models and algorithms for hyperspectral unmixing are presented. Unmixing algorithms are derived from an objective function. The objective function incorporates the linear mixture model for macroscopic unmixing and a nonlinear mixture model for microscopic unmixing. The nonlinear mixture model is derived from a bidirectional reflectance distribution function for microscopic mixtures. The algorithm is designed to unmix hyperspectral images composed of macroscopic or microscopic mixtures. The mixture types and abundances at each pixel can be estimated directly from the data without prior knowledge of mixture types. Endmembers can also be estimated. Results are presented using synthetic data sets of macroscopic and microscopic mixtures and usingwell-known, well-characterized laboratory data sets. The unmixing accuracy of this newphysics-based algorithm is compared to linear methods and to results published for other nonlinear models. The proposed method achieves the best unmixing accuracy.
A novel supervised dimensionality reduction method called orthogonal maximum margin discriminant projection (OMMDP) is proposed to cope with the high dimensionality, complex, various, irregular-shape plant leaf image data. OMMDP aims at learning a linear transformation. After projecting the original data into a low dimensional subspace by OMMDP, the data points of the same class get as near as possible while the data points of the differerent classes become as far as possible, thus the classification ability is enhanced. The main differences from linear discriminant analysis (LDA), discriminant locality preserving projections (DLPP) and other supervised manifold learning-based methods are as follows: (1) In OMMDP, Warshall algorithm is first applied to constructing both of the must-link and class-class scatter matrices, whose process is easily and quickly implemented without judging whether any pairwise points belong to the same class. (2) The neighborhood density is defined to construct the objective function of OMMDP, which makes OMMDP be robust to noise and outliers. Experimental results on two public plant leaf databases clearly demonstrate the effectiveness of the proposed method for classifying leaf images.
It is important to take account into both the spectral domain and spatial domain information for hyperspectral image analysis. Thus, how to effectively integrate both spectral and spatial information confronts us. Motived by the least square form of PCA, we extend it to a low-rank matrix approximation form for multi-feature dimensionality redu-ction. In addition, we use the ensemble manifold regularize-ation techniques to capture the complementary information provided by spectral-spatial features of hyperspectral image. Experimental results on public hyperspectral data set demonstrate the effectiveness of our proposed method.
Spectral–spatial feature combination for hyperspectral image analysis has become an important research topic in hyperspectral remote sensing applications. A simple and straightforward way to integrate spectral–spatial features is to concatenate heterogeneous features into a long vector. Then, the dimensionality reduction techniques, i.e., feature selection, are applied before subsequent utilizations. However, such representation can introduce redundancy and noise. Moreover, traditional single-feature selection methods treat different features equally and ignore their complementary properties. As a result, the performance of subsequent tasks, i.e., classification, would drop down. In this paper, we propose a novel approach to integrate the spectral–spatial features based on the concatenating strategy, termed discriminative sparse multimodal learning for feature selection (DSML-FS). In the proposed method, joint structured sparsity regularizations are used to exploit the intrinsic data structure and relationships among different features. Discriminative least squares regression is applied to enlarge the distance between classes. Therefore, the weight matrix incorporating the information of featurewise and individual properties is automatically learned for spectral–spatial feature selection. We develop an alternative iterative algorithm to solve the nonlinear optimization problem in DSML-FS with global convergence. We systematically evaluate the proposed algorithm on three available hyperspectral data sets, and the encouraging experimental results demonstrate the effectiveness of DSML-FS.
In hyperspectral image (HSI) classification, feature extraction is one important step. Traditional methods, e.g., principal component analysis (PCA) and locality preserving projection, usually neglect the information of within-class similarity and between-class dissimilarity, which is helpful to the improvement of classification. On the other hand, most of these methods, e.g., PCA and linear discriminative analysis, consider that the HSI data lie on a low-dimensional manifold or each class is on a submanifold. However, some class data of HSI may lie on a multimanifold. To avoid these problems, we propose a method for feature extraction in HSIs, assuming that a local region resides on a submainfold. In our method, we deal with the data region by region by taking into account the different discriminative locality information. Then, under the metric learning framework, a robust distance metric is learned. It aims to learn a subspace in which the samples in the same class are as near as possible while the samples in different classes are as far as possible. Encouraging experimental results on two available hyperspectral data sets indicate that our proposed algorithm outperforms many existing feature extract methods for HSI classification.
A method of incorporating the multi-mixture pixel model into hyperspectral endmember extraction is presented and
discussed. A vast majority of hyperspectral endmember extraction methods rely on the linear mixture model to describe
pixel spectra resulting from mixtures of endmembers. Methods exist to unmix hyperspectral pixels using nonlinear
models, but rely on severely limiting assumptions or estimations of the nonlinearity. This paper will present a
hyperspectral pixel endmember extraction method that utilizes the bidirectional reflectance distribution function to
model microscopic mixtures. Using this model, along with the linear mixture model to incorporate macroscopic
mixtures, this method is able to accurately unmix hyperspectral images composed of both macroscopic and microscopic
mixtures. The mixtures are estimated directly from the hyperspectral data without the need for a priori knowledge of the
mixture types. Results are presented using synthetic datasets, of multi-mixture pixels, to demonstrate the increased
accuracy in unmixing using this new physics-based method over linear methods. In addition, results are presented using
a well-known laboratory dataset.
A method of incorporating macroscopic and microscopic reflectance models into hyperspectral pixel unmixing is
presented and discussed. A vast majority of hyperspectral unmixing methods rely on the linear mixture model to
describe pixel spectra resulting from mixtures of endmembers. Methods exist to unmix hyperspectral pixels using
nonlinear models, but rely on severely limiting assumptions or estimations of the nonlinearity. This paper will present a
hyperspectral pixel unmixing method that utilizes the bidirectional reflectance distribution function to model
microscopic mixtures. Using this model, along with the linear mixture model to incorporate macroscopic mixtures, this
method is able to accurately unmix hyperspectral images composed of both macroscopic and microscopic mixtures. The
mixtures are estimated directly from the hyperspectral data without the need for a priori knowledge of the mixture types.
Results are presented using synthetic datasets, of macroscopic and microscopic mixtures, to demonstrate the increased
accuracy in unmixing using this new physics-based method over linear methods. In addition, results are presented using
a well-known laboratory dataset. Using these results, and other published results from this dataset, increased accuracy in
unmixing over other nonlinear methods is shown.
The availability of hyperspectral images expands the capability of using image classification to study detailed characteristics of objects, but at a cost of having to deal with huge data sets. This work studies the use of the principal component analysis as a preprocessing technique for the classification of hyperspectral images. Two hyperspectral data sets, HYDICE and AVIRIS, were used for the study. A brief presentation of the principal component analysis approach is followed by an examination of the infor-mation contents of the principal component image bands, which revealed that only the first few bands contain significant information. The use of the first few principal component images can yield about 70 percent correct classification rate. This study suggests the benefit and efficiency of using the principal component analysis technique as a preprocessing step for the classification of hyperspectral images.
The shortest path k-nearest neighbor classifier (SkNN), that utilizes nonlinear manifold learning, is proposed for analysis of hyperspectral data. In contrast to classifiers that deal with the high dimensional feature space directly, this approach uses the pairwise distance matrix over a nonlinear manifold to classify novel observations. Because manifold learning preserves the local pairwise distances and updates distances of a sample to samples beyond the user-defined neighborhood along the shortest path on the manifold, similar samples are moved into closer proximity. High classification accuracies are achieved by using the simple k-nearest neighbor (kNN) classifier. SkNN was applied to hyperspectral data collected by the Hyperion sensor on the EO-1 satellite over the Okavango Delta of Botswana. Classification accuracies and generalization capability are compared to those achieved by the best basis binary hierarchical classifier, the hierarchical support vector machine classifier, and the k-nearest neighbor classifier on both the original data and a subset of its principal components.
This paper considers the problem of dimensionality reduction by orthogonal projection techniques. The main feature of the proposed techniques is that they attempt to preserve both the intrinsic neighborhood geometry of the data samples and the global geometry. In particular we propose a method, named Orthogonal Neighborhood Preserving Projections, which works by first building an "affinity" graph for the data, in a way that is similar to the method of Locally Linear Embedding (LLE). However, in contrast with the standard LLE where the mapping between the input and the reduced spaces is implicit, ONPP employs an explicit linear mapping between the two. As a result, handling new data samples becomes straightforward, as this amounts to a simple linear transformation. We show how to define kernel variants of ONPP, as well as how to apply the method in a supervised setting. Numerical experiments are reported to illustrate the performance of ONPP and to compare it with a few competing methods.
A new algorithm for exploiting the nonlinear structure of hyperspectral imagery is developed and compared against the de facto standard of linear mixing. This new approach seeks a manifold coordinate system that preserves geodesic distances in the high-dimensional hyperspectral data space. Algorithms for deriving manifold coordinates, such as isometric mapping (ISOMAP), have been developed for other applications. ISOMAP guarantees a globally optimal solution, but is computationally practical only for small datasets because of computational and memory requirements. Here, we develop a hybrid technique to circumvent ISOMAP's computational cost. We divide the scene into a set of smaller tiles. The manifolds derived from the individual tiles are then aligned and stitched together to recomplete the scene. Several alignment methods are discussed. This hybrid approach exploits the fact that ISOMAP guarantees a globally optimal solution for each tile and the presumed similarity of the manifold structures derived from different tiles. Using land-cover classification of hyperspectral imagery in the Virginia Coast Reserve as a test case, we show that the new manifold representation provides better separation of spectrally similar classes than one of the standard linear mixing models. Additionally, we demonstrate that this technique provides a natural data compression scheme, which dramatically reduces the number of components needed to model hyperspectral data when compared with traditional methods such as the minimum noise fraction transform.
Scientists working with large volumes of high-dimensional data, such as global climate patterns, stellar spectra, or human
gene distributions, regularly confront the problem of dimensionality reduction: finding meaningful low-dimensional structures
hidden in their high-dimensional observations. The human brain confronts the same problem in everyday perception, extracting
from its high-dimensional sensory inputs—30,000 auditory nerve fibers or 106 optic nerve fibers—a manageably small number of perceptually relevant features. Here we describe an approach to solving dimensionality
reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set.
Unlike classical techniques such as principal component analysis (PCA) and multidimensional scaling (MDS), our approach is
capable of discovering the nonlinear degrees of freedom that underlie complex natural observations, such as human handwriting
or images of a face under different viewing conditions. In contrast to previous algorithms for nonlinear dimensionality reduction,
ours efficiently computes a globally optimal solution, and, for an important class of data manifolds, is guaranteed to converge
asymptotically to the true structure.
Existing manifold learning algorithms use Euclidean distance to measure the proximity of data points. However, in high-dimensional space, Minkowski metrics are no longer stable because the ratio of distance of nearest and farthest neighbors to a given query is almost unit. It will degrade the performance of manifold learning algorithms when applied to dimensionality reduction of high-dimensional data. We introduce a new distance function named shrinkage-divergence-proximity (SDP) to manifold learning, which is meaningful in any high-dimensional space. An improved locally linear embedding (LLE) algorithm named SDP-LLE is proposed in light of the theoretical result. Experiments are conducted on a hyperspectral data set and an image segmentation data set. Experimental results show that the proposed method can efficiently reduce the dimensionality while getting higher classification accuracy.
Many problems in information processing involve some form of dimensionality reduction. In this paper, we intro- duce Locality Preserving Projections (LPP). These are linear projective maps that arise by solving a variational problem that optimally preserves the neighborhood structure of the data set. LPP should be seen as an alternative to principal component analysis (PCA) ñ a classical linear technique that projects the data along the directions of maximal variance. When the high dimensional data lies on a low dimensional manifold embedded in the am- bient space, the Locality Preserving Projections are obtained by nding the optimal linear approximations to the eigenfunctions of the Laplace Beltrami operator on the manifold. As a result, LPP shares many of the data repre- sentation properties of non linear techniques such as Laplacian Eigenmap (4) or Locally Linear Embedding (5). This is borne out by illustrative examples on a couple of high dimensional image data sets.
In this paper, a new manifold learning method to reduce the dimension of hyperspectral data is proposed. In this method, ISOMAP algorithm is used to extract the inherent manifold of hyperspectral data to transform the high-dimensional space into a low-dimensional space. Experiments show that the method is effective, meaningful, and provides a new way for reducing the dimension of hyperspectral data while expands the application area of manifold learning in the hyperspectral data processing filed.
We present a new approach, called local discriminant embedding (LDE), to manifold learning and pattern classification. In our framework, the neighbor and class relations of data are used to construct the embedding for classification problems. The proposed algorithm learns the embedding for the submanifold of each class by solving an optimization problem. After being embedded into a low-dimensional subspace, data points of the same class maintain their intrinsic neighbor relations, whereas neighboring points of different classes no longer stick to one another. Via embedding, new test data are thus more reliably classified by the nearest neighbor rule, owing to the locally discriminating nature. We also describe two useful variants: two-dimensional LDE and kernel LDE. Comprehensive comparisons and extensive experiments on face recognition are included to demonstrate the effectiveness of our method.
In this paper, we study the Locally Linear Embedding (LLE) for nonlinear dimen-sionality reduction of hyperspectral data. We improve the existing LLE in terms of both compu-tational complexity and memory consumption by introducing a spatial neighbourhood window for calculating the k nearest neighbours. The improved LLE can process larger hyperspectral images than the existing LLE and it is also faster. We conducted experiments of endmember extraction to assess the effectiveness of the dimensionality reduction methods. Experimental results show that the improved LLE is better than PCA and the existing LLE in identifying endmembers. It finds more endmembers than PCA and the existing LLE when the Pixel Purity Index (PPI) based endmember extraction method is used. Also, better results are obtained for detection.
In the context of the appearance-based paradigm for object
recognition, it is generally believed that algorithms based on LDA
(linear discriminant analysis) are superior to those based on PCA
(principal components analysis). In this communication, we show that
this is not always the case. We present our case first by using
intuitively plausible arguments and, then, by showing actual results on
a face database. Our overall conclusion is that when the training data
set is small, PCA can outperform LDA and, also, that PCA is less
sensitive to different training data sets
Many areas of science depend on exploratory data analysis and visualization. The need to analyze large amounts of multivariate
data raises the fundamental problem of dimensionality reduction: how to discover compact representations of high-dimensional
data. Here, we introduce locally linear embedding (LLE), an unsupervised learning algorithm that computes low-dimensional,
neighborhood-preserving embeddings of high-dimensional inputs. Unlike clustering methods for local dimensionality reduction,
LLE maps its inputs into a single global coordinate system of lower dimensionality, and its optimizations do not involve local
minima. By exploiting the local symmetries of linear reconstructions, LLE is able to learn the global structure of nonlinear
manifolds, such as those generated by images of faces or documents of text.
Scientists working with large volumes of high-dimensional data, such as global climate patterns, stellar spectra, or human
gene distributions, regularly confront the problem of dimensionality reduction: finding meaningful low-dimensional structures
hidden in their high-dimensional observations. The human brain confronts the same problem in everyday perception, extracting
from its high-dimensional sensory inputs—30,000 auditory nerve fibers or 106 optic nerve fibers—a manageably small number of perceptually relevant features. Here we describe an approach to solving dimensionality
reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set.
Unlike classical techniques such as principal component analysis (PCA) and multidimensional scaling (MDS), our approach is
capable of discovering the nonlinear degrees of freedom that underlie complex natural observations, such as human handwriting
or images of a face under different viewing conditions. In contrast to previous algorithms for nonlinear dimensionality reduction,
ours efficiently computes a globally optimal solution, and, for an important class of data manifolds, is guaranteed to converge
asymptotically to the true structure.
In this paper, we propose a new nonlinear dimensionality reduction method by combining Locally Linear Embedding (LLE) with Laplacian Eigenmaps, and apply it to hyperspectral data. LLE projects high dimensional data into a low-dimensional Euclidean space while preserving local topological structures. However, it may not keep the relative distance between data points in the dimension-reduced space as in the original data space. Laplacian Eigenmaps, on the other hand, can preserve the locality characteristics in terms of distances between data points. By combining these two methods, a better locality preserving method is created for nonlinear dimensionality reduction. Experiments conducted in this paper confirms the feasibility of the new method for hyperspectral dimensionality reduction. The new method can find the same number of endmembers as PCA and LLE, but it is more accurate than them in terms of endmember location. Moreover, the new method is better than Laplacian Eigenmap alone because it identifies more pure mineral endmembers.
Recently there has been a lot of interest in geometrically motivated approaches to data analysis in high dimensional spaces. We consider the case where data is drawn from sampling a probability distribution that has support on or near a submanifold of Euclidean space. In this paper, we propose a novel subspace learning algorithm called neighborhood preserving embedding (NPE). Different from principal component analysis (PCA) which aims at preserving the global Euclidean structure, NPE aims at preserving the local neighborhood structure on the data manifold. Therefore, NPE is less sensitive to outliers than PCA. Also, comparing to the recently proposed manifold learning algorithms such as Isomap and locally linear embedding, NPE is defined everywhere, rather than only on the training data points. Furthermore, NPE may be conducted in the original space or in the reproducing kernel Hilbert space into which data points are mapped. This gives rise to kernel NPE. Several experiments on face database demonstrate the effectiveness of our algorithm.
Feature extraction is an indispensable preprocessing step for information extraction from hyperspectral remote sensing data. In this paper, we introduce a nonlinear feature extraction algorithm, called Locally Linear Embedding (LLE), and customize it for hyperspectral remote sensing applications. Unlike the linear feature extraction algorithms based on eigenvectors of data covariance matrix, LLE preserves local topology of hyperspectral data in the reduced space. This preservation is important to maintain the nonlinear properties of the input data that benefits further information extraction. To investigate its effectiveness for hyperspectral remote sensing applications, LLE was examined in terms of spatial information preservation and pure pixel identification. The preliminary result of this study demonstrated that it compared favorably with PCA on spatial information preservation. In addition, it exceeded PCA on pure pixel identification through scatter plots.
In the last decades, a large family of algorithms - supervised or unsupervised; stemming from statistic or geometry theory - have been proposed to provide different solutions to the problem of dimensionality reduction. In this paper, beyond the different motivations of these algorithms, we propose a general framework, graph embedding along with its linearization and kernelization, which in theory reveals the underlying objective shared by most previous algorithms. It presents a unified perspective to understand these algorithms; that is, each algorithm can be considered as the direct graph embedding or its linear/kernel extension of some specific graph characterizing certain statistic or geometry property of a data set. Furthermore, this framework is a general platform to develop new algorithm for dimensionality reduction. To this end, we propose a new supervised algorithm, Marginal Fisher Analysis (MFA), for dimensionality reduction by designing two graphs that characterize the intra-class compactness and inter-class separability, respectively. MFA measures the intra-class compactness with the distance between each data point and its neighboring points of the same class, and measures the inter-class separability with the class margins; thus it overcomes the limitations of traditional Linear Discriminant Analysis algorithm in terms of data distribution assumptions and available projection directions. The toy problem on artificial data and the real face recognition experiments both show the superiority of our proposed MFA in comparison to LDA.
We describe a method of processing hyperspectral images of natural scenes that uses a combination of k-means clustering and locally linear embedding (LLE). The primary goal is to assist anomaly detection by preserving spectral uniqueness among the pixels. In order to reduce redundancy among the pixels, adjacent pixels which are spectrally similar are grouped using the k-means clustering algorithm. Representative pixels from each cluster are chosen and passed to the LLE algorithm, where the high dimensional spectral vectors are encoded by a low dimensional mapping. Finally, monochromatic and tri-chromatic images are constructed from the k-means cluster assignments and LLE vector mappings. The method generates images where differences in the original spectra are reflected in differences in the output vector assignments. An additional benefit of mapping to a lower dimensional space is reduced data size. When spectral irregularities are added to a patch of the hyperspectral images, again the method successfully generated color assignments that detected the changes in the spectra.
The nonlinear dimensionality reduction and its effects on vector classification and segmentation of hyperspectral images are investigated in this letter. In particular, the way dimensionality reduction influences and helps classification and segmentation is studied. The proposed framework takes into account the nonlinear nature of high-dimensional hyperspectral images and projects onto a lower dimensional space via a novel spatially coherent locally linear embedding technique. The spatial coherence is introduced by comparing pixels based on their local surrounding structure in the image domain and not just on their individual values as classically done. This spatial coherence in the image domain across the multiple bands defines the high-dimensional local neighborhoods used for the dimensionality reduction. This spatial coherence concept is also extended to the segmentation and classification stages that follow the dimensionality reduction, introducing a modified vector angle distance. We present the underlying concepts of the proposed framework and experimental results showing the significant classification improvements
Drawing on the correspondence between the graph Laplacian, the Laplace-Beltrami operator on a manifold, and the connections to the heat equation, we propose a geometrically motivated algorithm for constructing a representation for data sampled from a low dimensional manifold embedded in a higher dimensional space. The algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering. Several applications are considered.
Available: http://www.math.ucla
- M C Fong