Fig 7 - uploaded by Zeyu Cao
Content may be subject to copyright.
Confusion matrix of 3DCSN on the SA dataset. The classes are labeled by numbers from 0 to 15, and the corresponding names are shown in Table 7.
Similar publications
![Figure 2.3: CBOW Model from Original Paper[15]](profile/Neel-Kanwal/publication/348944704/figure/fig2/AS:986493124366339@1612209697682/CBOW-Model-from-Original-Paper15_Q320.jpg)
![Figure 2.4: Skip-gram Model from Paper [15]](profile/Neel-Kanwal/publication/348944704/figure/fig3/AS:986493124370432@1612209697738/Skip-gram-Model-from-Paper-15_Q320.jpg)
![Figure 2.5: FastText Architecture from paper [40]](profile/Neel-Kanwal/publication/348944704/figure/fig4/AS:986493124366340@1612209697803/FastText-Architecture-from-paper-40_Q320.jpg)
![Figure 2.6: Attention Mechanism by Vinalys et al (2015) [44]](profile/Neel-Kanwal/publication/348944704/figure/fig5/AS:986493124349983@1612209697843/Attention-Mechanism-by-Vinalys-et-al-2015-44_Q320.jpg)
Citations
... These advanced CNN-based methods network by using 3D convolutional networks and combining contrast information and label information. HSI classification performs well when only a few training samples are available [20]. By combining EMP, Siamese CNNs and spectrum-space feature fusion, Huang et al. proposed an extended morphological profile-based method for HSI classification with limited training samples [21]. ...
... The core of the Siamese network structure is to map input sample pairs to the same feature space through two subnetworks, and the two subnetworks have shared weights. This allows the network to learn a common representation that makes similar inputs closer in the feature space [20]. The Siamese network is usually used for dealing with the problem of measurement learning and similarity comparison. ...
... To further verify the effectiveness of the proposed AL-MRIS method, several state-of-the-art classification methods, including DRIN [29], 3DCSN [20], S3Net [23], ALPN [26], FAAL [27], CFSL [16] and Gia-CFSL [17], were used for comparison. The corresponding classification maps are shown in Figures 9-12. ...
In hyperspectral image (HSI) classification scenarios, deep learning-based methods have achieved excellent classification performance, but often rely on large-scale training datasets to ensure accuracy. However, in practical applications, the acquisition of hyperspectral labeled samples is time consuming, labor intensive and costly, which leads to a scarcity of obtained labeled samples. Suffering from insufficient training samples, few-shot sample conditions limit model training and ultimately affect HSI classification performance. To solve the above issues, an active learning (AL)-based multipath residual involution Siamese network for few-shot HSI classification (AL-MRIS) is proposed. First, an AL-based Siamese network framework is constructed. The Siamese network, which has relatively low demand for sample data, is adopted for classification, and the AL strategy is integrated to select more representative samples to improve the model’s discriminative ability and reduce the costs of labeling samples in practice. Then, the multipath residual involution (MRIN) module is designed for the Siamese subnetwork to obtain the comprehensive features of the HSI. The involution operation was used to capture the fine-grained features and effectively aggregate the contextual semantic information of the HSI through dynamic weights. The MRIN module comprehensively considers the local features, dynamic features and global features through multipath residual connections, which improves the representation ability of HSIs. Moreover, a cosine distance-based contrastive loss is proposed for the Siamese network. By utilizing the directional similarity of high-dimensional HSI data, the discriminability of the Siamese classification network is improved. A large number of experimental results show that the proposed AL-MRIS method can achieve excellent classification performance with few-shot training samples, and compared with several state-of-the-art classification methods, the AL-MRIS method obtains the highest classification accuracy.
... There aren't many labelled samples available because collecting labelled samples for HSI is expensive. In recent years numbers of methods are proposed for HIS classification with limited labels training samples (9) . ...
... B. Gowthama et al. integrated principal component analysis (PCA)-based dimensionality reduction, the Siamese network framework and a CNN to achieve an improved HSI classification performance with a small sample [18]. Zeyu Cao proposed a Siamese network based on a 3DCNN, which combined contrastive information and label information to process small sample classification tasks [19]. Zhaohui Xue et al. utilized a spectral-spatial Siamese network consisting of a 1DCNN and a 2DCNN to extract spectral-spatial features [20]. ...
... To solve the above problems, inspired by 3DCSN [19] and Res2Net, which increases the receptive field by constructing hierarchical residual-like connections [24], a multipath and multiscale Siamese network based on spatial-spectral features for few-shot hyperspectral image classification (MMSN) is proposed. The MMSN is based on the Siamese network framework, which has low dependence on sample information. ...
... Inspired by 3DCSN [19] and Res2Net [24], the MMSN is proposed for his classification with few-shot training samples. The MMSN is based on the Siamese network framework, and each subnetwork branch mainly consists of a dilatation-cosine attention module (DCAM), residual-dense hybrid multipath (RDHM) and multikernel depth feature extraction (MDFE) module, as illustrated in the flowchart in Figure 5. ...
Deep learning has been demonstrated to be a powerful nonlinear modeling method with end-to-end optimization capabilities for hyperspectral Images (HSIs). However, in real classification cases, obtaining labeled samples is often time-consuming and labor-intensive, resulting in few-shot training samples. Based on this issue, a multipath and multiscale Siamese network based on spatial-spectral features for few-shot hyperspectral image classification (MMSN) is proposed. To conduct classification with few-shot training samples, a Siamese network framework with low dependence on sample information is adopted. In one subnetwork, a spatial attention module (DCAM), which combines dilated convolution and cosine similarity to comprehensively consider spatial-spectral weights, is designed first. Then, we propose a residual-dense hybrid module (RDHM), which merges three-path features, including grouped convolution-based local residual features, global residual features and global dense features. The RDHM can effectively propagate and utilize different layers of features and enhance the expression ability of these features. Finally, we construct a multikernel depth feature extraction module (MDFE) that performs multiscale convolutions with multikernel and hierarchical skip connections on the feature scales to improve the ability of the network to capture details. Extensive experimental evidence shows that the proposed MMSN method exhibits a superior performance on few-shot training samples, and its classification results are better than those of other state-of-the-art classification methods.
... By setting an appropriate loss function, the siamese network can map samples into a feature space with improved separability, where the distance between similar samples is reduced and the distance between dissimilar samples is increased. Two siamese networks have been proposed for HIC-SS in [35] and [63], respectively. These networks use the original training set to create positive and negative sample pairs for siamese network training. ...
Hyperspectral image (HSI) classification is one of the hotspots in remote sensing, and many methods have been continuously proposed in recent years. However, it is still challenging to achieve high accuracy classification in applications. One of the main reasons is the lack of labeled data. Due to the limitation of spatial resolution, manual labeling of HSI data is time-consuming and costly, so it is difficult to obtain a large amount of labeled data. In such a situation, many researchers turn their attention to the study of HSI classification with small samples. Focusing on this topic, this paper provides a systematic review of the research progress in recent years. Specifically, this paper contains three aspects. First, considering that the taxonomy used in previous review articles is not well-developed and confuses the reader, we propose a novel taxonomy based on the form of data utilization. This taxonomy provides a more accurate and comprehensive framework for categorizing the various approaches. Then, using the proposed taxonomy as a guideline, we analyze and summarize the existing methods, especially the latest research results (both deep and non-deep models) that were not included in the previous reviews, so that readers can understand the latest progress more clearly. Finally, we conduct several sets of experiments and present our opinions on current problems and future directions.
... In Liu et al. [43] a deep few-shot learning (DFSL) method is introduced using a 3-D CNN with residual blocks to learn the metric space and select the nearest neighbor (NN) or SVM classifier for classification. In Cao et al. [44], a 3-D Convolutional Siamese Network (3DCSN) is presented, which combines contrast information with label information for improved classification. In Alkhatib et al. [45], a Traditional CNN (Tri-CNN) approach to HSI classification was proposed, which is based on multi-scale 3D-CNN and three-branch feature fusion. ...
... The former include 2-D CNN models [18], 3-D CNN models [20], hierarchical residual models with attention mechanisms (HResNetAM) [37], and two-stream convolutional networks based on transfer learning (TWO-CNN) [50]. The latter include 2-D CNN-based relation networks (S-DMM) [46], metric-based learning classification (DFSL-NN) [43], 3-D Siamese networkbased 3DCSN [44], and 3D-CNN-based relation network (RN-FSC) [48] models. Again, 10 labeled samples per class were randomly selected for training the models. ...
Deep network models rely on sufficient training samples to perform reasonably well, which has inevitably constrained their application in classification of hyperspectral images (HSIs) due to the limited availability of labeled data. To tackle this particular challenge, we propose a hybrid relation network, H-RNet, by combining three-dimensional (3-D) convolution neural networks (CNN) and two-dimensional (2-D) CNN to extract the spectral–spatial features whilst reducing the complexity of the network. In an end-to-end relation learning module, the sample pairing approach can effectively alleviate the problem of few labeled samples and learn correlations between samples more accurately for more effective classification. Experimental results on three publicly available datasets have fully demonstrated the superior performance of the proposed model in comparison to a few state-of-the-art methods.
... The traditional siamese network can only obtain the similarity between samples, which makes it difficult to perform the classification task. To integrate the siamese network and the classifier into an end-to-end framework, Cao [37] added a classification structure to the siamese network, calculating the loss of each individual sample and training the classifier. Xue [38] designed a lightweight siamese structure for feature fitting and fast classification. ...
Hyperspectral image (HSI) classification has recently been successfully explored by using deep learning (DL) methods. However, DL models rely heavily on a large number of labeled samples, which are laborious to obtain. Therefore, finding a way to efficiently embed DL models in limited labeled samples is a hot topic in the field of HSI classification. In this paper, an active learning-based siamese network (ALSN) is proposed to solve the limited labeled samples problem in HSI classification. First, we designed a dual learning-based siamese network (DLSN), which consists of a contrastive learning module and a classification module. Secondly, in view of the problem that active learning is difficult to effectively sample under the extremely limited labeling cost, we proposed an adversarial uncertainty-based active learning (AUAL) method to query valuable samples, and to promote DLSN to learn a more complete feature distribution by fine-tuning. Finally, an active learning architecture, based on inter-class uncertainty (ICUAL), is proposed to construct a lightweight sample pair training set, fully extracting the inter-class information of sample pairs and improving classification accuracy. Experiments on three generic HSI datasets strongly demonstrated the effectiveness of ALSN for HSI classification, with performance improvements over other related DL methods.
... Recent studies in the past few years tried to develop novel methods in TL with very few annotated data. To this end, GNN model was presented in [76] for few-shot VHR RGB image classification, while Cao et al., [77] investigated 3D convolution Siamese model with a contrastive loss for • The pre-trained model may not be suitable for the target task, leading to suboptimal performance. • There might be a domain shift between the source and target domains, affecting the transferability of learned features. ...
An increasing availability of remote sensing data in the era of geo big-data makes producing well-represented, reliable training data to be more challenging and requires an excessive amount of human labor. In addition, the rapid increase in graphics processing unit (GPU) processing power has enabled the development of advanced deep learning (DL) algorithms, which achieve impressive results in the field of satellite image processing. However, they require a huge and comprehensive training dataset to avoid overfitting problems and to represent a generalizable model. Thus, moving toward the development of non-supervised deep learning (NSDL) models in different remote sensing applications is an inevitable need. To provide an initial response to that need, this paper performs a comprehensive review and systematic meta-analysis of recently published research articles focusing on the applications of NSDL for remote sensing data processing. In order to identify future research directions and formulate recommendations, we extract trends and highlight interesting approaches from this large body of literature. Consequently, current challenges, prospects, and recommendations are also discussed to uncover the trend. According to the results, there is a sharp increasing trend in the applicability of NSDL methods during these few years particularly, with the advent of new deep architectures, such as adversarial, graph, and transformer models. As a result, this review paper discusses different remote sensing data processing applications and challenges that can be addressed using NSDL approaches.
... The vectors are fed into projection head and prediction head. With reference to the setting of [42], we make the dimension of the output vector equal to 256, i.e. z ∈ R 256 , where z=h(f (x)). Therefore, the output of the prediction head is p ∈ R 256 , where p=g(h(f (x))). ...
Deep learning (DL) exhibits commendable performance in HSI classification because of its powerful feature expression ability. Siamese neural network further improves the performance of DL models by learning similarities within-class and differences between-class from sample pairs. However, there are still some limitations in siamese neural network. On the one hand, siamese neural network usually needs large number of negative pair samples in training process, leading to computing overhead. On the other hand, current models may lack interpretability because of complex network structure. To overcome the above limitations, we propose a spectral-spatial siamese neural network with Bag-of-Features (S3BoF) for HSI classification. Firstly, we use a siamese neural network with 3D and 2D convolutions to extract the spectral-spatial features. Secondly, we introduce stop-gradient operation and prediction head structure to make the siamese neural network work without negative pair samples, thus reducing computational burden. Thirdly, a Bag-of-Features (BoF) learning module is introduced to enhance the model interpretability and feature representation. Finally, a symmetric loss and a cross entropy loss are respectively used for contrastive learning and classification. Experiments results on four common hyperspectral data sets indicated that S3BoF performs better than the other traditional and state-of-the-art deep learning HSI classification methods in terms of classification accuracy and generalization performance, with improvements in terms of OA around 1.40%-30.01%, 0.27%-8.65%, 0.37%-6.27%, 0.22%-6.64% for Indian Pines, University of Pavia, Salinas, and Yellow River Delta data sets, respectively, under 5% labeled samples per class.
... Within the remote sensing community, semi-supervised learning has been long studied and enjoys applications in, e.g., hyperspectral image recognition and processing [19,20,21,22,23,24,25,26,27,28], multi-spectral image segmentation [29,30,31,32,33,34,35] and SAR-optical data fusion [36]. ...
In deep learning research, self-supervised learning (SSL) has received great attention, triggering interest within both the computer vision and remote sensing communities. While there has been big success in computer vision, most of the potential of SSL in the domain of Earth observation remains locked. In this article, we provide an introduction to and a review of the concepts and latest developments in SSL for computer vision in the context of remote sensing. Further, we provide a preliminary benchmark of modern SSL algorithms on popular remote sensing datasets, verifying the potential of SSL in remote sensing and providing an extended study on data augmentations. Finally, we identify a list of promising directions of future research in SSL for Earth observation (SSL4EO) to pave the way for the fruitful interaction of both domains.
... Metric learning is based on similarity prediction [14,16,21]. The network prediction of metric learning is fast because the weight of the network is shared, and it does not need to be adjusted for specific tasks [22][23][24]. By contrast, small sample learning based on Siamese learning has great flexibility in the number of categories. ...
... The sparse model realized by ReLU can better mine the relevant features and fit the training data. In Fig. 2, the data format of the original SAM image is NCHW (N stands for batch, C represents channel, H means height, and W is the width), which is [1,22], and after convolution of the first layer with the channel number of 8 and padding = 0, the format of the image becomes [1,8,20]. After embedding with the CBN module, the original SAM image is converted to the fused feature vector. ...
Flip chip has become one of the mainstream technologies in microelectronic packaging. Solder bumps play an important role in the interconnection of flip chips packages. The scanning acoustic microscopy (SAM) technology and a new network model were investigated for intelligent detection of flip chips. A new network model called CBN-S-Net was proposed based on a deep convolution network CBN and an optimized Siamese network. The CBN convolution network was used to extract the deep fusion features of solder bumps, and new triplet sample pairs were designed to measure the similarity between solder bumps. With the strategy of triplet sample pairs, the SAM images of the flip chip were used to verify the effectiveness of the designed network model. The results showed that the improved network has a high detection accuracy of 98.73%, and the proposed method is effective for the intelligent detection of solder bumps in high-density electronic packages.
