Figure 3 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Deep learning has become an effective method for hyperspectral image classification. However, the high band correlation and data volume associated with airborne hyperspectral images, and the insufficiency of training samples, present challenges to the application of deep learning in airborne image classification. Prototypical networks are practical...
Contexts in source publication
Context 1
... the nearest class prototype is found to classify the embedded query points. Figure 3 shows the schematic representation of the PrNet [40]. The figure shows samples in the projection space with three categories (C1, C2, and C3). ...
Citations
... Zhang et al. proposed a data-driven FSL approach for crop disease and pest detection based on target detection and transfer learning. On the other hand, the applications of FSL in forestry are fewer, and most of the studies are based on remote sensing images for classification, such as hyperspectral image classification of tree species [41][42][43]. ...
Efficient image recognition is important in crop and forest management. However, it faces many challenges, such as the large number of plant species and diseases, the variability of plant appearance, and the scarcity of labeled data for training. To address this issue, we modified a SOTA Cross-Domain Few-shot Learning (CDFSL) method based on prototypical networks and attention mechanisms. We employed attention mechanisms to perform feature extraction and prototype generation by focusing on the most relevant parts of the images, then used prototypical networks to learn the prototype of each category and classify new instances. Finally, we demonstrated the effectiveness of the modified CDFSL method on several plant and disease recognition datasets. The results showed that the modified pipeline was able to recognize several cross-domain datasets using generic representations, and achieved up to 96.95% and 94.07% classification accuracy on datasets with the same and different domains, respectively. In addition, we visualized the experimental results, demonstrating the model's stable transfer capability between datasets and the model's high visual correlation with plant and disease biological characteristics. Moreover, by extending the classes of different semantics within the training dataset, our model can be generalized to other domains, which implies broad applicability.
... However, due to the high input sample requirement of Mask R-CNN, the early recognition accuracy obtained by this method cannot meet the demand of PWD prevention and control. The prototypical network model has the advantage of small sample learning, and its simple network structure makes the classification process more efficient and achieves higher classification accuracy [41,42]. Based on the prototypical network classification algorithm, using the hyperspectral full bands dataset and the feature preferred bands dataset as model inputs, this study achieved an overall accuracy of 92.17% and 92.79%, respectively, for all stages of PWD detection, with early identification accuracy reaching 82.17% and 83.21%, respectively. ...
The pine wood nematode (PWN; Bursaphelenchus xylophilus) is a major invasive species in China, causing huge economic and ecological damage to the country due to the absence of natural enemies and the extremely rapid rate of infection and spread. Accurate monitoring of pine wilt disease (PWD) is a prerequisite for timely and effective disaster prevention and control. UAVs can carry hyperspectral sensors for near-ground remote sensing observations, which can obtain rich spatial and spectral information and have the potential for infected tree identification. Deep learning techniques can use rich multidimensional data to mine deep features in order to achieve tasks such as classification and target identification. Therefore, we propose an improved Mask R-CNN instance segmentation method and an integrated approach combining a prototypical network classification model with an individual tree segmentation algorithm to verify the possibility of deep learning models and UAV hyperspectral imagery for identifying infected individual trees at different stages of PWD. The results showed that both methods achieved good performance for PWD identification: the overall accuracy of the improved Mask R-CNN with the screened bands as input data was 71%, and the integrated method combining prototypical network classification model with individual tree segmentation obtained an overall accuracy of 83.51% based on the screened bands data, in which the early infected pine trees were identified with an accuracy of 74.89%. This study indicates that the improved Mask R-CNN and integrated prototypical network method are effective and practical for PWD-infected individual trees identification using UAV hyperspectral data, and the proposed integrated prototypical network enables early identification of PWD, providing a new technical guidance for early monitoring and control of PWD.
... Deep learning methods proved to be a robust alternative for remote sensing image classification, as they can learn optimal features and classification parameters to handle hyperspectral data (Signoroni et al., 2019). Different works successfully applied convolutional neural networks (CNN) (Krizhevsky et al., 2012) for tree species classification (Pölönen et al., 2018;Hartling et al., 2019;Fricker et al., 2019;Natesan et al., 2020;Mäyrä et al., 2021), including tropical and subtropical environments (Sothe et al., 2019(Sothe et al., , 2020Zhang et al., 2020;Tian et al., 2020;Abbas et al., 2021). Sothe et al. (2019) employed a CNN based on image patches classification to classify hyperspectral data pixels and reached significantly higher accuracies (84.4%) https://doi.org/10.1016/j.isprsjprs.2021.07.001 ...
This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 11% reaching an average user’s accuracy of 88.63% and an average producer’s accuracy of 88.59%, achieving state-of-art performance for tree species classification in tropical forests.
... Deep learning methods proved to be a robust alternative for remote sensing image classification, as they can learn optimal features and classification parameters to handle hyperspectral data (Signoroni et al., 2019). Different works successfully applied convolutional neural networks (CNN) (Krizhevsky et al., 2012) for tree species classification (Pölönen et al., 2018;Hartling et al., 2019;Fricker et al., 2019;Natesan et al., 2020;Mäyrä et al., 2021), including tropical and subtropical environments (Sothe et al., 2019(Sothe et al., , 2020Zhang et al., 2020;Tian et al., 2020;Abbas et al., 2021). Sothe et al. (2019) employed a CNN based on image patches classification to classify hyperspectral data pixels and reached significantly higher accuracies (84.4%) when compared to SVM (62.7%) and RF (59.2%). ...
This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 10% reaching an overall F1 score of 87.5% and an overall accuracy of 85.9%, achieving state-of-art performance for tree species classification in tropical forests.
... Compared with matching networks, it has fewer parameters and is more convenient to train. However, for the classification of hyperspectral images, the general prototypical networks structure is simple, and the problem of weak model generalization is prone to occur [39]. ...
... In the previous research, we have produced a complete set of sample data and constructed the classification framework of the prototypical networks [39]. The sample data set is based on hyperspectral images, as the data source, centered on the screen coordinate representation of the actual measured point's latitude and longitude, and clipped with different window sizes through the open source framework GDAL. ...
... Specifically, the neural network learns the nonlinear mapping of the input to the embedding space and uses the average value of the support set as the prototype of its class in the embedding space. Next, the nearest class prototype is found to classify the embedded query points [39]. The classification framework of the prototypical networks is shown in Figure 2, which mainly includes three parts: sample data input, image feature extraction, distance measurement and classification. ...
High-precision automatic identification and mapping of forest tree species composition is an important content of forest resource survey and monitoring. The airborne hyperspectral image contains rich spectral and spatial information, which provides the possibility of high-precision classification and mapping of forest tree species. Few-shot learning, as an application of deep learning, has become an effective method of image classification. Prototypical networks (P-Net) is a simple and practical deep learning network, which has significant advantages in solving few-shot classification problems. Considering the high band correlation and large data volume associated with airborne hyperspectral images, how to fully extract effective features, filter or reduce redundant features is the key to improving the classification accuracy of P-Net, in order to extract effective features in hyperspectral images and obtain a high-precision forest tree species classification model with limited samples. In this research, we embedded the convolutional block attention module (CBAM) between the convolution blocks of P-Net, the CBAM-P-Net was constructed, and a method to improve the feature extraction efficiency of the P-Net was proposed, although this method makes the network more complex and increases the computational cost to a certain extent. The results show that the combination strategy using Channel First for CBAM greatly improves the feature extraction efficiency of the model. In different sample windows, CBAM-P-Net has an average increase of 1.17% and 0.0129 in testing overall accuracy (OA) and kappa coefficient (Kappa). The optimal classification window is 17 × 17, the OA reaches 97.28%, and Kappa reaches 0.97, which is an increase of 1.95% and 0.0214 along with just 49 s of training time expended, respectively, compared with P-Net. Therefore, using a suitable sample window and applying the proposed CBAM-P-Net to classify airborne hyperspectral images can achieve high-precision classification and mapping of forest tree species.
Researchers and engineers have increasingly used Deep Learning (DL) for a variety of Remote Sensing (RS) tasks. However, data from local observations or via ground truth is often quite limited for training DL models, especially when these models represent key socio-environmental problems, such as the monitoring of extreme, destructive climate events, biodiversity, and sudden changes in ecosystem states. Such cases, also known as small data problems, pose significant methodological challenges. This review summarises these challenges in the RS domain and the possibility of using emerging DL techniques to overcome them. We show that the small data problem is a common challenge across disciplines and scales that results in poor model generalisability and transferability, yet this has not been investigated in a structured way. We first introduce ten emerging DL techniques: transfer learning, self-supervised learning, semi-supervised learning, few-shot learning, zero-shot learning, active learning, weakly supervised learning, multitask learning, process-aware learning, and ensemble learning; we also include a validation technique known as spatial k-fold cross validation. These techniques have shown promising potential in other scientific disciplines, but have been rarely applied in the RS domain. We also provide guidance on which learning technique to use in various cases, which helps to create a more methodologically robust DL application (and a greater number of them) that can be used to tackle socially important problems with limited data.
Hyperspectral image (HSI) consists of abundant spectral and spatial characteristics, which contribute to a more accurate identification of materials and land covers. However, most existing methods of hyperspectral image analysis primarily focus on spectral knowledge or coarse-grained spatial information while neglecting the fine-grained morphological structures. In the classification task of complex objects, spatial morphological differences can help to search for the boundary of fine-grained classes, e.g., forestry tree species. Focusing on subtle traits extraction, a spatial-logical aggregation network (SLA-NET) is proposed with morphological transformation for tree species classification. The morphological operators are effectively embedded with the trainable structuring elements, which contributes to distinctive morphological representations. We evaluate the classification performance of the proposed method on two tree species datasets, and the results demonstrate that the proposed SLA-NET significantly outperforms other state-of-the-art classifiers.
Achieving deep learning-based bearing fault diagnosis heavily relies on large labeled training samples. However, in real industry applications, labeled data are scarce or even impossible to obtain. In this study, we addressed a challenging few-shot bearing fault diagnosis problem with few or no training labeled samples of novel categories. To tackle this problem, we considered a semi-supervised prototype network based on few-shot bearing fault diagnosis with pseudo-labels. The existing prototypical networks with pseudo-label methods train a pseudo label model to label unlabeled samples using high-dimensional labeled data, which cannot eliminate the instability of the pseudo-label model caused by dimensional labeled features. To mitigate this issue, we used kernel principal component analysis to reduce the dimensions of and remove redundant information from high-dimensional data. Specifically, we used the pseudo-label prediction algorithm with probability distance to label unlabeled samples, aiming to improve the labeling accuracy. We applied two well-known bearing data sets for the validation experiments with symmetry parameters. The findings illustrated that the classification accuracy of the proposed method is higher than that of other existing methods.
