Figure 10 - uploaded by Ehsan Khoramshahi
Content may be subject to copyright.
Source publication
Interest in drone solutions in forestry applications is growing. Using drones, datasets can be captured flexibly and at high spatial and temporal resolutions when needed. In forestry applications, fundamental tasks include the detection of individual trees, tree species classification, bio-mass estimation, etc. Deep Neural Networks (DNN) have shown...
Similar publications
Interest in drone solutions in forestry applications is growing. Using drones, datasets can be captured flexibly and at high spatial and temporal resolutions when needed. In forestry applications, fundamental tasks include the detection of individual trees, tree species classification, biomass estimation, etc. Deep neural networks (DNN) have shown...
Citations
... Different technologies and types of sensors are used to identify tree species, with the capabilities of hyperspectral cameras showing great interest (Cao et al., 2018;2020a;Sothe et al., 2019;Saarinen et al., 2017;Tuominen et al., 2017;Nezami et al., 2020; Tang et al., 2022;Gimenez et al., 2021;Viinikka et al., 2020;Martin et al., 2020;Rizvi et al., 2020;Dabiri and Lang, 2018). This is due to an increase in the availability of multispectral cameras, a decrease in their cost, a decrease in size and weight, as well as the development of unmanned vehicles. ...
... Deep Learning (DL) is getting ubiquitous in state-of-the-art land cover classification [4]. Previous approaches to tree species classification in Norway either used classic machine learning approaches [2] or are drone-based and therefore have limited scalability [6]. One of the main challenges of successfully implementing a scalable deep learning approach for tree species mapping in Norway is the limited availability and quality of labeled data. ...
... In general results are expected to be similar for other land cover classification tasks, which are often characterized by large natural variation within classes and gradual changes between them. 6 ...
Tree species mapping of Norwegian production forests is a time-consuming process as forest associations largely rely on manual interpretation of earth observation data. Deep learning based image segmentation techniques have the potential to improve automated tree species classification, but a major challenge is the limited quality and availability of training data. Semi-supervised techniques could alleviate the need for training label and weak supervision enables handling coarse-grained and noisy labels. In this study, we evaluated the added value of semi-supervised deep learning methods in a weakly supervised setting. Specifically, consistency training and pseudo-labeling are applied for tree species classification from aerial ortho imagery in Norway. The techniques are generic and relevant for the wider earth observation domain, especially for other land cover segmentation tasks. The results show that consistency training gives a significant performance increase. Pseudo-labeling on the other hand does not, potentially this is due to varying convergence speeds for different classes causing confirmation bias or a partial violation of the cluster assumption.
... Trier et al. (2018) also used airborne hyperspectral data to classify pine, spruce, and birch trees in a boreal forest using a CNN. Nezami et al. (2020) showed very accurate results for classifying the same tree species testing CNNs with different combinations of hyperspectral and RGB imagery and canopy height models. Thus far, mapping tree species in forests often requires high spectral resolution data, which is cumbersome to access for non-specialist users. ...
... Atfer the removal of shadowed, low-, and non-vegetated pixels prior to CNNclassification, P. abies, P. sylvestris, and B. pendula have been mapped in boreal forests in airborne hyperspectral data (OA = 87%) and RGB data (OA = 74%) (Trier et al., 2018). The same species have been mapped with different combinations of hyperspectral data, RGB imagery, and canopy height models with highest accuracies (OA = 98%) (Nezami et al., 2020). CNNs have been successfully used to classify two Pinus species and non-Pinus in previously extracted tree crowns from UAVbased RGB imagery (F1 = 80%) (Natesan et al., 2019). ...
The use of unmanned aerial vehicles (UAVs) in vegetation remote sensing allows a time-flexible and cost-effective acquisition of very high-resolution imagery. Still, current methods for the mapping of forest tree species do not exploit the respective, rich spatial information. Here, we assessed the potential of convolutional neural networks (CNNs) and very high-resolution RGB imagery from UAVs for the mapping of tree species in temperate forests. We used multicopter UAVs to obtain very high-resolution (<2 cm) RGB imagery over 51 ha of temperate forests in the Southern Black Forest region, and the Hainich National Park in Germany. To fully harness the end-to-end learning capabilities of CNNs, we used a semantic segmentation approach (U-net) that concurrently segments and classifies tree species from imagery. With a diverse dataset in terms of study areas, site conditions, illumination properties, and phenology, we accurately mapped nine tree species, three genus-level classes, deadwood, and forest floor (mean F1-score 0.73). A larger tile size during CNN training negatively affected the model accuracies for underrepresented classes. Additional height information from normalized digital surface models slightly increased the model accuracy but increased computational complexity and data requirements. A coarser spatial resolution substantially reduced the model accuracy (mean F1-score of 0.26 at 32 cm resolution). Our results highlight the key role that UAVs can play in the mapping of forest tree species, given that air-and spaceborne remote sensing currently does not provide comparable spatial resolutions. The end-to-end learning capability of CNNs makes extensive preprocessing partly obsolete. The use of large and diverse datasets facilitate a high degree of generalization of the CNN, thus fostering transferability. The synergy of high-resolution UAV imagery and CNN provide a fast and flexible yet accurate means of mapping forest tree species.
Classifying grapevine varieties is crucial in precision viticulture, as it allows for accurate estimation of vineyard row growth for different varieties and ensures authenticity in the wine industry. This task can be performed with time-consuming destructive methods, including data collection and analysis in the laboratory. In contrast, unmanned aerial vehicles (UAVs) offer a markedly more efficient and less restrictive method for gathering hyperspectral data, even though they may yield data with higher levels of noise. Therefore, the first task is the processing of these data to correct and downsample large amounts of data. In addition, the hyperspectral signatures of grape varieties are very similar. In this study, we propose the use of a convolutional neural network (CNN) to classify seventeen different varieties of red and white grape cultivars. Instead of classifying individual samples, our approach involves processing samples alongside their surrounding neighborhood for enhanced accuracy. The extraction of spatial and spectral features is addressed with (1) a spatial attention layer and (2) inception blocks. The pipeline goes from data preparation to dataset elaboration, finishing with the training phase. The fitted model is evaluated in terms of response time, accuracy and data separability and is compared with other state-of-the-art CNNs for classifying hyperspectral data. Our network was proven to be much more lightweight by using a limited number of input bands (40) and a reduced number of trainable weights (560 k parameters). Hence, it reduced training time (1 h on average) over the collected hyperspectral dataset. In contrast, other state-of-the-art research requires large networks with several million parameters that require hours to be trained. Despite this, the evaluated metrics showed much better results for our network (approximately 99% overall accuracy), in comparison with previous works barely achieving 81% OA over UAV imagery. This notable OA was similarly observed over satellite data. These results demonstrate the efficiency and robustness of our proposed method across different hyperspectral data sources.
Timely and accurate information on tree species is of great importance for the sustainable management of natural resources, forest inventory, biodiversity detection, and carbon stock calculation. The advancement of remote sensing technology and artificial intelligence has facilitated the acquisition and analysis of remote sensing data, resulting in more precise and effective classification of tree species. A review of the remote sensing data and deep learning tree species classification methods is lacking in its analysis of unimodal and multimodal remote sensing data and classification methods in this field. To address this gap, we search for major trends in remote sensing data and tree species classification methods, provide a detailed overview of classic deep learning-based methods for tree species classification, and discuss some limitations of tree species classification.
Erannis jacobsoni Djak (Lepidoptera, Geometridae) is a leaf-feeding pest unique to Mongolia. Outbreaks of this pest can cause larch needles to shed slowly from the top until they die, leading to a serious imbalance in the forest ecosystem. In this work, to address the need for the low-cost, fast, and effective identification of this pest, we used field survey indicators and UAV images of larch forests in Binder, Khentii, Mongolia, a typical site of Erannis jacobsoni Djak pest outbreaks, as the base data, calculated relevant multispectral and red–green–blue (RGB) features, used a successive projections algorithm (SPA) to extract features that are sensitive to the level of pest damage, and constructed a recognition model of Erannis jacobsoni Djak pest damage by combining patterns in the RGB vegetation indices and texture features (RGBVI&TF) with the help of random forest (RF) and convolutional neural network (CNN) algorithms. The results were compared and evaluated with multispectral vegetation indices (MSVI) to explore the potential of UAV RGB images in identifying needle pests. The results show that the sensitive features extracted based on SPA can adequately capture the changes in the forest appearance parameters such as the leaf loss rate and the colour of the larch canopy under pest damage conditions and can be used as effective input variables for the model. The RGBVI&TF-RF440 and RGBVI&TF-CNN740 models have the best performance, with their overall accuracy reaching more than 85%, which is a significant improvement compared with that of the RGBVI model, and their accuracy is similar to that of the MSVI model. This low-cost and high-efficiency method can excel in the identification of Erannis jacobsoni Djak-infested regions in small areas and can provide an important experimental theoretical basis for subsequent large-scale forest pest monitoring with a high spatiotemporal resolution.
We present an evaluation of different deep learning and machine learning approaches for tree health classification in the Black Forest, the Harz Mountains, and the Göttinger Forest on a unique, highly accurate tree-level dataset. The multispectral UAV data were collected from eight forest plots with diverse tree species, mostly conifers. As ground truth data (GTD), nearly 1500 tree polygons with related attribute information on the health status of the trees were used. This data were collected during extensive fieldwork using a mobile application and subsequent individual tree segmentation. Extensive preprocessing included normalization, NDVI calculations, data augmentation to deal with the underrepresented classes, and splitting the data into training, validation, and test sets. We conducted several experiments using a classical machine learning approach (random forests), as well as different convolutional neural networks (CNNs)—ResNet50, ResNet101, VGG16, and Inception-v3—on different datasets and classes to evaluate the potential of these algorithms for tree health classification. Our first experiment was a binary classifier of healthy and damaged trees, which did not consider the degree of damage or tree species. The best results of a 0.99 test accuracy and an F1 score of 0.99 were obtained with ResNet50 on four band composites using the red, green, blue, and infrared bands (RGBI images), while VGG16 had the worst performance, with an F1 score of only 0.78. In a second experiment, we also distinguished between coniferous and deciduous trees. The F1 scores ranged from 0.62 to 0.99, with the highest results obtained using ResNet101 on derived vegetation indices using the red edge band of the camera (NDVIre images). Finally, in a third experiment, we aimed at evaluating the degree of damage: healthy, slightly damaged, and medium or heavily damaged trees. Again, ResNet101 had the best performance, this time on RGBI images with a test accuracy of 0.98 and an average F1 score of 0.97. These results highlight the potential of CNNs to handle high-resolution multispectral UAV data for the early detection of damaged trees when good training data are available.
Tree species identification is a critical component of forest resource monitoring, and timely and accurate acquisition of tree species information is the basis for sustainable forest management and resource assessment. Airborne hyperspectral images have rich spectral and spatial information and can detect subtle differences among tree species. To fully utilize the advantages of hyperspectral images, we propose a double-branch spatial–spectral joint network based on the SimAM attention mechanism for tree species classification. This method achieved high classification accuracy on three tree species datasets (93.31% OA value obtained in the TEF dataset, 95.7% in the Tiegang Reservoir dataset, and 98.82% in the Xiongan New Area dataset). The network consists of three parts: spectral branch, spatial branch, and feature fusion, and both branches make full use of the spatial–spectral information of pixels to avoid the loss of information. In addition, the SimAM attention mechanism is added to the feature fusion part of the network to refine the features to extract more critical features for high-precision tree species classification. To validate the robustness of the proposed method, we compared this method with other advanced classification methods through a series of experiments. The results show that: (1) Compared with traditional machine learning methods (SVM, RF) and other state-of-the-art deep learning methods, the proposed method achieved the highest classification accuracy in all three tree datasets. (2) Combining spatial and spectral information and incorporating the SimAM attention mechanism into the network can improve the classification accuracy of tree species, and the classification performance of the double-branch network is better than that of the single-branch network. (3) The proposed method obtains the highest accuracy under different training sample proportions, and does not change significantly with different training sample proportions, which are stable. This study demonstrates that high-precision tree species classification can be achieved using airborne hyperspectral images and the methods proposed in this study, which have great potential in investigating and monitoring forest resources.
The application of hyperspectral imagery coupled with deep learning shows vast promise in plant species discrimination. Reshaping one-dimensional (1D) leaf-level reflectance data (LLRD) into two-dimensional (2D) grayscale images as convolutional neural network (CNN) model input demonstrated marked effectiveness in plant species distinction. However, the impact of the image shape on CNN model performance remained unexplored. This study addressed this by reshaping data into fifteen distinct rectangular formats and creating nine CNN models to examine the effect of image structure. Results demonstrated that irrespective of CNN model structure, elongated narrow images yielded superior species identification results. The ‘l’-shaped images at 225 × 9 pixels outperformed other configurations based on 93.95% accuracy, 94.55% precision, and 0.94 F1 score. Furthermore, ‘l’-shaped hyperspectral images consistently produced high classification precision across species. The results suggest this image shape boosts robust predictive performance, paving the way for enhancing leaf trait estimation and proposing a practical solution for pixel-level categorization within hyperspectral imagery (HSIs).
Tree species information is important for forest management, especially in seedling stands. To mitigate the spectral admixture of understory reflectance with small and lesser foliaged seedling canopies, we proposed an image pre-processing step based on the canopy threshold (Cth) applied on drone-based multispectral images prior to feeding classifiers. This study focused on (1) improving the classification of seedlings by applying the introduced technique; (2) comparing the classification accuracies of the convolutional neural network (CNN) and random forest (RF) methods; and (3) improving classification accuracy by fusing vegetation indices to multispectral data. A classification of 5417 field-located seedlings from 75 sample plots showed that applying the Cth technique improved the overall accuracy (OA) of species classification from 75.7% to 78.5% on the Cth-affected subset of the test dataset in CNN method (1). The OA was more accurate in CNN (79.9%) compared to RF (68.3%) (2). Moreover, fusing vegetation indices with multispectral data improved the OA from 75.1% to 79.3% in CNN (3). Further analysis revealed that shorter seedlings and tensors with a higher proportion of Cth-affected pixels have negative impacts on the OA in seedling forests. Based on the obtained results, the proposed method could be used to improve species classification of single-tree detected seedlings in operational forest inventory.
