Figure - available from: GeoInformatica
This content is subject to copyright. Terms and conditions apply.
Source publication
The field of GeoAI or Geospatial Artificial Intelligence has undergone rapid development since 2017. It has been widely applied to address environmental and social science problems, from understanding climate change to tracking the spread of infectious disease. A foundational task in advancing GeoAI research is the creation of open, benchmark datas...
Citations
... https://github.com/ASUcicilab/ GeoImageNet Li et al. (2023) BigEarthNet A benchmark archive consisting of over 590k pairs of Sentinel-1 and Sentinel-2 image patches that were annotated with multi-labels of the CORINE Land Cover types to support deep learning studies in Earth remote sensing. ...
Generative Artificial Intelligence (GAI) represents an emerging field that promises the creation of synthetic data and outputs in different modalities. GAI has recently shown impressive results across a large spectrum of applications ranging from biology, medicine, education, legislation, computer science, and finance. As one strives for enhanced safety, efficiency, and sustainability, generative AI indeed emerges as a key differentiator and promises a paradigm shift in the field. This article explores the potential applications of generative AI and large language models in geoscience. The recent developments in the field of machine learning and deep learning have enabled the generative model's utility for tackling diverse prediction problems, simulation, and multi‐criteria decision‐making challenges related to geoscience and Earth system dynamics. This survey discusses several GAI models that have been used in geoscience comprising generative adversarial networks (GANs), physics‐informed neural networks (PINNs), and generative pre‐trained transformer (GPT)‐based structures. These tools have helped the geoscience community in several applications, including (but not limited to) data generation/augmentation, super‐resolution, panchromatic sharpening, haze removal, restoration, and land surface changing. Some challenges still remain, such as ensuring physical interpretation, nefarious use cases, and trustworthiness. Beyond that, GAI models show promises to the geoscience community, especially with the support to climate change, urban science, atmospheric science, marine science, and planetary science through their extraordinary ability to data‐driven modelling and uncertainty quantification.
... Terrain applications of machine learning are relatively new and range from terrain feature extraction to recognizing terrain-related hazards. For example, Li et al. (2022b) developed a labeled training dataset to detect natural features in the United States from imagery and elevation derivatives. They successfully tested this large-scale detection task using an object detection model Faster RCNN (Ren et al. 2015) with a CNN backbone -RetinaNet (Lin et al. 2017). ...
... Some geographic features, such as those having substantial elevation changes (for example, ridges and valleys) can be predicted better by multiple input layers. Previous work has demonstrated by loading a slope surface, a hillshade image, and satellite imagery into the RGB bands of the output training image (Figure 4) (Arundel et al. 2020, Li et al. 2022b. A fourth information layer can be loaded into the depth layer of an RGB image (RGB-D). ...
... The authors have been working on specific geomorphometry projects to advance the theory and best practices for cognition-driven automated mapping of topographically salient landforms (Arundel and Sinha 2018;Sinha and Arundel 2020;Arundel and Sinha 2020;Joly, Sinha, and Hassan 2022). In parallel, they are also studying the strengths and limitations of deep learning-based image analysis for delimiting landforms in the United States (Arundel, Li, and Wang 2020;Li et al. 2022). For example, hillshade, slope values, and natural colour imagery from the National Agricultural Imaging Program (NAIP) were used in a convolutional neural network (FasterRCNN and RetinaNet) to predict landform locations ( Figure 1). ...
... [11] generate saliency maps based on two model explanation methods: perturbation-based (manipulating input images) and gradient-based (visualizing model weights). Similarly, Li et al. [14] utilized the Shapley Additive exPlanations (SHAP) [16], to compare the spatial effects extracted by XGBoost with traditional statistical approaches like the spatial lag model and multi-scale geographically weighted regression (MGWR). And, Xing and Sieber [24] employed SHAP to visualized the importance of feature maps at different stages of the convolution process. ...
This paper presents our ongoing work towards XAI for Mobility Data Science applications, focusing on explainable models that can learn from dense trajectory data, such as GPS tracks of vehicles and vessels using temporal graph neural networks (GNNs) and counterfactuals. We review the existing GeoXAI studies, argue the need for comprehensible explanations with human-centered approaches, and outline a research path toward XAI for Mobility Data Science.
... To validate the produced forest canopy cover map, a validation dataset was acquired from 1/3 of the reference locations. The confusion matrix method, as described in previous studies by [14,15,16], was used to evaluate the map's accuracy using various metrics such as the producer's accuracy, user's accuracy, overall accuracy, F1 score, and weighted kappa. ...
... The limited availability of multimodal datasets and labeled training data is a major challenge in multimodal object detection for remote sensing [11]. However, recent efforts have introduced the GeoImageNet dataset [12]. Moreover, semantic segmentation datasets, e.g., ISPRS 2D Semantic Labeling Contest -Potsdam dataset 1 , can be converted into object detection datasets [13]. ...
... GeoImageNet [12] is a benchmark dataset specifically designed for GeoAI applications, focusing on natural features. It incorporates multi-source data, including RGB remote sensing images and Digital Elevation Model (DEM), which provides valuable spatial context for object detection and classification tasks. ...
... Besides, results in Table 1 demonstrate the strong performance of SuperYOLO with its compact multimodal fusion module across all datasets except GeoImageNet, because YOLOrs and SuperYOLO are specialized in small objects. However, remote sensing objects are not always small [12]. Furthermore, those algorithms do not learn any weighting schema for different modalities, which explains the deterioration of performances in YOLOrs and the slight improvement of results with SuperYOLO. ...
... The limited availability of multimodal datasets and labeled training data is a major challenge in multimodal object detection for remote sensing [11]. However, recent efforts have introduced the GeoImageNet dataset [12]. Moreover, semantic segmentation datasets, e.g., ISPRS 2D Semantic Labeling Contest -Potsdam dataset 1 , can be converted into object detection datasets [13]. ...
... GeoImageNet [12] is a benchmark dataset specifically designed for GeoAI applications, focusing on natural features. It incorporates multi-source data, including RGB remote sensing images and Digital Elevation Model (DEM), which provides valuable spatial context for object detection and classification tasks. ...
... Besides, results in Table 1 demonstrate the strong performance of SuperYOLO with its compact multimodal fusion module across all datasets except GeoImageNet, because YOLOrs and SuperYOLO are specialized in small objects. However, remote sensing objects are not always small [12]. Furthermore, those algorithms do not learn any weighting schema for different modalities, which explains the deterioration of performances in YOLOrs and the slight improvement of results with SuperYOLO. ...
Object detection in remote sensing is a crucial computer vision task that has seen significant advancements with deep learning techniques. However, most existing works in this area focus on the use of generic object detection and do not leverage the potential of multimodal data fusion. In this paper, we present a comparison of methods for multimodal object detection in remote sensing, survey available multimodal datasets suitable for evaluation, and discuss future directions.
Generative Artificial Intelligence (GAI) represents an emerging field that promises the creation of synthetic data and outputs in different modalities. GAI has recently shown impressive results across a large spectrum of applications ranging from biology, medicine, education, legislation, computer science, and finance. As one strives for enhanced safety, efficiency, and sustainability , generative AI indeed emerges as a key differentiator and promises a paradigm shift in the field. This paper explores the potential applications of generative AI and large language models in geoscience. The recent developments in the field of machine learning and deep learning have enabled the generative model's utility for tackling diverse prediction problems, simulation, and multi-criteria decision-making challenges related to geoscience and Earth system dynamics. This survey discusses several GAI models that have been used in geoscience comprising generative adversarial networks (GANs), physics-informed neural networks (PINNs), and generative pre-trained transformer (GPT)-based structures. These tools have helped the geoscience community in several applications, including (but not limited to) data gener-ation/augmentation, super-resolution, panchromatic sharpening, haze removal, restoration, and land surface changing. Some challenges still remain such as ensuring physical interpretation, nefarious use cases, and trustworthiness. Beyond that, GAI models show promises to the geoscience community, especially with the support to climate change, urban science, atmospheric science, marine science, and planetary science through their extraordinary ability to data-driven modeling and uncertainty quantification.
Improving the interpretability of geospatial artificial intelligence (GeoAI) models has become critically important to open the ‘black box’ of complex AI models, such as deep learning. This paper compares popular saliency map generation techniques and their strengths and weaknesses in interpreting GeoAI and deep learning models’ reasoning behaviors, particularly when applied to geospatial analysis and image processing tasks. We surveyed two broad classes of model explanation methods: perturbation-based and gradient-based methods. The former identifies important image areas, which help machines make predictions by modifying a localized area of the input image. The latter evaluates the contribution of every single pixel of the input image to the model’s prediction results through gradient backpropagation. In this study, three algorithms—the occlusion method, the integrated gradients method, and the class activation map method—are examined for a natural feature detection task using deep learning. The algorithms’ strengths and weaknesses are discussed, and the consistency between model-learned and human-understandable concepts for object recognition is also compared. The experiments used two GeoAI-ready datasets to demonstrate the generalizability of the research findings.
