Figure - available from: Remote Sensing
This content is subject to copyright.
Source publication
The available stereo matching algorithms produce large number of false positive matches or only produce a few true-positives across oblique stereo images with large baseline. This undesired result happens due to the complex perspective deformation and radiometric distortion across the images. To address this problem, we propose a novel affine invar...
Similar publications
The objective of this study was to evaluate the potential of the practical application of unmanned aerial vehicles and RGB vegetation indices (VIs) in the monitoring of a coffee crop. The study was conducted in an experimental coffee field over a 12-month period. An RGB digital camera coupled to a UAV was used. Nine VIs were evaluated in this study...
Vehicle re-identification (Re-ID) involves identifying the same vehicle captured by other cameras, given a vehicle image. It plays a crucial role in the development of safe cities and smart cities. With the rapid growth and implementation of unmanned aerial vehicles (UAVs) technology, vehicle Re-ID in UAV aerial photography scenes has garnered sign...
Unmanned aerial vehicles (UAVs) do not collide with obstacles, generate a path in real-time, and must fly to the target point. The sampling-based rapidly exploring random tree (RRT) algorithm has the advantages of fast computation and low computational complexity. It is suitable for real-time path generation, but the optimal path cannot be guarante...
With UAVs on the rise, accurate detection and identification are crucial. Traditional unmanned aerial vehicle (UAV) identification systems involve opaque decision-making, restricting their usability. This research introduces an RF-based Deep Learning (DL) framework for drone recognition and identification. We use cutting-edge eXplainable Artificial...
With the widespread application of unmanned aerial vehicle (UAV) formation technology, it is very important to maintain good communication quality with the limited power and spectrum resources that are available. To maximize the transmission rate and increase the successful data transfer probability simultaneously, the convolutional block attention...
Citations
... Before performing the enhancement matching of weak texture features, it is necessary to first pair and recognize the corresponding plane scenes in the left and right images for obtaining corresponding planes. The affine-invariant feature matching algorithm described in reference [22] can robustly extract corresponding features from plane scenes with large viewpoint variations. Therefore, in this section, we employ this algorithm to automatically recognize corresponding planes. ...
This paper proposes a quasi-dense feature matching algorithm that combines image semantic segmentation and local feature enhancement networks to address the problem of the poor matching of image features because of complex distortions, considerable occlusions, and a lack of texture on large oblique stereo images. First, a small amount of typical complex scene data are used to train the VGG16-UNet, followed by completing the semantic segmentation of multiplanar scenes across large oblique images. Subsequently, the prediction results of the segmentation are subjected to local adaptive optimization to obtain high-precision semantic segmentation results for each planar scene. Afterward, the LoFTR (Local Feature Matching with Transformers) strategy is used for scene matching, enabling enhanced matching for regions with poor local texture in the corresponding planes. The proposed method was tested on low-altitude large baseline stereo images of complex scenes and compared with five classical matching methods. Results reveal that the proposed method exhibits considerable advantages in terms of the number of correct matches, correct rate of matches, matching accuracy, and spatial distribution of corresponding points. Moreover, it is well-suitable for quasi-dense matching tasks of large baseline stereo images in complex scenes with considerable viewpoint variations.
... To solve the multi-view geometry problem, [11] used joint learning of detectors and descriptors to improve feature point stability and repeatability in various situations. For the false positive matching caused by the complex perspective distortion and radiation distortion of stereo matching algorithms, [12] proposed a novel affine invariant feature matching algorithm, which utilizes end-to-end convolutional neural network (CNN) to obtain affine invariant Hessian regions, improves the correlation accuracy between the corresponding features, and achieves an excellent matching effect. However, it needs a better adaptive ability in complex scenes. ...
Heterogeneous high-resolution remote sensing image matching will be disturbed by the differences in sensor type, imaging angle, height, and imaging time, and the matching difficulty is further increased in complex scenes with dense urban buildings and noticeable height differences. This paper proposes a method for matching heterogeneous high-resolution remote sensing images based on partitioned feature extraction and three-dimensional spatial constraints. First, this paper conducts image partitioning based on the geometric differences of ground objects. Two feature extraction methods, namely, adaptive phase threshold and weighted moment map, are employed to extract feature points independently. To address the issue of inaccurate feature descriptions caused by drastic changes in viewing angles in buildings, we construct a robust feature descriptor by combining a multi-scale phase weighted energy convolution histogram (MSPW-ECH) with a new gradient location orientation histogram (GLOH)-like local feature descriptor. Additionally, a new similarity measure incorporating three-dimensional spatial constraints and the Marginalizing Sample Consensus (MAGSAC) method is applied to eliminate mismatched point pairs, ensuring the acquisition of precise matching points. Based on the feature detection results of two different synthetic data sets, it is evident that the proposed detector outperforms the three classical detectors in terms of repeatability and uniformity. Ultimately, the matching performance is experimentally verified on six groups of heterogeneous high-resolution remote sensing images. The experimental results show that the proposed method significantly outperforms RIFT, HAPCG, and MS-HLMO methods and achieves the best matching accuracy results.
... The end-to-end matching strategy combines three different stages of image feature extraction, description, and matching into a single training system, making it easier to learn globally optimal model parameters and adaptively improve the performance of each stage [62]. Figure 7 shows an example of the detected and matched features. ...
... The detection results of IHesAffNet on the Graf1-6 stereo image dataset, with yellow and cyan ellipses indicating the detected and matched features, respectively [62] Although LIFT is an end-to-end network model, it employs a multi-stage training model based on backpropagation in network training, which reduces the model's training efficiency and practicality. Additionally, LIFT uses the SFM strategy and random spatial transformation to provide matching image blocks for training, which limits the descriptors' discriminative power. ...
Multi-viewpoint image stitching aims to stitch images taken from different viewpoints into pictures with a broader field of view. The stitched images are subject to artifacts, geometric distortion, and blur distortion due to the mismatch of feature points, inaccurate homography estimation, and improper fusion of the unstitched images. Deep learning has recently been increasingly applied to multi-viewpoint image stitching to overcome these problems. However, there has thus far been little related research to summarize the different deep learning techniques used for multi-viewpoint image stitching. Therefore, this review aims to explore the application of deep learning to multi-viewpoint image stitching. To better illustrate this topic, we first summarize the acquisition methods for multi-viewpoint images and the main challenges of image stitching. After which, deep learning techniques for multi-view image stitching with a single camera are sorted out. Subsequently, deep learning techniques for multi-view image stitching with camera arrays, including parallel-view multi-view image stitching and cross-view multi-view image stitching, are presented. Next, we summarize image stitching datasets, evaluation metrics, and experimental data of several leading stitching algorithms on public datasets. Finally, we discuss potential issues and future work on image stitching with multi-viewpoint images.
... Deep learning algorithms, based on convolutional neural networks (CNNs), constitute new methods for affine-invariant feature matching [16]. Compared with handcrafted methods, designers of deep learning algorithms for image matching do not need to artificially design an intuitive calculation model and its empirical parameters; instead, they need to construct a CNN and its loss function. ...
To promote the development of deep learning for feature matching, image registration, and three-dimensional reconstruction, we propose a method of constructing a deep learning benchmark dataset for affine-invariant feature matching. Existing images often have large viewpoint differences and areas with weak texture, which may cause difficulties for image matching, with respect to few matches, uneven distribution, and single matching texture. To solve this problem, we designed an algorithm for the automatic production of a benchmark dataset for affine-invariant feature matching. It combined two complementary algorithms, ASIFT (Affine-SIFT) and LoFTR (Local Feature Transformer), to significantly increase the types of matching patches and the number of matching features and generate quasi-dense matches. Optimized matches with uniform spatial distribution were obtained by the hybrid constraints of the neighborhood distance threshold and maximum information entropy. We applied this algorithm to the automatic construction of a dataset containing 20,000 images: 10,000 ground-based close-range images, 6000 satellite images, and 4000 aerial images. Each image had a resolution of 1024 × 1024 pixels and was composed of 128 pairs of corresponding patches, each with 64 × 64 pixels. Finally, we trained and tested the affine-invariant deep learning model, AffNet, separately on our dataset and the Brown dataset. The experimental results showed that the AffNet trained on our dataset had advantages, with respect to the number of matching points, match correct rate, and matching spatial distribution on stereo images with large viewpoint differences and weak texture. The results verified the effectiveness of the proposed algorithm and the superiority of our dataset. In the future, our dataset will continue to expand, and it is intended to become the most widely used benchmark dataset internationally for the deep learning of wide-baseline image matching.
... In the next study, more appropriate methods on leakage error correction can be considered to gradually improve the consistency between GRACE and GNSS monitoring results. The residual vertical displacement from GNSS in the SGN region still had periodic signals when deducting the sum of deformation caused by terrestrial water load and GAC load from the vertical deformation sequence of GNSS stations; however, the underlying reasons of this result need further study [43]. ...
The terrestrial water resources in Shaanxi–Gansu–Ningxia (SGN) region are relatively scarce, and its climate change is unstable. Research on the deformation driven by terrestrial water load is of great significance to the dynamic maintenance of reference station networks. In this paper, data derived from Gravity Recovery and Climate Experiment (GRACE) and Global Navigation Satellite System (GNSS) from 2010 to 2014 were combined to monitor the spatiotemporal characteristics of surface vertical deformation caused by terrestrial water load change. The single scale factor was calculated by comparing CPC, WGHM, and GLDAS hydrological model to restore filtering leakage signal. The singular spectrum analysis (SSA) method was used to extract the principal component of temporal vertical deformation, and its spatial distribution was analyzed. At the same time, in order to study the relationship between the terrestrial water load deformation from GRACE and that from GNSS, the first-order term correction, the Atmosphere and Ocean De-aliasing Level-1B product (GAC) correction, and the first-order load LOVE number correction for GRACE were adopted in this paper. In addition, a quantitative comparative analysis of both the monitoring results was carried out. The results show that the time-variable characteristics of surface vertical deformation characterized by the filtered three hydrological models were consistent with those of GRACE. The correlation coefficient and Nash–Sutcliffe efficiency coefficient (NSE) values were the highest in the GLDAS model and the GRACE model, respectively; the former index is 0.93, while the latter is 0.85. The crustal vertical deformation from terrestrial water load showed a declining rate from 2010 to 2014. Its spatial change rate showed an obvious ladder distribution, with the surface subsidence rate gradually decreasing from south to north. In addition, weighted root mean square (WRMS) contribution rate of the crustal vertical deformation resulting from GRACE with GAC correction between the different GNSS stations ranged from 18.52% to 54.82%. The correlation coefficient between them was close to 0.70. After deducting the mass load impact of GRACE only, the WRMS contribution rate of the corresponding stations decreased from −8.42% to 21.18%. The correlation coefficient between them reduced noticeably. Adding GAC back can increase the comparability with GRACE and GNSS in terms of monitoring the crustal vertical deformation. The annual amplitude and phase of surface vertical deformation resulting from GRACE with GAC correction were close to those of GNSS. The research results can help to explore the motion mechanism between water migration and surface deformation, which is of benefit in the protection of the water ecological environment in the region.
... In the next study, more appropriate methods on leakage error correction can be considered to gradually improve the consistency between GRACE and GNSS monitoring results. The residual vertical displacement from GNSS in the SGN region still had periodic signals when deducting the sum of deformation caused by terrestrial water load and GAC load from the vertical deformation sequence of GNSS stations; however, the underlying reasons of this result need further study [43]. ...
The terrestrial water resources in Shaanxi–Gansu–Ningxia (SGN) region are relatively scarce, and its climate change is unstable. Research on the deformation driven by terrestrial water load is of great significance to the dynamic maintenance of reference station networks. In this paper, data derived from Gravity Recovery and Climate Experiment (GRACE) and Global Navigation Satellite System (GNSS) from 2010 to 2014 were combined to monitor the spatiotemporal characteristics of surface vertical deformation caused by terrestrial water load change. The single scale factor was calculated by comparing CPC, WGHM, and GLDAS hydrological model to restore filtering leakage signal. The singular spectrum analysis (SSA) method was used to extract the principal component of temporal vertical deformation, and its spatial distribution was analyzed. At the same time, in order to study the relationship between the terrestrial water load deformation from GRACE and that from GNSS, the first-order term correction, the Atmosphere and Ocean De-aliasing Level-1B product (GAC) correction, and the first-order load LOVE number correction for GRACE were adopted in this paper. In addition, a quantitative comparative analysis of both the monitoring results was carried out. The results show that the time-variable characteristics of surface vertical deformation characterized by the filtered three hydrological models were consistent with those of GRACE. The correlation coefficient and Nash–Sutcliffe efficiency coefficient (NSE) values were the highest in the GLDAS model and the GRACE model, respectively; the former index is 0.93, while the latter is 0.85. The crustal vertical deformation from terrestrial water load showed a declining rate from 2010 to 2014. Its spatial change rate showed an obvious ladder distribution, with the surface subsidence rate gradually decreasing from south to north. In addition, weighted root mean square (WRMS) contribution rate of the crustal vertical deformation resulting from GRACE with GAC correction between the different GNSS stations ranged from 18.52% to 54.82%. The correlation coefficient between them was close to 0.70. After deducting the mass load impact of GRACE only, the WRMS contribution rate of the corresponding stations decreased from −8.42% to 21.18%. The correlation coefficient between them reduced noticeably. Adding GAC back can increase the comparability with GRACE and GNSS in terms of monitoring the crustal vertical deformation. The annual amplitude and phase of surface vertical deformation resulting from GRACE with GAC correction were close to those of GNSS. The research results can help to explore the motion mechanism between water migration and surface deformation, which is of benefit in the protection of the water ecological environment in the region.
... Digital image matching (DIM) techniques are now widely used to automatically generate Digital Surface Models and Digital Terrain Models from satellite or aerial images of different resolutions [62,63] and further to detect changes, e.g., as a result of the development of plant cover or movements [64]. The NDVIartifcial estimation method proposed by us can be interwoven into ML-based digital image correlation (DIC) procedures to find the same objects, e.g., in oblique photos, multi-view oblique images [65], or pairs of images even with large differences in viewpoints [66]. We expect that having information about the third dimension of objects can be used to develop expansions of known "two-dimensional" textural features into their "three-dimensional" versions [28], which will facilitate the recognition of objects, determining their shapes, and estimating their NDVIartificial on a level similar to NDVItrue. ...
Generative adversarial networks (GAN) opened new possibilities for image processing and analysis. Inpainting, dataset augmentation using artificial samples or increasing resolution of aerial imagery are only a few notable examples of utilizing GANs in remote sensing. This is due to a unique construction and training process expressed as a duel between GAN components. In the following research, GAN has been applied to enhance panchromatic images with Normalized Difference Vegetation Index (NDVI). Panchromatic orthoimagery dataset with NDVI ground-truth labels was prepared by combining RGB and NIR orthophoto. The dataset was then utilized as input for a conditional generative adversarial network (cGAN) to perform an image-to-image translation. The main goal of the neural network was to generate an artificial NDVI channel for each processed 256px × 256px patch using only information available in the panchromatic imagery. The network achieved 0.9869 ± 0.0099 SSIM, 47.1635643 ± 4.0527963 PNSR and 0.0048023 ± 0.0018756 RSME on the test set. Perceptual evaluation was performed to verify the usability of the method when working with a real-life scenario. The research confirms that the structure and texture of the panchromatic aerial remote sensing image contains sufficient information for NDVI estimation for various objects of urban space. Even though these results can be used to highlight areas rich in vegetation and distinguish them from urban background, there is still room for improvement in terms of accuracy of estimated values.
... Digital image matching (DIM) techniques are now widely used to automatically generate Digital Surface Models and Digital Terrain Models from satellite or aerial images of different resolution [61,62] and further to detect changes, e.g. as a result of the development of plant cover or movements [63]. The NDVIartifcial estimation method proposed by us can be interwoven into ML-based digital image correlation (DIC) procedures, to find the same objects e.g., in oblique photos, multi-view oblique images [64] or pairs of images even with large differences in viewpoints [65]. We expect that having information about the third dimension of objects can be used to develop expansions of known "two-dimensional" textural features into their "three-dimensional" versions [28], which will facilitate the recognition of objects, determining their shapes and estimating their NDVIartificial on a level similar to NDVItrue. ...
Generative adversarial networks (GAN) opened new possibilities for image processing and analysis. Inpainting, dataset augmentation using artificial samples or increasing resolution of aerial imagery are only a few notable examples of utilizing GANs in remote sensing. This is due to a unique construction and training process expressed as a duel between GAN components. In the following research, GAN has been applied to enhance panchromatic images with Normalized Difference Vegetation Index (NDVI). Panchromatic orthoimagery dataset with NDVI ground-truth labels was prepared by combining RGB and NIR orthophoto. The dataset was then utilized as input for a conditional generative adversarial network (cGAN) to perform an image-to-image translation. The main goal of the neural network was to generate an artificial NDVI channel for each processed 256px × 256px patch using only information available in the panchromatic imagery. The network achieved 0.9869 ± 0.0099 SSIM, 47.1635643 ± 4.0527963 PNSR and 0.0048023 ± 0.0018756 RSME on the test set. Perceptual evaluation was performed to verify the usability of the method when working with a real-life scenario. The research confirms that the structure and texture of the panchromatic aerial remote sensing image contains sufficient information for NDVI estimation for various objects of urban space. Even though these results can be used to highlight areas rich in vegetation and distinguish them from urban background, there is still room for improvement in terms of accuracy of estimated values.
... The former methods were primarily based on electromagnetic and acoustic signals, such as ultra-wideband (UWB) [2][3][4], Bluetooth [5,6], wireless fidelity (Wi-Fi) [7][8][9], radio frequency identification (RFID) [10][11][12], ultrasonic or acoustic [13,14], geo-magnetism [15,16], pseudolite [17,18], and so on. The latter ones were based on computer vision [19,20] and inertial navigation system (INS) or pedestrian dead reckoning (PDR) [21,22]. Multiple techniques strength of RPs and map position-domain and signal-domain distances into the same metrics, reserving intrinsic connection and avoiding zero value of signal-domain distance. ...
... In this situation, the degree of differentiation between ED, MD, and CD was considered to be consistent. TD (ED, MD, and CD) were simultaneously applied to calculate the positioning results, as shown in Equation (20). ...
A density-based spatial clustering of applications with noise (DBSCAN) and three distances (TD) integrated Wi-Fi positioning algorithm was proposed, aiming to enhance the positioning accuracy and stability of fingerprinting by the dynamic selection of signal-domain distance to obtain reliable nearest reference points (RPs). Two stages were included in this algorithm. One was the offline stage, where the offline fingerprint database was constructed and the other was the online positioning stage. Three distances (Euclidean distance, Manhattan distance, and cosine distance), DBSCAN, and high-resolution distance selection principle were combined to obtain more reliable nearest RPs and optimal signal-domain distance in the online stage. Fused distance, the fusion of position-domain and signal-domain distances, was applied for DBSCAN to generate the clustering results, considering both the spatial structure and signal strength of RPs. Based on the principle that the higher resolution the distance, the more clusters will be obtained, the high-resolution distance was used to compute positioning results. The weighted K-nearest neighbor (WKNN) considering signal-domain distance selection was used to estimate positions. Two scenarios were selected as test areas; a complex-layout room (Scenario A) for post-graduates and a typical large indoor environment (Scenario B) covering 3200 m2. In both Scenarios A and B, compared with support vector machine (SVM), Gaussian process regression (GPR) and rank algorithms, the improvement rates of positioning accuracy and stability of the proposed algorithm were up to 60.44 and 60.93%, respectively. Experimental results show that the proposed algorithm has a better positioning performance in complex and large indoor environments.
... However, the GRACE monitoring results show the same variation characteristics at different locations in the study area, indicating that the CORS network can reflect local characteristics when used to monitor the vertical deformation of the terrestrial water load [18]. GRACE is limited by its spatial resolution, as it struggles to identify details at such scales as the study area [46][47][48]. Table 4 shows that the correlation coefficient of crustal vertical deformation related to terrestrial water load, as monitored by CORS and GRACE, which reaches 0.64~0.74. The results derived from the CORS network and GRACE are generally close for the same period. ...
Citation: Li, W.; Dong, J.; Wang, W.; Wen, H.; Liu, H.; Guo, Q.; Yao, G.; Zhang, C. Regional Crustal Vertical Deformation Driven by Terrestrial Water Load Depending on CORS Network and Environmental Loading Data: A Case Study of Southeast Zhejiang. Sensors 2021, 21, 7699. Abstract: Monitoring regional terrestrial water load deformation is of great significance to the dynamic maintenance and hydrodynamic study of the regional benchmark framework. In view of the lack of a spatial interpolation method based on the GNSS (Global Navigation Satellite System) elevation time series for obtaining terrestrial water load deformation information, this paper proposes to employ a CORS (Continuously Operating Reference Stations) network combined with environmental loading data, such as ECMWF (European Centre for Medium-Range Weather Forecasts) atmospheric data, the GLDAS (Global Land Data Assimilation System) hydrological model, and MSLA (Mean Sea Level Anomaly) data. Based on the load deformation theory and spherical harmonic analysis method, we took 38 CORS stations in southeast Zhejiang province as an example and comprehensively determined the vertical deformation of the crust as caused by regional terrestrial water load changes from January 2015 to December 2017, and then compared these data with the GRACE (Gravity Recovery and Climate Experiment) satellite. The results show that the vertical deformation value of the terrestrial water load in southeast Zhejiang, as monitored by the CORS network, can reach a centimeter, and the amplitude changes from −1.8 cm to 2.4 cm. The seasonal change is obvious, and the spatial distribution takes a ladder form from inland to coastal regions. The surface vertical deformation caused by groundwater load changes in the east-west-south-north-central sub-regions show obvious fluctuations from 2015 to 2017, and the trends of the five sub-regions are consistent. The amplitude of surface vertical deformation caused by groundwater load change in the west is higher than that in the east. We tested the use of GRACE for the verification of CORS network monitoring results and found a relatively consistent temporal distribution between both data sets after phase delay correction on GRACE, except for in three months-November in 2015, and January and February in 2016. The results show that the comprehensive solution based on the CORS network can effectively improve the monitoring of crustal vertical deformation during regional terrestrial water load change.
