Figure 1 - uploaded by Chunping Qiu
Content may be subject to copyright.
Source publication
In this paper we discuss the potential and challenges regarding SAR-optical stereogrammetry for urban areas, using very-high-resolution (VHR) remote sensing imagery. Since we do this mainly from a geometrical point of view, we first analyze the height reconstruction accuracy to be expected for different stereogrammetric configurations. Then, we pro...
Contexts in source publication
Context 1
... remote sensing images were acquired. Details of the Munich dataset, which consists of images acquired by the spaceborne optical WorldView-2 (WV2) sensor, as well as the spaceborne SAR sensor TerraSAR-X (TSX) and the airborne SAR sensor MEMPHIS, are summarized in Tab. 1. The relation of the track orientations of the three acquisitions is shown in Fig. 10. ...
Context 2
... track orientation study area Figure 10: The track relation of the three sensors (in Google Earth) over Munich. ...
Context 3
... remote sensing images acquired over the city of Berlin, Germany, were used, whose details are summarized in Tab. 2. Again, the optical image was acquired by the spaceborne optical WorldView-2 (WV2) sensor, and the SAR image by the spaceborne SAR sensor TerraSAR-X (TSX). The relation of the track orientations of the two acquisitions is shown in Fig. 11. ...
Context 4
... track orientation study area Figure 11: The track relation of the two sensors (in Google Earth) over Berlin. ...
Context 5
... to dB and reduced to approximately square pixels. In addition the SAR images were rotated to an approximate north-aligned orientation. More details about MEMPHIS data preprocessing can be found in [26]. To illustrate the effect of the preprocessing, subsets showing the TUM main campus over Munich and main train station of Berlin are depicted in Fig. 12 and Fig. 13, respectively. For a quantitative evaluation of the stereogrammetric reconstruction results, we used dense LiDAR reference point clouds of centimeter and decimeter accuracy in Munich and Berlin, respectively ...
Context 6
... matching results relying on HOPC as similarity measure and the IMBLS search strategy without subsequent outlier removal are exemplarily displayed in Fig. 14 for WV2+MEMPHIS over Munich, Fig. 15 for WV2+TSX over Munich and Fig. 16 for WV2+TSX over Berlin. Considering conciseness and clarity, only a subset of the corresponding experimental scene is shown. In analogy, the results achieved using the outlier removal approach exploiting all similarity measures in a joint manner can be seen in ...
Context 7
... matching results relying on HOPC as similarity measure and the IMBLS search strategy without subsequent outlier removal are exemplarily displayed in Fig. 14 for WV2+MEMPHIS over Munich, Fig. 15 for WV2+TSX over Munich and Fig. 16 for WV2+TSX over Berlin. Considering conciseness and clarity, only a subset of the corresponding experimental scene is shown. In analogy, the results achieved using the outlier removal approach exploiting all similarity measures in a joint manner can be seen in Figs. 17, 18 and 19, respectively. ...
Context 8
... matching results relying on HOPC as similarity measure and the IMBLS search strategy without subsequent outlier removal are exemplarily displayed in Fig. 14 for WV2+MEMPHIS over Munich, Fig. 15 for WV2+TSX over Munich and Fig. 16 for WV2+TSX over Berlin. Considering conciseness and clarity, only a subset of the corresponding experimental scene is shown. In analogy, the results achieved using the outlier removal approach exploiting all similarity measures in a joint manner can be seen in Figs. 17, 18 and 19, respectively. ...
Context 9
... that the remaining difference between the individual similarity measures only reflects certain fine-positioning dif- ferences within the threshold window. Figure 21 additionally compares the results before outlier removal. Here, it can be seen that the feature-based similarity measures that combine HOG-derivatives and an L 2 -norm cost function slightly outperform the signal-based similarity measures. ...
Similar publications
Aimed at the problem that range-azimuth joint jamming can effectively generate high fidelity false targets in synthetic aperture radar (SAR) images, an anti-2D deceptive jamming method for multibaseline interferometric SAR is proposed in this article. The method consists of three steps: identification; classification; and suppression. First, the ta...
The spaceborne transmitter/missile-borne receiver (ST/MR) synthetic aperture radar (SAR) could provide several unique advantages, such as wide coverage, unrestricted geography, a small detection probability of the missile, and forward-looking imaging. However, it is also accompanied by problems in imaging, including geometric model establishment an...
While deep learning techniques have an increasing impact on many technical fields, gathering sufficient amounts of training data is a challenging problem in remote sensing. In particular, this holds for applications involving data from multiple sensors with heterogeneous characteristics. One example for that is the fusion of synthetic aperture rada...
While deep learning techniques have an increasing impact on many technical fields, gathering sufficient amounts of training data is a challenging problem in remote sensing. In particular, this holds for applications involving data from multiple sensors with heterogeneous characteristics. One example for that is the fusion of synthetic aperture rada...
Citations
... SAR images, acquired by active side-looking radar sensors, depict strong backscattering coefficients of targets, while optical images, acquired by passive top-viewing sensors, present radiometric properties resembling human eyes and clear structures of targets. These independent and complementary features provide a novel approach for downstream applications [1]. For instance, optical-SAR image fusion leverages the synergistic properties of both imaging systems to effectively remove clouds that appear opaque in all optical bands [2]. ...
Multi-sensor remote sensing applications require the registration of optical and Synthetic Aperture Radar (SAR) images, which presents challenges due to significant radiometric and geometric differences resulting from distinct imaging mechanisms. Although various algorithms have been proposed, including hand-crafted features and deep learning networks, most of them focus on matching radiometric-invariant features while ignoring geometric differences. Furthermore, these algorithms often achieve promising results on datasets that use manually labeled ground truths that may be less reliable for high-resolution SAR images affected by speckle noise. To address these issues, we propose a robust global-to-local registration algorithm consisting of four modules: geocoding, global matching, local matching, and refinement. We generate a geometry-invariant mask in the geocoding module to help the local matching module focus on valid areas, introduce a fast global matching method to solve large offsets, and use matching confidence to guide subsequent local matching based on the accuracy of global matching. We propose a feature based on multi-directional anisotropic Gaussian derivatives (MAGD) and embed it into the confidence-aware local matching with the geometry-invariant mask to reduce the effect of geometric differences. Finally, we refine correspondence positions and remove outliers. We also build a high-accuracy evaluation dataset with hundreds of image pairs, where the ground truth is obtained by meta poles, which have clear and reliable structures in both optical and SAR images. Experimental results on this dataset demonstrate the superiority of our proposed algorithm compared to several state-of-the-art methods.
... This scenario is common for SAR data because having a mapping between SAR and optical data can be very challenging. [16] outline the process of doing point matching between optical imagery from WV2 and two different types of SAR data. In particular, they found that it is very difficult to come up with a proper similarity metric between SAR and optical imagery as well as perform key point detection between co-registered images. ...
... Consequently, the corresponding point in the right image of one point in the left image should be located on the epipolar line. The search region can be reduced from 2D to 1D, which increases the matching accuracy and efficiency [38]. ...
Accurate optical and Synthetic Aperture Radar (SAR) image registration is crucial to multi-sensor remote sensing applications. Though several algorithms have been proposed, the practical implementation that directly matches large and high-resolution optical and SAR images remains under-explored by the community. Equipped with the satellite positioning parameters, optical and SAR images can be roughly registered based on geographic coordinates. However, the relative positioning accuracy is still dozens, even hundreds of pixels due to the inaccuracies of sensor parameters and elevation. Consequently, we propose a robust registration algorithm, which consists of two stages, where the horizontal positioning errors can be reduced in the first stage, and then we fine-tune the correspondences in the second stage. Specifically, we propose a novel template matching method based on the dilated convolutional feature (DCF) and epipolar-oriented phase correlation. DCF is constructed by a depthwise-separable dilated convolution with multi-channel gradients, which are generated by Sobel operator for optical images and ratio of exponentially weighted averages (ROEWA) operator for SAR images. Owing to the large reception field of dilated convolution, DCF can retain invariance even for large relative positioning errors. The epipolar-oriented rectangle template, which stretches along the epipolar line, is then proposed to capture more overlapping areas compared to square templates. Furthermore, the outlier removal is implemented in the coordinate system of optical images to avoid the effect of range compression in SAR images. Inliers are finally used to refine the rational polynomial coefficients (RPCs) based on the bundle adjustment technique. Experimental results on high-resolution optical and SAR image products of various scenarios demonstrate the effectiveness of the proposed registration framework. The relative positioning errors of the refined RPCs can be reduced from hundreds of pixels to sub-pixel level.
... Moreover, VHR SAR-optical image pairs have great potential for analyzing complex urban areas. If the accuracy of key points detection can be improved and the similarity between SAR images and optical images can be measured, then the accuracy of 3-D reconstruction can also be improved [12]. Therefore, how to generate high-quality epipolar images based on SAR-optical image pairs needs to be further studied. ...
Epipolar images can improve the efficiency and accuracy of dense matching by restricting the search range of correspondences from 2-D to 1-D, which play an important role in 3-D reconstruction. As most of the satellite images in archives are incidental collections, which do not have rigorous stereo properties, in this paper, we propose a general framework to generate epipolar images for both in-track and cross-track stereo images. We first investigate the theoretical epipolar constraints of single-sensor and multi-sensor images and then introduce the proposed framework in detail. Considering large elevation changes in mountain areas, the publicly available digital elevation model (DEM) is applied to reduce the initial offsets of two stereo images. The left image is projected into the image coordinate system of the right image using the rational polynomial coefficients (RPCs). By dividing the raw images into several blocks, the epipolar images of each block are parallel generated through a robust feature matching method and fundamental matrix estimation, in which way, the horizontal disparity can be drastically reduced while maintaining negligible vertical disparity for epipolar blocks. Then, stereo matching using the epipolar blocks can be easily implemented and the forward intersection method is used to generate the digital surface model (DSM). Experimental results on several in-track and cross-track images, including optical-optical, SAR-SAR, and SAR-optical pairs, demonstrate the effectiveness of the proposed framework, which not only has obvious advantages in mountain areas with large elevation changes but also can generate high-quality epipolar images for flat areas. The generated epipolar images of a ZiYuan-3 pair in Songshan are further utilized to produce a high-precision DSM.
... and Wegmuller 1998; Thiele et al. 2007;Brenner and Roessing 2008;Schmidt et al. 2010;Esch et al. 2011;Leinenkugel et al. 2011;Wu et al. 2011;Marin et al. 2015;Sorichetta et al. 2020) as well as 3D feature characterization (e.g. Soergel et al. 2003;Brunner et al. 2010;Buckreuss et al. 2018;Qiu et al. 2018) -for example, TDX data were employed for urban footprint delineation (Taubenböck et al. 2012) and to examine forest vegetation height (Qi and Dubayah 2016). In evaluating the utility of a variety of geospatial datasets for urban build-up at a continentalscale, Li et al. (2020) found SAR data to be the most useful for estimating building height, which further attests to the significance of SAR data for 3D urban analyses. ...
Remotely sensed data products such as satellite images have long been utilized to monitor the urban environment and quantify changes. While useful for certain studies, image‐based approaches often lack three‐dimensional (3D) information (the vertical dimension) for comprehensive analyses of vertical change (1D) in addition to lateral change (2D). This chapter introduces light detection and ranging (lidar)‐based modeling of the urban environment, specifically focusing on quantifying built‐up volume and modeling its change within urban areas using multi‐temporal data. In addition, the chapter introduces readers to recent advancements in incorporating spaceborne radar data (e.g. from satellite scatterometer), validated with lidar data, to monitor built‐up volume free from the limitations of high cost, and small area coverage. Finally, the chapter includes discussion of the physical basis and future extension of high‐resolution data synergistically from multiple satellite synthetic aperture radars (SARs) .
... Benefiting from the fast development of space transportation technology and sensor manufacturing technology, the acquisition capacity of remote sensing data has been largely enhanced [1]. Currently, it has formed a complete air-space-ground remote sensing data acquisition system which has the characteristics of being efficient and diverse (multi-platform, multisensor, multiscale) and having a high resolution (spectrum, spatial, time), providing massive unstructured remote sensing data [2]. ...
... where x i means a specific element of the feature vector, and x L i and x U i are calculated by statistical processing. To reduce the influence of orientation distortions caused by local deformations, we conducted convolution on the feature 3D cube with a kernel of d z = [1, 3,1] T in the z direction [43]. After that, we further normalized the feature vector using the L 2 norm with Equation (7). ...
Accurate geopositioning of optical satellite imagery is a fundamental step for many photogrammetric applications. Considering the imaging principle and data processing manner, SAR satellites can achieve high geopositioning accuracy. Therefore, SAR data can be a reliable source for providing control information in the orientation of optical satellite images. This paper proposes a practical solution for an accurate orientation of optical satellite images using SAR reference images to take advantage of the merits of SAR data. Firstly, we propose an accurate and robust multimodal image matching method to match the SAR and optical satellite images. This approach includes the development of a new structural-based multimodal applicable feature descriptor that employs angle-weighted oriented gradients (AWOGs) and the utilization of a three-dimensional phase correlation similarity measure. Secondly, we put forward a general optical satellite imagery orientation framework based on multiple SAR reference images, which uses the matches of the SAR and optical satellite images as virtual control points. A large number of experiments not only demonstrate the superiority of the proposed matching method compared to the state-of-the-art methods but also prove the effectiveness of the proposed orientation framework. In particular, the matching performance is improved by about 17% compared with the latest multimodal image matching method, namely, CFOG, and the geopositioning accuracy of optical satellite images is improved, from more than 200 to around 8 m.
... Metric learning, as a straightforward correlation analysis method, achieves improved results on simple similarity functions, e.g., Euclidean distance [9]. Nevertheless, with totally different imaging mechanism of remote sources and the scarcity of labeled patches, similarity function gets unwieldy to learn on multi-source data [10]; the effectiveness of parameters optimization involved in similarity estimation would be uncontrollable, due to insufficient analysis of the data volume diversity and nonlinear radiometric differences under limited training data. For multi-source remote sensing data correlation estimation, conventional hand-crafted approaches focus on matching human-engineering features. ...
Recent advancements in sensor technology have reflected promise in collaborative utilization; specifically, multi-source remote sensing data correspondence identification attracts increasing attention. In this paper, a domain-transfer learning based generative correspondence analysis (DT-GCA) scheme is proposed, which enables identifying corresponding data in optical and synthetic aperture radar (SAR) images with small-sized reference data. In the proposed architecture, an adversarial domain-translator is investigated as general-purpose domain transference solution to learn cross domain features. The optical-aided implicit representation, which is regarded as the clone of SAR, is adopted to estimate the correlation with SAR images. Particularly, the designed GCA integrates optical-generated features with SAR tightly instead of treating them separately and eliminates the discrepancy influence of different sensors. Experiments on cross-domain remote sensing data are validated, and extensive results demonstrate that the proposed DT-GCA yields substantial improvements over some state-of-the-art techniques when only limited training samples are available.
... To this end we train an Outlier Reduction Network (ORN) on̂to classify good and bad matches. The overall goal of the ORN is to reduce the overall number of inaccurate matches of the correspondence network, as a low false positive rate is more important than a high number of matches for many downstream applications of SAR-optical matching such as, for example, image coregistration or stereogrammetry (Müller et al., 2012;Merkle et al., 2017;Qiu et al., 2018;Bagheri et al., 2018). ...
... While the use of high goodness regions leads to improved matching performance, it comes at the cost of having fewer overall correspondences as the regions are significantly larger than those used to compute point features. This, however, is deemed to be an acceptable trade-off as many downstream tasks such as co-registration (Müller et al., 2012;Merkle et al., 2017) and SARoptical stereogrammetry (Qiu et al., 2018;Bagheri et al., 2018) favour accuracy and spatial diversity over the number of correspondences. ...
SAR and optical imagery provide highly complementary information about observed scenes. A combined use of these two modalities is thus desirable in many data fusion scenarios. However, any data fusion task requires measurements to be accurately aligned. While for both data sources images are usually provided in a georeferenced manner, the geo-localization of optical images is often inaccurate due to propagation of angular measurement errors. Many methods for the matching of homologous image regions exist for both SAR and optical imagery, however, these methods are unsuitable for SAR-optical image matching due to significant geometric and radiometric differences between the two modalities. In this paper, we present a three-step framework for sparse image matching of SAR and optical imagery, whereby each step is encoded by a deep neural network. We first predict regions in each image which are deemed most suitable for matching. A correspondence heatmap is then generated through a multi-scale, feature-space cross-correlation operator. Finally, outliers are removed by classifying the correspondence surface as a positive or negative match. Our experiments show that the proposed approach provides a substantial improvement over previous methods for SAR-optical image matching and can be used to register even large-scale scenes. This opens up the possibility of using both types of data jointly, for example for the improvement of the geo-localization of optical satellite imagery or multi-sensor stereogrammetry.
... It is also due to the peculiarities of the SAR images used for the production of GUF. The local speckle information and the texture information in the SAR images makes it possible to specifically detect vertical structures such as buildings (Klotz et al., 2016;Qiu et al., 2018). Specifically, buildings are characterized by stronger back-scattering signals than airport roads, even though they are made of the same materials. ...
Human settlement extent (HSE) information is a valuable indicator of world-wide urbanization as well as the resulting human pressure on the natural environment. Therefore, mapping HSE is critical for various environmental issues at local, regional, and even global scales. This paper presents a deep-learning-based framework to automatically map HSE from multi-spectral Sentinel-2 data using regionally available geo-products as training labels. A straightforward, simple, yet effective fully convolutional network-based architecture, Sen2HSE, is implemented as an example for semantic segmentation within the framework. The framework is validated against both manually labelled checking points distributed evenly over the test areas, and the OpenStreetMap building layer. The HSE mapping results were extensively compared to several baseline products in order to thoroughly evaluate the effectiveness of the proposed HSE mapping framework. The HSE mapping power is consistently demonstrated over 10 representative areas across the world. We also present one regional-scale and one country-wide HSE mapping example from our framework to show the potential for upscaling. The results of this study contribute to the generalization of the applicability of CNN-based approaches for large-scale urban mapping to cases where no up-to-date and accurate ground truth is available, as well as the subsequent monitor of global urbanization.
... In 2015, Zhang et al. used multisensor datasets from GeoEye-1 and TerraSAR-X for 3D building reconstruction, and the accuracy can reach meter-level based on a stereo mapping model with combinations of an angle model for optical images and range-doppler (RD) model for SAR images [7] [8]. Recently, Qiu et al. proposed a new strategy for automatic SAR-optical stereogrammetry with space-borne imagery from WorldView-2, TerraSAR-X and MEMPHIS sensors, and the reconstruction accuracy can also reach meterdomain in urban areas [9]. Unlike space-borne SAR sensors, orbit parameters for optical satellite images are not provided to users in most cases, which makes it impossible for the development of rigorous sensor models. ...
Nowadays, numerous earth-observation data are obtained from different kinds of platforms, in which data from synthetic aperture radar (SAR) sensors and optical sensors account for the vast majority. They have been widely used in different fields such as 3 dimensional (3D) geo-information reconstruction, object recognition and others. Researchers have investigated the suitability of the rational function model (RFM) for 3D geo-information production with stereo SAR-optical images. Compared with optical remote sensing data, parameters of the RFM for satellite SAR images are not provided to users in most cases, which increases the workload of users applying the SAR-optical stereo geolocation based on RFM. Moreover, the fitting accuracy of RFM to the Range Doppler (RD) model, which is considered as the rigorous sensor model for SAR imagery, do not always meet the requirements. Therefore, a new RD-RFM stereo geolocation model by integrating the RD model of SAR images with the RFM of optical images are proposed for 3D geo-information reconstruction with SAR-optical stereo image pairs. Experiments are conducted based on dataset from Gaofen-3 (GF-3) SAR satellite and Gaofen-2 (GF-2) optical satellite covering urban and mountainous areas in Xinyi City and Dengfeng City, Henan Province, China, respectively. Results indicated that our proposed model can achieve the best performance with different terrain and different convergence angle of stereo SAR-optical image pairs. Compared with the traditional stereo geolocation model based on the RFM for both SAR and optical data, our proposed model is more concise and efficient for the production of 3D geo-information with stereo SAR-optical image pairs, and it’s easy to be implemented for users.
