Hongliang Guan's research while affiliated with Capital Normal University and other places

Simultaneous localization and mapping (SLAM) based on RGB-D cameras has been widely used for robot localization and navigation in unknown environments. Most current SLAM methods are constrained by static environment assumptions and perform poorly in real-world dynamic scenarios. To improve the robustness and performance of SLAM systems in dynamic environments, this paper proposes a new RGB-D SLAM method for indoor dynamic scenes based on object detection. The method presented in this paper improves on the ORB-SLAM3 framework. First, we designed an object detection module based on YOLO v5 and relied on it to improve the tracking module of ORB-SLAM3 and the localization accuracy of ORB-SLAM3 in dynamic environments. The dense point cloud map building module was also included, which excludes dynamic objects from the environment map to create a static environment point cloud map with high readability and reusability. Full comparison experiments with the original ORB-SLAM3 and two representative semantic SLAM methods on the TUM RGB-D dataset show that: the method in this paper can run at 30+fps, the localization accuracy improved to varying degrees compared to ORB-SLAM3 in all four image sequences, and the absolute trajectory accuracy can be improved by up to 91.10%. The localization accuracy of the method in this paper is comparable to that of DS-SLAM, DynaSLAM and the two recent target detection-based SLAM algorithms, but it runs faster. The RGB-D SLAM method proposed in this paper, which combines the most advanced object detection method and visual SLAM framework, outperforms other methods in terms of localization accuracy and map construction in a dynamic indoor environment and has a certain reference value for navigation, localization, and 3D reconstruction.

The structure from motion (SfM) method has achieved great success in 3D sparse reconstruction, but it still faces serious challenges in large-scale scenes. Existing hybrid SfM methods usually do not fully consider the compactness between images and the connectivity between subclusters, resulting in a loose spatial distribution of images within subclusters, unbalanced connectivity between subclusters, and poor robustness in the merging stage. In this paper, an efficient and robust hybrid SfM method is proposed. First, the multifactor joint scene partition measure and the preassignment balanced image expansion algorithm among subclusters are constructed, which effectively solves the loose spatial distribution of images in subclusters problem and improves the degree of connection among subclusters. Second, the global GlobalACSfM method is used to complete the local sparse reconstruction of the subclusters under the cluster parallel framework. Then, a decentralized dynamic merging rule considering the connectivity of subclusters is proposed to realize robust merging among subclusters. Finally, public datasets and oblique photography datasets are used for experimental verification. The results show that the method proposed in this paper is superior to the state-of-the-art methods in terms of accuracy and robustness and has good feasibility and advancement prospects.

Most of the traditional image feature point extraction and matching methods are based on a series of light properties of images. These light properties easily conflict with the distinguishability of the image features. The traditional light imaging methods focus only on a fixed depth of the target scene, and subjects at other depths are often easily blurred. This makes the traditional image feature point extraction and matching methods suffer from a low accuracy and a poor robustness. Therefore, in this paper, a light field camera is used as a sensor to acquire image data and to generate a full-focus image with the help of the rich depth information inherent in the original image of the light field. The traditional ORB feature point extraction and matching algorithm is enhanced with the goal of improving the number and accuracy of the feature point extraction for the light field full-focus images. The results show that the improved ORB algorithm extracts not only most of the features in the target scene but also covers the edge part of the image to a greater extent and produces extracted feature points which are evenly distributed for the light field full-focus image. Moreover, the extracted feature points are not repeated in a large number in a certain part of the image, eliminating the aggregation phenomenon that exists in traditional ORB algorithms.

In the light field image saliency detection task, redundant cues are introduced due to computational methods. Inevitably, it leads to the inaccurate boundary segmentation of detection results and the problem of the chain block effect. To tackle this issue, we propose a method for salient object detection (SOD) in light field images that fuses focus and GrabCut. The method improves the light field focus calculation based on the spatial domain by performing secondary blurring processing on the focus image and effectively suppresses the focus information of out-of-focus areas in different focus images. Aiming at the redundancy of focus cues generated by multiple foreground images, we use the optimal single foreground image to generate focus cues. In addition, aiming at the fusion of various cues in the light field in complex scenes, the GrabCut algorithm is combined with the focus cue to guide the generation of color cues, which realizes the automatic saliency target segmentation of the image foreground. Extensive experiments are conducted on the light field dataset to demonstrate that our algorithm can effectively segment the salient target area and background area under the light field image, and the outline of the salient object is clear. Compared with the traditional GrabCut algorithm, the focus degree is used instead of artificial Interactively initialize GrabCut to achieve automatic saliency segmentation.

Garlic is one of the main economic crops in China. Accurate and timely extraction of the garlic planting area is critical for adjusting the agricultural planting structure and implementing rural policy actions. Crop extraction methods based on remote sensing usually use spectral-temporal features. Still, for garlic extraction, most methods simply combine all multi-temporal images. There has been a lack of research on each band's function in each multi-temporal image and optimal bands combination. To systematically explore the potential of the multi-temporal method for garlic extraction, we obtained a series of Sentinel-2 images in the whole garlic growth cycle. The importance of each band in all these images was ranked by the random forest (RF) method. According to the importance score of each band, eight different multi-temporal combination schemes were designed. The RF classifier was employed to extract garlic planting area, and the accuracy of the eight schemes was compared. The results show that (1) the Scheme VI (the top 39 bands in importance score) achieved the best accuracy of 98.65%, which is 6% higher than the optimal mono-temporal (February, wintering period) result, and (2) the red-edge band and the shortwave-infrared band played an essential role in accurate garlic extraction. This study gives inspiration in selecting the remotely sensed data source, the band, and phenology for accurately extracting garlic planting area, which could be transferred to other sites with larger areas and similar agriculture structures.

Change detection with very high resolution (VHR) satellite images is of great application values when evaluating and monitoring land use changes. However, intrinsic complexity of satellite images will introduce more difficulties to change detection tasks. In this study, a new change detection method is proposed by combining multi-scale simple linear iterative clustering-convolutional neural network (SLIC-CNN) with stacked convolutional auto-encoder (SCAE) features to improve change detection capabilities with VHR satellite images. First, the multi-scale SLIC-based image segmentation is performed on multi-temporal images to generate segment objects while keeping their edge information as much as possible. Second, the convolutional layers in a CNN architecture are used to generate change map, then, an SCAE feature-based classification procedure is performed to generate “from-to” change information. Finally, a Bayesian information criterion is used to optimize the results of change detection. In this study, the experiments carried out reveal that the multi-scale SLIC image segmentation algorithm affects the integrity of change regions; CNN features have an effect on the consistency of change regions; and SCAE features influence the performance of support vector machine (SVM) classifiers. And, features extracted from the architectures enhance the ability of information extraction from ground objects. Comparison results also show the superiority to other change detection methods.

Change detection has been a research hotspot in remote sensing fields for decades. However, the increasing use of very high-resolution (VHR) remote sensing images have introduced more difficulties in change detection because of the complex details these images contain. In this paper, we propose a novel deep learning architecture for change detection composed of a Trisiamese subnetwork and a long short-term memory (LSTM) subnetwork that fully utilizes the spatial, spectral and multiphase information and improves the change detection capabilities for VHR remote sensing images. Multiscale simple linear iterative clustering (SLIC)-based image segmentation is first performed on multitemporal images at different image scales to obtain edge information-based objects. A Trisiamese subnetwork with six inputs can extract abundant spectral-spatial feature representations; the LSTM subnetwork then uses the extracted image features to effectively analyse the multiphase information in bitemporal images. The proposed method has the following advantages: 1) it can fully utilize the significant spatial information to improve the detection task; 2) it combines the advantages of convolutional architectures for image feature representation and recurrent neural network (RNN) architectures for sequential data representation, unlike most of the algorithms that use either method or that merely use image differencing or stacking operations. The controlled experiments reveal that the multiphase information extracted by the LSTM subnetwork is important to improve the accuracy of the change detection results. The influence of the Trisiamese subnetwork on change detection is even more significant than that of the LSTM subnetwork. Comparisons with other state-of-the-art change detection methods indicate that in areas with clear surface features and limited interference, the proposed method obtains more competitive results compared to state-of-the-art methods, and in regions where the changed objects occur in complex patterns, the proposed method exhibited an ideal performance.