Skip to Content
You are currently on the new version of our website. Access the old version .
Remote SensingRemote Sensing
  • Article
  • Open Access

27 July 2022

19 Pages

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

,
and
1
College of Geological Engineering and Geomatics, Chang’an University, Xi’an 710064, China
2
School of Land Engineering, Chang’an University, Xi’an 710064, China
3
Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China
*
Author to whom correspondence should be addressed.

Abstract

Remote sensing image matching is the basis upon which to obtain integrated observations and complementary information representation of the same scene from multiple source sensors, which is a prerequisite for remote sensing tasks such as remote sensing image fusion and change detection. However, the intricate geometric and radiometric differences between the multimodal images render the registration quite challenging. Although multimodal remote sensing image matching methods have been developed in recent decades, most classical and deep learning based techniques cannot effectively extract high repeatable keypoints and discriminative descriptors for multimodal images. Therefore, we propose a two-step “detection + matching” framework in this paper, where each step consists of a deep neural network. A self-supervised detection network is first designed to generate similar keypoint feature maps between multimodal images, which is used to detect highly repeatable keypoints. We then propose a cross-fusion matching network, which aims to exploit global optimization and fusion information for cross-modal feature descriptors and matching. The experiments show that the proposed method has superior feature detection and matching performance compared with current state-of-the-art methods. Specifically, the keypoint repetition rate of the detection network and the NN mAP of the matching network are 0.435 and 0.712 on test datasets, respectively. The proposed whole pipeline framework is evaluated, which achieves an average M.S. and RMSE of 0.298 and 3.41, respectively. This provides a novel solution for the joint use of multimodal remote sensing images for observation and localization.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.