Search Results (51)

Accurate image registration is essential for synthetic aperture radar (SAR) applications such as change detection, image fusion, and deformation monitoring. However, SAR image registration faces challenges including speckle noise, low-texture regions, and the geometric transformation caused by topographic relief due to side-looking radar imaging. To address these issues, this paper proposes a novel two-stage registration method, consisting of pre-registration and fine registration. In the pre-registration stage, the scale-invariant feature transform for the synthetic aperture radar (SAR-SIFT) algorithm is integrated into an iterative optimization framework to eliminate large-scale geometric discrepancies, ensuring a coarse but reliable initial alignment. In the fine registration stage, a novel similarity measure is introduced by combining frequency-domain phase congruency and spatial-domain gradient features, which enhances the robustness and accuracy of template matching, especially in edge-rich regions. For the topographic relief in the SAR images, an adaptive local stretching transformation strategy is proposed to correct the undulating areas. Experiments on five pairs of SAR images containing flat and undulating regions show that the proposed method achieves initial alignment errors below 10 pixels and final registration errors below 1 pixel. Compared with other methods, our approach obtains more correct matching pairs (up to 100+ per image pair), higher registration precision, and improved robustness under complex terrains. These results validate the accuracy and effectiveness of the proposed registration framework. Full article

Brain medical image registration is a fundamental premise for the computer-assisted treatment of brain diseases. The brain is one of the most important and complex organs of the human body, and it is very challenging to perform accurate and fast registration on it. Aiming at the problem of voxel folding in the deformation field and low registration accuracy when facing complex and fine objects, this paper proposed a fully convolutional multi-constraint cascaded attention network (MCANet). The network is composed of two registration sub-network cascades and performs coarse-to-fine registration of input image pairs in an iterative manner. The registration subnetwork is called the dilated self-attention network (DSNet), which incorporates dilated convolution combinations with different dilation rates and attention gate modules. During the training of MCANet, a double regularization constraint was applied to punish, in a targeted manner, the excessive deformation problem, so that the network can generate relatively smooth deformation while having high registration accuracy. Experimental results on the Mindboggle101 dataset showed that the registration accuracy of MCANet was significantly better than several existing advanced registration methods, and the network can complete relatively smooth registration. Full article

The registration of aerial images and satellite images is a key step in leveraging complementary information from heterogeneous remote sensing images. Due to the significant intrinsic differences, such as scale, radiometric, and temporal differences, between the two types of images, existing multimodal registration methods tend to be either inaccurate or unstable when applied. This paper proposes a coarse-to-fine registration method for aerial images and satellite images based on the multiscale iteration method (MSIM). Firstly, an image pyramid is established, and feature points are extracted based on phase congruency. Secondly, the expression form of image descriptors is improved to more accurately describe image feature points, thereby increasing the matching success rate and achieving coarse registration between images. Finally, multiscale iterations are performed to find accurate matching points from top to bottom to achieve fine registration between images. In order to verify the effectiveness and accuracy of the algorithm, this paper also establishes a set of registration datasets of aerial and satellite captured images. Experimental results show that the proposed algorithm has high accuracy and good robustness, and effectively solves the problem of registration failure in existing algorithms when dealing with heterogeneous remote sensing images that have large scale differences. Full article

Automatic registration of remote sensing images is an important task, which requires the establishment of appropriate correspondence between the sensed image and the reference image. Nowadays, the trend of satellite remote sensing technology is shifting towards high-resolution hyperspectral imaging technology. Ever higher revisit cycles and image resolutions require higher accuracy and real-time performance for automatic registration. The push-broom payload is affected by the push-broom stability of the satellite platform and the elevation change of ground objects, and the obtained hyperspectral image may have distortions such as stretching or shrinking at different parts of the image. In order to solve this problem, a new automatic registration strategy for remote sensing hyperspectral images based on the combination of whole and local features of the image was established, and two granularity registrations were carried out, namely coarse-grained matching and fine-grained matching. The high-resolution spatial features are first employed for detecting scale-invariant features, while the spectral information is used for matching, and then the idea of image stitching is employed to fuse the image after fine registration to obtain high-precision registration results. In order to verify the proposed algorithm, a simulated on-orbit push-broom imaging experiment was carried out to obtain hyperspectral images with local complex distortions under different lighting conditions. The simulation results show that the proposed remote sensing hyperspectral image registration algorithm is superior to the existing automatic registration algorithms. The advantages of the proposed algorithm in terms of registration accuracy and real-time performance make it have a broad prospect for application in satellite ground application systems. Full article

Linear Variable Filter (LVF) hyperspectral cameras possess the advantages of high spectral resolution, compact size, and light weight, making them highly suitable for unmanned aerial vehicle (UAV) platforms. However, challenges arise in data registration due to the imaging characteristics of LVF data and the instability of UAV platforms. These challenges stem from the diversity of LVF data bands and significant inter-band differences. Even after geometric processing, adjacent flight lines still exhibit varying degrees of geometric deformation. In this paper, a progressive grouping-based strategy for iterative band selection and registration is proposed. In addition, an improved Scale-Invariant Feature Transform (SIFT) algorithm, termed the Double Sufficiency–SIFT (DS-SIFT) algorithm, is introduced. This method first groups bands, selects the optimal reference band, and performs coarse registration based on the SIFT method. Subsequently, during the fine registration stage, it introduces an improved position/scale/orientation joint SIFT registration algorithm (IPSO-SIFT) that integrates partitioning and the principle of structural similarity. This algorithm iteratively refines registration based on the grouping results. Experimental data obtained from a self-developed and integrated LVF hyperspectral remote sensing system are utilized to verify the effectiveness of the proposed algorithm. A comparison with classical algorithms, such as SIFT and PSO-SIFT, demonstrates that the registration of LVF hyperspectral data using the proposed method achieves superior accuracy and efficiency. Full article

The automatic registration of infrared and visible images in vehicular imaging systems remains challenging in vision-assisted driving systems because of differences in imaging mechanisms. Existing registration methods often fail to accurately register infrared and visible images in vehicular imaging systems due to numerous spurious points during feature extraction, unstable feature descriptions, and low feature matching efficiency. To address these issues, a robust and efficient registration of infrared and visible images for vehicular imaging systems is proposed. In the feature extraction stage, we propose a structural similarity point extractor (SSPE) that extracts feature points using the structural similarity between weighted phase congruency (PC) maps and gradient magnitude (GM) maps. This approach effectively suppresses invalid feature points while ensuring the extraction of stable and reliable ones. In the feature description stage, we design a rotation-invariant feature descriptor (RIFD) that comprehensively describes the attributes of feature points, thereby enhancing their discriminative power. In the feature matching stage, we propose an effective coarse-to-fine matching strategy (EC2F) that improves the matching efficiency through nearest neighbor matching and threshold-based fast sample consensus (FSC), while improving registration accuracy through coordinate-based iterative optimization. Registration experiments on public datasets and a self-established dataset demonstrate the superior performance of our proposed method, and also confirm its effectiveness in real vehicular environments. Full article

This paper presents a robust point cloud registration method based on a multi-scale covariance matrix descriptor and an accurate transformation estimation. Compared with state-of-the-art feature descriptors, such as FPH, 3DSC, spin image, etc., our proposed multi-scale covariance matrix descriptor is superior for dealing with registration problems in a higher noise environment since the mean operation in generating the covariance matrix can filter out most of the noise-damaged samples or outliers and also make itself robust to noise. Compared with transformation estimation, such as feature matching, clustering, ICP, RANSAC, etc., our transformation estimation is able to find a better optimal transformation between a pair of point clouds since our transformation estimation is a multi-level point cloud transformation estimator including feature matching, coarse transformation estimation based on clustering, and a fine transformation estimation based on ICP. Experiment findings reveal that our proposed feature descriptor and transformation estimation outperforms state-of-the-art feature descriptors and transformation estimation, and registration effectiveness based on our registration framework of point cloud is extremely successful in the Stanford 3D Scanning Repository, the SpaceTime dataset, and the Kinect dataset, where the Stanford 3D Scanning Repository is known for its comprehensive collection of high-quality 3D scans, and the SpaceTime dataset and the Kinect dataset are captured by a SpaceTime Stereo scanner and a low-cost Microsoft Kinect scanner, respectively. Full article

The registration of preoperative and follow-up brain MRI, which is crucial in illustrating patients’ responses to treatments and providing guidance for postoperative therapy, presents significant challenges. These challenges stem from the considerable deformation of brain tissue and the areas of non-correspondence due to surgical intervention and postoperative changes. We propose a stepwise corrected attention registration network grounded in convolutional neural networks (CNNs). This methodology leverages preoperative and follow-up MRI scans as fixed images and moving images, respectively, and employs a multi-level registration strategy that establishes a precise and holistic correspondence between images, from coarse to fine. Furthermore, our model introduces a corrected attention module into the multi-level registration network that can generate an attention map at the local level through the deformation fields of the upper-level registration network and pathological areas of preoperative images segmented by a mature algorithm in BraTS, serving to strengthen the registration accuracy of non-correspondence areas. A comparison between our scheme and the leading approach identified in the MICCAI’s BraTS-Reg challenge indicates a 7.5% enhancement in the target registration error (TRE) metric and improved visualization of non-correspondence areas. These results illustrate the better performance of our stepwise corrected attention registration network in not only enhancing the registration accuracy but also achieving a more logical representation of non-correspondence areas. Thus, this work contributes significantly to the optimization of the registration of brain MRI between preoperative and follow-up scans. Full article

Accurate registration between medical images and patient anatomy is crucial for surgical navigation systems in minimally invasive surgeries. This study introduces a novel deep learning-based refinement step to enhance the accuracy of surface registration without disrupting established workflows. The proposed method integrates a machine learning model between conventional coarse registration and ICP fine registration. A deep-learning model was trained using simulated anatomical landmarks with introduced localization errors. The model architecture features global feature-based learning, an iterative prediction structure, and independent processing of rotational and translational components. Validation with silicon-masked head phantoms and CT imaging compared the proposed method to both conventional registration and a recent deep-learning approach. The results demonstrated significant improvements in target registration error (TRE) across different facial regions and depths. The average TRE for the proposed method (1.58 ± 0.52 mm) was significantly lower than that of the conventional (2.37 ± 1.14 mm) and previous deep-learning (2.29 ± 0.95 mm) approaches (p < 0.01). The method showed a consistent performance across various facial regions and enhanced registration accuracy for deeper areas. This advancement could significantly enhance precision and safety in minimally invasive surgical procedures. Full article

Differences in field of view may occur during unmanned aerial remote sensing imaging applications with acousto-optic tunable filter (AOTF) spectral imagers using zoom lenses. These differences may stem from image size deformation caused by the zoom lens, image drift caused by AOTF wavelength switching, and drone platform jitter. However, they can be addressed using hyperspectral image registration. This article proposes a new coarse-to-fine remote sensing image registration framework based on feature and optical flow theory, comparing its performance with that of existing registration algorithms using the same dataset. The proposed method increases the structure similarity index by 5.2 times, reduces the root mean square error by 3.1 times, and increases the mutual information by 1.9 times. To meet the real-time processing requirements of the AOTF spectrometer in remote sensing, a development environment using VS2023+CUDA+OPENCV was established to improve the demons registration algorithm. The registration algorithm for the central processing unit+graphics processing unit (CPU+GPU) achieved an acceleration ratio of ~30 times compared to that of a CPU alone. Finally, the real-time registration effect of spectral data during flight was verified. The proposed method demonstrates that AOTF hyperspectral imagers can be used in real-time remote sensing applications on unmanned aerial vehicles. Full article

To address the matching challenge between the High Resolution Imaging Science Experiment (HiRISE) Digital Elevation Model (DEM) and the Mars Orbiter Laser Altimeter (MOLA) DEM, we propose a terrain matching framework based on the combination of point cloud coarse alignment and fine alignment methods. Firstly, we achieved global coarse localization of the HiRISE DEM through nearest neighbor matching of key Intrinsic Shape Signatures (ISS) points in the Fast Point Feature Histograms (FPFH) feature space. We introduced a graph matching strategy to mitigate gross errors in feature matching, employing a numerical method of non-cooperative game theory to solve the extremal optimization problem under Karush–Kuhn–Tucker (KKT) conditions. Secondly, to handle the substantial resolution disparities between the MOLA DEM and HiRISE DEM, we devised a smoothing weighting method tailored to enhance the Voxelized Generalized Iterative Closest Point (VGICP) approach for fine terrain registration. This involves leveraging the Euclidean distance between distributions to effectively weight loss and covariance, thereby reducing the results’ sensitivity to voxel radius selection. Our experiments show that the proposed algorithm improves the accuracy of terrain registration on the proposed Curiosity landing area’s, Mawrth Vallis, data by nearly 20%, with faster convergence and better algorithm robustness. Full article

This study introduces a pioneering multimodal fusion framework to enhance near-field 3D Synthetic Aperture Radar (SAR) imaging, crucial for applications like radar cross-section measurement and concealed object detection. Traditional near-field 3D SAR imaging struggles with issues like target–background confusion due to clutter and multipath interference, shape distortion from high sidelobes, and lack of color and texture information, all of which impede effective target recognition and scattering diagnosis. The proposed approach presents the first known application of multimodal fusion in near-field 3D SAR imaging, integrating LiDAR and optical camera data to overcome its inherent limitations. The framework comprises data preprocessing, point cloud registration, and data fusion, where registration between multi-sensor data is the core of effective integration. Recognizing the inadequacy of traditional registration methods in handling varying data formats, noise, and resolution differences, particularly between near-field 3D SAR and other sensors, this work introduces a novel three-stage registration process to effectively address these challenges. First, the approach designs a structure–intensity-constrained centroid distance detector, enabling key point extraction that reduces heterogeneity and accelerates the process. Second, a sample consensus initial alignment algorithm with SHOT features and geometric relationship constraints is proposed for enhanced coarse registration. Finally, the fine registration phase employs adaptive thresholding in the iterative closest point algorithm for precise and efficient data alignment. Both visual and quantitative analyses of measured data demonstrate the effectiveness of our method. The experimental results show significant improvements in registration accuracy and efficiency, laying the groundwork for future multimodal fusion advancements in near-field 3D SAR imaging. Full article

To generate an orthophoto mosaic from a collection of aerial images, the original images are first orthorectified individually using a Digital Surface Model (DSM). Then, they are stitched together along some determined seamlines to form the orthophoto mosaic. Determining appropriate seamlines is a critical process, as it affects the visual and geometric quality of the results. The stitching process can usually be done in frame-to-frame or multi-frame modes. Although the latter is more efficient, both still involve a lot of pre-processing, such as creating individual orthophotos, image registration, and overlap extraction. This paper presents a novel coarse-to-fine approach that directly determines the seamline network without such pre-processing. Our method has been specifically applied for UAV photogrammetry projects where, due to the large number of images and the corresponding overlaps, the orthophoto mosaic generation can be very challenging and time-consuming. We established the seamlines simultaneously for all the images through a two-step process. First, a DSM was generated, and a low-resolution grid was overlayed. Then, for each grid point, an optimal image was selected. Then, the grid cells are grouped into polygons based on their corresponding optimal image. Boundaries of these polygons established our seamline network. Thereafter, to generate the orthophoto mosaic, we overlayed a higher/full resolution grid on the top of the DSM, the optimal image of each point of which was quickly identified via our low-resolution polygons. In this approach, not only seamlines were automatically generated, but also were the need for the creation, registration, and overlap extraction of individual orthophotos. Our method was systematically compared with a conventional frame-to-frame (CF) technique from different aspects, including the number of double-mapped areas, discontinuities across the seamlines network, and the amount of processing time. The outcomes revealed a 46% decrease in orthophoto generation time and a notable reduction in the number of double-mapped areas, sawtooth effects, and object discontinuities within the constructed orthophoto mosaic. Full article

Infrared and visible remote sensing image registration is significant for utilizing remote sensing images to obtain scene information. However, it is difficult to establish a large number of correct matches due to the difficulty in obtaining similarity metrics due to the presence of radiation variation between heterogeneous sensors, which is caused by different imaging principles. In addition, the existence of sparse textures in infrared images as well as in some scenes and the small number of relevant trainable datasets also hinder the development of this field. Therefore, we combined data-driven and knowledge-driven methods to propose a Radiation-variation Insensitive, Zero-shot learning-based Registration (RIZER). First, RIZER, as a whole, adopts a detector-free coarse-to-fine registration framework, and the data-driven methods use a Transformer based on zero-shot learning. Next, the knowledge-driven methods are embodied in the coarse-level matches, where we adopt the strategy of seeking reliability by introducing the HNSW algorithm and employing a priori knowledge of local geometric soft constraints. Then, we simulate the matching strategy of the human eye to transform the matching problem into a model-fitting problem and employ a multi-constrained incremental matching approach. Finally, after fine-level coordinate fine tuning, we propose an outlier culling algorithm that only requires very few iterations. Meanwhile, we propose a multi-scene infrared and visible remote sensing image registration dataset. After testing, RIZER achieved a correct matching rate of 99.55% with an RMSE of 1.36 and had an advantage in the number of correct matches, as well as a good generalization ability for other multimodal images, achieving the best results when compared to some traditional and state-of-the-art multimodal registration algorithms. Full article

For Synthetic Aperture Radar (SAR) image registration, successive processes following feature extraction are required by both the traditional feature-based method and the deep learning method. Among these processes, the feature matching process—whose time and space complexity are related to the number of feature points extracted from sensed and reference images, as well as the dimension of feature descriptors—proves to be particularly time consuming. Additionally, the successive processes introduce data sharing and memory occupancy issues, requiring an elaborate design to prevent memory leaks. To address these challenges, this paper introduces the OptionEM-based reinforcement learning framework to achieve end-to-end SAR image registration. This framework outputs registered images directly without requiring feature matching and the calculation of the transformation matrix, leading to significant processing time savings. The Transformer architecture is employed to learn image features, while a correlation network is introduced to learn the correlation and transformation matrix between image pairs. Reinforcement learning, as a decision process, can dynamically correct errors, making it more-efficient and -robust compared to supervised learning mechanisms such as deep learning. We present a hierarchical reinforcement learning framework combined with Episodic Memory to mitigate the inherent problem of invalid exploration in generalized reinforcement learning algorithms. This approach effectively combines coarse and fine registration, further enhancing training efficiency. Experiments conducted on three sets of SAR images, acquired by TerraSAR-X and Sentinel-1A, demonstrated that the proposed method’s average runtime is sub-second, achieving subpixel registration accuracy. Full article