Search Results (15)

Sub-pixel matching of multimodal optical images is a critical step in the combined application of multiple sensors. However, structural noise and inconsistencies arising from variations in multimodal image responses usually limit the accuracy of matching. Phase congruency mutual-structure weighted least absolute deviation (PCWLAD) is developed as a coarse-to-fine framework. In the coarse matching stage, we preserve the complete structure and use an enhanced cross-modal similarity criterion to mitigate structural information loss by phase congruency (PC) noise filtering. In the fine matching stage, a mutual-structure filtering and weighted least absolute deviation-based method is introduced to enhance inter-modal structural consistency and to accurately estimate sub-pixel displacements adaptively. Experiments on three multimodal datasets—Landsat visible-infrared, short-range visible-near-infrared, and unmanned aerial vehicle (UAV) optical image pairs—show that PCWLAD achieves superior average performance compared with eight state-of-the-art methods, attaining an average matching accuracy of approximately 0.4 pixels. Full article

Multi-modal remote sensing image (MRSI) matching suffers from severe nonlinear radiometric distortions and geometric deformations, and conventional feature-based techniques are generally ineffective. This study proposes a novel and robust MRSI matching method using the side window filter (MSWF). First, a novel side window scale space is constructed based on the side window filter (SWF), which can preserve shared image contours and facilitate the extraction of feature points within this newly defined scale space. Second, noise thresholds in phase congruency (PC) computation are adaptively refined with the Weibull distribution; weighted phase features are then exploited to determine the principal orientation of each point, from which a maximum index map (MIM) descriptor is constructed. Third, coarse position, orientation, and scale information obtained through global matching are employed to estimate image-pair geometry, after which descriptors are recalculated for precise correspondence search. MSWF is benchmarked against eight state-of-the-art multi-modal methods—six hand-crafted (PSO-SIFT, LGHD, RIFT, RIFT2, HAPCG, COFSM) and two learning-based (CMM-Net, RedFeat) methods—on three public datasets. Experiments demonstrate that MSWF consistently achieves the highest number of correct matches (NCM) and the highest rate of correct matches (RCM) while delivering the lowest root mean square error (RMSE), confirming its superiority for challenging MRSI registration tasks. Full article

The application prospects of multi-temporal 3D models are broad. It is difficult to ensure that multi-temporal 3D models have a consistent spatial reference. In this study, a method for automatic alignment of multi-temporal 3D models based on phase congruency (PC) matching is proposed. Firstly, the texture image of the multi-temporal 3D model is obtained, and the key points are extracted from the texture image. Secondly, the affine model between the plane of the key point and its corresponding tile triangle is established, and the 2D coordinates of the key point are mapped to 3D spatial coordinates. Thirdly, multi-temporal 3D model matching is completed based on PC to obtain a large number of evenly distributed corresponding points. Finally, the parameters of the 3D transformation model are estimated based on the multi-temporal corresponding points, and the vertex update of the 3D model is completed. The first experiment demonstrates that the method proposed in this study performs remarkably well in improving the positioning accuracy of feature point coordinates, effectively reducing the mean error of the systematic error to below 0.001 m. The second experiment further reveals the significant impact of different 3D transformation models. The experimental results show that the coordinates obtained based on position and orientation system (POS) data have significant positioning errors, while the method proposed in this study can reduce the coordinate errors between the two-period models. Due to the fact that this method does not require obtaining ground control points (GCPs) and does not require manual measurement for 3D geometric registration, its application to multi-temporal 3D models can ensure high-precision spatial referencing for multi-temporal 3D models, streamlining processes to reduce resource intensity and enhancing economic efficiency. Full article

The automatic registration of infrared and visible images in vehicular imaging systems remains challenging in vision-assisted driving systems because of differences in imaging mechanisms. Existing registration methods often fail to accurately register infrared and visible images in vehicular imaging systems due to numerous spurious points during feature extraction, unstable feature descriptions, and low feature matching efficiency. To address these issues, a robust and efficient registration of infrared and visible images for vehicular imaging systems is proposed. In the feature extraction stage, we propose a structural similarity point extractor (SSPE) that extracts feature points using the structural similarity between weighted phase congruency (PC) maps and gradient magnitude (GM) maps. This approach effectively suppresses invalid feature points while ensuring the extraction of stable and reliable ones. In the feature description stage, we design a rotation-invariant feature descriptor (RIFD) that comprehensively describes the attributes of feature points, thereby enhancing their discriminative power. In the feature matching stage, we propose an effective coarse-to-fine matching strategy (EC2F) that improves the matching efficiency through nearest neighbor matching and threshold-based fast sample consensus (FSC), while improving registration accuracy through coordinate-based iterative optimization. Registration experiments on public datasets and a self-established dataset demonstrate the superior performance of our proposed method, and also confirm its effectiveness in real vehicular environments. Full article

Registration of optical and synthetic aperture radar (SAR) images is challenging because extracting located identically and unique features on both images are tricky. This paper proposes a novel optical and SAR image registration method based on relative total variation (RTV) and scale-invariant feature transform (SIFT), named RTV-SIFT, to extract feature points on the edges of structures and construct structural edge descriptors to improve the registration accuracy. First, a novel RTV-Harris feature point detection method by combining the RTV and the multiscale Harris algorithm is proposed to extract feature points on both images’ significant structures. This ensures a high repetition rate of the feature points. Second, the feature point descriptors are constructed on enhanced phase congruency edge (EPCE), which combines the Sobel operator and maximum moment of phase congruency (PC) to extract edges from structured images that enhance robustness to nonlinear intensity differences and speckle noise. Finally, after coarse registration, the position and orientation Euclidean distance (POED) between feature points is utilized to achieve fine feature point matching to improve the registration accuracy. The experimental results demonstrate the superiority of the proposed RTV-SIFT method in different scenes and image capture conditions, indicating its robustness and effectiveness in optical and SAR image registration. Full article

Optical and SAR image registration is the primary procedure to exploit the complementary information from the two different image modal types. Although extensive research has been conducted to narrow down the vast radiometric and geometric gaps so as to extract homogeneous characters for feature point matching, few works have considered the registration issue for non-flat terrains, which will bring in more difficulties for not only sparse feature point matching but also outlier removal and geometric relationship estimation. This article addresses these issues with a novel and effective optical-SAR image registration framework. Firstly, sparse feature points are detected based on the phase congruency moment map of the textureless SAR image (SAR-PC-Moment), which helps to identify salient local regions. Then a template matching process using very large local image patches is conducted, which increases the matching accuracy by a significant margin. Secondly, a mutual verification-based initial outlier removal method is proposed, which takes advantage of the different mechanisms of sparse and dense matching and requires no geometric consistency assumption within the inliers. These two procedures will produce a putative correspondence feature point (CP) set with a low outlier ratio and high reliability. In the third step, the putative CPs are used to segment the large input image of non-flat terrain into dozens of locally flat areas using a recursive random sample consensus (RANSAC) method, with each locally flat area co-registered using an affine transformation. As for the mountainous areas with sharp elevation variations, anchor CPs are first identified, and then optical flow-based pixelwise dense matching is conducted. In the experimental section, ablation studies using four precisely co-registered optical-SAR image pairs of flat terrain quantitatively verify the effectiveness of the proposed SAR-PC-Moment-based feature point detector, big template matching strategy, and mutual verification-based outlier removal method. Registration results on four 1 m-resolution non-flat image pairs prove that the proposed framework is able to produce robust and quite accurate registration results. Full article

The improved phase congruency (PC) algorithms have been successfully applied to optical and synthetic aperture radar (SAR) image registration since they are insensitive to nonlinear radiometric and geometric differences. However, most of the algorithms are sensitive to large-scale differences and rotation differences between optical and SAR images. To tackle this, we propose a PC framework to register optical and SAR images. It is compatible with large-scale and rotation invariance. Firstly, a multi-scale Harris keypoint extraction method based on the maximum moment of PC (named PC-Harris) is proposed. The scale space is constructed by combining PC with the log-Gabor filter. Secondly, we propose a PC model to construct the feature descriptors. The orientation and amplitude responses are obtained based on the PC model. Meanwhile, the novel descriptor is constructed based on the polar coordinate system and thus can handle the scale and rotation differences between optical and SAR images. Finally, outliers are removed by the fast sample consensus (FSC). The experiments conducted on several optical and SAR images verify the effectiveness of the proposed framework. Full article

Neutron imaging is an effective nondestructive testing (NDT) technique widely applied to detect structural defects and the enrichment of nuclear fuel elements due to its high penetration and nuclide-sensitive properties. Since the fuel element pellet is sealed in the cladding, the transmission imaging result is a superposition of the two parts. Therefore, the attenuation of neutrons by the cladding is interference that must be considered in the enrichment analysis. It is necessary to extract and separate cladding and pellets using an edge extraction method. However, the low neutron cross-section of the cladding material (e.g., aluminum and zirconium) leads to poor grayscale contrast at the cladding edge in the imaging result, and the intensity of the cladding edge is significantly lower than that of the pellet edge. In addition, affected by the noise from the imaging environment, the boundaries of targets are further blurred, making edge detection more challenging. Traditional detection algorithms extract the weak edges of cladding incompletely, and the results are discontinuous, with obvious edge breaks and missing areas. This paper proposes a method to extract edges in neutron images based on phase congruency (PC). This study utilized the classical perceptual field model to improve contrast at weak edges. The enriched edge map was generated using our PC model from six directions, allowing more weak edges to be detected accurately. The non-maximum suppression ensured precise localization and avoided edge breaks. Furthermore, the edge results were optimized by eliminating noise through morphological operations. The experimental results demonstrate that the proposed method effectively detects the weak edges of the cladding, is superior in accuracy and integrity to traditional detection, and is able to obtain stable and reliable results with different materials of neutron images. The edge integrity improved by 64.1%, and the edge localization accuracy reached 94.3%. The extracted edge information is useful in the next stage of the high-precision enrichment analysis. Full article

Image feature matching is essential in many computer vision applications, and the foundation of matching is feature detection, which is a crucial feature quantification process. This manuscript focused on detecting more features from underwater acoustic imageries for further ocean engineering applications of autonomous underwater vehicles (AUVs). Currently, the mainstream feature detection operators are developed for optical images, and there is not yet a feature detector oriented to underwater acoustic imagery. To better analyze the suitability of existing feature detectors for acoustic imagery and develop an operator that can robustly detect feature points in underwater imageries in the future, this manuscript compared the performance of well-established handcrafted feature detectors and that of the increasingly popular deep-learning-based detectors to fill the gap in the literature. The datasets tested are from the most commonly used side-scan sonars (SSSs) and forward-looking sonars (FLSs). Additionally, the detection idea of these detectors on the acoustic imagery phase congruency (PC) layer was innovatively proposed with the aim of finding a solution that balances detection accuracy and speed. The experimental results show that the ORB (Oriented FAST and Rotated BRIEF) and BRISK (Binary Robust Invariant Scalable Keypoints) detectors achieve the best overall performance, the FAST detector is the fastest, and the PC and Sobel layers are the most favorable for implementing feature detection. Full article

Feature matching is a fundamental procedure in several image processing methods applied in remote sensing. Multispectral sensors with different wavelengths can provide complementary information. In this work, we propose a multispectral line segment matching algorithm based on phase congruency and multiple local homographies (PC-MLH) for image pairs captured by the cross-spectrum sensors (visible spectrum and infrared spectrum) in man-made scenarios. The feature points are first extracted and matched according to phase congruency. Next, multi-layer local homographies are derived from clustered feature points via random sample consensus (RANSAC) to guide line segment matching. Moreover, three geometric constraints (line position encoding, overlap ratio, and point-to-line distance) are introduced in cascade to reduce the computational complexity. The two main contributions of our work are as follows: First, compared with the conventional line matching methods designed for single-spectrum images, PC-MLH is robust against nonlinear radiation distortion (NRD) and can handle the unknown multiple local mapping, two common challenges associated with multispectral feature matching. Second, fusion of line extraction results and line position encoding for neighbouring matching increase the number of matched line segments and speed up the matching process, respectively. The method is validated using two public datasets, CVC-multimodal and VIS-IR. The results show that the percentage of correct matches (PCM) using PC-MLH can reach 94%, which significantly outperforms other single-spectral and multispectral line segment matching methods. Full article

Automatic registration of optical and synthetic aperture radar (SAR) images is a challenging task due to the influence of SAR speckle noise and nonlinear radiometric differences. This study proposes a robust algorithm based on phase congruency to register optical and SAR images (ROS-PC). It consists of a uniform Harris feature detection method based on multi-moment of the phase congruency map (UMPC-Harris) and a local feature descriptor based on the histogram of phase congruency orientation on multi-scale max amplitude index maps (HOSMI). The UMPC-Harris detects corners and edge points based on a voting strategy, the multi-moment of phase congruency maps, and an overlapping block strategy, which is used to detect stable and uniformly distributed keypoints. Subsequently, HOSMI is derived for a keypoint by utilizing the histogram of phase congruency orientation on multi-scale max amplitude index maps, which effectively increases the discriminability and robustness of the final descriptor. Finally, experimental results obtained using simulated images show that the UMPC-Harris detector has a superior repeatability rate. The image registration results obtained on test images show that the ROS-PC is robust against SAR speckle noise and nonlinear radiometric differences. The ROS-PC can tolerate some rotational and scale changes. Full article

In multi-modality image fusion, source image decomposition, such as multi-scale transform (MST), is a necessary step and also widely used. However, when MST is directly used to decompose source images into high- and low-frequency components, the corresponding decomposed components are not precise enough for the following infrared-visible fusion operations. This paper proposes a non-subsampled contourlet transform (NSCT) based decomposition method for image fusion, by which source images are decomposed to obtain corresponding high- and low-frequency sub-bands. Unlike MST, the obtained high-frequency sub-bands have different decomposition layers, and each layer contains different information. In order to obtain a more informative fused high-frequency component, maximum absolute value and pulse coupled neural network (PCNN) fusion rules are applied to different sub-bands of high-frequency components. Activity measures, such as phase congruency (PC), local measure of sharpness change (LSCM), and local signal strength (LSS), are designed to enhance the detailed features of fused low-frequency components. The fused high- and low-frequency components are integrated to form a fused image. The experiment results show that the fused images obtained by the proposed method achieve good performance in clarity, contrast, and image information entropy. Full article

Infrared and visible image matching methods have been rising in popularity with the emergence of more kinds of sensors, which provide more applications in visual navigation, precision guidance, image fusion, and medical image analysis. In such applications, image matching is utilized for location, fusion, image analysis, and so on. In this paper, an infrared and visible image matching approach, based on distinct wavelength phase congruency (DWPC) and log-Gabor filters, is proposed. Furthermore, this method is modified for non-linear image matching with different physical wavelengths. Phase congruency (PC) theory is utilized to obtain PC images with intrinsic and affluent image features for images containing complex intensity changes or noise. Then, the maximum and minimum moments of the PC images are computed to obtain the corners in the matched images. In order to obtain the descriptors, log-Gabor filters are utilized and overlapping subregions are extracted in a neighborhood of certain pixels. In order to improve the accuracy of the algorithm, the moments of PCs in the original image and a Gaussian smoothed image are combined to detect the corners. Meanwhile, it is improper that the two matched images have the same PC wavelengths, due to the images having different physical wavelengths. Thus, in the experiment, the wavelength of the PC is changed for different physical wavelengths. For realistic application, BiDimRegression method is proposed to compute the similarity between two points set in infrared and visible images. The proposed approach is evaluated on four data sets with 237 pairs of visible and infrared images, and its performance is compared with state-of-the-art approaches: the edge-oriented histogram descriptor (EHD), phase congruency edge-oriented histogram descriptor (PCEHD), and log-Gabor histogram descriptor (LGHD) algorithms. The experimental results indicate that the accuracy rate of the proposed approach is 50% higher than the traditional approaches in infrared and visible images. Full article

The identification and monitoring of buildings from remotely sensed imagery are of considerable value for urbanization monitoring. Two outstanding issues in the detection of changes in buildings with composite structures and relief displacements are heterogeneous appearances and positional inconsistencies. In this paper, a novel patch-based matching approach is developed using densely connected conditional random field (CRF) optimization to detect building changes from bi-temporal aerial images. First, the bi-temporal aerial images are combined to obtain change information using an object-oriented technique, and then semantic segmentation based on a deep convolutional neural network is used to extract building areas. With the change information and extracted buildings, a graph-cuts-based segmentation algorithm is applied to generate the bi-temporal changed building proposals. Next, in the bi-temporal changed building proposals, corner and edge information are integrated for feature detection through a phase congruency (PC) model, and the structural feature descriptor, called the histogram of orientated PC, is used to perform patch-based roof matching. We determined the final change in buildings by gathering matched roof and bi-temporal changed building proposals using co-refinement based on CRF, which were further classified as “newly built,” “demolished”, or “changed”. Experiments were conducted with two typical datasets covering complex urban scenes with diverse building types. The results confirm the effectiveness and generality of the proposed algorithm, with more than 85% and 90% in overall accuracy and completeness, respectively. Full article

Edge detection in Synthetic Aperture Radar (SAR) images has been a challenging task due to the speckle noise. Ratio-based edge detectors are robust operators for SAR images that provide constant false alarm rates, but they are only optimal for step edges. Edge detectors developed by the phase congruency model provide the identification of different types of edge features, but they suffer from speckle noise. By combining the advantages of the two edge detectors, we propose a SAR phase congruency detector (SAR-PC). Firstly, an improved local energy model for SAR images is obtained by replacing the convolution of raw image and the quadrature filters by the ratio responses. Secondly, a new noise level is estimated for the multiplicative noise. Substituting the SAR local energy and the new noise level into the phase congruency model, SAR-PC is derived. Edge response corresponds to the max moment of SAR-PC. We compare the proposed detector with the ratio-based edge detectors and the phase congruency edge detectors. Receiver Operating Characteristic (ROC) curves and visual effects are used to evaluate the performance. Experimental results of simulated images and real-world images show that the proposed edge detector is robust to speckle noise and it provides a consecutive edge response. Full article