Search Results (46)

Image registration constitutes the fundamental basis for the joint interpretation of synthetic aperture radar (SAR) and optical images. However, robust image registration remains challenging due to significant regional heterogeneity in remote sensing scenes (e.g., co-existing urban and marine areas within a single image). To overcome this challenge, this article proposes a novel optical–SAR image registration method named Gradient and Standard Deviation Feature Weighted Fusion (GDWF). First, a Block-local standard deviation (Block-LSD) operator is proposed to extract block-based feature points with regional adaptability. Subsequently, a dual-modal feature description is developed, constructing both gradient-based descriptors and local standard deviation (LSD) descriptors for the neighborhoods surrounding the detected feature points. To further enhance matching robustness, a confidence-weighted feature fusion strategy is proposed. By establishing a reliability evaluation model for similarity measurement maps, the contribution weights of gradient features and LSD features are dynamically optimized, ensuring adaptive performance under varying conditions. To verify the effectiveness of the method, different optical and SAR datasets are used to compare it with the currently advanced algorithms MOGF, CFOG, and FED-HOPC. The experimental results demonstrate that the proposed GDWF algorithm achieves the best performance in terms of registration accuracy and robustness among all compared methods, effectively handling optical–SAR image pairs with significant regional heterogeneity. Full article

This study investigates the generation of synthetic CT (sCT) images from zero echo time (ZTE) MRI to support MR-only radiotherapy, which can reduce image registration errors and lower treatment planning costs. Since MRI lacks the electron density data required for accurate dose calculations, generating reliable sCTs is essential. ZTE MRI, offering high bone contrast, was used with two deep learning models: attention deep residual U-Net (ADR-Unet) and derived conditional generative adversarial network (cGAN). Data from 17 head and neck cancer patients were used to train and evaluate the models. ADR-Unet was enhanced with deep residual blocks and attention mechanisms to improve learning and reconstruction quality. Both models were implemented in-house and compared to standard U-Net and Unet++ architectures using image quality metrics, visual inspection, and dosimetric analysis. Volumetric modulated arc therapy (VMAT) planning was performed on both planning CT and generated sCTs. ADR-Unet achieved a mean absolute error of 55.49 HU and a Dice score of 0.86 for bone structures. All the models demonstrated Gamma pass rates above 99.4% and dose deviations within 2–3%, confirming clinical acceptability. These results highlight ADR-Unet and cGAN as promising solutions for accurate sCT generation, enabling effective MR-only radiotherapy. Full article

Multimodal remote sensing image registration aims to achieve effective fusion and analysis of information by accurately aligning image data obtained by different sensors, thereby improving the accuracy and application value of remote sensing data in engineering. However, current advanced registration frameworks are unable to accurately register large-scale rigid distortions, such as rotation or scaling, that occur in multi-source remote sensing images. This paper presents a stable and high-precision end-to-end registration network that incorporates dual-branch feature extraction to address the stringent registration requirements encountered in practical engineering applications. The deep neural network consists of three parts: dual-branch feature extraction, affine parameter regression, and spatial transformation network. In the upper branch of the dual-branch feature extraction module, we designed a combination of multi-scale convolution and Swin Transformer to fully extract features of remote sensing images at different scales and levels to better understand the global structure and context information. In the lower branch, we incorporate strip convolution blocks to capture remote contextual information from various directions in multimodal images. Additionally, we introduce an efficient and lightweight ResNet module to enhance global features. At the same time, we developed a strategy to parallelize various convolution kernels in affine parameter regression networks, aiming to enhance the accuracy of transformation parameters and the robustness of the model. We conducted experiments on panchromatic–multispectral, infrared–optical, and SAR–optical image pairs with large-scale rigid transformations. The experimental results show that our method achieves the best registration effect. Full article

In recent years, disaster chains caused by glacier movements have occurred frequently in the lower Yarlung Tsangpo River in southwest China. However, it is still unclear whether earthquakes significantly contribute to glacier movements and disaster chains. In addition, it is difficult to measure the high-frequency and large gradient displacement time series with optical remote sensing images due to cloud coverage. To this end, we take the Sedongpu disaster chain as an example, where the Milin earthquake, with an epicenter 11 km away, occurred on 18 November 2017. Firstly, to deal with the cloud coverage problem for single optical remote sensing analysis, we employed multiple platform optical images and conducted a cross-platform correlation technique to invert the two-dimensional displacement rate and the cumulative displacement time series of the Sedongpu glacier. To reveal the correlation between earthquakes and disaster chains, we divided the optical images into three classes according to the Milin earthquake event. Lastly, to increase the accuracy and reliability, we propose two strategies for displacement monitoring, that is, a four-quadrant block registration strategy and a multi-window fusion strategy. Results show that the RMSE reduction percentage of the proposed registration method reaches 80%, and the fusion method can retrieve the large magnitude displacements and complete displacement field. Secondly, the Milin earthquake accelerated the Sedongpu glacier movement, where the pre-seismic velocities were less than 0.5 m/day, the co-seismic velocities increased to 1 to 6 m/day, and the post-seismic velocities decreased to 0.5 to 3 m/day. Lastly, the earthquake had a triggering effect around 33 days on the Sedongpu disaster chain event on 21 December 2017. The failure pattern can be summarized as ice and rock collapse in the source area, large magnitude glacier displacement in the moraine area, and a large volume of sediment in the deposition area, causing a river blockage. Full article

The differences in sensor imaging mechanisms, observation angles, and scattering characteristics of terrestrial objects significantly limit the registration performance of synthetic aperture radar (SAR) and optical heterologous images. Traditional methods particularly struggle in weak feature regions, such as harbors and islands with substantial water coverage, as well as in desolate areas like deserts. This paper introduces a robust heterologous image registration technique based on region-adaptive keypoint selection that integrates image texture features, targeting two pivotal aspects: feature point extraction and matching point screening. Initially, a dual threshold criterion based on block region information entropy and variance products effectively identifies weak feature regions. Subsequently, it constructs feature descriptors to generate similarity maps, combining histogram parameter skewness with non-maximum suppression (NMS) to enhance matching point accuracy. Extensive experiments have been conducted on conventional SAR-optical datasets and typical SAR-optical images with different weak feature regions to assess the method’s performance. The findings indicate that this method successfully removes outliers in weak feature regions and completes the registration task of SAR and optical images with weak feature regions. Full article

Corrosion is one of the critical factors leading to the failure of steel structures. Ultrasonic C-scans are widely used to identify corrosion damage. Limited by the range of C-scans, multiple C-scans are usually required to cover the whole component. Thus, stitching multiple C-scans into a panoramic image of the area under detection is necessary for interpreting non-destructive testing (NDT) data. In this paper, an image mosaic method for ultrasonic C-scan based on scale invariant feature transform (SIFT) is proposed. Firstly, to improve the success rate of registration, the difference in the probe starting position in two scans is used to filter the matching pairs of feature points obtained by SIFT. Secondly, dynamic programming methods are used to search for the optimal seam path. Finally, the pixels in the overlapping area are fused by fade-in and fade-out fusion along the seam line. The improved method has a higher success rate of registration and lower image distortion than the conventional method in the mosaic of ultrasonic C-scan images. Experimental results show that the proposed method can stitch multiple C-scan images of a testing block containing artificial defects into a panorama image effectively. Full article

Thangka refers to a form of Tibetan Buddhist painting on a fabric, scroll, or Thangka, often depicting deities, scenes, or mandalas. Deep-learning-based super-resolution techniques have been applied to improve the spatial resolution of hyperspectral images (HSIs), especially for the preservation and analysis of Thangka cultural heritage. However, existing CNN-based methods encounter difficulties in effectively preserving spatial information, due to challenges such as registration errors and spectral variability. To overcome these limitations, we present a novel cross-sensor super-resolution (SR) framework that utilizes high-resolution RGBs (HR-RGBs) to enhance the spectral features in low-resolution hyperspectral images (LR-HSIs). Our approach utilizes spatial–spectral integration (SSI) blocks and spatial–spectral restoration (SSR) blocks to effectively integrate and reconstruct spatial and spectral features. Furthermore, we introduce a frequency multi-head self-attention (F-MSA) mechanism that treats high-, medium-, and low-frequency features as tokens, enabling self-attention computations across the frequency dimension. We evaluate our method on a custom dataset of ancient Thangka paintings and demonstrate its effectiveness in enhancing the spectral resolution in high-resolution hyperspectral images (HR-HSIs), while preserving the spatial characteristics of Thangka artwork with minimal information loss. Full article

This study aimed to achieve the accurate and real-time geographic positioning of UAV aerial image targets. We verified a method of registering UAV camera images on a map (with the geographic location) through feature matching. The UAV is usually in rapid motion and involves changes in the camera head, and the map is high-resolution and has sparse features. These reasons make it difficult for the current feature-matching algorithm to accurately register the two (camera image and map) in real time, meaning that there will be a large number of mismatches. To solve this problem, we used the SuperGlue algorithm, which has a better performance, to match the features. The layer and block strategy, combined with the prior data of the UAV, was introduced to improve the accuracy and speed of feature matching, and the matching information obtained between frames was introduced to solve the problem of uneven registration. Here, we propose the concept of updating map features with UAV image features to enhance the robustness and applicability of UAV aerial image and map registration. After numerous experiments, it was proved that the proposed method is feasible and can adapt to the changes in the camera head, environment, etc. The UAV aerial image is stably and accurately registered on the map, and the frame rate reaches 12 frames per second, which provides a basis for the geo-positioning of UAV aerial image targets. Full article

GaoFen 7 (GF-7) is China’s first submeter high-resolution stereo mapping satellite with dual-linear-array cameras and a laser altimeter system onboard for high-precision mapping. To further take advantage of the very high elevation accuracy of laser altimetry data and the high relative accuracy with stereo images, an innovative combined adjustment method for GF-7 stereo images with laser altimetry data is presented in this paper. In this method, two flexible and effective schemes were proposed to extract the elevation control point according to the registration of footprint images and stereo images and then utilized as vertical control in the block adjustment to improve the elevation accuracy without ground control points (GCPs). The validation experiments were conducted in Shandong, China, with different terrains. The results demonstrated that, after using the laser altimetry data, the root mean square error (RMSE) of elevation was dramatically improved from the original 2.15 m to 0.75 m, while the maximum elevation error was less than 1.6 m. Moreover, by integrating a few horizontal control points, the planar and elevation accuracy can be simultaneously improved. The results show that the method will be useful for reducing the need for field survey work and improving mapping efficiency. Full article

In this paper, the super-resolution structural point cloud matching (S²-PCM) framework is proposed for video synthetic aperture radar (SAR) inter-frame registration, which consists of a feature recurrence super-resolution network (FRSR-Net), structural point cloud extraction network (SPCE-Net) and robust point matching network (RPM-Net). FRSR-Net is implemented by integrating the feature recurrence structure and residual dense block (RDB) for super-resolution enhancement, SPCE-Net is implemented by training a U-Net with data augmentation, and RPM-Net is applied for robust point cloud matching. Experimental results show that compared with the classical SIFT-like algorithms, S²-PCM achieves higher registration accuracy for video-SAR images under diverse evaluation metrics, such as mutual information (MI), normalized mutual information (NMI), entropy correlation coefficient (ECC), structural similarity (SSIM), etc. The proposed FRSR-Net can significantly improve the quality of video-SAR images and point cloud extraction accuracy. Combining FRSR-Net with S²-PCM, we can obtain higher inter-frame registration accuracy, which is crucial for moving target detection and shadow tracking. Full article

The temporal subtraction technique is a useful tool for computer aided diagnosis (CAD) in visual screening. The technique subtracts the previous image set from the current one for the same subject to emphasize temporal changes and/or new abnormalities. However, it is difficult to obtain a clear subtraction image without subtraction image artifacts. VoxelMorph in deep learning is a useful method, as preparing large training datasets is difficult for medical image analysis, and the possibilities of incorrect learning, gradient loss, and overlearning are concerns. To overcome this problem, we propose a new method for generating temporal subtraction images of thoracic multi-detector row computed tomography (MDCT) images based on ResidualVoxelMorph, which introduces a residual block to VoxelMorph to enable flexible positioning at a low computational cost. Its high learning efficiency can be expected even with a limited training set by introducing residual blocks to VoxelMorph. We performed our method on 84 clinical images and evaluated based on three-fold cross-validation. The results showed that the proposed method reduced subtraction image artifacts on root mean square error (RMSE) by 11.3% (p < 0.01), and its effectiveness was verified. That is, the proposed temporal subtraction method for thoracic MDCT improves the observer’s performance. Full article

The geometric registration of mountainous remote sensing images is always a challenging project, as terrain fluctuations increase the complexity. Deep learning, with its superior computing power and data-driven nature, promises to solve this problem. However, the lack of an appropriate dataset limits the development of deep learning technology for mountainous remote sensing image registration, which it still an unsolved problem in photogrammetry and remote sensing. To remedy this problem, this paper presents a manually constructed imagery dataset of mountainous regions, called the MID (mountainous imagery dataset). To create the MID, we use 38 images from the Gaofen-2 satellite developed by China and generated 4093 pairs of reference and sensed image patches, making this the first real mountainous dataset to our knowledge. Simultaneously, we propose a fully unsupervised, convolutional-network-based iterative registration scheme for the MID. First, the large and global deformation of the reference and sensed images is reduced using an affine registration module, generating the coarse alignment. Then, the local and varied distortions are learned and eliminated progressively using a hybrid dilated convolution (HDC)-based encoder–decoder module with multistep iterations, achieving fine registration results. The HDC aims to increase the receptive field without blocking the artifacts, allowing for the continuous characteristics of the mountainous images of a local region to be represented. We provide a performance analysis of some typical registration algorithms and the developed approach for the MID. The proposed scheme gives the highest registration precision, achieving the subpixel alignment of mountainous remote sensing images. Additionally, the experimental results demonstrate the usability of the MID, which can lay a foundation for the development of deep learning technology in large mountainous remote sensing image registration tasks. Full article

This study aimed to evaluate the bone volume change at donor sites in patients who received mandibular body bone block grafts using intensity-based automatic image registration. A retrospective study was conducted with 32 patients who received mandibular bone block grafts between 2017 and 2019 at the Pusan National University Dental Hospital. Cone-beam computed tomography (CBCT) images were obtained before surgery (T0), 1 day after surgery (T1), and 4 months after surgery (T2). Scattered artefacts were removed by manual segmentation. The T0 image was used as the reference image for registration of T1 and T2 images using intensity-based registration. A total of 32 donor sites were analyzed three-dimensionally. The volume and pixel value of the bones were measured and analyzed. The mean regenerated bone volume rate on follow-up images (T2) was 34.87% ± 17.11%. However, no statistically significant differences of regenerated bone volume were noted among the four areas of the donor site (upper anterior, upper posterior, lower anterior, and lower posterior). The mean pixel value rate of the follow-up images (T2) was 78.99% ± 16.9% compared with that of T1, which was statistically significant (p < 0.05). Intensity-based registration with histogram matching showed that newly generated bone is generally qualitatively and quantitatively poorer than the original bone, thus revealing the feasibility of pixel value to evaluate bone quality in CBCT images. Considering the bone mass recovered in this study, 4 months may not be sufficient for a second harvesting, and a longer period of follow-up is required. Full article

Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operations, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying or mapping corresponding breast lesion features computed from unregistered multiple mammograms. This motivated us to leverage the architecture of Multi-view Vision Transformers to capture long-range relationships of multiple mammograms from the same patient in one examination. For this purpose, we employed local transformer blocks to separately learn patch relationships within four mammograms acquired from two-view (CC/MLO) of two-side (right/left) breasts. The outputs from different views and sides were concatenated and fed into global transformer blocks, to jointly learn patch relationships between four images representing two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of mammograms, which included 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our four-image (two-view-two-side) transformer-based model achieves case classification performance with an area under ROC curve (AUC = 0.818 ± 0.039), which significantly outperforms AUC = 0.784 ± 0.016 achieved by the state-of-the-art multi-view CNNs (p = 0.009). It also outperforms two one-view-two-side models that achieve AUC of 0.724 ± 0.013 (CC view) and 0.769 ± 0.036 (MLO view), respectively. The study demonstrates the potential of using transformers to develop high-performing computer-aided diagnosis schemes that combine four mammograms. Full article

Thermal imaging is an important source of information for geographic information systems (GIS) in various aspects of environmental research. This work contains a variety of experiences related to the use of the Yuneec E10T thermal imaging camera with a 320 × 240 pixel matrix and 4.3 mm focal length dedicated to working with the Yuneec H520 UAV in obtaining data on the natural environment. Unfortunately, as a commercial product, the camera is available without radiometric characteristics. Using the heated bed of the Omni3d Factory 1.0 printer, radiometric calibration was performed in the range of 18–100 °C (high sensitivity range–high gain settings of the camera). The stability of the thermal camera operation was assessed using several sets of a large number of photos, acquired over three areas in the form of aerial blocks composed of parallel rows with a specific sidelap and longitudinal coverage. For these image sets, statistical parameters of thermal images such as the mean, minimum and maximum were calculated and then analyzed according to the order of registration. Analysis of photos taken every 10 m in vertical profiles up to 120 m above ground level (AGL) were also performed to show the changes in image temperature established within the reference surface. Using the established radiometric calibration, it was found that the camera maintains linearity between the observed temperature and the measured brightness temperature in the form of a digital number (DN). It was also found that the camera is sometimes unstable after being turned on, which indicates the necessity of adjusting the device’s operating conditions to external conditions for several minutes or taking photos over an area larger than the region of interest. Full article