Search Results (17)

Convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have been widely applied to hyperspectral image classification tasks, but both exhibit certain limitations. To address these issues, this paper proposes a multi-scale feature fusion architecture (MCGNet). Symmetry serves as the core design principle of MCGNet, where its parallel CNN-GCN branches and multi-scale fusion mechanism strike a balance between local spectral-spatial features and global graph structural dependencies, effectively reducing redundancy and enhancing generalization capabilities. The architecture comprises four modules: the Spectral Noise Suppression (SNS) module enhances the signal-to-noise ratio of spectral features; the Local Spectral Extraction (LSE) module employs deep separable convolutions to extract local spectral-spatial features; Superpixel-level Graph Convolution (SGC), performing graph convolution on superpixel graphs to precisely capture dependencies between object regions; Pixel-level Graph Convolution (PGC), constructed via adaptive sparse pixel graphs based on spectral and spatial similarity to accurately capture irregular boundaries and fine-grained non-local relationships between pixels. These modules form a symmetric, hierarchical feature learning pipeline integrated within a unified framework. Experiments on three public datasets—Indian Pine, Pavia University, and Salinas—demonstrate that MCGNet outperforms baseline methods in overall accuracy, average precision, and Kappa coefficient. This symmetric design not only enhances classification performance but also endows the model with strong theoretical interpretability and cross-dataset robustness, highlighting the significance of symmetry principles in hyperspectral image analysis. Full article

In this study, we propose a novel pixel-level non-local similarity (PNS)-based reconstruction method for millimeter-wave interferometric synthetic aperture radiometer (InSAR) imaging. Unlike traditional compressed sensing (CS) methods, which rely on predefined sparse transforms and often introduce artifacts, our approach leverages structural redundancies in InSAR images through an enhanced sparse representation model with dynamically filtered coefficients. This design simultaneously preserves fine details and suppresses noise interference. Furthermore, an iterative refinement mechanism incorporates raw sampled data fidelity constraints, enhancing reconstruction accuracy. Simulation and physical experiments demonstrate that the proposed InSAR-PNS method significantly outperforms conventional techniques: it achieves a 1.93 dB average peak signal-to-noise ratio (PSNR) improvement over CS-based reconstruction while operating at reduced sampling ratios compared to Nyquist-rate fast fourier transform (FFT) methods. The framework provides a practical and efficient solution for high-fidelity millimeter-wave InSAR imaging under sub-Nyquist sampling conditions. Full article

Inpainting noisy remote-sensing images can reduce the cost of acquiring remote-sensing images (RSIs). Since RSIs contain complex land structure features and concentrated obscured areas, existing inpainting methods often produce color inconsistency and structural smoothing when applied to RSIs with a high missing ratio. To address these problems, inspired by tensor recovery, a lightweight image Inpainting Generative Adversarial Network (GAN) method combining low-rankness and local-smoothness (IGLL) is proposed. IGLL utilizes the low-rankness and local-smoothness characteristics of RSIs to guide the deep-learning inpainting. Based on the strong low rankness characteristic of the RSIs, IGLL fully utilizes the background information for foreground inpainting and constrains the consistency of the key ranks. Based on the low smoothness characteristic of the RSIs, learnable edges and structure priors are designed to enhance the non-smoothness of the results. Specifically, the generator of IGLL consists of a pixel-level reconstruction net (PIRN) and a perception-level reconstruction net (PERN). In PIRN, the proposed global attention module (GAM) establishes long-range pixel dependencies. GAM performs precise normalization and avoids overfitting. In PERN, the proposed flexible feature similarity module (FFSM) computes the similarity between background and foreground features and selects a reasonable feature for recovery. Compared with existing works, FFSM improves the fineness of feature matching. To avoid the problem of local-smoothness in the results, both the generator and discriminator utilize the structure priors and learnable edges to regularize large concentrated missing regions. Additionally, IGLL incorporates mathematical constraints into deep-learning models. A singular value decomposition (SVD) loss item is proposed to model the low-rankness characteristic, and it constrains feature consistency. Extensive experiments demonstrate that the proposed IGLL performs favorably against state-of-the-art methods in terms of the reconstruction quality and computation costs, especially on RSIs with high mask ratios. Moreover, our ablation studies reveal the effectiveness of GAM, FFSM, and SVD loss. Source code is publicly available on GitHub. Full article

Event-based clustering provides a low-power embedded solution for low-level feature extraction in a scene. The algorithm utilizes the non-uniform sampling capability of event-based image sensors to measure local intensity variations within a scene. Consequently, the clustering algorithm forms similar event groups while simultaneously estimating their attributes. This work proposes taking advantage of additional event information in order to provide new attributes for further processing. We elaborate on the estimation of the object velocity using the mean motion of the cluster. Next, we are examining a novel form of events, which includes intensity measurement of the color at the concerned pixel. These events may be processed to estimate the rough color of a cluster, or the color distribution in a cluster. Lastly, this paper presents some applications that utilize these features. The resulting algorithms are applied and exercised thanks to a custom event-based simulator, which generates videos of outdoor scenes. The velocity estimation methods provide satisfactory results with a trade-off between accuracy and convergence speed. Regarding color estimation, the luminance estimation is challenging in the test cases, while the chrominance is precisely estimated. The estimated quantities are adequate for accurately classifying objects into predefined categories. Full article

This paper presents a parallel implementation of a non-local transform-domain filter (BM4D). The effectiveness of the parallel implementation is demonstrated by denoising image series from computed tomography (CT) and magnetic resonance imaging (MRI). The basic idea of the filter is based on grouping and filtering similar data within the image. Due to the high level of similarity and data redundancy, the filter can provide even better denoising quality than current extensively used approaches based on deep learning (DL). In BM4D, cubes of voxels named patches are the essential image elements for filtering. Using voxels instead of pixels means that the area for searching similar patches is large. Because of this and the application of multi-dimensional transformations, the computation time of the filter is exceptionally long. The original implementation of BM4D is only single-threaded. We provide a parallel version of the filter that supports multi-core and many-core processors and scales on such versatile hardware resources, typical for high-performance computing clusters, even if they are concurrently used for the task. Our algorithm uses hybrid parallelisation that combines open multi-processing (OpenMP) and message passing interface (MPI) technologies and provides up to 283× speedup, which is a 99.65% reduction in processing time compared to the sequential version of the algorithm. In denoising quality, the method performs considerably better than recent DL methods on the data type that these methods have yet to be trained on. Full article

Image segmentation methods have received widespread attention in face image recognition, which can divide each pixel in the image into different regions and effectively distinguish the face region from the background for further recognition. Threshold segmentation, a common image segmentation method, suffers from the problem that the computational complexity shows exponential growth with the increase in the segmentation threshold level. Therefore, in order to improve the segmentation quality and obtain the segmentation thresholds more efficiently, a multi-threshold image segmentation framework based on a meta-heuristic optimization technique combined with Kapur’s entropy is proposed in this study. A meta-heuristic optimization method based on an improved grey wolf optimizer variant is proposed to optimize the 2D Kapur’s entropy of the greyscale and nonlocal mean 2D histograms generated by image computation. In order to verify the advancement of the method, experiments compared with the state-of-the-art method on IEEE CEC2020 and face image segmentation public dataset were conducted in this paper. The proposed method has achieved better results than other methods in various tests at 18 thresholds with an average feature similarity of 0.8792, an average structural similarity of 0.8532, and an average peak signal-to-noise ratio of 24.9 dB. It can be used as an effective tool for face segmentation. Full article

Structures in their service life are often damaged as a result of aging or extreme events such as earthquakes or storms. It is essential to detect damage in a timely fashion to ensure the safe operation of the structure. If left unchecked, subsurface damage (SSD) can cause significant internal damage and may result in premature structural failure. In this study, a Convolutional Neural Network (CNN) has been developed for SSD detection using surface strain measurements. The adopted network architecture is capable of pixel-level image segmentation, that is, it classifies each location of strain measurement as damaged or undamaged. The CNN which is fed full-field strain measurements as an input image of size 256 × 256 projects the SSD onto an output image of the same size. The data for network training is generated by numerical simulation of aluminum bars with different damage scenarios, including single damage and double damage cases at a random location, direction, length, and thickness. The trained network achieves an Intersection over Union (IoU) score of 0.790 for the validation set and 0.794 for the testing set. To check the applicability of the trained network on materials other than aluminum, testing is performed on a numerically generated steel dataset. The IoU score is 0.793, the same as the aluminum dataset, affirming the network’s capability to apply to materials exhibiting a similar stress–strain relationship. To check the generalization potential of the network, it is tested on triple damage cases; the IoU score is found to be 0.764, suggesting that the network works well for unseen damage patterns as well. The network was also found to provide accurate predictions for real experimental data obtained from Strain Sensing Smart Skin (S⁴). This proves the efficacy of the network to work in real-life scenarios utilizing the full potential of the novel full-field strain sensing methods such as S⁴. The performance of the proposed network affirms that it can be used as a non-destructive testing method for subsurface crack detection and localization. Full article

As clouds of different thicknesses block sunlight, large areas of cloud shadows with varying brightness can appear on the ground. Cloud shadows in high-resolution remote sensing images lead to uneven loss of image feature information. However, cloud shadows still retain feature information, and how to compensate for and restore unbalanced cloud shadow occlusion is of great significance in improving image quality. Though traditional shadow compensation methods can enhance the shaded brightness, the results are inconsistent in a single shadow region with over-compensated or insufficient compensation problems. Thus, this paper proposes a shadow-balanced compensation method combined with multi-level information. Multi-level information comprising the information of a shadow pixel, a local super-pixel centered with the pixel, the global cloud shadow region, and the global non-shadow region information, to comply with the cloud shadow’s internal difference. First, the original image is detected via the cloud shadow detection method and post-processing. The initial shadow is detected combined with designed complex shadow features and morphological shadow index features with threshold methods. Then, post-processing considering shadow area and morphological operation is applied to remove the small, non-cloud-shadow objects. Meanwhile, the initial image is also divided into super-pixel homogeneity regions using the super-pixel segmentation principle. A super-pixel region is between the pixel and the shadow area. Different from pixel and other window regions, it can provide a different measurement levels considering object homogeneity. Thus, a balanced compensation model is designed by combining the feature value of a shadow pixel and the mean and variance of a super-pixel, shadow region, and non-shadow region on the basis of the linear correlation correction principle. The super-pixel around the shadow pixel provides a local reliable homogenous region. It can reflect the internal difference inside the shadow region. Therefore, introducing a super-pixel in the proposed model can effectively compensate for the shaded information in a balanced way. Compared to those of only using pixel and shadow region information, the compensated results introduce super-pixel information, can deal with the homogenous region as a global one, and can be adaptive to the illustration differences in a cloud shadow. The experimental results show that compared to that of other reference methods, the quality of the proposed compensation result is better. The proposed method can enhance brightness and recover detailed information in shadow regions in a more balanced way. The issue of over-compensation and insufficient compensation inside a single shadow region can be resolved. Thus, the total result is similar to that of a non-shadow region. The proposed method can be used to recover the cloud shadow information more self-adaptively to improve image quality and usage in other applications. Full article

Freshwater ecosystems host high levels of biodiversity but are also highly vulnerable to biological invasions. Aquatic Invasive Alien Plant Species (aIAPS) can cause detrimental effects on freshwater ecosystems and their services to society, raising challenges to decision-makers regarding their correct management. Spatially and temporally explicit information on the occurrence of aIAPS in dynamic freshwater systems is essential to implement efficient regional and local action plans. The use of unmanned aerial vehicle imagery synchronized with free Sentinel-2 multispectral data allied with classifier fusion techniques may support more efficient monitoring actions for non-stationary aIAPS. Here, we explore the advantages of such a novel approach for mapping the invasive water-hyacinth (Eichhornia crassipes) in the Cávado River (northern Portugal). Invaded and non-invaded areas were used to explore the evolution of spectral attributes of Eichhornia crassipes through a time series (processed by a super-resolution algorithm) that covers March 2021 to February 2022 and to build an occurrence dataset (presence or absence). Analysis of the spectral behavior throughout the year allowed the detection of spectral regions with greater capacity to distinguish the target plant from the surrounding environment. Classifier fusion techniques were implemented in the biomod2 predictive modelling package and fed with selected spectral regions to firstly extract a spectral signature from the synchronized day and secondly to identify pixels with similar reflectance values over time. Predictions from statistical and machine-learning algorithms were ensembled to map invaded spaces across the whole study area during all seasons with classifications attaining high accuracy values (True Skill Statistic, TSS: 0.932; Area Under the Receiver Operating Curve, ROC: 0.992; Kappa: 0.826). Our results provide evidence of the potential of our approach to mapping plant invaders in dynamic freshwater systems over time, applicable in the assessment of the success of control actions as well as in the implementation of long-term strategic monitoring. Full article

Background: Distinguishing between the spinal cord and cerebrospinal fluid (CSF) non-invasively on CT is challenging due to their similar mass densities. We hypothesize that patch-based machine learning applied to dual-energy CT can accurately distinguish CSF from neural or other tissues based on the center voxel and neighboring voxels. Methods: 88 regions of interest (ROIs) from 12 patients’ dual-energy (100 and 140 kVp) lumbar spine CT exams were manually labeled by a neuroradiologist as one of 4 major tissue types (water, fat, bone, and nonspecific soft tissue). Four-class classifier convolutional neural networks were trained, validated, and tested on thousands of nonoverlapping patches extracted from 82 ROIs among 11 CT exams, with each patch representing pixel values (at low and high energies) of small, rectangular, 3D CT volumes. Different patch sizes were evaluated, ranging from 3 × 3 × 3 × 2 to 7 × 7 × 7 × 2. A final ensemble model incorporating all patch sizes was tested on patches extracted from six ROIs in a holdout patient. Results: Individual models showed overall test accuracies ranging from 99.8% for 3 × 3 × 3 × 2 patches (N = 19,423) to 98.1% for 7 × 7 × 7 × 2 patches (N = 1298). The final ensemble model showed 99.4% test classification accuracy, with sensitivities and specificities of 90% and 99.6%, respectively, for the water class and 98.6% and 100% for the soft tissue class. Conclusions: Convolutional neural networks utilizing local low-level features on dual-energy spine CT can yield accurate tissue classification and enhance the visualization of intraspinal neural tissue. Full article

Video super-resolution can generate corresponding to high-resolution video frames from a plurality of low-resolution video frames which have rich details and temporally consistency. Most current methods use two-level structure to reconstruct video frames by combining optical flow network and super-resolution network, but this process does not deeply mine the effective information contained in video frames. Therefore, we propose a video super-resolution method that combines non-local features and multi-scale features to extract more in-depth effective information contained in video frames. Our method obtains long-distance effective information by calculating the similarity between any two pixels in the video frame through the non-local module, extracts the local information covered by different scale convolution cores through the multi-scale feature fusion module, and fully fuses feature information using different connection modes of convolution cores. Experiments on different data sets show that the proposed method is superior to the existing methods in quality and quantity. Full article

Joint sparse representation classification (JSRC) is a representative spectral–spatial classifier for hyperspectral images (HSIs). However, the JSRC is inappropriate for highly heterogeneous areas due to the spatial information being extracted from a fixed-sized neighborhood block, which is often unable to conform to the naturally irregular structure of land cover. To address this problem, a superpixel-based JSRC with nonlocal weighting, i.e., superpixel-based nonlocal weighted JSRC (SNLW-JSRC), is proposed in this paper. In SNLW-JSRC, the superpixel representation of an HSI is first constructed based on an entropy rate segmentation method. This strategy forms homogeneous neighborhoods with naturally irregular structures and alleviates the inclusion of pixels from different classes in the process of spatial information extraction. Afterwards, the superpixel-based nonlocal weighting (SNLW) scheme is built to weigh the superpixel based on its structural and spectral information. In this way, the weight of one specific neighboring pixel is determined by the local structural similarity between the neighboring pixel and the central test pixel. Then, the obtained local weights are used to generate the weighted mean data for each superpixel. Finally, JSRC is used to produce the superpixel-level classification. This speeds up the sparse representation and makes the spatial content more centralized and compact. To verify the proposed SNLW-JSRC method, we conducted experiments on four benchmark hyperspectral datasets, namely Indian Pines, Pavia University, Salinas, and DFC2013. The experimental results suggest that the SNLW-JSRC can achieve better classification results than the other four SRC-based algorithms and the classical support vector machine algorithm. Moreover, the SNLW-JSRC can also outperform the other SRC-based algorithms, even with a small number of training samples. Full article

Semantic segmentation has been a fundamental task in interpreting remote sensing imagery (RSI) for various downstream applications. Due to the high intra-class variants and inter-class similarities, inflexibly transferring natural image-specific networks to RSI is inadvisable. To enhance the distinguishability of learnt representations, attention modules were developed and applied to RSI, resulting in satisfactory improvements. However, these designs capture contextual information by equally handling all the pixels regardless of whether they around edges. Therefore, blurry boundaries are generated, rising high uncertainties in classifying vast adjacent pixels. Hereby, we propose an edge distribution attention module (EDA) to highlight the edge distributions of leant feature maps in a self-attentive fashion. In this module, we first formulate and model column-wise and row-wise edge attention maps based on covariance matrix analysis. Furthermore, a hybrid attention module (HAM) that emphasizes the edge distributions and position-wise dependencies is devised combing with non-local block. Consequently, a conceptually end-to-end neural network, termed as EDENet, is proposed to integrate HAM hierarchically for the detailed strengthening of multi-level representations. EDENet implicitly learns representative and discriminative features, providing available and reasonable cues for dense prediction. The experimental results evaluated on ISPRS Vaihingen, Potsdam and DeepGlobe datasets show the efficacy and superiority to the state-of-the-art methods on overall accuracy (OA) and mean intersection over union (mIoU). In addition, the ablation study further validates the effects of EDA. Full article

Wetlands are one of the most important ecosystems that provide a desirable habitat for a great variety of flora and fauna. Wetland mapping and modeling using Earth Observation (EO) data are essential for natural resource management at both regional and national levels. However, accurate wetland mapping is challenging, especially on a large scale, given their heterogeneous and fragmented landscape, as well as the spectral similarity of differing wetland classes. Currently, precise, consistent, and comprehensive wetland inventories on a national- or provincial-scale are lacking globally, with most studies focused on the generation of local-scale maps from limited remote sensing data. Leveraging the Google Earth Engine (GEE) computational power and the availability of high spatial resolution remote sensing data collected by Copernicus Sentinels, this study introduces the first detailed, provincial-scale wetland inventory map of one of the richest Canadian provinces in terms of wetland extent. In particular, multi-year summer Synthetic Aperture Radar (SAR) Sentinel-1 and optical Sentinel-2 data composites were used to identify the spatial distribution of five wetland and three non-wetland classes on the Island of Newfoundland, covering an approximate area of 106,000 km². The classification results were evaluated using both pixel-based and object-based random forest (RF) classifications implemented on the GEE platform. The results revealed the superiority of the object-based approach relative to the pixel-based classification for wetland mapping. Although the classification using multi-year optical data was more accurate compared to that of SAR, the inclusion of both types of data significantly improved the classification accuracies of wetland classes. In particular, an overall accuracy of 88.37% and a Kappa coefficient of 0.85 were achieved with the multi-year summer SAR/optical composite using an object-based RF classification, wherein all wetland and non-wetland classes were correctly identified with accuracies beyond 70% and 90%, respectively. The results suggest a paradigm-shift from standard static products and approaches toward generating more dynamic, on-demand, large-scale wetland coverage maps through advanced cloud computing resources that simplify access to and processing of the “Geo Big Data.” In addition, the resulting ever-demanding inventory map of Newfoundland is of great interest to and can be used by many stakeholders, including federal and provincial governments, municipalities, NGOs, and environmental consultants to name a few. Full article

Feature-based matching methods have been widely used in remote sensing image matching given their capability to achieve excellent performance despite image geometric and radiometric distortions. However, most of the feature-based methods are unreliable for complex background variations, because the gradient or other image grayscale information used to construct the feature descriptor is sensitive to image background variations. Recently, deep learning-based methods have been proven suitable for high-level feature representation and comparison in image matching. Inspired by the progresses made in deep learning, a new technical framework for remote sensing image matching based on the Siamese convolutional neural network is presented in this paper. First, a Siamese-type network architecture is designed to simultaneously learn the features and the corresponding similarity metric from labeled training examples of matching and non-matching true-color patch pairs. In the proposed network, two streams of convolutional and pooling layers sharing identical weights are arranged without the manually designed features. The number of convolutional layers is determined based on the factors that affect image matching. The sigmoid function is employed to compute the matching and non-matching probabilities in the output layer. Second, a gridding sub-pixel Harris algorithm is used to obtain the accurate localization of candidate matches. Third, a Gaussian pyramid coupling quadtree is adopted to gradually narrow down the searching space of the candidate matches, and multiscale patches are compared synchronously. Subsequently, a similarity measure based on the output of the sigmoid is adopted to find the initial matches. Finally, the random sample consensus algorithm and the whole-to-local quadratic polynomial constraints are used to remove false matches. In the experiments, different types of satellite datasets, such as ZY3, GF1, IKONOS, and Google Earth images, with complex background variations are used to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed method, which can significantly improve the matching performance of multi-temporal remote sensing images with complex background variations, is better than the state-of-the-art matching methods. In our experiments, the proposed method obtained a large number of evenly distributed matches (at least 10 times more than other methods) and achieved a high accuracy (less than 1 pixel in terms of root mean square error). Full article