Search Results (37)

Multimodal remote sensing object detection has gained increasing attention due to its ability to leverage complementary information from different sensing modalities, particularly visible (RGB) and thermal infrared (TIR) imagery. However, existing methods typically depend on deep, computationally intensive backbones and complex fusion strategies, limiting their suitability for real-time applications. To address these challenges, we propose a lightweight and efficient detection framework named RGB-TIR Multimodal Fusion Network (RTMF-Net), which introduces innovations in both the backbone architecture and fusion mechanism. Specifically, RTMF-Net adopts a dual-stream structure with modality-specific enhancement modules tailored for the characteristics of RGB and TIR data. The visible-light branch integrates a Convolutional Enhancement Fusion Block (CEFBlock) to improve multi-scale semantic representation with low computational overhead, while the thermal branch employs a Dual-Laplacian Enhancement Block (DLEBlock) to enhance frequency-domain structural features and weak texture cues. To further improve cross-modal feature interaction, a Weighted Denoising Fusion Module is designed, incorporating an Enhanced Fusion Attention (EFA) attention mechanism that adaptively suppresses redundant information and emphasizes salient object regions. Additionally, a Shape-Aware Intersection over Union (SA-IoU) loss function is proposed to improve localization robustness by introducing an aspect ratio penalty into the traditional IoU metric. Extensive experiments conducted on the ODinMJ and LLVIP multimodal datasets demonstrate that RTMF-Net achieves competitive performance, with mean Average Precision (mAP) scores of 98.7% and 95.7%, respectively, while maintaining a lightweight structure of only 4.3M parameters and 11.6 GFLOPs. These results confirm the effectiveness of RTMF-Net in achieving a favorable balance between accuracy and efficiency, making it well-suited for real-time remote sensing applications. Full article

Medical image segmentation is crucial for disease diagnosis, as precise results aid clinicians in locating lesion regions. However, lesions often have blurred boundaries and complex shapes, challenging traditional methods in capturing clear edges and impacting accurate localization and complete excision. Small lesions are also critical but prone to detail loss during downsampling, reducing segmentation accuracy. To address these issues, we propose a novel adaptive scale thresholding network (AdSTNet) that acts as a post-processing lightweight network for enhancing sensitivity to lesion edges and cores through a dual-threshold adaptive mechanism. The dual-threshold adaptive mechanism is a key architectural component that includes a main threshold map for core localization and an edge threshold map for more precise boundary detection. AdSTNet is compatible with any segmentation network and introduces only a small computational and parameter cost. Additionally, Spatial Attention and Channel Attention (SACA), the Laplacian operator, and the Fusion Enhancement module are introduced to improve feature processing. SACA enhances spatial and channel attention for core localization; the Laplacian operator retains edge details without added complexity; and the Fusion Enhancement module adapts concatenation operation and Convolutional Gated Linear Unit (ConvGLU) to improve feature intensities to improve edge and small lesion segmentation. Experiments show that AdSTNet achieves notable performance gains on ISIC 2018, BUSI, and Kvasir-SEG datasets. Compared with the original U-Net, our method attains mIoU/mDice of 83.40%/90.24% on ISIC, 71.66%/80.32% on BUSI, and 73.08%/81.91% on Kvasir-SEG. Moreover, similar improvements are observed in the rest of the networks. Full article

Current depth map sensing technologies capture depth maps at low spatial resolution, rendering serious problems in various applications. In this paper, we propose a single depth map super-resolution method that combines the advantages of model-based methods and deep learning approaches. Specifically, we formulate a linear inverse problem which we solve by introducing a graph Laplacian regularizer. The regularization approach promotes smoothness and preserves the structural details of the observed depth map. We construct the graph Laplacian matrix by deploying latent features obtained from a pretrained deep learning model. The problem is solved with the Alternating Direction Method of Multipliers (ADMM). Experimental results show that the proposed approach outperforms existing optimization-based and deep learning solutions. Full article

Based on Graph Balancing Theory, this study proposes an anomaly detection algorithm, the Supply Chain Proof of Relation (PoR), applied to enterprise procurement networks formalized as weighted directed graphs. A mathematical framework is constructed by integrating Laplacian flow conservation and the Sheaf topological coherence principle to identify anomalous nodes whose local characteristics deviate significantly from the global features of the supply network. PoR was empirically implemented on a dataset comprising 856 Taiwanese enterprises, successfully detecting 56 entities exhibiting abnormal behavior. Anomaly intensity was visualized through trend plots, revealing nodes with rapidly increasing deviations. To validate the effectiveness of this detection, the study further analyzed the correlation between internal and external performance metrics. The results demonstrate that anomalous nodes exhibit near-zero correlations, in contrast to the significant correlations observed in normal nodes—indicating a disruption of information consistency. This research establishes a graph-theoretic framework for anomaly detection, presents a mathematical model independent of training data, and highlights the linkage between structural deviations and informational distortions. By incorporating Sheaf Theory, the study enhances the analytical depth of topological consistency. Moreover, this work demonstrates the observability of flow conservation violations within a highly complex, non-physical system such as the supply chain. It completes a logical integration of Sheaf Coherence, Graph Balancing, and High-Dimensional Anomaly Projection, and achieves a cross-mapping between Graph Structural Deviations and Statistical Inconsistencies in weighted directed graphs. This contribution advances the field of graph topology-based statistical anomaly detection, opening new avenues for the methodological integration between physical systems and economic networks. Full article

The steel strip is an important and ideal material for the automotive and aerospace industries due to its superior machinability, cost efficiency, and flexibility. However, surface defects such as inclusions, spots, and scratches can significantly impact product performance and durability. Accurately identifying these defects remains challenging due to the complex texture structures and subtle variations in the material. In order to tackle this challenge, we propose a Differentiable Edge-guided Pyramid Aggregation Network (DEPANet) to utilize edge information for improving segmentation performance. DEPANet adopts an end-to-end encoder-decoder framework, where the encoder consisting of three key components: a backbone network, a Differentiable Edge Feature Pyramid network (DEFP), and Edge-aware Feature Aggregation Modules (EFAMs). The backbone network is designed to extract overall features from the strip steel surface, while the proposed DEFP utilizes learnable Laplacian operators to extract multiscale edge information of defects across scales. In addition, the proposed EFAMs aggregate the overall features generating from the backbone and the edge information obtained from DEFP using the Convolutional Block Attention Module (CBAM), which combines channel attention and spatial attention mechanisms, to enhance feature expression. Finally, through the decoder, implemented as a Feature Pyramid Network (FPN), the multiscale edge-enhanced features are progressively upsampled and fused to reconstruct high-resolution segmentation maps, enabling precise defect localization and robust handling of defects across various sizes and shapes. DEPANet demonstrates superior segmentation accuracy, edge preservation, and feature representation on the SD-saliency-900 dataset, outperforming other state-of-the-art methods and delivering more precise and reliable defect segmentation. Full article

High-definition (HD) maps for autonomous driving rely on data from mobile mapping systems (MMS), but the high cost of MMS sensors has led researchers to explore cheaper alternatives like low-cost LiDAR sensors. While cost effective, these sensors produce sparser point clouds, leading to poor feature representation and degraded performance in deep learning techniques, such as convolutional neural networks (CNN), for tasks like road marking extraction and classification, which are essential for HD map generation. Examining common image segmentation workflows and the structure of U-Net, a CNN, reveals a source of performance loss in the succession of resizing operations, which further diminishes the already poorly represented features. Addressing this, we propose improving U-Net’s ability to extract and classify road markings from sparse-point-cloud-derived images by introducing a learnable resizer (LR) at the input stage and learnable resizer blocks (LRBs) throughout the network, thereby mitigating feature and localization degradation from resizing operations in the deep learning framework. Additionally, we incorporate Laplacian filters (LFs) to better manage activations along feature boundaries. Our analysis demonstrates significant improvements, with F1-scores increasing from below 20% to above 75%, showing the effectiveness of our approach in improving road marking extraction and classification from sparse-point-cloud-derived imagery. Full article

Satellite multi-view stereo (MVS) is a fundamental task in large-scale Earth surface reconstruction. Recently, learning-based multi-view stereo methods have shown promising results in this field. However, these methods are mainly developed by transferring the general learning-based MVS framework to satellite imagery, which lacks consideration of the specific terrain features of the Earth’s surface and results in inadequate accuracy. In addition, mainstream learning-based methods mainly use equal height interval partition, which insufficiently utilizes the height hypothesis surface, resulting in inaccurate height estimation. To address these challenges, we propose an end-to-end terrain feature-aware height estimation network named SA-SatMVS for large-scale Earth surface multi-view stereo, which integrates information across different scales. Firstly, we transform the Sobel operator into slope feature-aware kernels to extract terrain features, and a dual encoder–decoder architecture with residual blocks is applied to incorporate slope information and geometric structural characteristics to guide the reconstruction process. Secondly, we introduce a pixel-wise unequal interval partition method using a Laplacian distribution based on the probability volume obtained from other scales, resulting in more accurate height hypotheses for height estimation. Thirdly, we apply an adaptive spatial feature extraction network to search for the optimal fusion method for feature maps at different scales. Extensive experiments on the WHU-TLC dataset also demonstrate that our proposed model achieves the best MAE metric of 1.875 and an RMSE metric of 3.785, which constitutes a state-of-the-art performance. Full article

This study aims to evaluate the repeatability of radiomics and dosiomics features via image perturbation of patients with cervical cancer. A total of 304 cervical cancer patients with planning CT images and dose maps were retrospectively included. Random translation, rotation, and contour randomization were applied to CT images and dose maps before radiomics feature extraction. The repeatability of radiomics and dosiomics features was assessed using intra-class correlation of coefficient (ICC). Pearson correlation coefficient (r) was adopted to quantify the correlation between the image characteristics and feature repeatability. In general, the repeatability of dosiomics features was lower compared with CT radiomics features, especially after small-sigma Laplacian-of-Gaussian (LoG) and wavelet filtering. More repeatable features (ICC > 0.9) were observed when extracted from the original, Large-sigma LoG filtered, and LLL-/LLH-wavelet filtered images. Positive correlations were found between image entropy and high-repeatable feature number in both CT and dose (r = 0.56, 0.68). Radiomics features showed higher repeatability compared to dosiomics features. These findings highlight the potential of radiomics features for robust quantitative imaging analysis in cervical cancer patients, while suggesting the need for further refinement of dosiomics approaches to enhance their repeatability. Full article

In the acquisition process of 3D cultural relics, it is common to encounter noise. To facilitate the generation of high-quality 3D models, we propose an approach based on graph signal processing that combines color and geometric features to denoise the point cloud. We divide the 3D point cloud into patches based on self-similarity theory and create an appropriate underlying graph with a Markov property. The features of the vertices in the graph are represented using 3D coordinates, normal vectors, and color. We formulate the point cloud denoising problem as a maximum a posteriori (MAP) estimation problem and use a graph Laplacian regularization (GLR) prior to identifying the most probable noise-free point cloud. In the denoising process, we moderately simplify the 3D point to reduce the running time of the denoising algorithm. The experimental results demonstrate that our proposed approach outperforms five competing methods in both subjective and objective assessments. It requires fewer iterations and exhibits strong robustness, effectively removing noise from the surface of cultural relic point clouds while preserving fine-scale 3D features such as texture and ornamentation. This results in more realistic 3D representations of cultural relics. Full article

With the rapid advancement of remote-sensing technology, the spectral information obtained from hyperspectral remote-sensing imagery has become increasingly rich, facilitating detailed spectral analysis of Earth’s surface objects. However, the abundance of spectral information presents certain challenges for data processing, such as the “curse of dimensionality” leading to the “Hughes phenomenon”, “strong correlation” due to high resolution, and “nonlinear characteristics” caused by varying surface reflectances. Consequently, dimensionality reduction of hyperspectral data emerges as a critical task. This paper begins by elucidating the principles and processes of hyperspectral image dimensionality reduction based on manifold theory and learning methods, in light of the nonlinear structures and features present in hyperspectral remote-sensing data, and formulates a dimensionality reduction process based on manifold learning. Subsequently, this study explores the capabilities of feature extraction and low-dimensional embedding for hyperspectral imagery using manifold learning approaches, including principal components analysis (PCA), multidimensional scaling (MDS), and linear discriminant analysis (LDA) for linear methods; and isometric mapping (Isomap), locally linear embedding (LLE), Laplacian eigenmaps (LE), Hessian locally linear embedding (HLLE), local tangent space alignment (LTSA), and maximum variance unfolding (MVU) for nonlinear methods, based on the Indian Pines hyperspectral dataset and Pavia University dataset. Furthermore, the paper investigates the optimal neighborhood computation time and overall algorithm runtime for feature extraction in hyperspectral imagery, varying by the choice of neighborhood k and intrinsic dimensionality d values across different manifold learning methods. Based on the outcomes of feature extraction, the study examines the classification experiments of various manifold learning methods, comparing and analyzing the variations in classification accuracy and Kappa coefficient with different selections of neighborhood k and intrinsic dimensionality d values. Building on this, the impact of selecting different bandwidths t for the Gaussian kernel in the LE method and different Lagrange multipliers λ for the MVU method on classification accuracy, given varying choices of neighborhood k and intrinsic dimensionality d, is explored. Through these experiments, the paper investigates the capability and effectiveness of different manifold learning methods in feature extraction and dimensionality reduction within hyperspectral imagery, as influenced by the selection of neighborhood k and intrinsic dimensionality d values, identifying the optimal neighborhood k and intrinsic dimensionality d value for each method. A comparison of classification accuracies reveals that the LTSA method yields superior classification results compared to other manifold learning approaches. The study demonstrates the advantages of manifold learning methods in processing hyperspectral image data, providing an experimental reference for subsequent research on hyperspectral image dimensionality reduction using manifold learning methods. Full article

To address the lightweight and real-time issues of coal sorting detection, an intelligent detection method for coal and gangue, Our-v8, was proposed based on improved YOLOv8. Images of coal and gangue with different densities under two diverse lighting environments were collected. Then the Laplacian image enhancement algorithm was proposed to improve the training data quality, sharpening contours and boosting feature extraction; the CBAM attention mechanism was introduced to prioritize crucial features, enhancing more accurate feature extraction ability; and the EIOU loss function was added to refine box regression, further improving detection accuracy. The experimental results showed that Our-v8 for detecting coal and gangue in a halogen lamp lighting environment achieved excellent performance with a mean average precision (mAP) of 99.5%, was lightweight with FLOPs of 29.7, Param of 12.8, and a size of only 22.1 MB. Additionally, Our-v8 can provide accurate location information for coal and gangue, making it ideal for real-time coal sorting applications. Full article

Pan-sharpening is a fusion process that combines a low-spatial resolution, multi-spectral image that has rich spectral characteristics with a high-spatial resolution panchromatic (PAN) image that lacks spectral characteristics. Most previous learning-based approaches rely on the scale-shift assumption, which may not be applicable in the full-resolution domain. To solve this issue, we regard pan-sharpening as a multi-task problem and propose a Siamese network with Gradient-based Spatial Attention (GSA-SiamNet). GSA-SiamNet consists of four modules: a two-stream feature extraction module, a feature fusion module, a gradient-based spatial attention (GSA) module, and a progressive up-sampling module. In the GSA module, we use Laplacian and Sobel operators to extract gradient information from PAN images. Spatial attention factors, learned from the gradient prior, are multiplied during the feature fusion, up-sampling, and reconstruction stages. These factors help to keep high-frequency information on the feature map as well as suppress redundant information. We also design a multi-resolution loss function that guides the training process under the constraints of both reduced- and full-resolution domains. The experimental results on WorldView-3 satellite images obtained in Moscow and San Juan demonstrate that our proposed GSA-SiamNet is superior to traditional and other deep learning-based methods. Full article

Enhancing the generalization capability of time-series models for streamflow prediction using dimensionality reduction (DR) techniques remains a major challenge in water resources management (WRM). In this study, we investigated eight DR techniques and their effectiveness in mitigating the curse of dimensionality, which hinders the performance of machine learning (ML) algorithms in the field of WRM. Our study delves into the most non-linear unsupervised representative DR techniques, including principal component analysis (PCA), kernel PCA (KPCA), multi-dimensional scaling (MDS), isometric mapping (ISOMAP), locally linear embedding (LLE), t-distributed stochastic neighbor embedding (t-SNE), Laplacian eigenmaps (LE), and autoencoder (AE), examining their effectiveness in multi-step ahead (MSA) streamflow prediction. In this study, we conducted a conceptual comparison of these techniques. Subsequently, we focused on their performance in four different case studies in the USA. Moreover, we assessed the quality of the transformed feature spaces in terms of the MSA streamflow prediction improvement. Through our investigation, we gained valuable insights into the performance of different DR techniques within linear/dense/convolutional neural network (CNN)/long short-term memory neural network (LSTM) and autoregressive LSTM (AR-LSTM) architectures. This study contributes to a deeper understanding of suitable feature extraction techniques for enhancing the capabilities of the LSTM model in tackling high-dimensional datasets in the realm of WRM. Full article

Graph clustering is a fundamental and challenging task in unsupervised learning. It has achieved great progress due to contrastive learning. However, we find that there are two problems that need to be addressed: (1) The augmentations in most graph contrastive clustering methods are manual, which can result in semantic drift. (2) Contrastive learning is usually implemented on the feature level, ignoring the structure level, which can lead to sub-optimal performance. In this work, we propose a method termed Graph Clustering with High-Order Contrastive Learning (GCHCL) to solve these problems. First, we construct two views by Laplacian smoothing raw features with different normalizations and design a structure alignment loss to force these two views to be mapped into the same space. Second, we build a contrastive similarity matrix with two structure-based similarity matrices and force it to align with an identity matrix. In this way, our designed contrastive learning encompasses a larger neighborhood, enabling our model to learn clustering-friendly embeddings without the need for an extra clustering module. In addition, our model can be trained on a large dataset. Extensive experiments on five datasets validate the effectiveness of our model. For example, compared to the second-best baselines on four small and medium datasets, our model achieved an average improvement of 3% in accuracy. For the largest dataset, our model achieved an accuracy score of 81.92%, whereas the compared baselines encountered out-of-memory issues. Full article

This study focuses on improving the objective alignment of image quality assessment (IQA) algorithms with human visual perception. Existing methodologies, predominantly those based on the Laplacian of Gaussian (LoG) filter, often neglect the impact of color channels on human visual perception. Consequently, we propose a full-reference IQA method that integrates multi-channel visual information in color images. The methodology begins with converting red, green, blue (RGB) images into the luminance (L), red–green opponent color channel (M), blue–yellow opponent color channel (N) or LMN color space. Subsequently, the LoG filter is separately applied to the L, M, and N channels. The convoluted components are then fused to generate a contrast similarity map using the root-mean-square method, while the chromaticity similarity map is derived from the color channels. Finally, multi-channel LoG filtering, contrast, and chromaticity image features are connected. The standard deviation method is then used for sum pooling to create a full-reference IQA computational method. To validate the proposed method, distorted images from four widely used image databases were tested. The evaluation, based on four criteria, focused on the method’s prediction accuracy, computational complexity, and generalizability. The Pearson linear correlation coefficient (PLCC) values, recorded from the databases, ranged from 0.8822 (TID2013) to 0.9754 (LIVE). Similarly, the Spearman rank-order correlation coefficient (SROCC) values spanned from 0.8606 (TID2013) to 0.9798 (LIVE). In comparison to existing methods, the proposed IQA method exhibited superior visual correlation prediction accuracy, indicating its promising potential in the field of image quality assessment. Full article