Search Results (22)

Hyperspectral image (HSI) classification is central to environmental monitoring, yet real-time deployment of deep learning models on resource-constrained edge platforms remains challenging due to high spectral dimensionality and computational overhead. In this paper, we propose TinyCapsViT, an ultra-lightweight hybrid architecture that integrates convolutional feature extraction, transformer-based attention, and capsule-inspired representation learning for efficient HSI classification under strict TinyML constraints. The model employs a minimal convolutional stem using pointwise and depthwise separable convolutions to capture local spectral-spatial features, followed by a compact tokenization strategy and learnable positional embeddings. A lightweight self-attention module enables global context modeling with reduced computational complexity, while a capsule-inspired refinement block with squash nonlinearity and residual scaling enhances feature discrimination. The proposed architecture contains only 2781 trainable parameters, representing up to a 30× reduction compared with larger architectures such as ResNet and Vision Transformer, and approximately 11× and 9.5× fewer parameters than the CNN and 3D-CNN baselines, respectively. Extensive experiments across benchmark hyperspectral datasets demonstrate that TinyCapsViT achieves up to 99.61% overall accuracy and maintains competitive classification performance despite its substantially reduced model complexity. Although TinyCapsViT does not consistently achieve the highest classification accuracy compared with larger baseline models, it provides a favourable trade-off between classification performance and computational efficiency. These results demonstrate the potential of TinyCapsViT as a practical solution for real-time hyperspectral analysis on resource-constrained edge platforms and UAV-based environmental monitoring applications. Full article

Accurate classification and efficient retrieval of histopathological images are essential for the diagnosis of lung adenocarcinoma (LUAD). Existing deep learning approaches for Content-Based Histopathological Image Retrieval (CBHIR) typically generate single-scale embeddings, missing the richer spatial context from earlier network stages. We propose a unified framework composed of: (1) a ConvNeXt V2 backbone with an integrated Convolutional Block Attention Module (CBAM) for multi-scale feature extraction, (2) an Adaptive Weighted Fusion Neck with learnable softmax-normalized weights, and (3) a novel Hybrid Representation Head producing an 18,496-dimensional descriptor by concatenating global, spatial, and attention-weighted features. Evaluated on the WSSS4LUAD dataset (10,087 patches, four tissue classes), our model achieves 85.03% accuracy (5-fold CV: 83.35 ± 0.93%), F1-score of 0.8116, mean Average Precision (MAP) of 0.8323 for retrieval, and an Expected Calibration Error (ECE) of 0.0378. Ablation experiments confirm that all proposed modules contribute positively, with the Attention Branch being the most impactful (Δ = −2.13%). The framework further provides Gradient-weighted Class Activation Mapping (Grad-CAM) explainability for clinical interpretability. Full article

Inverse kinematics (IK) is fundamental to robot motion planning. Classical analytical solvers require complete Denavit–Hartenberg (DH) parameters that are often proprietary or degraded by mechanical wear, and numerical solvers based on damped least squares (DLS) are sensitive to initialization, particularly near singularities. We propose XGNN, an explainable graph neural network positioned as a model-free, interpretable warm-start initializer for downstream numerical IK refinement rather than as a standalone replacement for analytical solvers. Each IK query is encoded as a 12-node graph in which six pose nodes and six joint nodes are connected through bipartite pose-to-joint attention edges and chain edges along the kinematic structure. GATv2 message passing aggregates information at each joint node; two ablation-validated design contributions (a learnable node-type embedding and an angle-aware composite loss) enable training to convergence. Evaluated on 300,000 trajectory-style samples generated from the ABB IRB 2400 kinematic model, XGNN achieves ${3.66}^{\circ }$ joint mean absolute error (MAE), comparable to a multilayer perceptron baseline (${3.09}^{\circ }$) and a bidirectional LSTM (${3.14}^{\circ }$) under identical training. The standalone joint accuracy of all learned models is too coarse for direct industrial use, but XGNN provides the strongest warm start for DLS refinement: the convergence rate improves from $98.4\%$ to $100\%$, mean iterations drop from 14.6 to 3.2, and wall-clock time per pose drops $5.0\times$ on the IRB 2400. The benefit transfers cross-platform to the Universal Robots UR5 collaborative manipulator (convergence rate $82.2\%$ to $100\%$, $10.0\times$ speedup) and survives DH parameter perturbation of up to $\pm 10\%$, simulating calibration drift or mechanical wear. The GATv2 attention coefficients additionally provide an interpretability signal at zero inference cost. XGNN therefore complements analytical and numerical IK methods as an interpretable, calibration-robust warm start when DH parameters are unavailable, proprietary, or degraded. Full article

Robust defect classification is critical for intelligent process inspection and quality control in steel manufacturing, but it remains challenging when industrial tabular data are small, imbalanced, statistically skewed, and characterized by nonlinear inter-feature dependencies. This study proposes a robust steel plate defect classification framework based on a feature-embedded Transformer. A quantile-based transformation is first introduced to regularize heterogeneous and heavy-tailed process descriptors. Each numerical variable is then represented as a learnable feature token and processed by a Transformer encoder to model contextual interactions among positional, geometric, luminosity-related, and morphological attributes. Experiments were conducted on the Steel Plates Faults dataset, containing 1941 samples, 27 input features, and 7 defect categories. On the held-out test set, the model achieved an accuracy of 0.735, remaining competitive with XGBoost (0.794) and Random Forest (0.783). SHAP and self-attention analyses further indicate that the model captures distributed and interaction-aware defect representations, providing an interpretable solution for robust industrial defect classification. Full article

Accurate identification of karst caves from seismic data is crucial for carbonate reservoir characterization, as these caves often serve as primary hydrocarbon storage spaces and migration pathways. However, it remains challenging due to the highly nonlinear relationship between seismic waveforms and cave geometries, as well as the noise propagation in skip connections inherent to U-Net-based methods. To address these limitations, this paper proposes UKAN-CBAM, a novel 3D network that synergistically integrates Tokenized Kolmogorov–Arnold Network (Tok-KAN) modules and Convolutional Block Attention Modules (CBAM) within a U-shaped encoder–decoder architecture. Unlike U-Net, which relies on linear convolutional kernels, the Tok-KAN modules employ learnable spline-based activation functions to better capture the nonlinear relationships between seismic waveforms and cave geometries. Furthermore, CBAM embedded in each skip connection adaptively recalibrates features along the channel and spatial dimensions, thereby suppressing noise and sharpening cave boundaries. Trained on synthetic data and validated on physical modeling data from the Sichuan Basin and field data from the Tarim Basin, UKAN-CBAM consistently outperforms U-Net, ResUNet, UNet-CBAM, and coherence attributes across multiple evaluation metrics. The proposed network delineates caves with improved continuity and sharper boundaries while reducing false positives, demonstrating strong generalization capability. The results indicate that the synergistic design of KAN’s nonlinear modeling and CBAM’s attention mechanism effectively mitigates the limitations of traditional approaches for karst cave identification. Full article

Long-term time series forecasting (LTSF) is critical for modern power systems, energy management, and grid planning. Yet virtually all existing forecasting models employ stationary activation functions that apply identical nonlinear mappings regardless of temporal context—a fundamental mismatch with real-world load data, which exhibits strongly regime-dependent dynamics such as summer demand peaks, winter heating patterns, and overnight low-load periods. We address this gap by proposing TC-KAN (Time-Conditioned Kolmogorov–Arnold Network), the first forecasting architecture to augment KAN activation functions with position-aware coefficient parameterisation. The core innovation replaces the static polynomial coefficients in standard KAN activations with position-conditioned coefficients produced by a lightweight positional-embedding MLP, providing additional learnable capacity beyond standard KAN while adding negligible parameter overhead. TC-KAN further integrates a dual-pathway processing block—combining depthwise convolution for local temporal pattern extraction with the time-conditioned KAN layer for enhanced nonlinear transformation—within a channel-independent framework with Reversible Instance Normalisation. Experiments were conducted on four standard ETT benchmark datasets and the high-dimensional Weather dataset. TC-KAN achieves superior or competitive accuracy in most configurations while requiring merely 51K parameters—approximately 40% of DLinear and ∼100× fewer than iTransformer. On ETTh2, TC-KAN reduces the mean squared error by up to 61.4% over DLinear, and matches the current state-of-the-art iTransformer on ETTm2 at a fraction of the computational cost. This extreme parameter reduction circumvents the steep memory bottlenecks endemic to massive Transformer models, positioning TC-KAN as a highly practical architecture tailored precisely for resource-constrained edge deployments—such as on-device load forecasting inside smart grid sensors and industrial IoT controllers. Full article

Object detection in UAV-based fire rescue scenarios faces multiple challenges, including densely distributed small targets, severe occlusion, and interference from smoke and flames. Existing mainstream detection models, such as the YOLO series, often prioritize inference speed at the expense of modeling global context and spatial positional information, resulting in limited performance in such complex environments. To address these limitations, this paper proposes FirePM-YOLO, an object detection architecture optimized for fire rescue applications. Based on the YOLO framework, the proposed model introduces two key innovations: first, a Position-Aware Enhanced Mamba module (PEMamba) is designed, which incorporates a compact positional encoding mechanism, lightweight spatial enhancement, and an adaptive feature fusion strategy to significantly improve scene perception while maintaining computational efficiency. Second, a PEMBottleneck structure is constructed, which dynamically balances local convolutional features and global PEMamba features via learnable weights. This module is embedded into the shallow layers of the backbone network, forming an enhanced PEM-C3K2 module that captures long-range dependencies with linear complexity while preserving fine local details, thereby enabling holistic contextual understanding of fireground environments. Experimental results on the self-built “FireRescue” dataset demonstrate that compared with the original YOLOv12 and other mainstream detectors, the proposed model achieves improvements in both mean average precision (mAP) and recall while maintaining real-time inference capability. Notably, it exhibits superior detection performance on challenging samples, such as small-scale and partially occluded professional firefighting vehicles. Full article

Breast ultrasound imaging functions as a vital radiation-free detection tool for breast cancer, yet its low contrast, speckle noise, and interclass variability make automated interpretation difficult. In this paper, we introduce UltraScanNet as a specific deep learning backbone that addresses breast ultrasound classification needs. The proposed architecture combines a convolutional stem with learnable 2D positional embeddings, followed by a hybrid stage that unites MobileViT blocks with spatial gating and convolutional residuals and two progressively global stages that use a depth-aware composition of three components: (1) UltraScanUnit (a state-space module with selective scan gated convolutional residuals and low-rank projections), (2) ConvAttnMixers for spatial channel mixing, and (3) multi-head self-attention blocks for global reasoning. This research includes a detailed ablation study to evaluate the individual impact of each architectural component. The results demonstrate that UltraScanNet reaches 91.67% top-1 accuracy, a precision score of 0.9072, a recall score of 0.9174, and an F1-score of 0.9096 on the BUSI dataset, which make it a very competitive option among multiple state-of-the-art models, including ViT-Small (91.67%), MaxViT-Tiny (91.67%), MambaVision (91.02%), Swin-Tiny (90.38%), ConvNeXt-Tiny (89.74%), and ResNet-50 (85.90%). On top of this, the paper provides an extensive global and per-class analysis of the performance of these models, offering a comprehensive benchmark for future work. The code will be publicly available. Full article

We propose a Transformer-based data augmentation framework with a time-series dual-stream architecture to address performance degradation in encrypted network traffic classification caused by class imbalance between attack and benign traffic. The proposed framework independently processes the complete flow’s sequential packet information and statistical characteristics by extracting and normalizing a local channel (comprising packet size, inter-arrival time, and direction) and a set of six global flow-level statistical features. These are used to generate a fixed-length multivariate sequence and an auxiliary vector. The sequence and vector are then fed into an encoder-only Transformer that integrates learnable positional embeddings with a FiLM + context token-based injection mechanism, enabling complementary representation of sequential patterns and global statistical distributions. Large-scale experiments demonstrate that the proposed method reduces reconstruction RMSE and additional feature restoration MSE by over 50%, while improving accuracy, F1-Score, and AUC by 5–7%p compared to classification on the original imbalanced datasets. Furthermore, the augmentation process achieves practical levels of processing time and memory overhead. These results show that the proposed approach effectively mitigates class imbalance in encrypted traffic classification and offers a promising pathway to achieving more robust model generalization in real-world deployment scenarios. Full article

The knowledge tracing method based on deep learning is used to assess learners’ cognitive states, laying the foundation for personalized education. However, deep learning methods are inefficient when processing long-term series data and are prone to overfitting. To improve the accuracy of cognitive state prediction, we design a Multi-layer Attention Self-supervised Knowledge Tracing Method (MASKT) using self-supervised learning and the Transformer method. In the pre-training stage, MASKT uses a random forest method to filter out positive and negative correlation feature embeddings; then, it reuses noise-processed restoration tasks to extract more learnable features and enhance the learning ability of the model. The Transformer in MASKT not only solves the problem of long-term dependencies between input and output using an attention mechanism, but also has parallel computing capabilities that can effectively improve the learning efficiency of the prediction model. Finally, a multidimensional attention mechanism is integrated into cross-attention to further optimize prediction performance. The experimental results show that, compared with various knowledge tracing models on multiple datasets, MASKT’s prediction performance remains 2 percentage points higher. Compared with the multidimensional attention mechanism of graph neural networks, MASKT’s time efficiency is shortened by nearly 30%. Due to the improvement in prediction accuracy and performance, this method has broad application prospects in the field of cognitive diagnosis in intelligent education. Full article

Specific emitter identification (SEI) is a core technology for wireless device security that plays a crucial role in protecting wireless communication systems from various security threats. However, current deep learning-based SEI methods heavily rely on large amounts of labeled data for supervised training, facing challenges in non-cooperative communication scenarios. To address these issues, this paper proposes a novel contrastive asymmetric masked learning-based SEI (CAML-SEI) method, effectively solving the problem of SEI under scarce labeled samples. The proposed method constructs an asymmetric auto-encoder architecture, comprising an encoder network based on channel squeeze-and-excitation residual blocks to capture radio frequency fingerprint (RFF) features embedded in signals, while employing a lightweight single-layer convolutional decoder for masked signal reconstruction. This design promotes the learning of fine-grained local feature representations. To further enhance feature discriminability, a learnable non-linear mapping is introduced to compress high-dimensional encoded features into a compact low-dimensional space, accompanied by a contrastive loss function that simultaneously achieves feature aggregation of positive samples and feature separation of negative samples. Finally, the network is jointly optimized by combining signal reconstruction and feature contrast tasks. Experiments conducted on real-world ADS-B and Wi-Fi datasets demonstrate that the proposed method effectively learns generalized RFF features, and the results show superior performance compared with other SEI methods. Full article

To address the challenges of high missed detection rates for minute transmission line defects, strong complex background interference, and limited computational power on edge devices in UAV-assisted power line inspection, this paper proposes a lightweight improved YOLOv12 real-time detection model. First, a Bidirectional Weighted Feature Fusion Network (BiFPN) is introduced to enhance bidirectional interaction between shallow localization information and deep semantic features through learnable feature layer weighting, thereby improving detection sensitivity for line defects. Second, a Cross-stage Channel-Position Collaborative Attention (CPCA) module is embedded in the BiFPN’s cross-stage connections, jointly modeling channel feature significance and spatial contextual relationships to effectively suppress complex background noise from vegetation occlusion and metal reflections while enhancing defect feature representation. Furthermore, the backbone network is reconstructed using ShuffleNetV2’s channel rearrangement and grouped convolution strategies to reduce model complexity. Experimental results demonstrate that the improved model achieved 98.7% mAP@0.5 on our custom transmission line defect dataset, representing a 3.0% improvement over the baseline YOLOv12, with parameters compressed to 2.31M (8.3% reduction) and real-time detection speed reaching 142.7 FPS. This method effectively balances detection accuracy and inference efficiency, providing reliable technical support for unmanned intelligent inspection of transmission lines. Full article

The style change detection (SCD) task asks to find the positions of authors’ style changes within multi-authored texts. It has several application areas, such as forensics, cybercrime, and literary analysis. Since 2017, SCD solutions in English have been actively investigated. However, to the best of our knowledge, this task has not yet been investigated in Arabic text. Moreover, most existing SCD solutions represent boundaries surrounding segments by concatenating them. This shallow concatenation may lose style patterns within each segment and also increase input lengths while several embedding models restrict these lengths. This study seeks to bridge these gaps by introducing an Edge Convolutional Neural Network for the Arabic SCD task (ECNN-ASCD) solution. It represents boundaries as standalone learnable parameters across layers based on graph neural networks. ECNN-ASCD was trained on an Arabic dataset containing three classes of instances according to difficulty level: easy, medium, and hard. The results show that ECNN-ASCD achieved a high $F_1$ score of 0.9945%, 0.9381%, and 0.9120% on easy, medium, and hard instances, respectively. The ablation experiments demonstrated the effectiveness of ECNN-ASCD components. As the first publicly available solution for Arabic SCD, ECNN-ASCD would open the door for more active research on solving this task and contribute to boosting research in Arabic NLP. Full article

The aim of sequential recommendation (SR) is to predict a user’s next interaction by analyzing their historical behavioral sequences. The proposed framework leverages the inherent symmetry in user behavior patterns, where temporal and spectral representations exhibit complementary structures that can be harmonized for robust recommendation. Conventional SR methods predominantly utilize implicit feedback (e.g., clicks and views) as model inputs, whereby observed interactions are treated as positive instances, while unobserved ones are considered negative samples. However, the inherent randomness and diversity in user behaviors inevitably introduce noise into such implicit feedback, potentially compromising the accuracy of recommendations. Recent studies have explored noise mitigation through two primary approaches: temporal-domain methods that reweight interactions to distill clean samples for comprehensive user preference modeling, and frequency-domain techniques that purify item embeddings to reduce the propagation of noise. While temporal approaches excel in sample refinement, frequency-based methods demonstrate superior capability in learning noise-resistant representations through spectral analysis. Motivated by the desire to synergize these complementary advantages, we propose SR-DFN, a novel framework that systematically addresses noise interference through coordinated time–frequency processing. Self-guided sample purification is implemented in the temporal domain of our architecture via adaptive interaction weighting, effectively distilling behaviorally significant patterns. The refined sequence is then transformed into the frequency domain, where learnable spectral filters operate to further attenuate residual noise components while preserving essential preference signals. Drawing on the convolution theorem’s revelation regarding frequency-domain operations capturing behavioral periodicity, we critically examine conventional position encoding schemes and propose an efficient parameterization strategy that eliminates redundant positional embeddings without compromising temporal awareness. Comprehensive experiments conducted on four real-world benchmark datasets demonstrate SR-DFN’s superior performance over state-of-the-art baselines, with ablation studies validating the effectiveness of our dual-domain denoising mechanism. Our findings suggest that coordinated time–frequency processing offers a principled solution for noise-resilient sequential recommendation while challenging conventional assumptions about positional encoding requirements in spectral-based approaches. The symmetry principles underlying our dual-domain approach demonstrate how the balanced processing of temporal and frequency domains can achieve superior noise resilience. Full article

Seismic data are easily contaminated by random noise, impairing subsequent geological interpretation tasks. Existing denoising methods like low-rank approximation (LRA) and deep learning (DL) show promising denoising capabilities but still have limitations; for instance, LRA performance is parameter-sensitive, and DL networks lack interpretation. As an alternative, this paper introduces the low-rank tensor network (LRTNet), an innovative approach that integrates low-rank tensor approximation (LRTA) with DL. Our method involves constructing a noise attenuation model that leverages LRTA, total variation (TV) regularization, and weighted tensor nuclear norm minimization (WTNNM). By applying the alternating direction method of multipliers (ADMM), we solve the model and transform the iterative schemes into a DL framework, where each iteration corresponds to a network layer. The key learnable parameters, including weights and thresholds, are optimized using labeled data to enhance performance. Quantitative evaluations on synthetic data reveal that LRTNet achieves an average signal-to-noise ratio (SNR) of 9.37 dB on the validation set, outperforming Pyseistr (6.46 dB) and TNN-SSTV (6.10 dB) by 45.0% and 53.6%, respectively. Furthermore, tests on real field datasets demonstrate consistent enhancements in noise suppression while preserving critical stratigraphic structures and fault discontinuities. The embedded LRTA mechanism not only improves network interpretability, but also reduces parameter sensitivity compared to conventional LRA methods. These findings position LRTNet as a robust, physics-aware solution for seismic data restoration. Full article