Search Results (391)

DC bias will cause the magnetization working point of the transformer core to shift and cause local saturation, and generate abnormal vibration through the magnetostrictive effect, which threatens the safe operation of the transformer. Aiming at the problem that the time–frequency characteristics of transformer vibration signals under DC bias are complex and the adjacent bias levels are difficult to distinguish, this paper proposes a 2DWT-CNN-Transformer diagnostic method that combines two-dimensional discrete wavelet transform, a convolutional neural network, and Transformer Encoder. Firstly, the multi-physical-field finite element model of three-phase three-column transformer is established, and the L0–L5 six-class DC bias dataset is constructed. Secondly, the one-dimensional vibration signal is reconstructed into a two-dimensional matrix, and the multi-subband time–frequency features of LL, LH, HL, and HH are extracted by two-dimensional discrete wavelet transform. The local texture features are extracted by the CNN, and the multi-head self-attention mechanism of Transformer Encoder is introduced to establish the global dependence and enhance the discrimination ability of adjacent bias levels. Compared with the traditional time–frequency-feature deep learning model, the proposed method achieves higher accuracy, especially in the high-noise environment of 15 dB, where it can still maintain accuracy of 96.23%. The visualization results further show that the model can form a more compact intra-class aggregation and a clearer inter-class boundary. This also provides an effective solution for the identification and evaluation of transformer DC bias states based on vibration signals in complex environments in the future. Full article

Oilfield production forecasting serves as the decision-making basis for monthly production allocation and injection–production system optimization. Existing mainstream prediction methods face significant limitations: the Arps decline curve extrapolates historical production trends, yet its accuracy degrades rapidly following adjustments to injection–production regimes; machine learning methods such as XGBoost can leverage extensive dynamic data but rely heavily on manual feature engineering and offer limited decision interpretability. This paper proposes the FTT-Transformer prediction model, whose core innovation is the Feature-Time Tokenizer (FTT). The FTT projects every scalar pair (time step, feature) in a multivariate time series matrix into a token of uniform dimensionality, superimposing three types of positional information—time embedding, feature embedding, and global position encoding. On this foundation, a multi-head self-attention mechanism performs end-to-end, full-capacity learning of nonlinear interactions across both the temporal and feature dimensions. The model is lightweight, requiring only 35,361 parameters for 13 input features and is readily deployable. Validation was conducted using production data from two independent waterflooding oilfields. On Dataset 2 (60 wells, 2012–2026), the model achieved an R² of 0.819, achieving performance on par with XGBoost (0.813; DM test p = 0.620, indicating no statistically significant difference) and substantially outperforming temporal Transformer baselines PatchTST (R² = 0.683) and iTransformer (R² = 0.786). On Dataset 1 (96 wells), it reached an R² of 0.930, statistically indistinguishable from XGBoost’s 0.943 (DM test p = 0.611). Five-fold temporal cross-validation yielded a mean R² of 0.840 ± 0.036, confirming the model’s stability. Ablation experiments revealed that global position encoding contributed most significantly, with its removal causing a 3.9 percentage point reduction in R² on Dataset 2. Composite feature-importance analysis showed that monthly liquid production and water cut are identified by the model as the two most predictive features, contributing 31.93% and 30.17% of the total importance, respectively. Multi-step forecasting results demonstrated that the model retains an R² of 0.640 for predictions two months ahead, spanning one complete decision cycle of monthly production reallocation. The proposed architecture is not domain-specific; by adapting the feature embeddings and time encoding, it could potentially be extended to diverse multivariate time series forecasting applications. However, cross-domain validation remains future work. Full article

Automatic speech recognition (ASR) systems based on Conformer architectures achieve strong performance by jointly modeling local acoustic patterns and global contextual dependencies. However, their interleaved convolution–attention design leads to progressive entanglement of fine-grained acoustic features and global semantic representations, which weakens monotonic alignment in speech recognition and degrades performance in long utterances. To address this limitation, we propose an AS-Split Conformer–Mamba framework that decouples local and global modeling into two explicit stages. First, a stage-wise encoder is introduced, where a dedicated local modeling stage extracts phonetic-level acoustic features using SE-enhanced convolution, followed by a global modeling stage that captures long-range dependencies via multi-head self-attention and temporal convolution. Second, a Transition Fusion Block (TFB) is designed as an adaptive transition module that transforms local acoustic representations before they enter the global modeling stage. Third, intermediate CTC supervision is introduced to explicitly strengthen monotonic alignment at shallow representations. Finally, a hybrid Transformer–Mamba decoder is adopted, in which the Mamba block provides O(N) state-space computation within the replaced FFN sublayer while retaining Transformer attention mechanisms for acoustic–text alignment. Experiments conducted on AISHELL-1, THCHS-30, and ST-CMDS demonstrate that the proposed method achieves consistent improvements over strong baselines. On AISHELL-1, our model reduces Character Error Rate (CER) from 5.7% to 4.8% and Sentence Error Rate (SER) from 24.8% to 20.5%, while maintaining competitive computational efficiency. Full article

Stock market price prediction remains a persistent challenge owing to the non-stationarity, high noise content, and intricate spatiotemporal dependencies that characterize financial time series. Existing approaches typically excel at either local pattern extraction or long-range dependency modeling, yet seldom reconcile both within a unified framework. This paper introduces a CNN–Transformer dual-channel architecture equipped with a dynamic attention fusion module for stock price forecasting. The convolutional channel applies hierarchical dilated convolutions to distill fine-grained local patterns from multi-indicator sequences while suppressing high-frequency noise. Simultaneously, the Transformer channel employs multi-head self-attention to capture long-distance temporal correlations and regime-shift dynamics. A learnable gating mechanism then fuses the two feature streams by adaptively weighting local detail against global trend information according to market conditions. Experiments conducted on four real-world stock datasets spanning the S&P 500, CSI 300, NASDAQ Composite, and Hang Seng Index show that the proposed model reduces mean absolute error by 9.7–15.3% and root mean square error by 9.5–13.8% relative to competitive baselines including LSTM, CNN–LSTM, Informer, and PatchTST. Ablation studies further indicate that both channels and the fusion module contribute to prediction accuracy, and the architecture remains effective across markets with differing volatility profiles. Full article

Reliable state of health (SOH) estimation provides important support for safety management and efficient operation of lithium-ion battery energy storage systems. To address the limitations of existing SOH estimation methods in terms of feature interpretability, joint modeling of local variations and overall morphological characteristics within each feature segment, and the rationality of model hyperparameter configuration, this study proposes a hybrid data-driven SOH estimation method integrating interpretable feature construction, an improved Transformer, and Bayesian optimization (BO). Specifically, raw charging data are first converted into incremental capacity (IC) curves based on incremental capacity analysis, and IC peaks are dynamically located in different cycles. Local voltage–capacity segments around the IC peak voltage are then extracted as interpretable input features closely related to battery aging. Subsequently, the Transformer encoder is improved by incorporating patch embedding and a multi-head differential self-attention mechanism, thereby enhancing the model’s ability to jointly capture local variations and overall morphological characteristics within each cycle-wise feature segment. BO is further employed to adaptively optimize key model hyperparameters. Experimental results on the CALCE-CS2 and CALCE-CX2 battery datasets show that the proposed BO-PatchDiffFormer model can provide accurate and stable SOH estimation in both comparative and generalization experimental scenarios. Compared with the best-performing baseline model, the maximum reductions in RMSE and MAE on the four CS2 test batteries reach 31.81% and 23.76%, respectively. In the generalization experiments, the average RMSE and MAE on the two CX2 test batteries are 1.6039% and 1.3081%, respectively. In addition, the maximum model size is only 2.07 MB, and the single-sample inference time remains within 0.84–1.18 ms, indicating good potential for practical deployment and application. Full article

Single-image snow removal remains a challenging, ill-posed inverse problem in computer vision due to the highly variable appearance of snow degradation. Existing CNN-based methods are limited by local receptive fields and cannot model globally distributed snow patterns, while Transformer-based methods achieve strong performance at a prohibitive computational cost. To address both limitations, GLA-DesnowNet is proposed as a lightweight hybrid architecture that introduces a novel Gated Transformer (GT) module, a gated global attention mechanism designed specifically for physics-driven image desnowing. The Gated Transformer applies global multi-head self-attention with a learned convolutional gate, enabling long-range spatial dependencies to be captured while remaining computationally efficient. An adaptive three-channel snow-mask generation strategy is introduced to automatically produce pixel-level supervision from paired images without manual annotation. A compound loss combining L1 reconstruction with multi-scale pyramid loss is employed to ensure consistent restoration across spatial scales. Evaluated on the Snow100K benchmark, the proposed method achieves 29.30 dB PSNR and 0.93 SSIM in only 160 training epochs, with 0.53 M parameters and 4.44 GFLOPs per inference—significantly fewer than existing state-of-the-art (SOTA) methods—while maintaining competitive restoration quality. On the Comprehensive Snow Dataset (CSD), the identical model achieves a 27.79 dB PSNR and 0.90 SSIM. These results confirm a strong efficiency–accuracy trade-off and cross-dataset generalization suited for resource-constrained and real-time deployment. Full article

Hyperspectral image classification is a core task in remote sensing image analysis and understanding. Existing Transformer-based methods have achieved excellent performance but are limited by the quadratic computational complexity of the self-attention mechanism, while the high-dimensional redundancy of hyperspectral data and the difficulty in deeply integrating spatial–spectral features also restrict further performance improvement. To address these issues, we introduce the Mamba architecture based on state-space models into hyperspectral image classification and propose the DFMamba model. The main innovations include (1) constructing a Hyperspectral Spatial Attention Embed (HSAE) to achieve efficient channel compression and feature extraction via adaptive grouped convolution, depth-wise separable convolution, and spatial attention; (2) proposing a spatial–spectral dual-branch collaborative modeling mechanism, EnhancedBothMamba, which separately models global dependencies in the spatial and spectral branches and integrates their outputs through softmax-normalized learnable global weights together with a learnable residual scaling factor; and (3) building an improved classification head, ClsHead, with a multi-scale branch fusion strategy to fully exploit local and global feature information. The experimental results on four standard hyperspectral datasets demonstrate that DFMamba achieves overall accuracy (OA) of 97.41% on the Pavia University dataset, 92.25% on the HanChuan dataset, 95.12% on the HongHu dataset, and 94.98% on the Houston dataset. Under the adopted evaluation protocol, DFMamba obtains higher mean OA than MambaHSI and the other compared methods while retaining favorable computational efficiency. Full article

Accurate short-term electrical load forecasting is of paramount importance for economic dispatch, reliable grid operation, and efficient demand-side management. However, hybrid forecasting frameworks constructed with deep learning models exhibit strong sensitivity to hyperparameter settings. Moreover, swarm-intelligence optimization algorithms are prone to premature convergence when tuning the hyperparameters of forecasting models, thereby degrading prediction performance. In addition, complex load sequences contain local fluctuations and key temporal segments that are difficult to capture using a single recurrent architecture. To address these challenges, this paper proposes a short-term electrical load forecasting method based on a BiGRU-Attention network optimized by an improved multi-strategy black-winged kite algorithm (IMBKA). The BiGRU extracts bidirectional temporal dependencies from historical load windows, while the attention module assigns adaptive weights to informative time steps and suppresses redundant historical information. To improve hyperparameter optimization, IMBKA introduces Sobol sequence initialization and adaptive elite differential mutation. Sobol sequence initialization enhances population coverage, and adaptive elite differential mutation strengthens information exchange among high-quality individuals. Experimental results on electrical load datasets from Singapore, Australia, and Belgium show that IMBKA-BiGRU-Attention achieves favorable forecasting performance among the compared models. The proposed model obtains RMSE values of 70.07 MW, 159.49 MW, 231.82 MW, and 163.43 MW in the Singapore, Australian, Belgian weekday, and Belgian weekend experiments, respectively. Compared with the best-performing model among the evaluated baselines in each experiment, the RMSE is reduced by 4.65%, 16.48%, 3.34%, and 11.39%, respectively. Full article

Fault signals in rotating machinery typically manifest as long time-series data embedded with local high-frequency impulses. Traditional deep learning methods often struggle to simultaneously capture these transient local impacts and model long-term global degradation features. To address this challenge, this paper proposes a novel GCU-SAM enhanced Transformer for intelligent fault diagnosis. The proposed network integrates a Gated Convolutional Unit (GCU) with a Self-Attention Mechanism (SAM). By introducing the GCU as a local inductive bias prior to the global attention module, the model dynamically captures and purifies local impulse responses via reset and update gates. Subsequently, a cascaded multi-head self-attention mechanism models long-sequence global evolution trends, forming an integrated framework for local fine-grained perception and global correlation modeling. Validated on the CWRU bearing and SEU gearbox datasets, the proposed architecture achieves superior diagnostic performance with an average F1-score of 99.01%. Compared to representative baselines, including 1D-CNN, BiLSTM, and the standard Transformer, the GCU-SAM significantly boosts diagnostic accuracy and effectively overcomes the early-stage optimization oscillations inherent in pure attention mechanisms. By achieving a deep multi-scale fusion of local abrupt changes and global degradation trends, the model exhibits exceptional feature-clustering discriminative capability and convergence stability, providing a robust and highly accurate solution for the intelligent fault diagnosis of complex rotating machinery. Full article

In large-scale cattle farming, accurate behavior recognition is central to achieving intensive health monitoring and animal welfare assessment. To address challenges such as background interference from fences, feed troughs, and stains in real-world barns, and overlapping of cattle coupled with the inability of single-RGB modalities to capture physical spatial structure, which leads to issues like blurred detection boundaries and significant noise interference—we propose a cattle behavior detection method based on RGB-Depth dual-modal information fusion. This approach jointly models the texture information from RGB images and the spatial structural information from depth images. Within this framework, this paper constructs three collaborative optimization modules: first, the CDSAM module is developed, which evaluates neuron importance through a parameter-free attention mechanism and combines dynamic convolutions to adapt to the cattle’s variable postures, effectively suppressing complex background noise. Second, we propose the C2BRA module based on a two-layer routed attention mechanism. By adopting a two-stage modeling approach of “region-level routing—intra-region fine-grained attention,” it adapts to changes in target scale and enhances the model’s ability to represent spatial context for multi-scale semantic information. Finally, in the prediction stage, a lightweight shared convolutional detection head (LSCD) is introduced. By sharing convolutional parameters across scales and decoupling the classification and regression architectures, it reduces computational overhead while maintaining accuracy. Experimental results show that the improved model achieves a mAP@0.5 of 90.3% on our self-built cattle behavior dataset, representing a 4.3 percentage point increase compared to the baseline model, while reducing GFLOPs from 11.0 G to 9.6 G, a decrease of 12.7%; Visualization results indicate that the improved model can focus more accurately on cattle body contours and key behavioral regions, thereby reducing false negatives and enhancing detection accuracy. Concurrently, the model achieves an optimal balance between detection performance and computational complexity, providing robust technical support for automated cattle behavior monitoring on smart farms. Full article

Accurate wind power forecasting is imperative for ensuring grid stability and facilitating the large-scale integration of renewable energy—both central pillars of the global energy transition and the Dual Carbon strategic goals. However, existing methods often fail to fully capture the spatial heterogeneity and interdependencies among individual turbines, limiting their effectiveness for sustainable grid operation. To address this gap, this paper proposes an ultra-short-term wind power forecasting framework that incorporates explicit multi-dimensional spatial features. At the feature level, a 12-dimensional spatial feature system is constructed to quantify the microscale topology of wind farms. These static spatial attributes are seamlessly fused with dynamic temporal data using a dimensionality-balance factor strategy. Finally, a hybrid deep learning network comprising a multi-scale CNN, a multi-layer BiLSTM, and a multi-head self-attention mechanism is developed to capture complex spatiotemporal patterns. Experimental results on three real-world datasets show that the proposed method significantly outperforms baseline models, reducing the Mean Absolute Percentage Error by up to 11.09% and improving the coefficient of determination R² up to 0.9120. By improving forecast accuracy and robustness, the method directly supports more reliable grid dispatching, reduces curtailment of wind energy, and thus contributes to the sustainable utilization of renewable resources. These findings demonstrate that incorporating explicit spatial correlation effectively enhances the accuracy and robustness of ultra-short-term wind power forecasting, providing robust decision support for power grid dispatching and advancing the sustainability of modern power systems. Full article

This paper proposes a SOC-SOH joint estimation method based on adaptive weighted multi-channel LSTM Transformer fusion network (MLTA-Net). The proposed method constructs a battery health factor set which covers multi-level features, and the aging trend of batteries can be characterized from multiple dimensions. The MLTA-Net model adopts a multi-channel parallel architecture, which can analyze the different types of battery data characteristics. Short-term temporal dependencies are captured by LSTM encoder, and global operating characteristics are analyzed using Transformer multi head self-attention mechanism. Based on adaptive weighted fusion layer for feature fusion, high-precision estimation of battery state can be achieved. Experimental results on CATL-1 and CATL-2 datasets show that the proposed MLTA-Net achieves superior SOH estimation accuracy, with RMSE values of 0.286 and 0.287, MAE values of 0.151 and 0.162, MAPE values of 0.053 and 0.056, and R² values of 0.997 and 0.997, respectively. Compared with CNN-GRU, MLP-Attention, Transformer, MLP, RNN, and SVR models, the proposed method exhibits lower prediction errors and better robustness. Full article

The rapid advancement of enterprise and cloud networks has significantly increased the complexity and scale of cyber threats, making conventional signature-based Intrusion Detection Systems (IDSs) insufficient. This paper presents the concept of an Intelligent Hybrid-Inference Network Intrusion Detection System (IHI-NIDS) powered by AI for real-time threat detection in high-velocity enterprise and cloud environments. IHI-NIDS combines Gated Recurrent Units (GRUs) for temporal sequence modelling with a multi-head self-attention mechanism, accounting for both short-range packet dynamics and long-range dependencies that signal multi-stage attacks. The system supports hybrid datasets that combine enterprise packet flows (PCAP/NetFlow) and cloud telemetry (VPC Flow Logs). It uses strong scaling and dimensionality reduction via PCA as the main approach to guarantee computational efficiency. Assessment of a composite dataset of 1.2 million flows indicates a detection rate of 97.8, recall of 96.1, F1-score of 96.5, and false alarm rate of 2.1, validating the capability to detect both enterprise-level and cloud-based threats. SHAP-based explainability highlights key features that affect predictions, which security analysts can use. Based on latency measurements, it can be deployed in real time to high-throughput networks, whereas temporal generalisation tests demonstrate its ability to withstand unobserved attack sequences. This article makes IHI-NIDS a scalable, interpretable, high-performance solution for current hybrid network security, filling the gap between operational viability and predictive AI-based defence. Full article

With the rapid development of social media and online interactive platforms, online public opinion has become a vital information source for public emotional expression, social risk perception, and decision support. However, public opinion texts are typically characterized by short length, obscure semantics, complex emotional expressions, and strong context dependence, making it difficult for traditional lexicon-based or shallow neural network methods to achieve stable and robust performance in sentiment discrimination tasks. To address these issues, this paper proposes BERT-BiLSTM-MHSA-Capsule (BBMC), hereafter referred to as BBMC, an online public opinion sentiment analysis model based on multi-level semantic feature fusion. The model first utilizes the pretrained language model BERT to extract dynamic semantic representations with context-aware capabilities; subsequently, a Bidirectional Long Short-Term Memory (BiLSTM) network is employed to model the bidirectional temporal dependencies within the texts, while a Multi-Head Self-Attention (MHSA) mechanism is introduced to achieve adaptive focusing on key emotional information. Building upon this, a three-layer cascaded capsule network is constructed to achieve structured modeling of high-order emotional attributes through vector neurons and dynamic routing mechanisms, effectively mitigating the loss of spatial feature information caused by traditional pooling and fully connected structures. Experimental results on a manually annotated online public opinion dataset show that BBMC achieves better performance than the evaluated baseline models in terms of accuracy, recall, and F1-score. These results indicate the empirical effectiveness of the proposed task-oriented feature-integration strategy and capsule-based classification head for online public opinion sentiment analysis. Full article

Rapid and accurate identification of various faults occurring in transmission lines is essential for restoring normal line operation. However, existing transmission line fault diagnosis methods still face challenges in terms of noise immunity and diagnostic accuracy. To address these issues, this paper proposes a deep learning method based on recurrence plots and a convolutional neural network–bidirectional gated recurrent unit–attention mechanism model. The voltage and current signals of transmission lines are transformed into recurrence plots in both the time and frequency domains. Parallel convolutional neural networks are then employed to extract local features from the two domains, while bidirectional gated recurrent units are used to capture temporal dependencies. Furthermore, multi-head self-attention and cross-attention mechanisms are introduced to enhance key features within each domain and achieve adaptive fusion of inter-domain feature information. A transmission line model is established in Simulink to collect data under various fault conditions and influencing factors, thereby verifying the effectiveness and adaptability of the proposed method. Experimental results show that the proposed method achieves fault recognition accuracies of 99.63%, 96.68%, and 75.38% under NL1, NL2, and NL3 Gaussian-noise conditions, respectively, and maintains accuracies of 99.02%, 95.93%, and 72.43% under mixed-noise conditions. Compared with other deep learning models, the proposed method demonstrates higher diagnostic accuracy and stronger robustness. Full article