Search Results (1,090)

With the escalating global cyber threats, Distributed Denial of Service (DDoS) attacks have become one of the most disruptive and prevalent network attacks. Traditional DDoS detection systems face significant challenges due to the unpredictable nature, diverse protocols, and coupled behavioral patterns of attack traffic. To address this issue, this paper proposes a novel approach for DDoS attack detection by leveraging the Transformer architecture to model both temporal dependencies and behavioral patterns, significantly improving detection accuracy. We utilize the global attention mechanism of the Transformer to effectively capture long-range temporal correlations in network traffic, and the model’s ability to process multiple traffic features simultaneously enables it to identify nonlinear interactions. By reconstructing the CIC-DDoS2019 dataset, we strengthen the representation of attack behaviors, enabling the model to capture dynamic attack patterns and subtle traffic anomalies. This approach represents a key contribution by applying Transformer-based self-attention mechanisms to accurately model DDoS attack traffic, particularly in handling complex and dynamic attack patterns. Experimental results demonstrate that the proposed method achieves 99.9% accuracy, with 100% precision, recall, and F1 score, showcasing its potential for high-precision, low-false-alarm automated DDoS attack detection. This study provides a new solution for real-time DDoS detection and holds significant practical implications for cybersecurity systems. Full article

Accurate inversion of the T₂ spectrum of shale oil reservoir fluids is crucial for reservoir evaluation. However, traditional nuclear magnetic resonance inversion methods face challenges in extracting features from multi-exponential decay signals. This study proposed an inversion method that combines autoencoder (AE) and Fourier transform, aiming to enhance the accuracy and stability of T₂ spectrum estimation for shale oil reservoirs. The autoencoder is employed to automatically extract deep features from the echo train, while the Fourier transform is used to enhance frequency domain features of multi-exponential decay information. Furthermore, this paper designs a customized weighted loss function based on a self-attention mechanism to focus the model’s learning capability on peak regions, thereby mitigating the negative impact of zero-value regions on model training. Experimental results demonstrate significant improvements in inversion accuracy, noise resistance, and computational efficiency compared to traditional inversion methods. This research provides an efficient and reliable new approach for precise evaluation of the T₂ spectrum in shale oil reservoirs. Full article

This paper presents a comprehensive study on automated defect detection in complex structures using phased array ultrasonic testing data, focusing on both traditional signal processing and advanced deep learning methods. As a non-AI baseline, the well-known signal-to-noise ratio algorithm was improved by introducing automatic depth gate calculation using derivative analysis and eliminated the need for manual parameter tuning. Even though this method demonstrates robust flaw indication, it faces difficulties for automatic defect detection in highly noisy data or in cases with large pore zones. Considering this, multiple DL architectures—including fully connected networks, convolutional neural networks, and a novel Convolutional Attention Temporal Transformer for Sequences—are developed and trained on diverse datasets comprising simulated CIVA data and real-world data files from welded and composite specimens. Experimental results show that while the FCN architecture is limited in its ability to model dependencies, the CNN achieves a strong performance with a test accuracy of 94.9%, effectively capturing local features from PAUT signals. The CATT-S model, which integrates a convolutional feature extractor with a self-attention mechanism, consistently outperforms the other baselines by effectively modeling both fine-grained signal morphology and long-range inter-beam dependencies. Achieving a remarkable accuracy of 99.4% and a strong F1-score of 0.905 on experimental data, this integrated approach demonstrates significant practical potential for improving the reliability and efficiency of NDT in complex, heterogeneous materials. Full article

Landslide displacement prediction is vital for geohazard early warning and infrastructure safety. To address the challenges of modeling nonstationary, nonlinear, and multiscale behaviors inherent in GNSS time series, this study proposes a hybrid predicting framework that integrates Past Decomposable Mixing with an inverted Transformer architecture (PDM-iTransformer). The PDM module decomposes the original sequence into multi-resolution trend and seasonal components, using structured bottom-up and top-down mixing strategies to enhance feature representation. The iTransformer then models each variable’s time series independently, applying cross-variable self-attention to capture latent dependencies and using feed-forward networks to extract local dynamic features. This design enables simultaneous modeling of long-term trends and short-term fluctuations. Experimental results on GNSS monitoring data demonstrate that the proposed method significantly outperforms traditional models, with R² increased by 16.2–48.3% and RMSE and MAE reduced by up to 1.33 mm and 1.08 mm, respectively. These findings validate the framework’s effectiveness and robustness in predicting landslide displacement under complex terrain conditions. Full article

With the rapid development of social media, short-text data have become increasingly important in fields such as public opinion monitoring, user feedback analysis, and intelligent recommendation systems. However, existing short-text sentiment analysis models often suffer from limited cross-domain adaptability and poor generalization performance. To address these challenges, this study proposes a novel short-text sentiment classification model based on the Bidirectional Encoder Representations from Transformers (BERTs) and a dual-stream Transformer gated attention mechanism. This model first employs Bidirectional Encoder Representations from Transformers (BERTs) and the Chinese Robustly Optimized BERT Pretraining Approach (Chinese-RoBERTa) to achieve data augmentation and multilevel semantic mining, thereby expanding the training corpus and enhancing minority class coverage. Second, a dual-stream Transformer gated attention mechanism was developed to dynamically adjust feature fusion weights, enhancing adaptability to heterogeneous texts. Finally, the model integrates a Bidirectional Gated Recurrent Unit (BiGRU) with Multi-Head Self-Attention (MHSA) to strengthen sequence information modeling and global context capture, enabling the precise identification of key sentiment dependencies. The model’s superior performance in handling data imbalance and complex textual sentiment logic scenarios is demonstrated by the experimental results, achieving significant improvements in accuracy and F1 score. The F1 score reached 92.4%, representing an average increase of 8.7% over the baseline models. This provides an effective solution for enhancing the performance and expanding the application scenarios of short-text sentiment analysis models. Full article

The preservation of bridge infrastructure has become increasingly critical as aging assets face accelerated deterioration due to climate change, environmental loading, and operational stressors. This issue is particularly pronounced in regions with limited maintenance budgets, where delayed interventions compound structural vulnerabilities. Although traditional bridge inspections generate detailed condition ratings, these are often viewed as isolated snapshots rather than part of a continuous structural health timeline, limiting their predictive value. To overcome this, recent studies have employed various Artificial Intelligence (AI) models. However, these models are often restricted by fixed input sizes and specific report formats, making them less adaptable to the variability of real-world data. Thus, this study introduces a Transformer architecture inspired by Natural Language Processing (NLP), treating condition ratings, and other features as tokens within temporally ordered inspection “sentences” spanning 1993–2024. Due to the self-attention mechanism, the model effectively captures long-range dependencies in patterns, enhancing forecasting accuracy. Empirical results demonstrate 96.88% accuracy for short-term prediction and 86.97% across seven years, surpassing the performance of comparable time-series models such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRUs). Ultimately, this approach enables a data-driven paradigm for structural health monitoring, enabling bridges to “speak” through inspection data and empowering engineers to “listen” with enhanced precision. Full article

We review the evolution of the concept of utility in economics, addressing the conceptual and terminological fragmentation that characterises the interdisciplinary debate. This study adopts the scoping review framework to systematically analyse the main theoretical approaches, ranging from utility as preference to utility as subjective satisfaction and well-being. Particular attention is paid to procedural utility, i.e., the utility derived from the way decisions are made and interactions develop, divided into three areas: individual, linked to autonomy and self-determination; interpersonal, related to the quality of social relations; and institutional, referring to participation and recognition. The analysis is based on three aspects: (i) how different theoretical traditions have interpreted utility and well-being; (ii) what convergences and divergences emerge in the contemporary literature; (iii) and what implications these factors have for research and public policy. We highlight the complementarity between approaches and suggest extending economic reflection to dimensions that are central to the well-being of individuals and societies. The insights of this study have public policy implications, indicating that, through well-defined institutions, distributive justice, and welfare systems, taxpayers’ hedonic utility can be transformed into the chrematistic utility of beneficiaries. Full article

Climate change is transforming energy use at the residential level by increasing temperature fluctuations and sustaining extreme weather events. This study proposes a climate-reactive, multi-objective approach to integrate the demand response (DR) with distributed generation (DG) and power quality improvement under a multi-objective framework of an integrated climate-adaptive approach to residential energy management. A cognitive neural network combination model with bidirectional long short-term memory networks (bidirectional) and a self-attention mechanism was used to successfully predict temperature-sensitive loads. The hybrid deep learning solution, which applies convolutional and bidirectional long short-term memory (LSTM) networks with attention, predicted the temperature-dependent load profiles optimized with an enhanced modified grey wolf optimizer (MGWO). The results of the experimental studies indicated significant gains in performance: in energy expenditure, the studies reduced it by 32.7%; in peak demand, they were able to reduce it by 45.2%; and in self-generated renewable energy, the results were 28.9% higher. The solution reliability rate provided by the MGWO was 94.5%, and it converged more quickly, thus providing better diversity in the Pareto-optimal frontier than that of traditional metaheuristic algorithms. Sensitivity tests with climate conditions of +2 °C and +4 °C showed strategy changes as high as 18.3%, thus establishing the flexibility of the system. Empirical evidence indicates that the energy and peak demand are to be cut, renewable integration is enhanced, and performance is strong in fluctuating climate conditions, highlighting the adaptability of the system to future resilient smart homes. Full article

The Denoising Wavelet Transform Convolutional Long Short-Term Memory Network (DWTConvLSTM) is a novel ionospheric total electron content (TEC) spatiotemporal prediction model proposed in 2025 that can simultaneously consider high-frequency and low-frequency features while suppressing noise. However, it also has flaws as it only considers unidirectional temporal features in spatiotemporal prediction. To address this issue, this paper adopts a bidirectional structure and designs a bidirectional DWTConvLSTM model that can simultaneously extract bidirectional spatiotemporal features from TEC maps. Furthermore, we integrate a lightweight attention mechanism called Convolutional Additive Self-Attention (CASA) to enhance important features and attenuate unimportant ones. The final model was named CASA-BiDWTConvLSTM. We validated the effectiveness of each improvement through ablation experiments. Then, a comprehensive comparison was performed on the 11-year Global Ionospheric Maps (GIMs) dataset, involving the proposed CASA-BiDWTConvLSTM model and several other state-of-the-art models such as C1PG, ConvGRU, ConvLSTM, and PredRNN. In this experiment, the dataset was partitioned into 7 years for training, 2 years for validation, and the final 2 years for testing. The experimental results indicate that the RMSE of CASA-BiDWTConvLSTM is lower than those of C1PG, ConvGRU, ConvLSTM, and PredRNN. Specifically, the decreases in RMSE during high solar activity years are 24.84%, 16.57%, 13.50%, and 10.29%, respectively, while the decreases during low solar activity years are 26.11%, 16.83%, 11.68%, and 7.04%, respectively. In addition, this article also verified the effectiveness of CASA-BiDWTConvLSTM from spatial and temporal perspectives, as well as on four geomagnetic storms. Full article

Inshore ship detection in synthetic aperture radar (SAR) imagery poses significant challenges due to the high density and diversity of ships. However, low inter-object backscatter contrast and blurred boundaries of docked ships often result in performance degradation for traditional object detection methods, especially under complex backgrounds and low signal-to-noise ratio (SNR) conditions. To address these issues, this paper proposes a novel detection framework, the Dynamic Weighted Joint Time–Frequency Feature Fusion DEtection TRansformer (DETR) Model (DWTF-DETR), specifically designed for SAR-based ship detection in inshore areas. The proposed model integrates a Dual-Domain Feature Fusion Module (DDFM) to extract and fuse features from both SAR images and their frequency-domain representations, enhancing sensitivity to both high- and low-frequency target features. Subsequently, a Dual-Path Attention Fusion Module (DPAFM) is introduced to dynamically weight and fuse shallow detail features with deep semantic representations. By leveraging an attention mechanism, the module adaptively adjusts the importance of different feature paths, thereby enhancing the model’s ability to perceive targets with ambiguous structural characteristics. Experiments conducted on a self-constructed inshore SAR ship detection dataset and the public HRSID dataset demonstrate that DWTF-DETR achieves superior performance compared to the baseline RT-DETR. Specifically, the proposed method improves mAP@50 by 1.60% and 0.72%, and F1-score by 0.58% and 1.40%, respectively. Moreover, comparative experiments show that the proposed approach outperforms several state-of-the-art SAR ship detection methods. The results confirm that DWTF-DETR is capable of achieving accurate and robust detection in diverse and complex maritime environments. Full article

Accurate load forecasting for community-level heating, ventilation, and air conditioning (HVAC) plays an important role in determining an efficient strategy for demand response (DR) and the operation of the power grid. However, community-level HVAC includes various building-level HVACs, whose usage patterns and standard parameters vary, causing the challenge of load forecasting. To this end, a novel deep learning model, multiscale and cross-variable transformer (MSCVFormer), is proposed to achieve accurate community-level HVAC probabilistic load forecasting by capturing the various influences of multivariables on the load pattern, providing effective information for the grid operators to develop DR and operation strategies. This approach is combined with the multiscale attention (MSA) and cross-variable attention (CVA) mechanism, capturing the complex temporal patterns of the aggregated load. Specifically, by embedding the time series decomposition into the self-attention mechanism, MSA enables the model to capture the critical features of time series while considering the correlation between multiscale time series. Then, CVA calculates the correlations between the exogenous variable and aggregated load, explicitly utilizing the exogenous variables to enhance the model’s understanding of the temporal pattern. This differs from the usual methods, which do not fully consider the relationship between the exogenous variable and aggregated load. To test the effectiveness of the proposed method, two datasets from Germany and China are used to conduct the experiment. Compared to the benchmarks, the proposed method achieves outperforming probabilistic load forecasting results, where the prediction interval coverage probability (PICP) deviation with the nominal coverage and prediction interval normalized averaged width (PINAW) are reduced by 46.7% and 5.25%, respectively. Full article

Financial markets exhibit fundamental asymmetries in temporal causality, where policy interventions create asymmetric transmission patterns that traditional symmetric modeling approaches fail to capture. This work introduces a mathematical framework that exploits the inherent symmetries of transformer architectures while preserving essential asymmetric temporal relationships in financial causal inference. We develop CausalFormer, a symmetry-aware neural architecture that maintains the permutation equivariance properties of self-attention mechanisms while enforcing strict temporal asymmetry constraints for causal discovery. The framework incorporates three mathematically principled components: (1) a symmetric attention matrix construction with asymmetric temporal masking that preserves the mathematical elegance of transformer operations while ensuring causal consistency, (2) a multi-scale convolution module with symmetric kernel initialization but asymmetric temporal receptive fields that captures policy transmission effects across heterogeneous time horizons, and (3) enhanced Nelson–Siegel decomposition that maintains the symmetric factor structure while modeling the evolution dynamics of asymmetric factors. Our mathematical formulation establishes the formal symmetry properties of the attention mechanism under temporal transformations while proving asymmetric convergence behaviors in policy transmission scenarios. The integration of symmetric optimization landscapes with asymmetric causal constraints enables simultaneous achievement of mathematical elegance and economic interpretability. Comprehensive experiments on monetary policy datasets demonstrate that the symmetry-aware design achieves a 15.3% improvement in the accuracy of causal effect estimations and a 12.7% enhancement in the predictive performance compared to those for existing methods while maintaining 91.2% causal consistency scores. The framework successfully identifies asymmetric policy transmission mechanisms, revealing that monetary tightening exhibits 40% faster propagation than easing policies, establishing new mathematical insights into the temporal asymmetries in financial systems. This work demonstrates how principled exploitation of architectural symmetries combined with domain-specific asymmetric constraints opens up new directions for mathematically rigorous yet economically interpretable deep learning in financial econometrics, with broad applications spanning computational finance, economic forecasting, and policy analysis. Full article

The reliable operation of photovoltaic (PV) systems is often compromised by surface soiling and structural damage, which reduce energy efficiency and complicate large-scale monitoring. To address this challenge, we propose a two-tiered image-classification framework that combines Vision Transformer (ViT) models, lightweight convolutional neural networks (CNNs), and knowledge distillation (KD). In Tier 1, a DINOv2 ViT-Base model is fine-tuned to provide robust high-level categorization of solar-panel images into three classes: Normal, Soiled, and Damaged. In Tier 2, two enhanced EfficientNetB0 models are introduced: (i) a KD-based student model distilled from a DINOv2 ViT-S/14 teacher, which improves accuracy from 96.7% to 98.67% for damage classification and from 90.7% to 92.38% for soiling classification, and (ii) an EfficientNetB0 augmented with Multi-Head Self-Attention (MHSA), which achieves 98.73% accuracy for damage and 93.33% accuracy for soiling. These results demonstrate that integrating transformer-based representations with compact CNN architectures yields a scalable and efficient solution for automated monitoring of the condition of PV systems, offering high accuracy and real-time applicability in inspections on solar farms. Full article

Non-Intrusive Load Monitoring (NILM) aims to disaggregate a household’s total aggregated power consumption into appliance-level usage, enabling intelligent energy management without the need for intrusive metering. While deep learning has improved NILM significantly, existing NILM models struggle to capture load patterns across both longer time intervals and subtle timings for appliances involving brief or overlapping usage patterns. In this paper, we propose a novel GRU+BERT hybrid architecture, exploring both unidirectional (GRU+BERT) and bidirectional (Bi-GRU+BERT) variants. Our model combines Gated Recurrent Units (GRUs) to capture sequential temporal dependencies with Bidirectional Encoder Representations from Transformers (BERT), which is a transformer-based model that captures rich contextual information across the sequence. The bidirectional variant (Bi-GRU+BERT) processes input sequences in both forward (past to future) and backward (future to past) directions, enabling the model to learn relationships between power consumption values at different time steps more effectively. The unidirectional variant (GRU+BERT) provides an alternative suited for appliances with structured, sequential multi-phase usage patterns, such as dishwashers. By placing the Bi-GRU or GRU layer before BERT, our models first capture local time-based load patterns and then use BERT’s self-attention to understand the broader contextual relationships. This design addresses key limitations of both standalone recurrent and transformer-based models, offering improved performance on transient and irregular appliance loads. Evaluated on the UK-DALE and REDD datasets, the proposed Bi-GRU+BERT and GRU+BERT models show competitive performance compared to several state-of-the-art NILM models while maintaining a comparable model size and training time, demonstrating their practical applicability for real-time energy disaggregation, including potential edge and cloud deployment scenarios. Full article

To address the limitations of the traditional methods that are used to extract features from non-stationary signals and capture temporal dependency relationships, a rolling bearing fault diagnosis method combining variational mode decomposition (VMD) and deep learning is proposed. A hybrid VMD-CNN-Transformer model is constructed, where VMD is used to adaptively decompose bearing vibration signals into multiple intrinsic mode functions (IMFs). The convolutional neural network (CNN) captures the local features of each modal time series, while the multi-head self-attention mechanism of the Transformer captures the global dependencies of each mode, enabling the global analysis and fusion of features from each mode. Finally, a fully connected layer is used to classify the 10 fault types. The experimental results on the Case Western Reserve University bearing dataset demonstrate that the model achieves a fault diagnosis accuracy of 99.48%, which is significantly higher than that of single or traditional combined methods, providing a new technical path for the intelligent diagnosis of rolling bearing faults. Full article