Search Results (300)

Multivariate Time Series Forecasting (MTSF) has been innovated with a series of models in the last two decades, ranging from traditional statistical approaches to RNN-based models. However, recent contributions from deep learning to time series problems have made huge progress with a series of Transformer-based models. Despite the breakthroughs by attention mechanisms applied to deep learning areas, many challenges remain to be solved with more sophisticated models. Existing Transformers known as attention-based models outperform classical models with abilities to capture temporal dependencies and better strategies for learning dependencies among variables as well as in the time domain in an efficient manner. Aiming to solve those issues, we propose a novel Transformer—SegmentedCrossformer (SCF), a Transformer-based model that considers both time and dependencies among variables in an efficient manner. The model is built upon the encoder–decoder architecture in different scales and compared with the previous state of the art. Experimental results on different datasets show the effectiveness of SCF with unique advantages and efficiency. Full article

The Solar Agro Savior (SAS) is an innovative solution that is assisted by drones for the sustainable utilization of water and plant disease observation in the agriculture sector. This system integrates an alerting mechanism for humidity, moisture, and temperature variations, which affect the plants’ health and optimization in water utilization, which enhances plant yield productivity. A significant feature of the system is the efficient monitoring system in a larger region through drones’ high-resolution cameras, which enables real-time, efficient response and alerting for environmental fluctuations to the authorities. The machine learning algorithm, particularly recurrent neural networks, which is a pioneer with agriculture and pest control, is incorporated for intelligent monitoring systems. The proposed system incorporates a specialized form of a recurrent neural network, Long Short-Term Memory (LSTM), which effectively addresses the vanishing gradient problem. It also utilizes an attention-based mechanism that enables the model to assign meaningful weights to the most important parts of the data sequence. This algorithm not only enhances water utilization efficiency but also boosts plant yield and strengthens pest control mechanisms. This system also provides sustainability through the re-utilization of water and the elimination of electric energy through solar panel systems for powering the inbuilt irrigation system. A comparative analysis of variant algorithms in the agriculture sector with a machine learning approach was also illustrated, and the proposed system yielded 99% yield accuracy, a 97.8% precision value, 98.4% recall, and a 98.4% F1 score value. By encompassing solar irrigation and artificial intelligence-driven analysis, the proposed algorithm, Solar Argo Savior, established a sustainable framework in the latest agricultural sectors and promoted sustainability to protect our environment and community. Full article

Background and Objective: Temperature management is key for reliable operation of permanent magnet synchronous motors (PMSMs). The lumped-parameter thermal network (LPTN) is fast and interpretable but struggles with nonlinear behavior under high power density. We propose OLTEM, a physics-informed deep model that combines LPTN with a thermal neural network (TNN) to improve prediction accuracy while keeping physical meaning. Methods: OLTEM embeds LPTN into a recurrent state-space formulation and learns three parameter sets: thermal conductance, inverse thermal capacitance, and power loss. Two additions are introduced: (i) a state-conditioned squeeze-and-excitation (SC-SE) attention that adapts feature weights using the current temperature state, and (ii) an enhanced power-loss sub-network that uses a deep MLP with SC-SE and non-negativity constraints. The model is trained and evaluated on the public Electric Motor Temperature dataset (Paderborn University/Kaggle). Performance is measured by mean squared error (MSE) and maximum absolute error across permanent-magnet, stator-yoke, stator-tooth, and stator-winding temperatures. Results: OLTEM tracks fast thermal transients and yields lower MSE than both the baseline TNN and a CNN–RNN model for all four components. On a held-out generalization set, MSE remains below 4.0 °C² and the maximum absolute error is about 4.3–8.2 °C. Ablation shows that removing either SC-SE or the enhanced power-loss module degrades accuracy, confirming their complementary roles. Conclusions: By combining physics with learned attention and loss modeling, OLTEM improves PMSM temperature prediction while preserving interpretability. This approach can support motor thermal design and control; future work will study transfer to other machines and further reduce short-term errors during abrupt operating changes. Full article

Inter-turn short-circuit (ITSC) faults in permanent magnet synchronous machines (PMSMs) present a significant reliability challenge in electric vehicle (EV) drivetrains, particularly under non-stationary operating conditions characterized by inverter-driven transients, variable loads, and magnetic saturation. Existing diagnostic approaches, including motor current signature analysis (MCSA) and wavelet-based methods, are primarily designed for steady-state conditions and rely on manual feature selection, limiting their applicability in real-time embedded systems. Furthermore, the lack of publicly available, high-fidelity datasets capturing the transient dynamics and nonlinear flux-linkage behaviors of PMSMs under fault conditions poses an additional barrier to developing data-driven diagnostic solutions. To address these challenges, this study introduces a simulation framework that generates a comprehensive dataset using finite element method (FEM) models, incorporating magnetic saturation effects and inverter-driven transients across diverse EV operating scenarios. Time-frequency features extracted via Discrete Wavelet Transform (DWT) from stator current signals are used to train a Transformer model for automated ITSC fault detection. The Transformer model, leveraging self-attention mechanisms, captures both local transient patterns and long-range dependencies within the time-frequency feature space. This architecture operates without sequential processing, in contrast to recurrent models such as LSTM or RNN models, enabling efficient inference with a relatively low parameter count, which is advantageous for embedded applications. The proposed model achieves 97% validation accuracy on simulated data, demonstrating its potential for real-time PMSM fault detection. Additionally, the provided dataset and methodology contribute to the facilitation of reproducible research in ITSC diagnostics under realistic EV operating conditions. Full article

With the increasing scale of industrial devices and the growing complexity of multi-source heterogeneous sensor data, traditional methods struggle to address challenges in fault detection, data security, and trustworthiness. Ensuring tamper-proof data storage and improving prediction accuracy for imbalanced anomaly detection for potential deployment in the Industrial Internet of Things (IIoT) remain critical issues. This study proposes a blockchain-powered Long Short-Term Memory Network (LSTM)–Attention hybrid model: an LSTM-based Encoder–Attention–Decoder (LEAD) for industrial device anomaly detection. The model utilizes an encoder–attention–decoder architecture for processing multivariate time series data generated by industrial sensors and smart contracts for automated on-chain data verification and tampering alerts. Experiments on real-world datasets demonstrate that the LEAD achieves an $F_{0.1}$ score of 0.96, outperforming baseline models (Recurrent Neural Network (RNN): 0.90; LSTM: 0.94; and Bi-directional LSTM (Bi-LSTM, 0.94)). We simulate the system using a private FISCO-BCOS network with a multi-node setup to demonstrate contract execution, anomaly data upload, and tamper alert triggering. The blockchain system successfully detects unauthorized access and data tampering, offering a scalable solution for device monitoring. Full article

With the growing global demand for clean energy, the accuracy of wind-power forecasting plays a vital role in ensuring the stable operation of power systems. However, wind-power generation is significantly influenced by meteorological conditions and is characterized by high uncertainty and multiscale fluctuations. Traditional recurrent neural network (RNN) and long short-term memory (LSTM) models, although capable of handling sequential data, struggle with modeling long-term temporal dependencies due to the vanishing gradient problem; thus, they are now rarely used. Recently, Transformer models have made notable progress in sequence modeling compared to RNNs and LSTM models. Nevertheless, when dealing with long wind-power sequences, their quadratic computational complexity (O(L²)) leads to low efficiency, and their global attention mechanism often fails to capture local periodic features accurately, tending to overemphasize redundant information while overlooking key temporal patterns. To address these challenges, this paper proposes a wind-power forecasting method based on dimension-transformed collaborative multidimensional multiscale attention (DTCMMA). This method first employs fast Fourier transform (FFT) to automatically identify the main periodic components in wind-power data, reconstructing the one-dimensional time series as a two-dimensional spatiotemporal representation, thereby explicitly encoding periodic features. Based on this, a collaborative multidimensional multiscale attention (CMMA) mechanism is designed, which hierarchically integrates channel, spatial, and pixel attention to adaptively capture complex spatiotemporal dependencies. Considering the geometric characteristics of the reconstructed data, asymmetric convolution kernels are adopted to enhance feature extraction efficiency. Experiments on multiple wind-farm datasets and energy-related datasets demonstrate that DTCMMA outperforms mainstream methods such as Transformer, iTransformer, and TimeMixer in long-sequence forecasting tasks, achieving improvements in MSE performance by 34.22%, 2.57%, and 0.51%, respectively. The model’s training speed also surpasses that of the fastest baseline by 300%, significantly improving both prediction accuracy and computational efficiency. This provides an efficient and accurate solution for wind-power forecasting and contributes to the further development and application of wind energy in the global energy mix. Full article

This study proposes an interpretable hybrid deep learning framework for classifying attention deficit hyperactivity disorder (ADHD) using EEG signals recorded during cognitively demanding tasks. The core architecture integrates convolutional neural networks (CNNs), gated recurrent units (GRUs), and long short-term memory (LSTM) layers to jointly capture spatial and temporal dynamics. In addition to the final hybrid architecture, the CNN–GRU–LSTM model alone demonstrates excellent accuracy (99.63%) with minimal variance, making it a strong baseline for clinical applications. To evaluate the role of global attention mechanisms, transformer encoder models with two and three attention blocks, along with a spatiotemporal transformer employing 2D positional encoding, are benchmarked. A hybrid CNN–RNN–transformer model is introduced, combining convolutional, recurrent, and transformer-based modules into a unified architecture. To enhance interpretability, SHapley Additive exPlanations (SHAP) are employed to identify key EEG channels contributing to classification outcomes. Experimental evaluation using stratified five-fold cross-validation demonstrates that the proposed hybrid model achieves superior performance, with average accuracy exceeding 99.98%, F1-scores above 0.9999, and near-perfect AUC and Matthews correlation coefficients. In contrast, transformer-only models, despite high training accuracy, exhibit reduced generalization. SHAP-based analysis confirms the hybrid model’s clinical relevance. This work advances the development of transparent and reliable EEG-based tools for pediatric ADHD screening. Full article

Accurate forecasting of river water flow helps to warn of floods and droughts in advance, provides a basis for the rational allocation of water resources, and at the same time, offers important support for the safe operation of hydropower stations and water conservancy projects. Water flow is characterized by time series, but the existing models focus on the positive series when LSTM is applied, without considering the different contributions of the water flow series to the model at different moments. In order to solve this problem, this study proposes a river water flow prediction model, named AT-BiLSTM, which mainly consists of a bidirectional layer and an attention layer. The bidirectional layer is able to better capture the long-distance dependencies in the sequential data by combining the forward and backward information processing capabilities. In addition, the attention layer focuses on key parts and ignores irrelevant information when processing water flow data series. The effectiveness of the proposed method was validated against an actual dataset from the Shizuishan monitoring station on the Yellow River in China. The results confirmed that compared with the RNN model, the proposed model significantly reduced the MAE, MSE, and RMSE on the dataset by 27.16%, 42.01%, and 23.85%, respectively, providing the best predictive performance among the six compared models. Moreover, this attention mechanism enables the model to show good performance in 72 h (3 days) forecast, keeping the average prediction error below 6%. This implies that the proposed hybrid model could provide a decision base for river flow flood control and resource allocation. Full article

Personality is a fundamental psychological trait that exerts a long-term influence on human behavior patterns and social interactions. Automatic personality recognition (APR) has exhibited increasing importance across various domains, including Human–Robot Interaction (HRI), personalized services, and psychological assessments. In this study, we propose a multimodal personality recognition model that classifies the Big Five personality traits by extracting features from three heterogeneous sources: audio processed using Wav2Vec2, video represented as Skeleton Landmark time series, and text encoded through Bidirectional Encoder Representations from Transformers (BERT) and Doc2Vec embeddings. Each modality is handled through an independent Self-Attention block that highlights salient temporal information, and these representations are then summarized and integrated using a late fusion approach to effectively reflect both the inter-modal complementarity and cross-modal interactions. Compared to traditional recurrent neural network (RNN)-based multimodal models and unimodal classifiers, the proposed model achieves an improvement of up to 12 percent in the F1-score. It also maintains a high prediction accuracy and robustness under limited input conditions. Furthermore, a visualization based on t-distributed Stochastic Neighbor Embedding (t-SNE) demonstrates clear distributional separation across the personality classes, enhancing the interpretability of the model and providing insights into the structural characteristics of its latent representations. To support real-time deployment, a lightweight thread-based processing architecture is implemented, ensuring computational efficiency. By leveraging deep learning-based feature extraction and the Self-Attention mechanism, we present a novel personality recognition framework that balances performance with interpretability. The proposed approach establishes a strong foundation for practical applications in HRI, counseling, education, and other interactive systems that require personalized adaptation. Full article

Blind source separation (BSS) of underwater acoustic mixed signals aims to improve signal clarity by separating noise components from aliased underwater signal sources. This enhancement directly increases target detection accuracy in underwater acoustic perception systems, particularly in scenarios involving multi-vessel interference or biological sound coexistence. Deep learning-based BSS methods have gained wide attention for their superior nonlinear modeling capabilities. However, existing approaches in underwater acoustic scenarios still face two key challenges: limited feature discrimination and inadequate robustness against non-stationary noise. To overcome these limitations, we propose a novel Robust Cross-Band Network (RCBNet) for the BSS of underwater acoustic mixed signals. To address insufficient feature discrimination, we decompose mixed signals into sub-bands aligned with ship noise harmonics. For intra-band modeling, we apply a parallel gating mechanism that strengthens long-range dependency learning so as to enhance robustness against non-stationary noise. For inter-band modeling, we design a bidirectional-frequency RNN to capture the global dependency relationships of the same signal across sub-bands. Our experiment demonstrates that RCBNet achieves a 0.779 dB improvement in the SDR compared to the advanced model. Additionally, the anti-noise experiment demonstrates that RCBNet exhibits satisfactory robustness across varying noise environments. Full article

Short-term precipitation forecasting is a core task in meteorological science, aiming to achieve accurate predictions by modeling the spatiotemporal evolution of radar echo sequences, thereby supporting meteorological services and disaster warning systems. However, existing spatiotemporal sequence prediction methods still struggle to disentangle complex spatiotemporal dependencies effectively and fail to capture the nonlinear chaotic characteristics of precipitation systems. This often results in ambiguous predictions, attenuation of echo intensity, and spatial localization errors. To address these challenges, this paper proposes a unified spatiotemporal sequence prediction framework based on spatiotemporal masking, which comprises two stages: self-supervised pre-training and task-oriented fine-tuning. During pre-training, the model learns global structural features of meteorological systems from sparse contexts by randomly masking local spatiotemporal regions of radar images. In the fine-tuning stage, considering the importance of the temporal dimension in short-term precipitation forecasting and the complex long-range dependencies in spatiotemporal evolution of precipitation systems, we design an RNN-based cyclic temporal mask self-encoder model (MAE-RNN) and a transformer-based spatiotemporal attention model (STMT). The former focuses on capturing short-term temporal dynamics, while the latter simultaneously models long-range dependencies across space and time via a self-attention mechanism, thereby avoiding the smoothing effects on high-frequency details that are typical of conventional convolutional or recurrent structures. The experimental results show that STMT improves 3.73% and 2.39% in CSI and HSS key indexes compared with the existing advanced models, and generates radar echo sequences that are closer to the real data in terms of air mass morphology evolution and reflection intensity grading. Full article

In this paper, we propose TransConvNet, a hybrid model combining Convolutional Neural Networks (CNNs), self-attention mechanisms, and transfer learning for wireless signal recognition under challenging conditions. The model effectively addresses challenges such as low signal-to-noise ratio (SNR), low sampling rates, and limited labeled data. The CNN module extracts local features and suppresses noise, while the self-attention mechanism within the Transformer encoder captures long-range dependencies in the signal. To enhance performance with limited data, we incorporate transfer learning by leveraging pre-trained models, ensuring faster convergence and improved generalization. Extensive experiments were conducted on a six-class wireless signal dataset, downsampled to 1 MSPS to simulate real-world constraints. The proposed TransConvNet achieved 92.1% accuracy, outperforming baseline models such as LSTM, CNN, and RNN across multiple evaluation metrics, including RMSE and $R^2$. The model demonstrated strong robustness under varying SNR conditions and exhibited superior discriminative ability, as confirmed by Precision–Recall and ROC curves. These results validate the effectiveness and robustness of the TransConvNet model for wireless signal recognition, particularly in resource-constrained and noisy environments. Full article

In the field of intelligent decision-making, time-series data collected by sensors serves as the core carrier for interaction between the physical and digital worlds. Accurate analysis is the cornerstone of decision-making in critical scenarios, such as industrial monitoring and intelligent transportation. However, the inherent spatio-temporal coupling characteristics and cross-period long-range dependency of sensor data cause traditional time-series prediction methods to face performance bottlenecks in feature decoupling and multi-scale modeling. This study innovatively proposes a Spatio-Temporal Attention-Enhanced Network (TSEBG). Breaking through traditional structural designs, the model employs a Squeeze-and-Excitation Network (SENet) to reconstruct the convolutional layers of the Temporal Convolutional Network (TCN), strengthening the feature expression of key time steps through dynamic channel weight allocation to address the redundancy issue of traditional causal convolutions in local pattern capture. A Bidirectional Gated Recurrent Unit (BiGRU) variant based on a global attention mechanism is designed, leveraging the collaboration between gating units and attention weights to mine cross-period long-distance dependencies and effectively alleviate the gradient disappearance problem of Recurrent Neural Network (RNN-like) models in multi-scale time-series analysis. A hierarchical feature fusion architecture is constructed to achieve multi-dimensional alignment of local spatial and global temporal features. Through residual connections and the dynamic adjustment of attention weights, hierarchical semantic representations are output. Experiments show that TSEBG outperforms current dominant models in time-series single-step prediction tasks in terms of accuracy and performance, with a cross-dataset R² standard deviation of only 3.7%, demonstrating excellent generalization stability. It provides a novel theoretical framework for feature decoupling and multi-scale modeling of complex time-series data. Full article

The ionospheric total electron content (TEC) has complex spatiotemporal variations, making its spatiotemporal prediction challenging. Capturing long-range spatial dependencies is of great significance for improving the spatiotemporal prediction accuracy of TEC. Existing work based on Convolutional Long Short-Term Memory (ConvLSTM) primarily relies on convolutional operations for spatial feature extraction, which are effective at capturing local spatial correlations, but struggle to model long-range dependencies, limiting their predictive performance. Self-Attention Convolutional Long Short-Term Memory (SA-ConvLSTM) can selectively store and focus on long-range spatial dependencies, but it requires the input length and output length to be the same due to its “n vs. n” structure, limiting its application. To solve this problem, this paper proposes an encoder-decoder SA-ConvLSTM, abbreviated as ED-SA-ConvLSTM. It can effectively capture long-range spatial dependencies using SA-ConvLSTM and achieve unequal input-output lengths through encoder–decoder structure. To verify its performance, the proposed ED-SA-ConvLSTM was compared with C1PG, ConvLSTM, and PredRNN from multiple perspectives in the area of 12.5° S–87.5° N, 25° E–180° E, including overall quantitative comparison, comparison across different months, comparison at different latitude regions, visual comparisons, and comparison under extreme situations. The results have shown that, in the vast majority of cases, the proposed ED-SA-ConvLSTM outperforms the comparative models. Full article

In recent years, advances in artificial intelligence, machine vision, and the Internet of Things have significantly impacted sports analytics, particularly basketball, where accurate measurement and analysis of player performance have become increasingly important. This study proposes a real-time goal state recognition system based on inertial measurement unit (IMU) sensors, focusing on four shooting scenarios: rebounds, swishes, other shots, and misses. By installing IMU sensors around the basketball net, the system captures real-time data on acceleration, angular velocity, and angular changes to comprehensively analyze the fluency and success rate of shooting execution, utilizing five deep learning models—convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory (LSTM), CNN-LSTM, and CNN-LSTM-Attention—to classify shot types. Experimental results indicate that the CNN-LSTM-Attention model outperformed other models with an accuracy of 87.79% in identifying goal states. This result represents a commanding level of real-time goal state recognition, demonstrating the robustness and efficiency of the model in complex sports environments. This high accuracy not only supports the application of the system in skill analysis and sports performance evaluation but also lays a solid foundation for the development of intelligent basketball training equipment, providing an efficient and practical solution for athletes and coaches. Full article