Sign in to use this feature.

Years

Between: -

Subjects

remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline
remove_circle_outline

Journals

Article Types

Countries / Regions

Search Results (3)

Search Parameters:
Keywords = SWiGLU

Order results
Result details
Results per page
Select all
Export citation of selected articles as:
26 pages, 4789 KB  
Article
EMAT: Enhanced Multi-Aspect Attention Transformer for Financial Time Series Forecasting
by Yingjun Chen, Wenfeng Shen, Han Liu and Xiaolin Cao
Entropy 2025, 27(10), 1029; https://doi.org/10.3390/e27101029 - 1 Oct 2025
Viewed by 504
Abstract
Financial time series prediction remains a challenging task due to the inherent non-stationarity, noise, and complex temporal dependencies present in market data. Traditional forecasting methods often fail to capture the multifaceted nature of financial markets, where temporal proximity, trend dynamics, and volatility patterns [...] Read more.
Financial time series prediction remains a challenging task due to the inherent non-stationarity, noise, and complex temporal dependencies present in market data. Traditional forecasting methods often fail to capture the multifaceted nature of financial markets, where temporal proximity, trend dynamics, and volatility patterns simultaneously influence price movements. To address these limitations, this paper proposes the Enhanced Multi-Aspect Transformer (EMAT), a novel deep learning architecture specifically designed for stock market prediction. EMAT incorporates a Multi-Aspect Attention Mechanism that simultaneously captures temporal decay patterns, trend dynamics, and volatility regimes through specialized attention components. The model employs an encoder–decoder architecture with enhanced feed-forward networks utilizing SwiGLU activation, enabling superior modeling of complex non-linear relationships. Furthermore, we introduce a comprehensive multi-objective loss function that balances point-wise prediction accuracy with volatility consistency. Extensive experiments on multiple stock market datasets demonstrate that EMAT consistently outperforms a wide range of state-of-the-art baseline models, including various recurrent, hybrid, and Transformer architectures. Our ablation studies further validate the design, confirming that each component of the Multi-Aspect Attention Mechanism makes a critical and quantifiable contribution to the model’s predictive power. The proposed architecture’s ability to simultaneously model these distinct financial characteristics makes it a particularly effective and robust tool for financial forecasting, offering significant improvements in accuracy compared to existing approaches. Full article
(This article belongs to the Special Issue Entropy, Artificial Intelligence and the Financial Markets)
Show Figures

Figure 1

14 pages, 346 KB  
Article
An ELECTRA-Based Model for Power Safety Named Entity Recognition
by Peng Liu, Zhenfu Sun and Biao Zhou
Appl. Sci. 2024, 14(20), 9410; https://doi.org/10.3390/app14209410 - 15 Oct 2024
Cited by 1 | Viewed by 1432
Abstract
Power safety named entity recognition (NER) is essential for determining the cause of faults, assessing potential risks, and planning maintenance schedules, contributing to the comprehension and analysis of power safety documentation content and structure. Such analysis is crucial for the development of a [...] Read more.
Power safety named entity recognition (NER) is essential for determining the cause of faults, assessing potential risks, and planning maintenance schedules, contributing to the comprehension and analysis of power safety documentation content and structure. Such analysis is crucial for the development of a knowledge graph within the power safety domain and the augmentation of the associated dataset. This paper introduces a power safety NER model using efficiently learning an encoder that classifies token replacements accurately (ELECTRA) model. This model employs root mean square layer normalization (RMSNorm) and the switched gated linear unit (SwiGLU) activation function, which substitutes the conventional layer normalization (LayerNorm) and the Gaussian error linear units (GeLU). This model also integrates bidirectional long short-term memory (BiLSTM) with conditional random fields (CRF) to bolster performance in NER tasks. Experimental results show that the improved ELECTRA model achieved an F1 value of 93% on the constructed power safety NER dataset. It outperforms the BERT-BiLSTM-CRF model, achieving a 3.3% performance improvement. Full article
Show Figures

Figure 1

30 pages, 4017 KB  
Article
Identification of Homeobox Transcription Factors in a Dimorphic Fungus Talaromyces marneffei and Protein-Protein Interaction Prediction of RfeB
by Monsicha Pongpom, Nopawit Khamto, Panwarit Sukantamala, Thitisuda Kalawil and Tanaporn Wangsanut
J. Fungi 2024, 10(10), 687; https://doi.org/10.3390/jof10100687 - 30 Sep 2024
Viewed by 1682
Abstract
Talaromyces marneffei is a thermally dimorphic fungus that can cause life-threatening systemic mycoses, particularly in immunocompromised individuals. Fungal homeobox transcription factors control various developmental processes, including the regulation of sexual reproduction, morphology, metabolism, and virulence. However, the function of homeobox proteins in T. [...] Read more.
Talaromyces marneffei is a thermally dimorphic fungus that can cause life-threatening systemic mycoses, particularly in immunocompromised individuals. Fungal homeobox transcription factors control various developmental processes, including the regulation of sexual reproduction, morphology, metabolism, and virulence. However, the function of homeobox proteins in T. marneffei has not been fully explored. Here, we searched the T. marneffei genome for the total homeobox transcription factors and predicted their biological relevance by performing gene expression analysis in different cell types, including conidia, mycelia, yeasts, and during phase transition. RfeB is selected for further computational analysis since (i) its transcripts were differentially expressed in different phases of T. marneffei, and (ii) this protein contains the highly conserved protein-protein interaction region (IR), which could be important for pathobiology and have therapeutic application. To assess the structure-function of the IR region, in silico alanine substitutions were performed at three-conserved IR residues (Asp276, Glu279, and Gln282) of RfeB, generating a triple RfeB mutated protein. Using 3D modeling and molecular dynamics simulations, we compared the protein complex formation of wild-type and mutated RfeB proteins with the putative partner candidate TmSwi5. Our results demonstrated that the mutated RfeB protein exhibited increased free binding energy, elevated protein compactness, and a reduced number of atomic contacts, suggesting disrupted protein stability and interaction. Notably, our model revealed that the IR residues primarily stabilized the RfeB binding sites located in the central region (CR). This computational approach for protein mutagenesis could provide a foundation for future experimental studies on the functional characterization of RfeB and other homeodomain-containing proteins in T. marneffei. Full article
(This article belongs to the Special Issue Current Trends in Mycological Research in Southeast Asia)
Show Figures

Figure 1

Back to TopTop