Skip to Content

1,755 Results Found

  • Article
  • Open Access
2 Citations
2,978 Views
16 Pages

MoE-World: A Mixture-of-Experts Architecture for Multi-Task World Models

  • Cong Tang,
  • Yuang Liu,
  • Yueling Wu,
  • Wence Han and ,
  • Qian Yin,
  • Xin Zheng,
  • Wenyi Zeng and
  • Qiuli Zhang
  • + 3 authors+ 0 author

11 December 2025

World models are currently a mainstream approach in model-based deep reinforcement learning. Given the widespread use of Transformers in sequence modeling, they have provided substantial support for world models. However, world models often face the...

(This article belongs to the Special Issue Reinforcement Learning: Sample Efficiency, Generalisation, and AI Applications)
  • Article
  • Open Access
  • Best Paper Award
45 Citations
5,038 Views
15 Pages

15 January 2023

Most of the Artificial Intelligence (AI) models currently used in energy forecasting are traditional and deterministic. Recently, a novel deep learning paradigm, called ‘transformer’, has been developed, which adopts the mechanism of self...

(This article belongs to the Special Issue Applications of Machine Learning and Soft Computing in Energy Use Forecasting)
  • Article
  • Open Access
84 Citations
14,851 Views
14 Pages

23 August 2024

In recent years, transformer-based models have gained prominence in multivariate long-term time series forecasting (LTSF), demonstrating significant advancements despite facing challenges such as high computational demands, difficulty in capturing te...

(This article belongs to the Section AI Systems: Theory and Applications)
  • Article
  • Open Access
65 Citations
11,549 Views
23 Pages

Detection of Depression Severity in Social Media Text Using Transformer-Based Models

  • Amna Qasim,
  • Gull Mehak,
  • Nisar Hussain,
  • Alexander Gelbukh and
  • Grigori Sidorov

7 February 2025

Depression, a serious mental health disorder, requires accurate classification for effective intervention. Existing methods often fail to capture nuanced emotional and linguistic cues, leading to suboptimal classification of depression severity. This...

(This article belongs to the Special Issue Application of Machine Learning in Data Science and Computational Intelligence)
  • Article
  • Open Access
12 Citations
3,781 Views
24 Pages

21 April 2025

Software quality assurance is a critical aspect of software engineering, directly impacting maintainability, extensibility, and overall system performance. Traditional machine-learning techniques, such as gradient boosting and support vector machines...

(This article belongs to the Special Issue AI in Software Engineering: Challenges, Solutions and Applications)
  • Article
  • Open Access
2 Citations
2,077 Views
25 Pages

14 October 2025

This paper provides a detailed breakdown of a minimalist, fundamental Transformer-based architecture for forecasting univariate time series. It describes each processing step in detail, from input embedding and positional encoding to self-attention m...

(This article belongs to the Special Issue 2024 and 2025 Selected Papers from Algorithms Editorial Board Members)
  • Article
  • Open Access
28 Citations
7,433 Views
21 Pages

16 October 2024

This paper introduces a novel approach to pavement material crack detection, classification, and segmentation using advanced deep learning techniques, including multi-scale feature aggregation and transformer-based attention mechanisms. The proposed...

  • Article
  • Open Access
30 Citations
5,342 Views
19 Pages

Background/Objectives: As reported by the World Health Organization, Mpox (monkeypox) is an important disease present in 110 countries, mostly in South Asia and Africa. The number of Mpox cases has increased rapidly, and the medical world is worried...

(This article belongs to the Section Machine Learning and Artificial Intelligence in Diagnostics)
  • Article
  • Open Access
8 Citations
2,829 Views
15 Pages

Enhancing Lymph Node Metastasis Risk Prediction in Early Gastric Cancer Through the Integration of Endoscopic Images and Real-World Data in a Multimodal AI Model

  • Donghoon Kang,
  • Han Jo Jeon,
  • Jie-Hyun Kim,
  • Sang-Il Oh and ,
  • Ye Seul Seong,
  • Jae Young Jang,
  • Jung-Wook Kim,
  • Joon Sung Kim,
  • Seung-Joo Nam and
  • Hyuk Soon Choi
  • + 6 authors+ 1 author

3 March 2025

Objectives: The accurate prediction of lymph node metastasis (LNM) and lymphovascular invasion (LVI) is crucial for determining treatment strategies for early gastric cancer (EGC). This study aimed to develop and validate a deep learning-based clinic...

(This article belongs to the Collection Artificial Intelligence and Machine Learning in Cancer Research)
  • Article
  • Open Access
862 Views
17 Pages

We propose a transformer-based neural network for predicting credit card default using raw multivariate credit data represented as a 2D time series, eliminating the need for manual feature engineering. Unlike existing state-of-the-art (SOTA) tree-bas...

(This article belongs to the Section Computer Science & Engineering)
  • Article
  • Open Access
29 Citations
5,047 Views
17 Pages

A Method of Modern Standardized Apple Orchard Flowering Monitoring Based on S-YOLO

  • Xinzhu Zhou,
  • Guoxiang Sun,
  • Naimin Xu,
  • Xiaolei Zhang and ,
  • Jiaqi Cai,
  • Yunpeng Yuan and
  • Yinfeng Huang
  • + 2 authors+ 0 author

Monitoring fruit tree flowering information in the open world is more crucial than in the research-oriented environment for managing agricultural production to increase yield and quality. This work presents a transformer-based flowering period monito...

(This article belongs to the Special Issue 'Eyes', 'Brain', 'Feet' and 'Hands' of Efficient Harvesting Machinery)
  • Article
  • Open Access
44 Citations
4,306 Views
19 Pages

5 April 2023

Tomato leaf diseases can incur significant financial damage by having adverse impacts on crops and, consequently, they are a major concern for tomato growers all over the world. The diseases may come in a variety of forms, caused by environmental str...

(This article belongs to the Special Issue Applications of Video Processing and Computer Vision Sensor II)
  • Article
  • Open Access
26 Citations
4,110 Views
28 Pages

Grape Disease Detection Using Transformer-Based Integration of Vision and Environmental Sensing

  • Weixia Li,
  • Bingkun Zhou,
  • Yinzheng Zhou,
  • Chenlu Jiang and ,
  • Mingzhuo Ruan,
  • Tangji Ke,
  • Huijun Wang and
  • Chunli Lv
  • + 3 authors+ 0 author

27 March 2025

This study proposes a novel Transformer-based multimodal fusion framework for grape disease detection, integrating RGB images, hyperspectral data, and environmental sensor readings. Unlike traditional single-modal approaches, the proposed method leve...

(This article belongs to the Special Issue Advanced Machine Learning in Agriculture)
  • Article
  • Open Access
26 Citations
5,268 Views
24 Pages

15 August 2025

Machine learning (ML) has greatly improved intrusion detection in enterprise networks. However, ML models remain vulnerable to adversarial attacks, where small input changes cause misclassification. This study evaluates the robustness of a Random For...

(This article belongs to the Special Issue Recent Advances in Information Security and Data Privacy)
  • Feature Paper
  • Article
  • Open Access
1 Citations
487 Views
27 Pages

6 July 2026

The accelerating adoption of electric vehicles (EVs) presents significant challenges for maintaining grid stability and optimizing charging infrastructure. Accurate short-term forecasting of EV charging demand is therefore critical to support reliabl...

(This article belongs to the Special Issue Advancements in Vehicle-to-Grid Technology for Smart Energy Systems)
  • Article
  • Open Access
469 Views
39 Pages

3 September 2026

This work proposes a comparative framework for malware classification that evaluates the synergy between traditional feature engineering and modern deep learning architectures. Our methodology follows two primary paths: first, we integrate opcode n-g...

(This article belongs to the Special Issue AI in Cybersecurity, 3rd Edition)
  • Article
  • Open Access
694 Views
37 Pages

10 September 2026

This study proposes EQAS (Empowered Question-Answering System), a hybrid framework designed to support context-aware question-answering and intelligent feedback generation in domain-specific knowledge environments. EQAS integrates fine-tuned transfor...

  • Article
  • Open Access
21 Citations
7,500 Views
15 Pages

18 April 2023

Artificial intelligence, particularly machine learning, has begun to permeate various real-world applications and is continually being explored in automatic music generation. The approaches to music generation can be broadly divided into two categori...

(This article belongs to the Special Issue Artificial Intelligence and Machine Learning Based Methods and Applications)
  • Article
  • Open Access
4,883 Views
24 Pages

Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation

  • Phanwadee Sinthong,
  • Nam Nguyen,
  • Vijay Ekambaram,
  • Arindam Jati and ,
  • Jayant Kalagnanam and
  • Peeravit Koad
  • + 1 author+ 0 author

5 June 2025

Transformer-based time series models are being increasingly employed for time series data analysis. However, their training remains memory intensive, especially with high-dimensional data and extended look-back windows, while model-level memory optim...

(This article belongs to the Section Parallel and Distributed Algorithms)
  • Article
  • Open Access
2 Citations
2,705 Views
13 Pages

Transformer-Based Deep Learning for Multiplanar Cervical Spine MRI Interpretation: Comparison with Spine Surgeons and Radiologists

  • Aric Lee,
  • Junran Wu,
  • Changshuo Liu,
  • Andrew Makmur and ,
  • Yong Han Ting,
  • You Jun Lee,
  • Wilson Ong,
  • Tricia Kuah,
  • Juncheng Huang and
  • James Thomas Patrick Decourcy Hallinan
  • + 17 authors+ 12 authors

27 November 2025

Background: Degenerative cervical spondylosis (DCS) is a common and potentially debilitating condition, with surgery indicated in selected patients. Deep learning models (DLMs) can improve consistency in grading DCS neural stenosis on magnetic resona...

  • Article
  • Open Access
1 Citations
617 Views
23 Pages

2 March 2026

Transformer-based language models have been increasingly adopted to enhance semantic awareness in web service selection systems. However, the computational cost of large transformer encoders poses significant challenges for real-time and resource-con...

(This article belongs to the Section Information Systems)
  • Article
  • Open Access
11 Citations
6,042 Views
23 Pages

24 December 2024

The chicken is the world’s most farmed animal. In this work, we introduce the Chicks4FreeID dataset, the first publicly available dataset focused on the reidentification of individual chickens. We begin by providing a comprehensive overview of...

(This article belongs to the Special Issue Automated Monitoring of Livestock and Poultry with Machine Learning Technology, Volume II)
  • Article
  • Open Access
13 Citations
3,225 Views
37 Pages

27 November 2025

Sentiment analysis is essential for understanding consumer opinions, yet selecting the optimal models and embedding methods remains challenging, especially when handling ambiguous expressions, slang, or mismatched sentiment–rating pairs. This s...

  • Article
  • Open Access
1 Citations
4,761 Views
21 Pages

Receipt Information Extraction with Joint Multi-Modal Transformer and Rule-Based Model

  • Xandru Mifsud,
  • Leander Grech,
  • Adriana Baldacchino,
  • Léa Keller and ,
  • Gianluca Valentino and
  • Adrian Muscat
  • + 1 author+ 0 author

A receipt information extraction task requires both textual and spatial analyses. Early receipt analysis systems primarily relied on template matching to extract data from spatially structured documents. However, these methods lack generalizability a...

  • Article
  • Open Access
4 Citations
2,525 Views
17 Pages

12 August 2025

Multivariate time series forecasting requires modeling complex and evolving spatio-temporal dependencies as well as frequency-domain patterns; however, the existing Transformer-based approaches often struggle to effectively capture dynamic inter-seri...

  • Article
  • Open Access
3 Citations
1,324 Views
24 Pages

9 December 2025

Sequential recommendation aims to model evolving user preferences based on historical interactions. Transformer-based architectures have achieved strong performance by focusing on user-level sequential patterns, yet global item–item relationshi...

  • Article
  • Open Access
5 Citations
1,434 Views
24 Pages

A Mechanical Fault Diagnosis Method for On-Load Tap Changers Based on GOA-Optimized FMD and Transformer

  • Ruifeng Wei,
  • Zhenjiang Chen,
  • Qingbo Wang,
  • Yongsheng Duan and ,
  • Hui Wang,
  • Feiming Jiang,
  • Daoyuan Liu and
  • Xiaolong Wang
  • + 3 authors+ 0 author

19 July 2025

Mechanical failures frequently occur in On-Load Tap Changers (OLTCs) during operation, potentially compromising the reliability and stability of power systems. The goal of this study is to develop an intelligent and accurate diagnostic approach for O...

(This article belongs to the Special Issue Fault Diagnosis and Simulations for Power Transformers, Converter Transformers, and High-Frequency Transformers)
  • Feature Paper
  • Article
  • Open Access
2 Citations
3,996 Views
20 Pages

17 August 2022

The introduction and ever-growing size of the transformer deep-learning architecture have had a tremendous impact not only in the field of natural language processing but also in other fields. The transformer-based language models have contributed to...

(This article belongs to the Special Issue Computational Intelligence and Soft Computing: Recent Applications)
  • Article
  • Open Access
1,094 Views
22 Pages

BERT-Based Models for Normalization of Adverse Drug Event Expressions in Social Media to Standard Medical Terminology for Drug Safety Analysis

  • Fan Dong,
  • Wenjing Guo,
  • Jie Liu,
  • Ann Varghese and ,
  • Weida Tong,
  • Tucker A. Patterson and
  • Huixiao Hong
  • + 2 authors+ 0 author

Social media platforms host abundant and timely descriptions of medication experiences that can complement traditional pharmacovigilance systems. Yet the linguistic informality of these data presents a major challenge for mapping adverse drug event (...

(This article belongs to the Section Data Mining and Machine Learning)
  • Article
  • Open Access
1,681 Views
21 Pages

In recent years, Transformer-based methods have demonstrated proficiency in capturing complex patterns for time series forecasting. However, their quadratic complexity relative to input sequence length poses a significant bottleneck for scalability a...

(This article belongs to the Section Artificial Intelligence)
  • Article
  • Open Access
1 Citations
2,138 Views
26 Pages

30 July 2025

Recent studies on network intrusion detection using deep learning primarily focus on detecting attacks or classifying attack types, but they often overlook the challenge of attributing each attack to its specific source among many potential adversari...

(This article belongs to the Section Electrical, Electronics and Communications Engineering)
  • Article
  • Open Access
7 Citations
2,552 Views
15 Pages

Background/Objectives: Glaucoma is the second-leading cause of irreversible blindness globally. Retinal images such as color fundus photography have been widely used to detect glaucoma. However, little is known about the effectiveness of using raw cl...

(This article belongs to the Section Machine Learning and Artificial Intelligence in Diagnostics)
  • Article
  • Open Access
4 Citations
1,849 Views
22 Pages

23 June 2025

Background: Colorectal cancer (CRC) remains a significant global health burden, with early detection and intervention crucial for improving patient outcomes. This study aims to develop and evaluate a novel proof-of-concept ensemble framework combinin...

(This article belongs to the Section Gastroenterology & Hepatopancreatobiliary Medicine)
  • Article
  • Open Access
46 Citations
12,627 Views
15 Pages

25 August 2024

Most current methods use spatial–temporal graph neural networks (STGNNs) to analyze complex spatial–temporal information from traffic data collected from hundreds of sensors. STGNNs combine graph neural networks (GNNs) and sequence models...

(This article belongs to the Section Intelligent Sensors)
  • Article
  • Open Access
25 Citations
6,076 Views
19 Pages

A Novel Artificial-Intelligence-Based Approach for Classification of Parkinson’s Disease Using Complex and Large Vocal Features

  • Rahul Nijhawan,
  • Mukul Kumar,
  • Sahitya Arya,
  • Neha Mendirtta and ,
  • Sunil Kumar,
  • S. K. Towfek,
  • Doaa Sami Khafaga,
  • Hend K. Alkahtani and
  • Abdelaziz A. Abdelhamid
  • + 4 authors+ 0 author

Parkinson’s disease (PD) affects a large proportion of elderly people. Symptoms include tremors, slow movement, rigid muscles, and trouble speaking. With the aging of the developed world’s population, this number is expected to rise. The...

(This article belongs to the Special Issue Artificial Intelligence (AI) 2.0)
  • Article
  • Open Access
8 Citations
3,013 Views
26 Pages

In recent years, Android malware have been evolving and becoming more sophisticated at an alarming rate, highlighting the need for robust and evolving detection schemes. Despite the popularity of artificial intelligence-based approaches, they still s...

  • Article
  • Open Access
7 Citations
2,598 Views
18 Pages

Transformer-Based Decomposition of Electrodermal Activity for Real-World Mental Health Applications

  • Charalampos Tsirmpas,
  • Stasinos Konstantopoulos,
  • Dimitris Andrikopoulos,
  • Konstantina Kyriakouli and
  • Panagiotis Fatouros

15 July 2025

Decomposing Electrodermal Activity (EDA) into phasic (short-term, stimulus-linked responses) and tonic (longer-term baseline) components is essential for extracting meaningful emotional and physiological biomarkers. This study presents a comparative...

(This article belongs to the Section Biosensors)
  • Article
  • Open Access
4 Citations
4,130 Views
32 Pages

21 February 2026

In modern organizations, IT Service Management (ITSM) relies on the efficient handling of large volumes of unstructured textual data, such as support tickets and incident reports. This study investigates the automated classification of IT support req...

(This article belongs to the Section Artificial Intelligence and Digital Systems Engineering)
  • Article
  • Open Access
1 Citations
2,757 Views
42 Pages

11 May 2025

Product engagement detection in retail environments is critical for understanding customer preferences through nonverbal cues such as gaze and hand movements. This study presents a system leveraging a 360-degree top-view fisheye camera combined with...

(This article belongs to the Special Issue Feature Papers in "Sensing and Imaging" Section 2025&2026)
  • Article
  • Open Access
12 Citations
4,857 Views
18 Pages

Synonyms, Antonyms and Factual Knowledge in BERT Heads

  • Lorenzo Serina,
  • Luca Putelli,
  • Alfonso Emilio Gerevini and
  • Ivan Serina

In recent years, many studies have been devoted to discovering the inner workings of Transformer-based models, such as BERT, for instance, attempting to identify what information is contained within them. However, little is known about how these mode...

(This article belongs to the Section Big Data and Augmented Intelligence)
  • Article
  • Open Access
14 Citations
3,000 Views
28 Pages

DESAT: A Distance-Enhanced Strip Attention Transformer for Remote Sensing Image Super-Resolution

  • Yujie Mao,
  • Guojin He,
  • Guizhou Wang,
  • Ranyu Yin and ,
  • Yan Peng and
  • Bin Guan
  • + 1 author+ 0 author

14 November 2024

Transformer-based methods have demonstrated impressive performance in image super-resolution tasks. However, when applied to large-scale Earth observation images, the existing transformers encounter two significant challenges: (1) insufficient consid...

(This article belongs to the Special Issue Deep Learning for Remote Sensing Image Enhancement)
  • Article
  • Open Access
2 Citations
1,886 Views
22 Pages

17 March 2026

In real-world data, the structural symmetry of time series across temporal scales is often disrupted by the entanglement of trend, seasonal, and high-frequency components. This poses significant challenges to reliable time-series forecasting (TSF) in...

(This article belongs to the Section A: Computer Science)
  • Review
  • Open Access
45 Citations
7,612 Views
13 Pages

17 July 2024

The global transportation system’s need for electrification is driving research efforts to overcome the drawbacks of battery electric vehicles (BEVs). The accurate and reliable estimation of the states of charge (SOC) and health (SOH) of Li-Ion...

(This article belongs to the Section E: Electric Vehicles)
  • Review
  • Open Access
54 Citations
12,584 Views
34 Pages

Survey of Transformer-Based Malicious Software Detection Systems

  • Mohammed Alshomrani,
  • Aiiad Albeshri,
  • Badraddin Alturki,
  • Fouad Shoie Alallah and
  • Abdulaziz A. Alsulami

27 November 2024

In the recent past, the level of cyber threats has changed drastically, leading to the current transformation of the cybersecurity landscape. For example, emerging threats like Zero-day and polymorphic malware cannot be detected by conventional detec...

(This article belongs to the Special Issue AI-Based Solutions for Cybersecurity)
  • Review
  • Open Access
11 Citations
7,890 Views
29 Pages

Innovative Approaches to Traffic Anomaly Detection and Classification Using AI

  • Borja Pérez,
  • Mario Resino,
  • Teresa Seco,
  • Fernando García and
  • Abdulla Al-Kaff

15 May 2025

Video anomaly detection plays a crucial role in intelligent transportation systems by enhancing urban mobility and safety. This review provides a comprehensive analysis of recent advancements in artificial intelligence methods applied to traffic anom...

(This article belongs to the Special Issue Computer Vision and Pattern Recognition: Advanced Techniques and Applications)
  • Article
  • Open Access
5 Citations
2,218 Views
19 Pages

Automated Weld Defect Classification Enhanced by Synthetic Data Augmentation in Industrial Ultrasonic Images

  • Amir-M. Naddaf-Sh,
  • Vinay S. Baburao,
  • Zina Ben-Miled and
  • Hassan Zargarzadeh

3 December 2025

Automated ultrasonic testing (AUT) serves as a vital method for evaluating critical infrastructure in industries such as oil and gas. However, a significant challenge in deploying artificial intelligence (AI)-based interpretation methods for AUT data...

(This article belongs to the Special Issue Machine Learning for Object Detection and Scene Description in Images and Videos)
  • Article
  • Open Access
11 Citations
4,268 Views
17 Pages

Extracting financial events from numerous financial announcements is very important for investors to make right decisions. However, it is still challenging that event arguments always scatter in multiple sentences in a financial announcement, while m...

(This article belongs to the Section Computer Science & Engineering)
  • Article
  • Open Access
682 Views
18 Pages

23 March 2026

Transformer-based detectors have demonstrated exceptional accuracy in visible-object detection tasks. However, adversarial patches, specific types of adversarial examples, can disrupt these detectors by introducing unrestricted perturbations into spe...

(This article belongs to the Section Electronic Sensors)
  • Article
  • Open Access
6 Citations
4,232 Views
28 Pages

Comparative Evaluation of Sequential Neural Network (GRU, LSTM, Transformer) Within Siamese Networks for Enhanced Job–Candidate Matching in Applied Recruitment Systems

  • Mateusz Łępicki,
  • Tomasz Latkowski,
  • Izabella Antoniuk,
  • Michał Bukowski and ,
  • Bartosz Świderski,
  • Grzegorz Baranik,
  • Bogusz Nowak,
  • Robert Zakowicz,
  • Łukasz Dobrakowski and
  • Jarosław Kurek
  • + 6 authors+ 1 author

26 May 2025

Job–candidate matching is pivotal in recruitment, yet traditional manual or keyword-based methods can be laborious and prone to missing qualified candidates. In this study, we introduce the first Siamese framework that systematically contrasts...

(This article belongs to the Special Issue Innovations in Artificial Neural Network Applications)
  • Review
  • Open Access
10 Citations
3,754 Views
27 Pages

Segmentation and Classification of Lung Cancer Images Using Deep Learning

  • Xiaoli Yang,
  • Angchao Duan,
  • Ziyan Jiang,
  • Xiao Li and ,
  • Chenchen Wang,
  • Jiawen Wang and
  • Jiayi Zhou
  • + 2 authors+ 0 author

7 January 2026

Lung cancer ranks among the world’s most prevalent and deadly diseases. Early detection is crucial for improving patient survival rates. Computed tomography (CT) is a common method for lung cancer screening and diagnosis. With the advancement o...

of 36