Skip to Content

18,821 Results Found

  • Article
  • Open Access
4 Citations
4,572 Views
17 Pages

A Multi-Modal Entity Alignment Method with Inter-Modal Enhancement

  • Song Yuan,
  • Zexin Lu,
  • Qiyuan Li and
  • Jinguang Gu

Due to inter-modal effects hidden in multi-modalities and the impact of weak modalities on multi-modal entity alignment, a Multi-modal Entity Alignment Method with Inter-modal Enhancement (MEAIE) is proposed. This method introduces a unique modality...

  • Article
  • Open Access
14 Citations
4,059 Views
17 Pages

Multi-Modal Spatio-Temporal Knowledge Graph of Ship Management

  • Yitao Zhang,
  • Ruiqing Xu,
  • Wangping Lu,
  • Wolfgang Mayer and ,
  • Da Ning,
  • Yucong Duan,
  • Xi Zeng and
  • Zaiwen Feng
  • + 3 authors+ 0 author

18 August 2023

In modern maritime activities, the quality of ship communication directly impacts the safety, efficiency, and economic viability of ship operations. Therefore, predicting and analyzing ship communication status has become a crucial task to ensure the...

(This article belongs to the Special Issue Purpose-Driven Data–Information–Knowledge–Wisdom (DIKWP)-Based Artificial General Intelligence Models and Applications)
  • Article
  • Open Access
12 Citations
4,522 Views
21 Pages

Gated Fusion Networks for Multi-Modal Violence Detection

  • Bilal Ahmad,
  • Mustaqeem Khan and
  • Muhammad Sajjad

3 October 2025

Public safety and security require an effective monitoring system to detect violence through visual, audio, and motion data. However, current methods often fail to utilize the complementary benefits of visual and auditory modalities, thereby reducing...

(This article belongs to the Special Issue Deep Learning Technologies and Their Applications in Image Processing, Computer Vision, and Computational Intelligence)
  • Article
  • Open Access
2 Citations
3,990 Views
19 Pages

6 September 2021

Over the last few years, there has been an increase in the studies that consider experiential (visual) information by building multi-modal language models and representations. It is shown by several studies that language acquisition in humans starts...

(This article belongs to the Special Issue Current Approaches and Applications in Natural Language Processing)
  • Article
  • Open Access
4 Citations
2,529 Views
11 Pages

10 October 2024

Integrating multi-modal data in natural language processing has opened new pathways for the enhancement of dialogue summarization. However, existing models often struggle to effectively synthesize textual, auditory, and visual inputs. This paper intr...

(This article belongs to the Special Issue Artificial Intelligence in Transportation Safety and Traffic Management)
  • Article
  • Open Access
4 Citations
1,086 Views
14 Pages

RMP: Robust Multi-Modal Perception Under Missing Condition

  • Xin Ma,
  • Xuqi Cai,
  • Yuansheng Song,
  • Yu Liang and ,
  • Gang Liu and
  • Yijun Yang
  • + 1 author+ 0 author

26 December 2025

Multi-modal perception is a core technology for edge devices to achieve safe and reliable environmental understanding in autonomous driving scenarios. In recent years, most approaches have focused on integrating complementary signals from diverse sen...

(This article belongs to the Special Issue Hardware and Software Co-Design in Intelligent Systems)
  • Article
  • Open Access
16 Citations
4,928 Views
19 Pages

Mixture of Attention Variants for Modal Fusion in Multi-Modal Sentiment Analysis

  • Chao He,
  • Xinghua Zhang,
  • Dongqing Song,
  • Yingshan Shen and ,
  • Chengjie Mao,
  • Huosheng Wen,
  • Dingju Zhu and
  • Lihua Cai
  • + 3 authors+ 0 author

With the popularization of better network access and the penetration of personal smartphones in today’s world, the explosion of multi-modal data, particularly opinionated video messages, has created urgent demands and immense opportunities for...

(This article belongs to the Special Issue Research Progress in Artificial Intelligence and Social Network Analysis)
  • Article
  • Open Access
9 Citations
3,875 Views
13 Pages

Multi-Modal Entity Alignment Method Based on Feature Enhancement

  • Huansha Wang,
  • Qinrang Liu,
  • Ruiyang Huang and
  • Jianpeng Zhang

1 June 2023

Multi-modal entity alignment refers to identifying equivalent entities between two different multi-modal knowledge graphs that consist of multi-modal information such as structural triples and descriptive images. Most previous multi-modal entity alig...

(This article belongs to the Topic Recent Advances in Data Mining)
  • Article
  • Open Access
4 Citations
1,577 Views
14 Pages

16 October 2025

Multi-modal knowledge graph completion (MMKGC) aims to complete knowledge graphs by integrating structural information with multi-modal (e.g., visual, textual, and numerical) features and leveraging cross-modal reasoning within a unified semantic spa...

  • Article
  • Open Access
6 Citations
4,760 Views
21 Pages

Multi-modal retrieval is a challenge due to heterogeneous gap and a complex semantic relationship between different modal data. Typical research map different modalities into a common subspace with a one-to-one correspondence or similarity/dissimilar...

(This article belongs to the Special Issue Deep Neural Networks and Their Applications)
  • Article
  • Open Access
1,633 Views
21 Pages

22 November 2025

Multi-modal object tracking (MMOT) has received widespread attention for the ability to overcome single-sensor perception limitations. However, existing methods encounter several critical challenges. Representation learning and generalization capabil...

(This article belongs to the Section Computer Vision and Pattern Recognition)
  • Article
  • Open Access
38 Citations
7,089 Views
17 Pages

On Robustness of Multi-Modal Fusion—Robotics Perspective

  • Michal Bednarek,
  • Piotr Kicki and
  • Krzysztof Walas

The efficient multi-modal fusion of data streams from different sensors is a crucial ability that a robotic perception system should exhibit to ensure robustness against disturbances. However, as the volume and dimensionality of sensory-feedback incr...

(This article belongs to the Special Issue Artificial Intelligence and Ambient Intelligence)
  • Article
  • Open Access
1 Citations
974 Views
21 Pages

Multi-modal sentiment analysis (MSA) aims to accurately identify users’ emotional states by integrating textual, acoustic, and visual modalities. However, existing methods often suffer from insufficient cross-modal interaction, rigid fusion str...

(This article belongs to the Section Artificial Intelligence)
  • Article
  • Open Access
7 Citations
3,001 Views
17 Pages

Multi-Modal Haptic Rendering Based on Genetic Algorithm

  • Yucheng Li,
  • Fei Wang,
  • Liangze Tao and
  • Juan Wu

24 November 2022

Multi-modal haptic rendering is an important research direction to improve realism in haptic rendering. It can produce various mechanical stimuli that render multiple perceptions, such as hardness and roughness. This paper proposes a multi-modal hapt...

  • Article
  • Open Access
5 Citations
2,091 Views
19 Pages

Multi-Level Cross-Modal Interactive-Network-Based Semi-Supervised Multi-Modal Ship Classification

  • Xin Song,
  • Zhikui Chen,
  • Fangming Zhong,
  • Jing Gao and ,
  • Jianning Zhang and
  • Peng Li
  • + 1 author+ 0 author

15 November 2024

Ship image classification identifies the type of ships in an input image, which plays a significant role in the marine field. To enhance the ship classification performance, various research focuses on studying multi-modal ship classification, which...

(This article belongs to the Section Sensor Networks)
  • Article
  • Open Access
38 Citations
4,171 Views
16 Pages

Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification

  • Wenjie Pan,
  • Linhan Huang,
  • Jianbao Liang,
  • Lan Hong and
  • Jianqing Zhu

23 April 2023

Multi-modal (i.e., visible, near-infrared, and thermal-infrared) vehicle re-identification has good potential to search vehicles of interest in low illumination. However, due to the fact that different modalities have varying imaging characteristics,...

(This article belongs to the Topic Applications in Image Analysis and Pattern Recognition)
  • Article
  • Open Access
8 Citations
4,710 Views
16 Pages

Deep learning-based methods have achieved good performance in various recognition benchmarks mostly by utilizing single modalities. As different modalities contain complementary information to each other, multi-modal based methods are proposed to imp...

(This article belongs to the Special Issue Deep Learning Based Object Detection II)
  • Article
  • Open Access
27 Citations
6,550 Views
18 Pages

12 August 2021

Emotion recognition is an important research field for human–computer interaction. Audio–video emotion recognition is now attacked with deep neural network modeling tools. In published papers, as a rule, the authors show only cases of the superiority...

(This article belongs to the Special Issue Emotion Recognition Based on Sensors)
  • Article
  • Open Access
10 Citations
4,441 Views
17 Pages

Multi-Task Federated Split Learning Across Multi-Modal Data with Privacy Preservation

  • Yipeng Dong,
  • Wei Luo,
  • Xiangyang Wang,
  • Lei Zhang and ,
  • Lin Xu,
  • Zehao Zhou and
  • Lulu Wang
  • + 2 authors+ 0 author

3 January 2025

With the advancement of federated learning (FL), there is a growing demand for schemes that support multi-task learning on multi-modal data while ensuring robust privacy protection, especially in applications like intelligent connected vehicles. Trad...

(This article belongs to the Section Vehicular Sensing)
  • Article
  • Open Access
1 Citations
2,856 Views
14 Pages

An Algorithm for Generating a Diverse Set of Multi-Modal Journeys

  • Federico Mosquera,
  • Pieter Smet and
  • Greet Vanden Berghe

7 November 2022

A direct way of reducing the number of cars on the road is to dissuade individuals from exclusively using their car and instead integrate public transport into their daily routine. Planning multi-modal journeys is a complex task for which individuals...

(This article belongs to the Section Combinatorial Optimization, Graph, and Network Algorithms)
  • Article
  • Open Access
9 Citations
3,256 Views
23 Pages

With the development of internet technology, fake news has become a multi-modal collection. The current news detection methods cannot fully extract semantic information between modalities and ignore the rumor properties of fake news, making it diffic...

  • Data Descriptor
  • Open Access
10 Citations
7,933 Views
9 Pages

Korean Tourist Spot Multi-Modal Dataset for Deep Learning Applications

  • Changhoon Jeong,
  • Sung-Eun Jang,
  • Sanghyuck Na and
  • Juntae Kim

12 October 2019

Recently, deep learning-based methods for solving multi-modal tasks such as image captioning, multi-modal classification, and cross-modal retrieval have attracted much attention. To apply deep learning for such tasks, large amounts of data are needed...

  • Article
  • Open Access
9 Citations
3,589 Views
21 Pages

4 May 2021

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human la...

(This article belongs to the Special Issue Feature Papers in Physical Sensors Section 2020)
  • Article
  • Open Access
27 Citations
4,535 Views
18 Pages

15 September 2023

As the number of modalities in biomedical data continues to increase, the significance of multi-modal data becomes evident in capturing complex relationships between biological processes, thereby complementing disease classification. However, the cur...

(This article belongs to the Special Issue Artificial Intelligence (AI) in Biomedicine)
  • Article
  • Open Access
3 Citations
3,552 Views
30 Pages

MERGE: A Modal Equilibrium Relational Graph Framework for Multi-Modal Knowledge Graph Completion

  • Yuying Shang,
  • Kun Fu,
  • Zequn Zhang,
  • Li Jin and ,
  • Zinan Liu,
  • Shensi Wang and
  • Shuchao Li
  • + 2 authors+ 0 author

28 November 2024

The multi-modal knowledge graph completion (MMKGC) task aims to automatically mine the missing factual knowledge from the existing multi-modal knowledge graphs (MMKGs), which is crucial in advancing cross-modal learning and reasoning. However, few me...

(This article belongs to the Section Intelligent Sensors)
  • Article
  • Open Access
53 Citations
7,859 Views
15 Pages

A Spam Filtering Method Based on Multi-Modal Fusion

  • Hong Yang,
  • Qihe Liu,
  • Shijie Zhou and
  • Yang Luo

19 March 2019

In recent years, the single-modal spam filtering systems have had a high detection rate for image spamming or text spamming. To avoid detection based on the single-modal spam filtering systems, spammers inject junk information into the multi-modality...

(This article belongs to the Special Issue Advances in Deep Learning)
  • Article
  • Open Access
35 Citations
7,812 Views
13 Pages

User Identification from Gait Analysis Using Multi-Modal Sensors in Smart Insole

  • Sang-Il Choi,
  • Jucheol Moon,
  • Hee-Chan Park and
  • Sang Tae Choi

31 August 2019

Recent studies indicate that individuals can be identified by their gait pattern. A number of sensors including vision, acceleration, and pressure have been used to capture humans’ gait patterns, and a number of methods have been developed to r...

(This article belongs to the Section Physical Sensors)
  • Article
  • Open Access
1,948 Views
21 Pages

25 October 2024

Magnetic resonance (MR) imaging is widely used in the clinical field due to its non-invasiveness, but the long scanning time is still a bottleneck for its popularization. Using the complementary information between multi-modal imaging to accelerate i...

(This article belongs to the Topic Deep Learning for Medical Image Analysis and Medical Natural Language Processing)
  • Article
  • Open Access
382 Views
15 Pages

21 August 2026

Multi-modal image fusion integrates complementary information from different modalities to generate a unified representation that is informative for human perception and beneficial to downstream vision tasks. However, existing methods often inefficie...

(This article belongs to the Special Issue Advances in Image Processing and Analysis)
  • Article
  • Open Access
17 Citations
4,752 Views
17 Pages

Multi-Modal Graph Interaction for Multi-Graph Convolution Network in Urban Spatiotemporal Forecasting

  • Lingyu Zhang,
  • Xu Geng,
  • Zhiwei Qin,
  • Hongjun Wang and ,
  • Xiao Wang,
  • Ying Zhang,
  • Jian Liang,
  • Guobin Wu,
  • Xuan Song and
  • Yunhai Wang
  • + 5 authors+ 0 author

29 September 2022

Graph convolution network-based approaches have been recently used to model region-wise relationships in region-level prediction problems in urban computing. Each relationship represents a kind of spatial dependency, such as region-wise distance or f...

(This article belongs to the Special Issue Big Data, Information and AI for Smart Urban)
  • Article
  • Open Access
16 Citations
3,503 Views
16 Pages

Multi-Modal Learning-Based Equipment Fault Prediction in the Internet of Things

  • Xin Nan,
  • Bo Zhang,
  • Changyou Liu,
  • Zhenwen Gui and
  • Xiaoyan Yin

6 September 2022

The timely detection of equipment failure can effectively avoid industrial safety accidents. The existing equipment fault diagnosis methods based on single-mode signal not only have low accuracy, but also have the inherent risk of being misled by sig...

(This article belongs to the Special Issue Parallel and Edge Computing with Artificial Intelligence for Sensor Network)
  • Article
  • Open Access
21 Citations
7,661 Views
12 Pages

Multi-Modal Sarcasm Detection with Sentiment Word Embedding

  • Hao Fu,
  • Hao Liu,
  • Hongling Wang,
  • Linyan Xu and ,
  • Jiali Lin and
  • Dazhi Jiang
  • + 1 author+ 0 author

23 February 2024

Sarcasm poses a significant challenge for detection due to its unique linguistic phenomenon where the intended meaning is often opposite of the literal expression. Current sarcasm detection technology primarily utilizes multi-modal processing, but th...

(This article belongs to the Special Issue New Advances in Affective Computing)
  • Article
  • Open Access
6 Citations
3,579 Views
13 Pages

13 June 2024

Online education review data have strong statistical and predictive power but lack efficient and accurate analysis methods. In this paper, we propose a multi-modal emotion analysis method to analyze the online education of college students based on e...

(This article belongs to the Special Issue Applied and Innovative Computational Intelligence Systems: 3rd Edition)
  • Article
  • Open Access
6 Citations
3,835 Views
16 Pages

23 May 2024

As a prominent topic in food computing, cross-modal recipe retrieval has garnered substantial attention. However, the semantic alignment across food images and recipes cannot be further enhanced due to the lack of intra-modal alignment in existing so...

(This article belongs to the Special Issue Applications of Artificial Intelligence in Food Industry)
  • Article
  • Open Access
10 Citations
4,616 Views
12 Pages

Designing Multi-Modal Embedding Fusion-Based Recommender

  • Anna Wróblewska,
  • Jacek Dąbrowski,
  • Michał Pastuszak,
  • Andrzej Michałowski and ,
  • Michał Daniluk,
  • Barbara Rychalska,
  • Mikołaj Wieczorek and
  • Sylwia Sysko-Romańczuk
  • + 3 authors+ 0 author

Recommendation systems have lately been popularised globally. However, often they need to be adapted to particular data and the use case. We have developed a machine learning-based recommendation system, which can be easily applied to almost any item...

(This article belongs to the Special Issue Recommender Systems: Approaches, Challenges and Applications)
  • Article
  • Open Access
1 Citations
428 Views
22 Pages

In multi-UAV mountain search and rescue scenarios, the perception system of multi-UAV suffers from low utilization of noise resources, poor collaboration of multi-modal data, and a persistent imbalance between speed and detection accuracy. The paper...

(This article belongs to the Special Issue Advances in Intelligent Motion Planning, Control, and Perception for Robotic Systems)
  • Article
  • Open Access
13 Citations
5,208 Views
10 Pages

Multi-Modal Sentiment Analysis Based on Interactive Attention Mechanism

  • Jun Wu,
  • Tianliang Zhu,
  • Xinli Zheng and
  • Chunzhi Wang

16 August 2022

In recent years, multi-modal sentiment analysis has become more and more popular in the field of natural language processing. Multi-modal sentiment analysis mainly concentrates on text, image and audio information. Previous work based on BERT utilize...

(This article belongs to the Special Issue Selected Papers from FCPAE2022 and 4th International Conference on Artificial Intelligence and Advanced Manufacturing (AIAM2022))
  • Article
  • Open Access
6 Citations
3,816 Views
13 Pages

Multi-Modal 3D Shape Clustering with Dual Contrastive Learning

  • Guoting Lin,
  • Zexun Zheng,
  • Lin Chen,
  • Tianyi Qin and
  • Jiahui Song

22 July 2022

3D shape clustering is developing into an important research subject with the wide applications of 3D shapes in computer vision and multimedia fields. Since 3D shapes generally take on various modalities, how to comprehensively exploit the multi-moda...

(This article belongs to the Special Issue Advances and Application of Intelligent Video Surveillance System)
  • Article
  • Open Access
19 Citations
5,613 Views
13 Pages

A Multi-Modal and Multi-Atlas Integrated Framework for Identification of Mild Cognitive Impairment

  • Zhuqing Long,
  • Jie Li,
  • Haitao Liao,
  • Li Deng and ,
  • Yukeng Du,
  • Jianghua Fan,
  • Xiaofeng Li,
  • Jichang Miao,
  • Shuang Qiu and
  • Bin Jing
  • + 6 authors+ 1 author

Background: Multi-modal neuroimaging with appropriate atlas is vital for effectively differentiating mild cognitive impairment (MCI) from healthy controls (HC). Methods: The resting-state functional magnetic resonance imaging (rs-fMRI) and structural...

(This article belongs to the Topic Applications of Biomedical Technology and Molecular Biological Approach in Brain Diseases)
  • Article
  • Open Access
2 Citations
3,647 Views
22 Pages

Optimal Routing of Wide Multi-Modal Energy and Infrastructure Corridors

  • Mehdi Salamati,
  • Xin Wang,
  • Jennifer Winter and
  • Hamidreza Zareipour

A multi-modal corridor accommodates multiple modes of energy and transportation infrastructure within the same right-of-way. The existing literature on corridor routing in raster space often focuses on one mode with no consideration of the width. Thi...

  • Article
  • Open Access
36 Citations
5,721 Views
16 Pages

Multi-Modal Deep Learning for Assessing Surgeon Technical Skill

  • Kevin Kasa,
  • David Burns,
  • Mitchell G. Goldenberg,
  • Omar Selim and ,
  • Cari Whyne and
  • Michael Hardisty
  • + 1 author+ 0 author

27 September 2022

This paper introduces a new dataset of a surgical knot-tying task, and a multi-modal deep learning model that achieves comparable performance to expert human raters on this skill assessment task. Seventy-two surgical trainees and faculty were recruit...

(This article belongs to the Special Issue Sensor Systems for Gesture Recognition II)
  • Article
  • Open Access
13 Citations
4,575 Views
14 Pages

17 July 2024

Multi-modal object re-identification (ReID) is a challenging task that seeks to identify objects across different image modalities by leveraging their complementary information. Traditional CNN-based methods are constrained by limited receptive field...

(This article belongs to the Section Sensing and Imaging)
  • Article
  • Open Access
298 Views
17 Pages

Multi-Modal Semantic-Enhanced Architecture with L1* Regularization for Novelty Detection

  • Zeqiu Chen,
  • Kaiyi Zhao,
  • Huiyu Mu,
  • Ruizhi Sun and ,
  • Yi Peng and
  • Shuze Jia
  • + 1 author+ 0 author

3 October 2026

Novelty detection is the identification (normal or anomalous) of test data differing from the training data (all are normal). It has gained significant attention in many fields, due to the infrequency and unpredictability of anomalies in reality. Mos...

  • Review
  • Open Access
418 Views
33 Pages

15 October 2026

The sensitive detection of tumor markers is of great significance for clinical diagnosis. Due to the extremely low abundance of early tumor biomarkers and severe background interference from biological matrices, traditional single-modal atomolar biol...

(This article belongs to the Section Biosensor and Bioelectronic Devices)
  • Article
  • Open Access
9 Citations
3,127 Views
21 Pages

8 March 2024

Efficient flotation beneficiation heavily relies on accurate flotation condition recognition based on monitored froth video. However, the recognition accuracy is hindered by limitations of extracting temporal features from froth videos and establishi...

(This article belongs to the Section Complexity)
  • Article
  • Open Access
466 Citations
24,647 Views
15 Pages

Over the past decade, convolutional neural networks (CNN) have shown very competitive performance in medical image analysis tasks, such as disease classification, tumor segmentation, and lesion detection. CNN has great advantages in extracting local...

(This article belongs to the Special Issue Machine Learning for Computer-Aided Diagnosis in Biomedical Imaging)
  • Article
  • Open Access
4 Citations
2,527 Views
22 Pages

Multi-Modal Entity Alignment Based on Enhanced Relationship Learning and Multi-Layer Feature Fusion

  • Huayu Li,
  • Yujie Hou,
  • Jing Liu,
  • Peiying Zhang and ,
  • Cuicui Wang and
  • Kai Liu
  • + 1 author+ 0 author

23 June 2025

Entity alignment is a critical technique for integrating diverse knowledge graphs. Although existing methods have achieved impressive success in traditional entity alignment, they may struggle to handle the complexities arising from interactions and...

(This article belongs to the Section A: Computer Science)
  • Article
  • Open Access
4 Citations
2,741 Views
22 Pages

Research on a PTSD Risk Assessment Model Using Multi-Modal Data Fusion

  • Youxi Luo,
  • Yucui Shang,
  • Dongfeng Zhu,
  • Tian Zhang and
  • Chaozhu Hu

5 June 2025

Post-traumatic stress disorder (PTSD) is a complex psychological disorder caused by multiple factors, which are not only related to individual psychological states but also closely linked to physiological responses, social environments, and personal...

(This article belongs to the Section D: Statistics and Operational Research)
  • Article
  • Open Access
1 Citations
1,142 Views
26 Pages

3 December 2025

In the field of remote sensing, accurate semantic segmentation is crucial for applications such as environmental monitoring and urban planning. Effective fusion of multi-modal data is a key factor in improving land cover classification accuracy. To a...

(This article belongs to the Section A: Computer Science)
  • Article
  • Open Access
30 Citations
7,136 Views
17 Pages

Multi-Modal Fake News Detection via Bridging the Gap between Modals

  • Peng Liu,
  • Wenhua Qian,
  • Dan Xu,
  • Bingling Ren and
  • Jinde Cao

4 April 2023

Multi-modal fake news detection aims to identify fake information through text and corresponding images. The current methods purely combine images and text scenarios by a vanilla attention module but there exists a semantic gap between different scen...

(This article belongs to the Topic Machine and Deep Learning)

of 377