Skip to Content

658 Results Found

  • Article
  • Open Access
3 Citations
2,487 Views
19 Pages

16 February 2023

In the rapid serial visual presentation (RSVP) paradigm, sound affects participants’ recognition of targets. Although many studies have shown that sound improves cross-modal processing, researchers have not yet explored the effects of sound sem...

(This article belongs to the Section Neurolinguistics)
  • Article
  • Open Access
27 Citations
5,233 Views
23 Pages

Cross-Modal Transformer-Based Streaming Dense Video Captioning with Neural ODE Temporal Localization

  • Shakhnoza Muksimova,
  • Sabina Umirzakova,
  • Murodjon Sultanov and
  • Young Im Cho

24 January 2025

Dense video captioning is a critical task in video understanding, requiring precise temporal localization of events and the generation of detailed, contextually rich descriptions. However, the current state-of-the-art (SOTA) models face significant c...

(This article belongs to the Section Sensing and Imaging)
  • Article
  • Open Access
4 Citations
3,345 Views
18 Pages

29 August 2022

A human can infer the magnitude of interaction force solely based on visual information because of prior knowledge in human–robot interaction (HRI). A method of reconstructing tactile information through cross-modal signal processing is propose...

(This article belongs to the Special Issue Sensing, Estimating, and Analyzing Human Movements for Human–Robot Interaction)
  • Article
  • Open Access
5 Citations
1,318 Views
18 Pages

15 December 2025

Recent advancements in multimodal large language models (MLLMs) have demonstrated remarkable capabilities in processing diverse data types, yet significant disparities persist between human cognitive processes and computational approaches to multimod...

  • Article
  • Open Access
4 Citations
1,504 Views
16 Pages

Identifying instruments in polyphonic audio is challenging due to overlapping spectra and variations in timbre and playing styles. This task is central to music information retrieval, with applications in transcription, recommendation, and indexing....

  • Article
  • Open Access
3 Citations
2,504 Views
22 Pages

Accurate comprehension of others’ thoughts and intentions is crucial for smooth social interactions, wherein understanding their perceptual experiences serves as a fundamental basis for this high-level social cognition. However, previous resear...

(This article belongs to the Section Sensory and Motor Neuroscience)
  • Article
  • Open Access
7 Citations
2,560 Views
20 Pages

2 September 2025

Hands-free computer interaction is a key topic in assistive technology, with camera-based and voice-based systems being the most common methods. Recent camera-based solutions leverage facial expressions or head movements to simulate mouse clicks or k...

(This article belongs to the Section Sensing and Imaging)
  • Article
  • Open Access
7 Citations
2,814 Views
22 Pages

Double Augmentation: A Modal Transforming Method for Ship Detection in Remote Sensing Imagery

  • Fangli Mou,
  • Zide Fan,
  • Chuan’ao Jiang,
  • Yidan Zhang and ,
  • Lei Wang and
  • Xinming Li
  • + 1 author+ 0 author

5 February 2024

Ship detection in remote sensing images plays an important role in maritime surveillance. Recently, convolution neural network (CNN)-based methods have achieved state-of-the-art performance in ship detection. Even so, there are still two problems tha...

  • Article
  • Open Access
4 Citations
3,875 Views
13 Pages

Cross-Modal Interactions and Movement-Related Tactile Gating: The Role of Vision

  • Maria Casado-Palacios,
  • Alessia Tonelli,
  • Claudio Campus and
  • Monica Gori

Background: When engaging with the environment, multisensory cues interact and are integrated to create a coherent representation of the world around us, a process that has been suggested to be affected by the lack of visual feedback in blind individ...

(This article belongs to the Section Sensory and Motor Neuroscience)
  • Review
  • Open Access
9 Citations
3,444 Views
24 Pages

21 October 2025

Gear defect detection directly affects the operational reliability of critical equipment in fields such as automotive and aerospace. Gear defect detection technology based on machine vision, leveraging the advantages of non-contact measurement, high...

(This article belongs to the Special Issue Simulation, Modeling, and Decision-Making Processes in Manufacturing Systems and Industrial Engineering, 2nd Edition)
  • Article
  • Open Access
664 Views
22 Pages

18 March 2026

Sound plays a fundamental role in human perception, conveying information about events, objects, and spatial dynamics that may not be visually accessible. However, current technologies such as Acoustic Event Detection typically reduce complex soundsc...

(This article belongs to the Section Computing and Artificial Intelligence)
  • Article
  • Open Access
1 Citations
1,261 Views
38 Pages

14 April 2026

Classifying which operation a Computer Numerical Control (CNC) machine is executing, not just detecting whether it is functioning correctly, is a monitoring challenge that existing sensor-based studies rarely address. Unlike tool wear estimation, ope...

(This article belongs to the Special Issue Sensors and IoT Technologies for the Smart Industry)
  • Article
  • Open Access
1 Citations
2,025 Views
19 Pages

1 September 2025

Presenting information to multiple sensory modalities often facilitates or interferes with processing, yet the mechanisms remain unclear. Using a Stroop-like task, the two reported experiments examined how semantic congruency and incongruency in one...

  • Article
  • Open Access
6 Citations
4,045 Views
12 Pages

1 April 2022

The brain’s ability to create a unified conscious representation of an object by integrating information from multiple perception pathways is called perceptual binding. Binding is crucial for normal cognitive function. Some perceptual binding e...

(This article belongs to the Special Issue Dynamics and Bifurcations in Mathematical Neuroscience: Analysis, Modeling and Applications)
  • Review
  • Open Access
2 Citations
3,130 Views
14 Pages

26 October 2021

In this article, we detail the advantages of self-face identification latencies over more traditional tests of mirror self-recognition. Using reaction time latencies (measured in milliseconds) to identify different dimensions of the self, instead of...

(This article belongs to the Section Neuropsychology)
  • Hypothesis
  • Open Access
1 Citations
3,609 Views
12 Pages

Cross-Modal Tinnitus Remediation: A Tentative Theoretical Framework

  • Antoine J. Shahin,
  • Mariel G. Gonzales and
  • Andrew Dimitrijevic

19 January 2024

Tinnitus is a prevalent hearing-loss deficit manifested as a phantom (internally generated by the brain) sound that is heard as a high-frequency tone in the majority of afflicted persons. Chronic tinnitus is debilitating, leading to distress, sleep d...

(This article belongs to the Special Issue Recent Advances in Hearing Impairment)
  • Article
  • Open Access
40 Citations
4,215 Views
18 Pages

1 February 2024

Insulator defect detection of transmission line insulators is an important task for unmanned aerial vehicle (UAV) inspection, which is of immense importance in ensuring the stable operation of transmission lines. Transmission line insulators exist in...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
24 Citations
7,823 Views
23 Pages

AVaTER: Fusing Audio, Visual, and Textual Modalities Using Cross-Modal Attention for Emotion Recognition

  • Avishek Das,
  • Moumita Sen Sarma,
  • Mohammed Moshiul Hoque,
  • Nazmul Siddique and
  • M. Ali Akber Dewan

10 September 2024

Multimodal emotion classification (MEC) involves analyzing and identifying human emotions by integrating data from multiple sources, such as audio, video, and text. This approach leverages the complementary strengths of each modality to enhance the a...

(This article belongs to the Special Issue Emotion Recognition Based on Sensors (3rd Edition))
  • Article
  • Open Access
326 Views
20 Pages

30 September 2026

Fine-grained sentiment information about specific aspects is the target of aspect-based sentiment analysis (ABSA). Nevertheless, when real-world situations involve sentiment expressed through multiple modalities, traditional text-centric approaches a...

(This article belongs to the Special Issue Multimodal Content and Sentiment Analysis)
  • Article
  • Open Access
894 Views
23 Pages

LiDAR point cloud semantic segmentation is essential for autonomous driving, yet LiDAR-only methods remain constrained by sparsity and limited texture cues. We propose Cross-Modal Collaborative Manifold Distillation (CMCMD), which transfers open-worl...

  • Article
  • Open Access
849 Views
29 Pages

IC-SAM: Segment Anything for Industrial Integrated Circuit Chip Inspection

  • Fuqin Deng,
  • Zhi Xu,
  • Yihang Shi,
  • Nannan Li and ,
  • Qingshan Xia and
  • Lanhui Fu
  • + 1 author+ 0 author

Industrial integrated circuit (IC) chip defect inspection is hindered by diverse topologies, micro-scale structures, and the insufficient precision of traditional vision paradigms. While the Segment Anything Model (SAM) offers strong zero-shot capabi...

(This article belongs to the Topic Computer Vision and Image Processing, 3rd Edition)
  • Article
  • Open Access
1 Citations
1,818 Views
27 Pages

An AI-Driven Multimodal Sensor Fusion Framework for Fraud Perception in Short-Video and Live-Streaming Platforms

  • Ruixiang Zhao,
  • Xuanhao Zhang,
  • Jinfan Yang,
  • Haofei Li and ,
  • Zhengjia Lu,
  • Wenrui Xu and
  • Manzhou Li
  • + 2 authors+ 0 author

28 February 2026

With the rapid proliferation of short-video platforms and live-streaming commerce ecosystems, marketing activities are increasingly manifested through complex multimodal sensing signals. These heterogeneous sensor data streams exhibit strong temporal...

(This article belongs to the Special Issue Artificial Intelligence-Driven Sensing)
  • Article
  • Open Access
10 Citations
3,569 Views
17 Pages

CMID: Crossmodal Image Denoising via Pixel-Wise Deep Reinforcement Learning

  • Yi Guo,
  • Yuanhang Gao,
  • Bingliang Hu,
  • Xueming Qian and
  • Dong Liang

20 December 2023

Removing noise from acquired images is a crucial step in various image processing and computer vision tasks. However, the existing methods primarily focus on removing specific noise and ignore the ability to work across modalities, resulting in limit...

(This article belongs to the Section Sensing and Imaging)
  • Article
  • Open Access
8 Citations
3,964 Views
22 Pages

25 June 2023

Cross-modal sentiment analysis is an emerging research area in natural language processing. The core task of cross-modal fusion lies in cross-modal relationship extraction and joint feature learning. The existing research methods of cross-modal senti...

  • Article
  • Open Access
7 Citations
2,706 Views
21 Pages

14 May 2024

Fine-grained representation is fundamental to species classification based on deep learning, and in this context, cross-modal contrastive learning is an effective method. The diversity of species coupled with the inherent contextual ambiguity of natu...

(This article belongs to the Section Sensing and Imaging)
  • Review
  • Open Access
7 Citations
8,713 Views
26 Pages

4 April 2023

Cross-modal retrieval aims to elucidate information fusion, imitate human learning, and advance the field. Although previous reviews have primarily focused on binary and real-value coding methods, there is a scarcity of techniques grounded in deep re...

(This article belongs to the Special Issue Advances in Intelligent Information Systems and AI Applications)
  • Review
  • Open Access
62 Citations
15,076 Views
12 Pages

We review evidence for cross-modal cortical re-organization in clinical populations with hearing loss. Cross-modal plasticity refers to the ability for an intact sensory modality (e.g., vision or somatosensation) to recruit cortical brain regions fro...

  • Article
  • Open Access
1 Citations
732 Views
21 Pages

A Multi-Stage Cross-Modal Fusion Framework for Visible–Infrared Object Detection

  • Anfu Zhu,
  • Yinbing Chen,
  • Heng Guo,
  • Zhizeng Zhang and ,
  • Yaning Yang,
  • Qinghua Jiang,
  • Yueyong Li and
  • Yi Yang
  • + 3 authors+ 0 author

8 June 2026

In visible–infrared object detection under complex environments, cross-modal fusion often suffers from spatial misalignment, semantic inconsistency, and unstable feature responses under varying illumination conditions. To address these issues,...

(This article belongs to the Special Issue Applied Computer Vision and Deep Learning)
  • Article
  • Open Access
2 Citations
2,361 Views
19 Pages

25 June 2024

Emotion is a significant factor influencing education and teaching, closely intertwined with learners’ cognitive processing. Conducting analysis of learners’ emotions based on cross-modal data is beneficial for achieving personalized guid...

(This article belongs to the Special Issue Artificial Intelligence Technologies for Education: Advancements, Challenges, and Impacts)
  • Article
  • Open Access
3 Citations
3,614 Views
18 Pages

Soft Contrastive Cross-Modal Retrieval

  • Jiayu Song,
  • Yuxuan Hu,
  • Lei Zhu,
  • Chengyuan Zhang and ,
  • Jian Zhang and
  • Shichao Zhang
  • + 1 author+ 0 author

27 February 2024

Cross-modal retrieval plays a key role in the Natural Language Processing area, which aims to retrieve one modality to another efficiently. Despite the notable achievements of existing cross-modal retrieval methodologies, the complexity of the embedd...

(This article belongs to the Section Computing and Artificial Intelligence)
  • Article
  • Open Access
7 Citations
3,366 Views
15 Pages

28 April 2022

Our senses are constantly reached by a multitude of stimuli from all different sensory modalities. To create a coherent representation of the environment, we must integrate the various unimodal inputs that refer to the same object into a single multi...

(This article belongs to the Special Issue Selected Papers from the Symposium on Perception and Cognition and Kanizsa Lecture 2021)
  • Article
  • Open Access
1 Citations
2,282 Views
16 Pages

Graph Sampling-Based Multi-Stream Enhancement Network for Visible-Infrared Person Re-Identification

  • Jinhua Jiang,
  • Junjie Xiao,
  • Renlin Wang,
  • Tiansong Li and ,
  • Wenfeng Zhang,
  • Ruisheng Ran and
  • Sen Xiang
  • + 2 authors+ 0 author

18 September 2023

With the increasing demand for person re-identification (Re-ID) tasks, the need for all-day retrieval has become an inevitable trend. Nevertheless, single-modal Re-ID is no longer sufficient to meet this requirement, making Multi-Modal Data crucial i...

(This article belongs to the Special Issue Multi-Modal Data Sensing and Processing)
  • Article
  • Open Access
15 Citations
5,288 Views
21 Pages

Unified Transformer with Cross-Modal Mixture Experts for Remote-Sensing Visual Question Answering

  • Gang Liu,
  • Jinlong He,
  • Pengfei Li,
  • Shenjun Zhong and ,
  • Hongyang Li and
  • Genrong He
  • + 1 author+ 0 author

24 September 2023

Remote-sensing visual question answering (RSVQA) aims to provide accurate answers to remote sensing images and their associated questions by leveraging both visual and textual information during the inference process. However, most existing methods i...

(This article belongs to the Special Issue Advanced Application of Artificial Intelligence and Machine Vision in Remote Sensing II)
  • Article
  • Open Access
1 Citations
2,594 Views
11 Pages

23 December 2022

The development of intelligent information technology provides an effective way for cross-modal learning analytics and promotes the realization of procedural and scientific educational evaluation. To accurately capture the emotional changes of learne...

(This article belongs to the Special Issue STEAM Education and the Innovative Pedagogies in the Intelligence Era)
  • Article
  • Open Access
24 Citations
4,956 Views
20 Pages

24 June 2024

In complex environments a single visible image is not good enough to perceive the environment, this paper proposes a novel dual-stream real-time detector designed for target detection in extreme environments such as nighttime and fog, which is able t...

(This article belongs to the Section Remote Sensors)
  • Review
  • Open Access
13 Citations
5,670 Views
25 Pages

28 January 2023

Person re-identification (Re-ID) aims to retrieve a particular pedestrian’s identification from a surveillance system consisting of non-overlapping cameras. In recent years, researchers have begun to focus on open-world person Re-ID tasks based...

(This article belongs to the Special Issue Advances in Mathematical Methods for Image Processing and Pattern Recognition)
  • Article
  • Open Access
531 Views
28 Pages

20 August 2026

Optical–elevation data fusion is widely used in aerial remote sensing semantic segmentation, as optical imagery provides rich spectral and textural information, while DSM or DEM data offer complementary elevation-related structural cues. Howeve...

(This article belongs to the Section Remote Sensing Image Processing)
  • Article
  • Open Access
308 Views
35 Pages

TPA-NET: A Progressive Cross-Modal Alignment and Confidence Fusion Network for Object Detection in Low-Resolution Optical-SAR Remote Sensing Imagery

  • Xiaofeng Zhao,
  • Chenxiao Li,
  • Hui Zhang,
  • Kehao Wang and ,
  • Fan Zhang,
  • Hanshuo Huo and
  • Zhili Zhang
  • + 2 authors+ 0 author

9 October 2026

Object detection in low-resolution optical-SAR remote sensing imagery is challenged by degraded object structures, residual cross-modal spatial inconsistency, and unreliable SAR responses. To address these issues, we propose TPA-NET, a progressive cr...

(This article belongs to the Special Issue Advanced Object Detection, Classification and Recognition in VIR Optical and SAR Remote Sensing Imagery)
  • Article
  • Open Access
236 Views
21 Pages

Observer-Related Versus Cross-Modality Variability in Ventricular Quantification by Cardiac CT and Cardiovascular Magnetic Resonance

  • Răzvan-Andrei Licu,
  • Alexandra-Mihaela Țolescu-Dobra,
  • Gheorghe-Marian Rotariu,
  • Kincső-Zsófia Lőrincz and ,
  • Claudia-Raluca Mariean,
  • Marian Pop,
  • Dragoș Constantin Cucoranu,
  • Liviu Moraru,
  • Andrei Manea and
  • Anca Bacârea
  • + 7 authors+ 2 authors

9 October 2026

Background/Objectives: Ventricular measurements from cardiac computed tomography (CCT) and cardiovascular magnetic resonance (CMR) may differ due to modality-specific, physiological, or observer-dependent reasons. We compared cross-modality disagreem...

(This article belongs to the Special Issue Optimization of Clinical Imaging: From Diagnosis to Prognosis, 2nd Edition)
  • Article
  • Open Access
3 Citations
2,430 Views
20 Pages

Medical report generation has made significant progress in recent years. However, generated reports still suffer from issues such as poor readability, incomplete and inaccurate descriptions of lesions, and challenges in capturing fine-grained abnorma...

(This article belongs to the Topic Intelligent Image Processing Technology)
  • Article
  • Open Access
37 Citations
5,218 Views
22 Pages

CMANet: Cross-Modality Attention Network for Indoor-Scene Semantic Segmentation

  • Longze Zhu,
  • Zhizhong Kang,
  • Mei Zhou,
  • Xi Yang and ,
  • Zhen Wang,
  • Zhen Cao and
  • Chenming Ye
  • + 2 authors+ 0 author

5 November 2022

Indoor-scene semantic segmentation is of great significance to indoor navigation, high-precision map creation, route planning, etc. However, incorporating RGB and HHA images for indoor-scene semantic segmentation is a promising yet challenging task,...

(This article belongs to the Special Issue LiDAR Sensor Hardware, Algorithm Development and Its Application)
  • Article
  • Open Access
5 Citations
3,445 Views
26 Pages

10 July 2025

Although multimodal feature fusion technology in fake news detection can integrate complementary information from different modal data, the semantic inconsistency of multimodal features will lead to feature fusion difficulties. And there is the probl...

(This article belongs to the Special Issue Algorithms for Feature Selection (3rd Edition))
  • Article
  • Open Access
2,998 Views
16 Pages

Deep Adversarial Learning Triplet Similarity Preserving Cross-Modal Retrieval Algorithm

  • Guokun Li,
  • Zhen Wang,
  • Shibo Xu,
  • Chuang Feng and ,
  • Xiaohan Yang,
  • Nannan Wu and
  • Fuzhen Sun
  • + 2 authors+ 0 author

25 July 2022

The cross-modal retrieval task can return different modal nearest neighbors, such as image or text. However, inconsistent distribution and diverse representation make it hard to directly measure the similarity relationship between different modal sam...

(This article belongs to the Special Issue Artificial Intelligence and Natural Computing: Theory, Methodology and Applications)
  • Article
  • Open Access
3,263 Views
29 Pages

Research on fMRI Image Generation from EEG Signals Based on Diffusion Models

  • Xiaoming Sun,
  • Yutong Sun,
  • Junxia Chen,
  • Bochao Su and ,
  • Tuo Nie and
  • Ke Shui
  • + 1 author+ 0 author

13 November 2025

Amidrapid advances in intelligent medicine, decoding brain activity from electroencephalogram (EEG) signals has emerged as a critical technical frontier for brain–computer interfaces and medical AI systems. Given the inherent spatial resolution...

(This article belongs to the Special Issue Deep Learning in Image Processing and Pattern Recognition, 2nd Edition)
  • Article
  • Open Access
4 Citations
2,609 Views
16 Pages

RGB-D salient object detection (SOD) aims at locating the most eye-catching object in visual input by fusing complementary information of RGB modality and depth modality. Most of the existing RGB-D SOD methods integrate multi-modal features to genera...

(This article belongs to the Section Microwave and Wireless Communications)
  • Article
  • Open Access
3 Citations
3,004 Views
19 Pages

21 September 2021

Cross-modal hashing technology is a key technology for real-time retrieval of large-scale multimedia data in real-world applications. Although the existing cross-modal hashing methods have achieved impressive accomplishment, there are still some limi...

  • Article
  • Open Access
443 Views
26 Pages

10 September 2026

Visible and synthetic aperture radar (SAR) images exhibit substantial nonlinear radiometric differences and geometric deformations due to their fundamentally different imaging mechanisms, making high-precision registration between the two modalities...

(This article belongs to the Section Remote Sensing Image Processing)
  • Article
  • Open Access
7 Citations
4,377 Views
22 Pages

15 December 2022

Cross-Modal Hashing (CMH) retrieval methods have garnered increasing attention within the information retrieval research community due to their capability to deal with large amounts of data thanks to the computational efficiency of hash-based methods...

(This article belongs to the Section AI in Imaging)
  • Article
  • Open Access
718 Views
21 Pages

HDCA: Heterogeneous Dual-Path Contrastive Architecture for Action Recognition

  • Shilu Kang,
  • Hua Huo,
  • Lan Ma,
  • Jinxuan Wang and
  • Aokun Mei

30 November 2025

We propose Heterogeneous Dual-path Contrastive Architecture (HDCA) for action recognition. Our model involves a spatial pathway and a temporal pathway; these two pathways employ distinct backbone networks and input formats, tailored to the specific p...

(This article belongs to the Special Issue Advances in Artificial Intelligence and Computer Vision Based on Deep Learning)
  • Article
  • Open Access
3 Citations
2,615 Views
18 Pages

30 October 2023

Given a textual query, text-based person re-identification is supposed to search for the targeted pedestrian images from a large-scale visual database. Due to the inherent heterogeneity between different modalities, it is challenging to measure the c...

(This article belongs to the Section Computing and Artificial Intelligence)

of 14