Background/Objectives: Neural decoding methods are often limited by the performance of brain encoders, which map complex brain signals into a latent representation space of perception information. These brain encoders are constrained by the limited a...
Pilot visual behavior safety assessment is a cross-disciplinary technology that analyzes pilots’ gaze behavior and neurocognitive responses. This paper proposes a multimodal analysis method for pilot visual behavior safety, specifically for cog...
Chronic pain is a dynamic, brain-wide condition that eludes effective management by conventional, static treatment approaches. Transcutaneous Electrical Nerve Stimulation (TENS), traditionally perceived as a simple and generic modality, is on the ver...
Robust decoding of latent cognitive states from non-stationary physiological time series is a challenging high-dimensional signal processing problem. Traditional unimodal frameworks based only on electroencephalography often show covariate shift and...
Turning visual evidence into coherent sentences depends on how visual and linguistic representations are combined before decoding. We present two decoder-specific generation strategies, CLS-LSTM and CLS-T5, built on a common CLS-guided Swin–BER...
Brain–computer interface (BCI) technology has shown potential for future rehabilitation-related and assistive control applications. Nevertheless, single-modality electroencephalography-based motor imagery (EEG-MI) signals are susceptible to interfere...
Humans’ performance varies due to the mental resources that are available to successfully pursue a task. To monitor users’ current cognitive resources in naturalistic scenarios, it is essential to not only measure demands induced by the t...
Perivascular adipose tissue (PVAT) has emerged as a central regulator of vascular homeostasis in large arteries, with its dysfunction driving the pathogenesis of atherosclerosis, hypertension, and metabolic diseases. In the past, our understanding of...
Multi-modal feature fusion and effectively exploiting high-level semantic information are critical in salient object detection (SOD). However, the depth maps complementing RGB image fusion strategies cannot supply effective semantic information when...
Stereotactic brain tumor segmentation based on 3D neuroimaging data is a challenging task due to the complexity of the brain architecture, extreme heterogeneity of tumor malformations, and the extreme variability of intensity signal and noise distrib...
Real-time monitoring of urban building development provides a basis for urban planning and management. Remote sensing change detection is a key technology for achieving this goal. Intelligent change detection based on deep learning of remote sensing...
Background: Programming Deep Brain Stimulation (DBS) for pediatric dystonia remains largely empirical, as no validated electrophysiological biomarker currently guides parameter selection, and most studies treat each signal modality separately rather...
Improving the decoding accuracy of biological signals has been a research focus for decades to advance health, automation, and robotic industries. However, challenges like inter-subject variability, data scarcity, and multifunctional variability caus...
Vehicle trajectory prediction at urban intersections is challenging due to strong maneuver ambiguity, dense agent interactions, and complex lane branching structures. Existing methods frequently suffer from mode collapse or fail to provide sufficient...
Due to the inadequacy in utilizing complementary information from different modalities and the biased estimation of degraded parameters, the unsupervised hyperspectral super-resolution algorithm suffers from low precision and limited applicability. T...
Human performance in critical environments is frequently degraded by mistimed communication delivered during periods of visual–cognitive saturation. In such settings, failures arise not only from individual limitations but also from poor coordi...
Decoding mental states from non-invasive neural recordings is central to brain-computer interface research. Multimodal acquisition that combines electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) couples the high temporal...
Multi-agent trajectory prediction serves as a critical component in autonomous driving systems, bridging environment perception, behavior understanding, and motion planning. Its outputs not only affect candidate trajectory evaluation and interactive...
Deep learning is increasingly integrated into oral rehabilitation workflows, particularly in implant planning, prosthodontic design automation, and peri-implant diagnosis. However, reported performance is heterogeneous and difficult to compare across...
Multiple Internet of Healthcare Things (IoHT)-based devices have been utilized as sensing methodologies for human locomotion decoding to aid in applications related to e-healthcare. Different measurement conditions affect the daily routine monitoring...
Urban housing rents are central to socioeconomic dynamics and urban sustainability, shaping affordability and quality of life. Existing research largely relies on linear models and focuses on economic, demographic, and locational factors, often negle...
Motor Imagery (MI) decoding based on electroencephalogram (EEG) is promising for Brain–Computer Interface (BCI) applications, yet existing methods generally suffer from two major limitations. (1) Global EEG signal analysis overlooks region-spec...
Sea fog can seriously affect schedules and safety by reducing visibility during marine transportation. Therefore, the forecasting of sea fog is an important issue in preventing accidents. Recently, in order to forecast sea fog, several deep learning...
The encoder–decoder framework is the main frame of image captioning. The convolutional neural network (CNN) is usually used to extract grid-level features of the image, and the graph convolutional neural network (GCN) is used to extract the ima...
Image captioning is oriented towards describing an image with the best possible use of words that can provide a semantic, relatable meaning of the scenario inscribed. Different models can be used to accomplish this arduous task depending on the conte...
Semantic segmentation plays a critical role in understanding complex urban environments, particularly for autonomous driving applications. However, existing approaches face significant challenges under low-light and adverse weather conditions. To add...
Referring image segmentation aims to segment an object described by a natural language expression from an image. Existing methods perform multi-modal fusion during encoding, typically integrating image and text features before predicting masks via up...
The recent advancements in multimodal dialogue systems have been gaining importance in several domains such as retail, travel, fashion, among others. Several existing works have improved the understanding and generation of multimodal dialogues. Howev...
Co-registered multimodal magnetic resonance imaging (MRI) contains redundancy within each volume and across an ordered set of contrasts. A lossless codec can exploit this dependence only if a standalone decoder can reproduce every prediction, context...
The accuracy and real-time performance of 3D object detection are key factors limiting its widespread application. While cameras capture detailed color and texture features, they lack depth information compared to LiDAR. Multimodal detection combinin...
The rapid development of sensor technology has made multi-modal remote sensing data valuable for land cover classification due to its diverse and complementary information. Many feature extraction methods for multi-modal data, combining light detecti...
Ulva prolifera detection is of great significance for marine ecological monitoring and green tide disaster prevention and control. Current single-modal detection methods have inherent limitations. Optical RGB imagery is highly vulnerable to cloud occ...
Magnetic resonance (MR) imaging is widely used in the clinical field due to its non-invasiveness, but the long scanning time is still a bottleneck for its popularization. Using the complementary information between multi-modal imaging to accelerate i...
Multivariate pattern analysis (MVPA) has played an extensive role in interpreting brain activity, which has been applied in studies with modalities such as functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG) and Electroencephal...
Breast ultrasound imaging is widely used for the detection and characterization of breast abnormalities; however, generating detailed and consistent radiological reports remains a labor-intensive and subjective process. Recent advances in deep learni...
Multimodal medical image fusion integrates complementary information from heterogeneous imaging modalities to provide comprehensive visual support for clinical analysis. Most existing methods adopt an “encode–fuse–decode” para...
The multimode underwater acoustic communication network is a novel form of underwater acoustic communication that adjusts its communication mode to enhance overall performance. Current performance analysis methods are primarily applied to single-mode...
AI-empowered smart sensor networks are driving Internet of Things (IoT) systems from passive data acquisition toward human-centric dynamic perception. In such scenarios, multimodal emotion recognition is expected to infer subtle affective states from...
Brain–computer interface (BCI) technologies for language decoding have emerged as a transformative bridge between neuroscience and artificial intelligence (AI), enabling direct neural–computational communication. The current literature pr...
Multimodal summarization (MS) generates text summaries from multimedia articles with textual and visual content. Therefore, MS can suffer from the multimodal factual inconsistency problem, where the generated summaries may distort or deviate from bot...
Directly applying brain signals to operate a mobile manned platform, such as a vehicle, may help people with neuromuscular disorders regain their driving ability. In this paper, we developed a novel electroencephalogram (EEG) signal-based driver&ndas...
Background: Sensor-based systems increasingly allow post-stroke rehabilitation devices to infer voluntary movement intention and respond to active effort, but approaches vary widely in sensor modality, target movement, recognition method, and applica...
Medical lesion segmentation remains challenging due to ambiguous boundaries, heterogeneous appearances, and the limited ability of visual features alone to capture clinically meaningful semantics. Existing vision-language segmentation methods have de...
Multimodal named entity recognition (MNER) leverages both textual and visual information to improve entity recognition, particularly in unstructured scenarios such as social media. While existing approaches predominantly rely on raster images (e.g.,...
The Trump era marked a turning point for political satire, where The Daily Show’s coverage of the last two election cycles reveals a dynamic interplay between satire, ideology, and democratic processes. This study examines how The Daily Show ha...
Remote sensing image semantic segmentation (RSISS) aims to assign a correct class label to each pixel in remote sensing images and has wide applications. With the development of artificial intelligence, RSISS based on deep learning has made significa...
Multi-modal ocular biometrics has recently garnered significant attention due to its potential in enhancing the security and reliability of biometric identification systems in non-constrained scenarios. However, accurately and efficiently segmenting...
The generalization of safety mechanisms from text-only LLMs to Large Vision–Language Models (LVLMs) remains poorly understood, especially concerning their vulnerability to typographic adversarial prompts. In this paper we ask where, inside a mo...
The widespread adoption of heartbeat monitoring sensors has increased the demand for secure and trustworthy multimodal cardiac monitoring systems capable of accurate heartbeat pattern recognition. While existing systems offer convenience, they often...
Trajectory prediction plays a crucial role in level autonomous driving systems, as real-time and accurate trajectory predictions can significantly enhance the safety of autonomous vehicles and the robustness of the autonomous driving system. We propo...
of 6