- Article
Image–text multimodal deep semantic segmentation leverages the fusion and alignment of image and text information and provides more prior knowledge for segmentation tasks. It is worth exploring image–text multimodal semantic segmentation...
Image–text multimodal deep semantic segmentation leverages the fusion and alignment of image and text information and provides more prior knowledge for segmentation tasks. It is worth exploring image–text multimodal semantic segmentation...
Demand forecasting for emergency supplies is critical for public health emergency response, but demand sequences are strongly affected by external factors such as epidemic progression, traffic control, medical resource adjustments, and weather condit...
In the context of highly interconnected global markets and the rapid dissemination of multilingual information, traditional risk prediction methods that rely on single numerical sequences or monolingual text are insufficient for achieving early perce...
With the proliferation of social media platforms, misinformation has evolved toward more diverse modalities and complex cross-semantic correlations. Accurately detecting such content, particularly under conditions of semantic inconsistency and uneven...
Multimodal entity alignment identifies equivalent entities referring to the same real-world object across heterogeneous multimodal knowledge graphs and is essential for multi-source knowledge integration. Existing methods often rely on manually label...
Multimodal Knowledge Graphs (MMKGs) offer a promising paradigm for integrating heterogeneous sources into a unified, queryable, semantically structured representation. However, existing MMKG construction pipelines remain predominantly text-centric, e...
3D multi-object tracking (MOT) in complex and dynamic environments remains challenging due to the time-varying reliability of sensor modalities, severe occlusions, and the difficulty of distinguishing instances with similar appearances. Existing meth...
Under the growing demand for processing multimodal and cross-lingual information, traditional retrieval systems have encountered substantial limitations when handling heterogeneous inputs such as images, textual layouts, and multilingual language exp...
Multimodal amed entity recognition (MNER) is a natural language-processing technique that integrates text and visual modalities to detect and segment entity boundaries and their types from unstructured multimodal data. Although existing methods allev...
Recent vision-language pre-training models, like CLIP, have been shown to generalize well across a variety of multitask modalities. Nonetheless, their generalization for downstream tasks is limited. As a lightweight adaptation approach, prompt learni...
Multimodal sentiment analysis aims to infer sentiment polarity by jointly modeling textual and visual information. Despite recent advances in pretrained language and vision encoders, sentiment prediction from social media posts remains challenging be...
Multimodal sentiment analysis (MSA) benefits from integrating diverse modalities (e.g., text, video, and audio). However, challenges remain in effectively aligning non-text features and mitigating redundant information, which may limit potential perf...
In the era of big-data-driven multi-platform and multimodal health information dissemination, the rapid spread of false and misleading content poses a critical threat to public health awareness and decision making. To address this issue, a dual-strea...
Text-to-motion generation aims to synthesize realistic human motion sequences that accurately reflect natural language descriptions. While recent approaches have improved motion quality, achieving strong semantic alignment between text and motion, es...
Semantic segmentation of urban scenes from red–green–blue and thermal infrared imagery enables per-pixel categorization, delivering precise environmental understanding for autonomous driving and urban planning. However, existing methods s...
Farmland ecosystems present complex pest–predator co-occurrence patterns, posing significant challenges for image-based multitarget recognition and ecological modeling in sensor-driven computer vision tasks. To address these issues, this study...
Joint classification of hyperspectral images (HSI) and light detection and ranging (LiDAR) data is important for land-cover recognition, as it can exploit both spectral discrimination and structural elevation information. However, existing methods ma...
Tomato leaf diseases exhibit subtle inter-class differences and substantial intra-class variability, making accurate identification challenging for conventional deep learning models, especially under real-world conditions with diverse lighting, occlu...
Optical–elevation data fusion is widely used in aerial remote sensing semantic segmentation, as optical imagery provides rich spectral and textural information, while DSM or DEM data offer complementary elevation-related structural cues. Howeve...
Recent advances in Large Vision–Language Models (LVLMs) have demonstrated strong cross-modal reasoning capabilities, offering new opportunities for decision-making in autonomous driving. However, existing end-to-end approaches still suffer from...
Entity alignment is used to determine whether entities from different sources refer to the same object in the real world. It is one of the key technologies for constructing large-scale knowledge graphs and is widely used in the fields of knowledge gr...
Complex tasks in the real world involve different modal models, such as visual question answering (VQA). However, traditional multimodal learning requires a large amount of aligned data, such as image text pairs, and constructing a large amount of tr...
Multimodal Named Entity Recognition (MNER) aims to integrate textual and visual information to identify entities with specific semantic categories. However, existing methods often suffer from insufficient intra-modal semantic modeling, coarse cross-m...
To improve the accuracy and reliability of LiDAR semantic segmentation, previous studies have introduced multi-modal approaches that utilize additional modalities, such as 2D RGB images, to provide complementary information. However, these methods in...
Multimodal knowledge graph entity alignment is a key basic task of knowledge fusion and integration, which is used to identify entities with semantic equivalent but different representation forms in different knowledge graphs. Previous entity alignme...
The rapid development of social media has driven the need for opinion mining and sentiment analysis based on multimodal samples. As a fine-grained task within multimodal sentiment analysis, aspect-based multimodal sentiment analysis (ABMSA) enables t...
Remote sensing image semantic segmentation (RSISS) aims to assign a correct class label to each pixel in remote sensing images and has wide applications. With the development of artificial intelligence, RSISS based on deep learning has made significa...
Video-based pedestrian re-identification (Re-ID) is used to re-identify the same person across different camera views. One of the key problems is to learn an effective representation for the pedestrian from video. However, it is difficult to learn an...
Multimodal Arabic sentiment analysis has gained increasing attention due to the growing volume of user-generated multimedia content. However, effectively integrating textual, acoustic, and visual modalities remains challenging because of modality imb...
Existing seizure detection methods cannot fully exploit the spatiotemporal features of multimodal signals. They also fail to capture deep associations among cross-modal features. This limits their ability to learn unified representations of spatiotem...
Multimodal large language models and diffusion transformers have made text-to-video generation accessible to film production, advertising and social-media editing, but the assessment of generated video aesthetics remains poorly aligned with how viewe...
Multi-modal deep learning methods have achieved great improvements in visual grounding; their objective is to localize text-specified objects in images. Most of the existing methods can localize and classify objects with significant appearance differ...
The rapid growth of social media and online information-sharing platforms facilitates the spread of rumors. Accurate rumor detection to minimize manual verification efforts remains a critical research challenge. While multimodal rumor detection lever...
Domain shifts pose significant challenges for cross-domain semantic segmentation in high-resolution remote sensing imagery. Inspired by the cognitive mechanisms of the human brain, we propose a Brain-Inspired Style Transfer and Semantic Segmentation...
Multimodal Sequential recommendation alleviates the semantic insufficiency and data sparsity of item-ID-based models by incorporating side information such as text and images. However, multimodal systems face the dual challenges of feature-space hete...
The proliferation of multimodal misinformation on social media has become a critical concern. Although detection methods have advanced, feature representation and cross-modal semantic alignment challenges continue to hinder the effective use of multi...
The symmetry between different representation spaces plays a crucial role in effectively modeling complex multimodal data. To address the challenge of equipment knowledge graphs containing hierarchical relationships that cannot be fully represented i...
Image–text multimodal sentiment analysis aims to integrate textual and visual information to comprehensively understand sentiment expressions in complex scenarios. However, existing methods focus on cross-modal feature interaction and fusion, a...
Traditional multimodal contrastive learning brings text and its corresponding image closer together as a positive pair, where the text typically consists of fixed sentence structures or specific descriptive statements, and the image features are gene...
Multimodal Named Entity Recognition (MNER) and multimodal Relationship Extraction (MRE) play an important role in processing multimodal data and understanding entity relationships across textual and visual domains. However, irrelevant image informati...
With the increasing content richness of social media platforms, Multimodal Named Entity Recognition (MNER) faces the dual challenges of heterogeneous feature fusion and accurate entity recognition. Aiming at the key problems of inconsistent distribut...
Few-shot multimodal sentiment analysis (FMSA) has garnered substantial attention due to the proliferation of multimedia applications, especially given the frequent difficulty in obtaining large quantities of training samples. Previous works have dire...
Multimodal named entity recognition (MNER) leverages both textual and visual information to improve entity recognition, particularly in unstructured scenarios such as social media. While existing approaches predominantly rely on raster images (e.g.,...
Fine-grained Multimodal Named-Entity Recognition and Grounding (FMNERG) aims to extract entity–type–object triples from image–text pairs. However, existing methods still struggle with two major challenges: distinguishing semanticall...
Background: Drug–drug interactions (DDIs) account for about 30% of adverse drug reactions and 5–10% of hospital deaths. Combination therapy increases DDI risks, yet extracting DDIs from biomedical text remains challenging: existing method...
Diffusion-based image super-resolution (SR) has recently achieved impressive perceptual quality by progressively generating plausible high-resolution details. However, its restoration performance still depends strongly on the reliability of the condi...
Background: Integrating human neural signals with computational vision systems offers a promising route toward more robust visual recognition, yet supporting mixed-granularity recognition, where both coarse- and fine-grained categories must be distin...
The semantic segmentation of textured 3D meshes is a critical step in constructing city-scale realistic 3D models. Compared to colored point clouds, textured 3D meshes have the advantage of high-resolution texture image patches embedded on each mesh...
Multi-modal entity alignment (MMEA) identifies identical real-world entities across two multi-modal knowledge graphs. Most existing methods heavily rely on costly manually labeled seed alignments; in response, self-supervised MMEA has emerged to redu...
As a prominent topic in food computing, cross-modal recipe retrieval has garnered substantial attention. However, the semantic alignment across food images and recipes cannot be further enhanced due to the lack of intra-modal alignment in existing so...
of 7