Skip to Content

12,605 Results Found

  • Editorial
  • Open Access
1 Citations
4,364 Views
4 Pages

Acknowledgment to Reviewers of Multimodal Technologies and Interaction in 2020

  • Multimodal Technologies and Interaction Editorial Office

Peer review is the driving force of journal development, and reviewers are gatekeepers who ensure that Multimodal Technologies and Interaction maintains its standards for the high quality of its published papers [...]

  • Article
  • Open Access
9 Citations
5,320 Views
14 Pages

Using Augmented Small Multimodal Models to Guide Large Language Models for Multimodal Relation Extraction

  • Wentao He,
  • Hanjie Ma,
  • Shaohua Li,
  • Hui Dong,
  • Haixiang Zhang and
  • Jie Feng

10 November 2023

Multimodal Relation Extraction (MRE) is a core task for constructing Multimodal Knowledge images (MKGs). Most current research is based on fine-tuning small-scale single-modal image and text pre-trained models, but we find that image-text datasets fr...

  • Review
  • Open Access
39 Citations
21,312 Views
32 Pages

Multimodal Interaction, Interfaces, and Communication: A Survey

  • Elias Dritsas,
  • Maria Trigka,
  • Christos Troussas and
  • Phivos Mylonas

Multimodal interaction is a transformative human-computer interaction (HCI) approach that allows users to interact with systems through various communication channels such as speech, gesture, touch, and gaze. With advancements in sensor technology an...

  • Article
  • Open Access
1,035 Views
15 Pages

Lightweight Multimodal Adapter for Visual Object Tracking

  • Vasyl Borsuk,
  • Vitaliy Yakovyna and
  • Nataliya Shakhovska

Visual object tracking is a fundamental computer vision task recently extended to multimodal settings, where natural language descriptions complement visual information. Existing multimodal trackers typically rely on large-scale transformer architect...

  • Article
  • Open Access
1 Citations
2,323 Views
22 Pages

26 September 2025

The widespread emergence of multimodal data on social platforms has presented new opportunities for sentiment analysis. However, previous studies have often overlooked the issue of detail loss during modal interaction fusion. They also exhibit limita...

  • Article
  • Open Access
20 Citations
5,899 Views
16 Pages

Multimodal Dynamic Journey-Planning

  • Kalliopi Giannakopoulou,
  • Andreas Paraskevopoulos and
  • Christos Zaroliagis

13 October 2019

In this paper, a new model, known as the multimodal dynamic timetable model (DTM), is presented for computing optimal multimodal journeys in schedule-based public transport systems. The new model constitutes an extension of the dynamic timetable mode...

  • Article
  • Open Access
8 Citations
3,267 Views
19 Pages

1 July 2022

With the increasing growth of multimedia data on the Internet, multimodal image aesthetic assessment has attracted a great deal of attention in the image processing community. However, traditional multimodal methods often have the following two probl...

  • Article
  • Open Access
1 Citations
2,265 Views
19 Pages

GC4MRec: Generative-Contrastive for Multimodal Recommendation

  • Lei Wang,
  • Yingjie Li,
  • Heran Wang and
  • Jun Li

27 March 2025

The rapid growth of information technology has led to an explosion of data, posing a significant challenge for data processing. Recommendation systems aim to address this by providing personalized content recommendations to users from vast datasets....

  • Article
  • Open Access
1 Citations
2,117 Views
19 Pages

29 December 2023

Multimodal interaction systems can provide users with natural and compelling interactive experiences. Despite the availability of various sensing devices, only some commercial multimodal applications are available. One reason may be the need for a mo...

  • Review
  • Open Access
17 Citations
2,721 Views
20 Pages

Multimodal Biosensing of Foodborne Pathogens

  • Najeeb Ullah,
  • Tracy Ann Bruce-Tagoe,
  • George Adu Asamoah and
  • Michael K. Danquah

Microbial foodborne pathogens present significant challenges to public health and the food industry, requiring rapid and accurate detection methods to prevent infections and ensure food safety. Conventional single biosensing techniques often exhibit...

  • Article
  • Open Access
5 Citations
3,582 Views
20 Pages

Fusion of Multimodal Imaging and 3D Digitization Using Photogrammetry

  • Roland Ramm,
  • Pedro de Dios Cruz,
  • Stefan Heist,
  • Peter Kühmstedt and
  • Gunther Notni

3 April 2024

Multimodal sensors capture and integrate diverse characteristics of a scene to maximize information gain. In optics, this may involve capturing intensity in specific spectra or polarization states to determine factors such as material properties or a...

  • Review
  • Open Access
100 Citations
21,010 Views
21 Pages

Multimodal Federated Learning: A Survey

  • Liwei Che,
  • Jiaqi Wang,
  • Yao Zhou and
  • Fenglong Ma

6 August 2023

Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image an...

  • Article
  • Open Access
41 Citations
16,282 Views
27 Pages

A Survey on Multimodal Knowledge Graphs: Construction, Completion and Applications

  • Yong Chen,
  • Xinkai Ge,
  • Shengli Yang,
  • Linmei Hu,
  • Jie Li and
  • Jinwen Zhang

11 April 2023

As an essential part of artificial intelligence, a knowledge graph describes the real-world entities, concepts and their various semantic relationships in a structured way and has been gradually popularized in a variety practical scenarios. The major...

  • Article
  • Open Access
11 Citations
5,585 Views
21 Pages

Multimodal Emotional Classification Based on Meaningful Learning

  • Hajar Filali,
  • Jamal Riffi,
  • Chafik Boulealam,
  • Mohamed Adnane Mahraz and
  • Hamid Tairi

Emotion recognition has become one of the most researched subjects in the scientific community, especially in the human–computer interface field. Decades of scientific research have been conducted on unimodal emotion analysis, whereas recent co...

  • Review
  • Open Access
105 Citations
19,925 Views
26 Pages

30 November 2020

Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Report...

  • Review
  • Open Access
4 Citations
5,193 Views
26 Pages

Antibody Aggregate Removal by Multimodal Chromatography

  • Veronika Rupčíková,
  • Tomáš Molnár,
  • Tomáš Kurák and
  • Milan Polakovič

The growing demand for therapeutic monoclonal antibodies (mAbs) has heightened the need for efficient and scalable purification strategies. A major challenge in downstream processing is the removal of antibody aggregates, which can compromise drug sa...

  • Review
  • Open Access
48 Citations
10,271 Views
21 Pages

Deep Multimodal Emotion Recognition on Human Speech: A Review

  • Panagiotis Koromilas and
  • Theodoros Giannakopoulos

28 August 2021

This work reviews the state of the art in multimodal speech emotion recognition methodologies, focusing on audio, text and visual information. We provide a new, descriptive categorization of methods, based on the way they handle the inter-modality an...

  • Article
  • Open Access
6 Citations
5,451 Views
16 Pages

28 July 2024

Research on recommendation methods using multimodal graph information presents a significant challenge within the realm of information services. Prior studies in this area have lacked precision in the purification and denoising of multimodal informat...

  • Article
  • Open Access
2 Citations
2,248 Views
16 Pages

21 July 2023

For multimodal multi-objective optimization problems (MMOPs), there are multiple equivalent Pareto optimal solutions in the decision space that are corresponding to the same objective value. Therefore, the main tasks of multimodal multi-objective opt...

  • Article
  • Open Access
1 Citations
2,332 Views
16 Pages

Multimodal emotion recognition has emerged as a prominent field in affective computing, offering superior performance compared to single-modality methods. Among various physiological signals, EEG signals and EOG data are highly valued for their compl...

  • Data Descriptor
  • Open Access
27 Citations
12,250 Views
8 Pages

MultimodalGasData: Multimodal Dataset for Gas Detection and Classification

  • Parag Narkhede,
  • Rahee Walambe,
  • Pulkit Chandel,
  • Shruti Mandaokar and
  • Ketan Kotecha

12 August 2022

The detection of gas leakages is a crucial aspect to be considered in the chemical industries, coal mines, home applications, etc. Early detection and identification of the type of gas is required to avoid damage to human lives and the environment. T...

  • Article
  • Open Access
122 Citations
26,757 Views
16 Pages

Effective Techniques for Multimodal Data Fusion: A Comparative Analysis

  • Maciej Pawłowski,
  • Anna Wróblewska and
  • Sylwia Sysko-Romańczuk

21 February 2023

Data processing in robotics is currently challenged by the effective building of multimodal and common representations. Tremendous volumes of raw data are available and their smart management is the core concept of multimodal learning in a new paradi...

  • Article
  • Open Access
2,380 Views
18 Pages

An Optimal-Transport-Based Multimodal Big Data Clustering

  • Zheng Yang,
  • Chongyang Shi and
  • Ying Guan

Multimodal clustering achieves outstanding performance in various applications by aggregating information from heterogeneous devices. However, previous methods rely on strong-notion distances to fuse crossmodal complementary knowledge, established on...

  • Article
  • Open Access
1 Citations
2,732 Views
19 Pages

11 May 2023

In classification tasks, such as face recognition and emotion recognition, multimodal information is used for accurate classification. Once a multimodal classification model is trained with a set of modalities, it estimates the class label by using t...

  • Article
  • Open Access
1,171 Views
29 Pages

CAGMC-Defence: A Cross-Attention-Guided Multimodal Collaborative Defence Method for Multimodal Remote Sensing Image Target Recognition

  • Jiahao Cui,
  • Hang Cao,
  • Lingquan Meng,
  • Wang Guo,
  • Keyi Zhang,
  • Qi Wang,
  • Cheng Chang and
  • Haifeng Li

25 September 2025

With the increasing diversity of remote sensing modalities, multimodal image fusion improves target recognition accuracy but also introduces new security risks. Adversaries can inject small, imperceptible perturbations into a single modality to misle...

  • Review
  • Open Access
23 Citations
15,593 Views
34 Pages

Multimodal Artificial Intelligence in Medical Diagnostics

  • Bassem Jandoubi and
  • Moulay A. Akhloufi

The integration of artificial intelligence into healthcare has advanced rapidly in recent years, with multimodal approaches emerging as promising tools for improving diagnostic accuracy and clinical decision making. These approaches combine heterogen...

  • Article
  • Open Access
88 Citations
19,160 Views
16 Pages

Multimodal Fake News Detection

  • Isabel Segura-Bedmar and
  • Santiago Alonso-Bartolome

Over the last few years, there has been an unprecedented proliferation of fake news. As a consequence, we are more susceptible to the pernicious impact that misinformation and disinformation spreading can have on different segments of our society. Th...

  • Review
  • Open Access
48 Citations
12,524 Views
30 Pages

Reviewing Multimodal Machine Learning and Its Use in Cardiovascular Diseases Detection

  • Mohammad Moshawrab,
  • Mehdi Adda,
  • Abdenour Bouzouane,
  • Hussein Ibrahim and
  • Ali Raad

Machine Learning (ML) and Deep Learning (DL) are derivatives of Artificial Intelligence (AI) that have already demonstrated their effectiveness in a variety of domains, including healthcare, where they are now routinely integrated into patients&rsquo...

  • Article
  • Open Access
1 Citations
2,787 Views
15 Pages

20 October 2024

Multimodal summarization, a rapidly evolving field within multimodal learning, focuses on generating cohesive summaries by integrating information from diverse modalities, such as text and images. Unlike traditional unimodal summarization, multimodal...

  • Article
  • Open Access
1 Citations
2,354 Views
29 Pages

30 June 2023

In social interactions, people who are perceived as competent win more chances, tend to have more opportunities, and perform better in both personal and professional aspects of their lives. However, the process of evaluating competence is still poorl...

  • Review
  • Open Access
10 Citations
12,289 Views
37 Pages

A Comprehensive Review of Multimodal Analysis in Education

  • Jared D. T. Guerrero-Sosa,
  • Francisco P. Romero,
  • Víctor H. Menéndez-Domínguez,
  • Jesus Serrano-Guerrero,
  • Andres Montoro-Montarroso and
  • Jose A. Olivas

23 May 2025

Multimodal learning analytics (MMLA) has become a prominent approach for capturing the complexity of learning by integrating diverse data sources such as video, audio, physiological signals, and digital interactions. This comprehensive review synthes...

  • Article
  • Open Access
6 Citations
3,394 Views
21 Pages

28 November 2022

Multimodal sentiment analysis, which aims to recognize the emotions expressed in multimodal data, has attracted extensive attention in both academia and industry. However, most of the current studies on user-generated reviews classify the overall sen...

  • Article
  • Open Access
56 Citations
17,518 Views
18 Pages

A Robust Approach to Multimodal Deepfake Detection

  • Davide Salvi,
  • Honggu Liu,
  • Sara Mandelli,
  • Paolo Bestagini,
  • Wenbo Zhou,
  • Weiming Zhang and
  • Stefano Tubaro

The widespread use of deep learning techniques for creating realistic synthetic media, commonly known as deepfakes, poses a significant threat to individuals, organizations, and society. As the malicious use of these data could lead to unpleasant sit...

  • Article
  • Open Access
10 Citations
5,391 Views
31 Pages

Multimodal Classification of Safety-Report Observations

  • Georgios Paraskevopoulos,
  • Petros Pistofidis,
  • Georgios Banoutsos,
  • Efthymios Georgiou and
  • Vassilis Katsouros

7 June 2022

Modern businesses are obligated to conform to regulations to prevent physical injuries and ill health for anyone present on a site under their responsibility, such as customers, employees and visitors. Safety officers (SOs) are engineers, who perform...

  • Article
  • Open Access
6 Citations
3,450 Views
14 Pages

Transformer-Based Multimodal Infusion Dialogue Systems

  • Bo Liu,
  • Lejian He,
  • Yafei Liu,
  • Tianyao Yu,
  • Yuejia Xiang,
  • Li Zhu and
  • Weijian Ruan

20 October 2022

The recent advancements in multimodal dialogue systems have been gaining importance in several domains such as retail, travel, fashion, among others. Several existing works have improved the understanding and generation of multimodal dialogues. Howev...

  • Article
  • Open Access
11 Citations
9,440 Views
17 Pages

This article is concerned with digital, multimodal feedback that supports learning and assessment within education. Drawing on the research literature alongside a case study from a postgraduate program in digital education, I argue that approaching f...

  • Review
  • Open Access
9 Citations
5,995 Views
11 Pages

An Overview of Multimodal Neuroimaging Using Nanoprobes

  • Sriram Sridhar,
  • Sachin Mishra,
  • Miklós Gulyás,
  • Parasuraman Padmanabhan and
  • Balázs Gulyás

Nanomaterials have gained tremendous significance as contrast agents for both anatomical and functional preclinical bio-imaging. Contrary to conventional medical practices, molecular imaging plays an important role in exploring the affected cells, th...

  • Article
  • Open Access
7 Citations
5,923 Views
21 Pages

27 October 2023

Spam detection has been a topic of extensive research; however, there has been limited focus on multimodal spam detection. In this study, we introduce a novel approach for multilingual multimodal spam detection, presenting the Multilingual and Multim...

  • Article
  • Open Access
213 Views
24 Pages

14 February 2026

Multimodal named-entity recognition (MNER) aims to identify entity information by leveraging multimodal features. With recent research shifting to multi-image scenarios, existing methods overlook modality noise and lack effective cross-modal interact...

  • Article
  • Open Access
17 Citations
7,515 Views
29 Pages

19 February 2021

Multimodal freight transport in cities is a complex, valid, and vitally important problem. It is more seldom underlined in scientific studies and included in cities’ strategies that devote more attention to passenger transport than freight transport....

  • Article
  • Open Access
4 Citations
3,485 Views
22 Pages

Research on Multimodal Transport of Electronic Documents Based on Blockchain

  • Xueqi Qian,
  • Lixin Shen,
  • Dong Yang,
  • Zhiwen Zhang and
  • Zhihong Jin

Multimodal transport document collaboration is the foundation of multimodal transport operations. Blockchain technology can effectively address issues such as a lack of trust and difficulties in information sharing in current multimodal transport doc...

  • Article
  • Open Access
1 Citations
2,297 Views
20 Pages

Historically, the field of discourse marker research has moved from relying on intuition to more and more ecological data, with written, spoken, and now multimodal corpora available to study these pervasive pragmatic devices. For some topics, video i...

  • Review
  • Open Access
86 Citations
20,527 Views
34 Pages

Technologies for Multimodal Interaction in Extended Reality—A Scoping Review

  • Ismo Rakkolainen,
  • Ahmed Farooq,
  • Jari Kangas,
  • Jaakko Hakulinen,
  • Jussi Rantala,
  • Markku Turunen and
  • Roope Raisamo

When designing extended reality (XR) applications, it is important to consider multimodal interaction techniques, which employ several human senses simultaneously. Multimodal interaction can transform how people communicate remotely, practice for tas...

  • Review
  • Open Access
8 Citations
5,698 Views
14 Pages

Data Governance in Multimodal Behavioral Research

  • Zhehan Jiang,
  • Zhengzhou Zhu and
  • Shucheng Pan

In the digital era, multimodal behavioral research has emerged as a pivotal discipline, integrating diverse data sources to comprehensively understand human behavior. This paper defines and distinguishes data governance from mere data management with...

  • Article
  • Open Access
11 Citations
5,961 Views
18 Pages

23 January 2014

A numerical algorithm to compute the topological entropy of multimodal maps is proposed. This algorithm results from a closed formula containing the so-called min-max symbols, which are closely related to the kneading symbols. Furthermore, it simplif...

  • Article
  • Open Access
5 Citations
2,734 Views
14 Pages

25 September 2024

This paper explores the challenges of finding robust shortest paths in multimodal transportation networks. With the increasing complexity and uncertainties in modern transportation systems, developing efficient and reliable routing strategies that ca...

of 253