You are currently viewing a new version of our website. To view the old version click .

18 Results Found

  • Article
  • Open Access
4 Citations
3,208 Views
14 Pages

Transformer-Based Multimodal Infusion Dialogue Systems

  • Bo Liu,
  • Lejian He,
  • Yafei Liu,
  • Tianyao Yu,
  • Yuejia Xiang,
  • Li Zhu and
  • Weijian Ruan

20 October 2022

The recent advancements in multimodal dialogue systems have been gaining importance in several domains such as retail, travel, fashion, among others. Several existing works have improved the understanding and generation of multimodal dialogues. Howev...

  • Systematic Review
  • Open Access
1 Citations
6,043 Views
28 Pages

In the current context, the use of technologies in applications for multimodal dialogue systems with computers and emotion recognition through artificial intelligence continues to grow rapidly. Consequently, it is challenging for researchers to ident...

  • Article
  • Open Access
1 Citations
1,270 Views
22 Pages

Influence of Personality Traits and Demographics on Rapport Recognition Using Adversarial Learning

  • Wenqing Wei,
  • Sixia Li,
  • Candy Olivia Mawalim,
  • Xiguang Li,
  • Kazunori Komatani and
  • Shogo Okada

The automatic recognition of user rapport at the dialogue level for multimodal dialogue systems (MDSs) is a critical component of effective dialogue system management. Both the dialogue systems and their evaluations need to be based on user expressio...

  • Article
  • Open Access
2,029 Views
14 Pages

19 April 2025

Sentiment analysis is pivotal in advancing human–computer interaction (HCI) systems as it enables emotionally intelligent responses. While existing models show potential for HCI applications, current conversational datasets exhibit critical lim...

  • Article
  • Open Access
2 Citations
7,532 Views
17 Pages

Mitigating LLM Hallucinations Using a Multi-Agent Framework

  • Ahmed M. Darwish,
  • Essam A. Rashed and
  • Ghada Khoriba

21 June 2025

The rapid advancement of Large Language Models (LLMs) has led to substantial investment in enhancing their capabilities and expanding their feature sets. Despite these developments, a critical gap remains between model sophistication and their depend...

  • Article
  • Open Access
24 Citations
6,696 Views
19 Pages

A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly

  • César Montenegro,
  • Asier López Zorrilla,
  • Javier Mikel Olaso,
  • Roberto Santana,
  • Raquel Justo,
  • Jose A. Lozano and
  • María Inés Torres

This paper presents a dialogue act taxonomy designed for the development of a conversational agent for elderly. The main goal of this conversational agent is to improve life quality of the user by means of coaching sessions in different topics. In co...

  • Article
  • Open Access
28 Citations
5,386 Views
30 Pages

Modelling Multimodal Dialogues for Social Robots Using Communicative Acts

  • Enrique Fernández-Rodicio,
  • Álvaro Castro-González,
  • Fernando Alonso-Martín,
  • Marcos Maroto-Gómez and
  • Miguel Á. Salichs

18 June 2020

Social Robots need to communicate in a way that feels natural to humans if they are to effectively bond with the users and provide an engaging interaction. Inline with this natural, effective communication, robots need to perceive and manage multimod...

  • Article
  • Open Access
7 Citations
3,401 Views
15 Pages

MMATERIC: Multi-Task Learning and Multi-Fusion for AudioText Emotion Recognition in Conversation

  • Xingwei Liang,
  • You Zou,
  • Xinnan Zhuang,
  • Jie Yang,
  • Taiyu Niu and
  • Ruifeng Xu

The accurate recognition of emotions in conversations helps understand the speaker’s intentions and facilitates various analyses in artificial intelligence, especially in human–computer interaction systems. However, most previous methods need more ab...

  • Article
  • Open Access
2 Citations
2,396 Views
20 Pages

An Architecture for Intelligent Tutoring in Virtual Reality: Integrating LLMs and Multimodal Interaction for Immersive Learning

  • Mohamed El Hajji,
  • Tarek Ait Baha,
  • Anas Berka,
  • Hassan Ait Nacer,
  • Houssam El Aouifi and
  • Youssef Es-Saady

29 June 2025

Immersive learning has been recognized as a promising paradigm for enhancing educational experiences through the integration of VR. We propose an architecture for intelligent tutoring in immersive VR environments that employs LLM-based non-playable c...

  • Article
  • Open Access
34 Citations
4,562 Views
14 Pages

Dialogue Enhanced Extended Reality: Interactive System for the Operator 4.0

  • Manex Serras,
  • Laura García-Sardiña,
  • Bruno Simões,
  • Hugo Álvarez and
  • Jon Arambarri

7 June 2020

The nature of industrial manufacturing processes and the continuous need to adapt production systems to new demands require tools to support workers during transitions to new processes. At the early stage of transitions, human error rate is often hig...

  • Review
  • Open Access
24 Citations
6,890 Views
31 Pages

On the Gap between Domestic Robotic Applications and Computational Intelligence

  • Junpei Zhong,
  • Chaofan Ling,
  • Angelo Cangelosi,
  • Ahmad Lotfi and
  • Xiaofeng Liu

Aspired to build intelligent agents that can assist humans in daily life, researchers and engineers, both from academia and industry, have kept advancing the state-of-the-art in domestic robotics. With the rapid advancement of both hardware (e.g., hi...

  • Review
  • Open Access
1 Citations
3,564 Views
22 Pages

Microglia are exceptionally dynamic resident innate immune cells within the central nervous system, existing on a continuum of morphologies and functions throughout their lifespan. They play vital roles in response to injuries and infections, clearin...

  • Article
  • Open Access
1,944 Views
27 Pages

“Found in Translation”: An Evolutionary Framework for Auditory–Visual Relationships

  • Ana Rodrigues,
  • Bruna Sousa,
  • Amílcar Cardoso and
  • Penousal Machado

22 November 2022

The development of computational artifacts to study cross-modal associations has been a growing research topic, as they allow new degrees of abstraction. In this context, we propose a novel approach to the computational exploration of relationships b...

  • Article
  • Open Access
16 Citations
3,638 Views
26 Pages

Investigating Methods for Cognitive Workload Estimation for Assistive Robots

  • Ayca Aygun,
  • Thuan Nguyen,
  • Zachary Haga,
  • Shuchin Aeron and
  • Matthias Scheutz

9 September 2022

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing mod...

  • Article
  • Open Access
1 Citations
1,466 Views
28 Pages

Alquist 5.0: Dialogue Trees Meet Generative Models, a Novel Approach for Enhancing SocialBot Conversations

  • Ondrej Kobza,
  • David Herel,
  • Jan Cuhel,
  • Tommaso Gargiani,
  • Petr Marek and
  • Jan Sedivy

21 September 2024

This article introduces Alquist 5.0, our SocialBot that was designed for the Alexa Prize SocialBot Grand Challenge 5. Building upon previous iterations, we present the integration of our novel neural response generator (NRG) Barista within a hybrid a...

  • Article
  • Open Access
4 Citations
3,152 Views
20 Pages

Designing Social Robots with LLMs for Engaging Human Interaction

  • Maria Pinto-Bernal,
  • Matthijs Biondina and
  • Tony Belpaeme

5 June 2025

Large Language Models (LLMs), particularly those enhanced through Reinforcement Learning from Human Feedback, such as ChatGPT, have opened up new possibilities for natural and open-ended spoken interaction in social robotics. However, these models ar...

  • Article
  • Open Access
42 Citations
6,406 Views
15 Pages

21 January 2020

In Europe, the share of people aged 65 years and above is expected to increase exponentially, and in 2050, for the first time in human history, the number of older people will be larger than the number of children under the age of 15. Aging is associ...