Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.3
2.5
Journals
MTI
Special Issues
Multimodal Conversational Interaction and Interfaces, Volume II
share announcement

Multimodal Conversational Interaction and Interfaces, Volume II

A special issue of Multimodal Technologies and Interaction (ISSN 2414-4088).

Deadline for manuscript submissions: closed (30 June 2022) | Viewed by 7392

Special Issue Editors

Prof. Dr. Gabriel Murray

E-Mail Website
Guest Editor
School of Computing, University of the Fraser Valley, Abbotsford, BC V2S 7M8, Canada
Interests: natural language processing; speech processing; small group interaction; A.I. for health and wellness
Special Issues, Collections and Topics in MDPI journals
Prof. Dr. Catharine Oertel

E-Mail Website
Guest Editor
Department of Intelligent Systems, Interactive Intelligence, Delft University of Technology, 2628 XE Delft, The Netherlands
Interests: group interaction; social signal processing; social robot; educational technologies
Prof. Dr. Yukiko I. Nakano

E-Mail Website
Guest Editor
Department of Computer and Information Science, Faculty of Science and Technology, Seikei University, 3-3-1 Kichijoji-Kitamachi, Musashino-shi, Tokyo 180-8633, Japan
Interests: multimodal/multiparty interaction; intelligent user interfaces; conversational agents/robots; dialogue systems
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

We are pleased to announce the call for papers for a second Special Issue on Multimodal Conversational Interaction and Interfaces. This is prompted by the successful first Issue and the positive responses to the research featured therein. We thank all of the authors and reviewers who participated in the first Issue. The published papers included work on predicting the next speakers in multi-party conversations, predicting meeting participation based on turn-taking dynamics, identifying salient utterances for automatic meeting summarization, analysis of collaboration in small-group interactions, work on developing conversational robots, and analysis of how humans interact with museum exhibits. 

In conversational interactions, communicative behaviors between participants include verbal information as well as nonverbal signals such as gestures, facial expressions, and gaze. The combination and co-occurrence of verbal and nonverbal behaviors make conversations rich in complexity, and patterns of multimodal information become even more complex when moving from dyads to multi-party groups. Research on multimodal conversational interaction has frequently used machine learning in order to shed light on the complex processes and patterns that are inherent to these group communicative interactions. Sample applications of these technologies include artificial conversational agents, intelligent meeting assistants, and enhanced computer-mediated conversations. 

The purpose of this second Special Issue is to solicit contributions from both theoretical and practical perspectives, and to envision the future directions of research on multimodal interaction and its application to multimodal conversational interfaces. We encourage authors to submit original research articles on topics including but not limited to the following:

  • Theoretical and computational models that shed light on the process and characteristics of multimodal interaction;
  • New data-driven methodologies for investigating multimodal interactional data;
  • Interpretable machine-learning models for multimodal conversational interaction;
  • Virtual agents and humanoid robots with multimodal and/or multiparty conversational functionality;
  • Communication support systems that facilitate multimodal and/or multiparty conversation in computer-mediated communication;
  • Tools and platforms that contribute to research on multimodal interaction and building novel multimodal conversational interfaces.

Prof. Dr. Gabriel Murray
Prof. Dr. Catharine Oertel
Prof. Dr. Yukiko I. Nakano
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Multimodal Technologies and Interaction is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • verbal and nonverbal information
  • multiparty interaction
  • interpretable machine learning for conversational interaction
  • computational and statistical models
  • conversational virtual agents
  • communication robots
  • multimodal interfaces for human-human communication
  • social signal processing

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

18 pages, 3078 KiB  
Article
Inhibitors and Enablers to Explainable AI Success: A Systematic Examination of Explanation Complexity and Individual Characteristics
by Carolin Wienrich, Astrid Carolus, David Roth-Isigkeit and Andreas Hotho
Multimodal Technol. Interact. 2022, 6(12), 106; https://doi.org/10.3390/mti6120106 - 28 Nov 2022
Cited by 1 | Viewed by 1702
Abstract
With the increasing adaptability and complexity of advisory artificial intelligence (AI)-based agents, the topics of explainable AI and human-centered AI are moving close together. Variations in the explanation itself have been widely studied, with some contradictory results. These could be due to users’ [...] Read more.
With the increasing adaptability and complexity of advisory artificial intelligence (AI)-based agents, the topics of explainable AI and human-centered AI are moving close together. Variations in the explanation itself have been widely studied, with some contradictory results. These could be due to users’ individual differences, which have rarely been systematically studied regarding their inhibiting or enabling effect on the fulfillment of explanation objectives (such as trust, understanding, or workload). This paper aims to shed light on the significance of human dimensions (gender, age, trust disposition, need for cognition, affinity for technology, self-efficacy, attitudes, and mind attribution) as well as their interplay with different explanation modes (no, simple, or complex explanation). Participants played the game Deal or No Deal while interacting with an AI-based agent. The agent gave advice to the participants on whether they should accept or reject the deals offered to them. As expected, giving an explanation had a positive influence on the explanation objectives. However, the users’ individual characteristics particularly reinforced the fulfillment of the objectives. The strongest predictor of objective fulfillment was the degree of attribution of human characteristics. The more human characteristics were attributed, the more trust was placed in the agent, advice was more likely to be accepted and understood, and important needs were satisfied during the interaction. Thus, the current work contributes to a better understanding of the design of explanations of an AI-based agent system that takes into account individual characteristics and meets the demand for both explainable and human-centered agent systems. Full article
(This article belongs to the Special Issue Multimodal Conversational Interaction and Interfaces, Volume II)
►▼ Show Figures

Figure 1

14 pages, 5048 KiB  
Article
Towards Emotionally Expressive Virtual Human Agents to Foster L2 Production: Insights from a Preliminary Woz Experiment
by Emmanuel Ayedoun and Masataka Tokumaru
Multimodal Technol. Interact. 2022, 6(9), 77; https://doi.org/10.3390/mti6090077 - 08 Sep 2022
Cited by 1 | Viewed by 1888
Abstract
In second-language communication, emotional feedbacks play a preponderant role in instilling positive emotions and thereby facilitating the production of the target language by second-language learners. In contrast, facial expressions help convey emotion, intent, and sometimes even desired actions more effectively. Additionally, according to [...] Read more.
In second-language communication, emotional feedbacks play a preponderant role in instilling positive emotions and thereby facilitating the production of the target language by second-language learners. In contrast, facial expressions help convey emotion, intent, and sometimes even desired actions more effectively. Additionally, according to the facial feedback hypothesis, a major component of several contemporary theories of emotion, facial expressions can regulate emotional behavior and experience. The aim of this study was to determine whether and to what extent emotional expressions reproduced by virtual agents could provide empathetic support to second-language learners during communication tasks. To do so, using the Facial Coding Action System, we implemented a prototype virtual agent that can display a collection of nonverbal feedbacks, including Ekman’ six basic universal emotions and gazing and nodding behaviors. Then, we designed a Wizard of Oz experiment in which second-language learners were assigned independent speaking tasks with a virtual agent. In this paper, we outline our proposed method and report on an initial experimental evaluation which validated the meaningfulness of our approach. Moreover, we present our next steps for improving the system and validating its usefulness through large-scale experiments. Full article
(This article belongs to the Special Issue Multimodal Conversational Interaction and Interfaces, Volume II)
►▼ Show Figures

Figure 1

23 pages, 364 KiB  
Article
Perspectives on Socially Intelligent Conversational Agents
by Luisa Brinkschulte, Stephan Schlögl, Alexander Monz, Pascal Schöttle and Matthias Janetschek
Multimodal Technol. Interact. 2022, 6(8), 62; https://doi.org/10.3390/mti6080062 - 25 Jul 2022
Viewed by 2801
Abstract
The propagation of digital assistants is consistently progressing. Manifested by an uptake of ever more human-like conversational abilities, respective technologies are moving increasingly away from their role as voice-operated task enablers and becoming rather companion-like artifacts whose interaction style is rooted in anthropomorphic [...] Read more.
The propagation of digital assistants is consistently progressing. Manifested by an uptake of ever more human-like conversational abilities, respective technologies are moving increasingly away from their role as voice-operated task enablers and becoming rather companion-like artifacts whose interaction style is rooted in anthropomorphic behavior. One of the required characteristics in this shift from a utilitarian tool to an emotional character is the adoption of social intelligence. Although past research has recognized this need, more multi-disciplinary investigations should be devoted to the exploration of relevant traits and their potential embedding in future agent technology. Aiming to lay a foundation for further developments, we report on the results of a Delphi study highlighting the respective opinions of 21 multi-disciplinary domain experts. Results exhibit 14 distinctive characteristics of social intelligence, grouped into different levels of consensus, maturity, and abstraction, which may be considered a relevant basis, assisting the definition and consequent development of socially intelligent conversational agents. Full article
(This article belongs to the Special Issue Multimodal Conversational Interaction and Interfaces, Volume II)
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop