Special Issue "Semantics of Multimodal Social Interaction"

A special issue of Multimodal Technologies and Interaction (ISSN 2414-4088).

Deadline for manuscript submissions: closed (1 July 2019).

Special Issue Editors

Dr. Volha Petukhova
E-Mail Website
Guest Editor
Saarland Univeristy, Saarbrücken, Germany
Interests: multimodal dialogue modelling; artificial intelligence; human–computer interaction; semantic annotations
Prof. Harry Bunt
E-Mail Website
Guest Editor
Tilburg Center for Cognition and Communication (TiCC), Tilburg University
Interests: computational semantics; dialogue modelling; semantic annotation; human–computer interaction; computational pragmatics
Prof. Laurent Prévot
E-Mail Website
Guest Editor
Laboratoire Parole et Langage, Aix-Marseille Université, France
Interests: discourse; dialogue; Natural Language Processing; computational linguistics; language resources

Special Issue Information

Dear Colleagues,

The increasing complexity of modern human–computer systems and interfaces results in an increasing demand for intelligent social interaction that is natural to users and exploits the full potential of spoken and multimodal communication. Today’s users have high expectations and require real-time engagement with highly relevant personalized content in a way which is comparable to natural human behaviour, and with the ability to adapt to a user’s cognitive and emotional states. Many real-life interactive situations involve more than the exchange of information, decision making, or problem-solving; they involve a wide range of aspects related to feelings, emotions, social status, power, interpersonal relations, and context. A comprehensive account of these aspects in computational models of multimodal interaction may form the foundation for a new generation of interactive social systems.

Over the past decade, a substantial amount of research has been devoted to the understanding of mechanisms governing multimodal social interaction. This Special Issue aims at presenting interdisciplinary research at the interface of artificial intelligence, computational linguistics, psycholinguistics, cognitive science, and human–computer interaction, focussing on the semantic and cognitive modelling of human social multimodal behaviour.

We invite contributions that deal with the modelling of human social behaviour and its interpretation. Topics range from multimodal behaviour recognition and social signals processing, analysis and annotation of multimodal interactive patterns, and semantic and pragmatic interpretation of multimodal interactive behaviour, to computational models of social cognition and multimodal dialogue.

Dr. Volha Petukhova
Prof. Harry Bunt
Prof. Laurent Prévot
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Multimodal Technologies and Interaction is an international peer-reviewed open access quarterly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1000 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.


  • Semantics and pragmatics of multimodal dialogue
  • Interpretation and reasoning in multimodal dialogue systems
  • Multimodal behaviour in social interaction: recognition, annotation, and analysis
  • Datasets, tools, and methodology
  • Multimodal dialogue modelling and management
  • Multimodal dialogue and social interactive systems
  • Cognitive modelling and multimodal interaction
  • Machine learning for multimodal interaction
  • Natural language interfaces for human–computer interaction
  • Multimodal behaviour interpretation and generation
  • New modalities and `beyond desktop' interfaces
  • Social multimodal interaction and industrial challenges

Published Papers (1 paper)

Order results
Result details
Select all
Export citation of selected articles as:


Open AccessArticle
A Dialogue-Act Taxonomy for a Virtual Coach Designed to Improve the Life of Elderly
Multimodal Technologies Interact. 2019, 3(3), 52; https://doi.org/10.3390/mti3030052 - 11 Jul 2019
Cited by 1
This paper presents a dialogue act taxonomy designed for the development of a conversational agent for elderly. The main goal of this conversational agent is to improve life quality of the user by means of coaching sessions in different topics. In contrast to [...] Read more.
This paper presents a dialogue act taxonomy designed for the development of a conversational agent for elderly. The main goal of this conversational agent is to improve life quality of the user by means of coaching sessions in different topics. In contrast to other approaches such as task-oriented dialogue systems and chit-chat implementations, the agent should display a pro-active attitude, driving the conversation to reach a number of diverse coaching goals. Therefore, the main characteristic of the introduced dialogue act taxonomy is its capacity for supporting a communication based on the GROW model for coaching. In addition, the taxonomy has a hierarchical structure between the tags and it is multimodal. We use the taxonomy to annotate a Spanish dialogue corpus collected from a group of elder people. We also present a preliminary examination of the annotated corpus and discuss on the multiple possibilities it presents for further research. Full article
(This article belongs to the Special Issue Semantics of Multimodal Social Interaction)
Show Figures

Figure 1

Back to TopTop