Next Article in Journal
A Survey of the Techniques for The Identification and Classification of Human Actions from Visual Data
Next Article in Special Issue
Transmission Optimization of Social and Physical Sensor Nodes via Collaborative Beamforming in Cyber-Physical-Social Systems
Previous Article in Journal
Erratum: Zalameda, J. et al. Detection and Characterization of Damage in Quasi-Static Loaded Composite Structures using Passive Thermography. Sensors 2018, 18, 3562
Previous Article in Special Issue
Robust Iterative Distributed Minimum Total MSE Algorithm for Secure Communications in the Internet of Things Using Relays
Article Menu
Issue 11 (November) cover image

Export Article

Open AccessArticle
Sensors 2018, 18(11), 3978; https://doi.org/10.3390/s18113978

EmoTour: Estimating Emotion and Satisfaction of Users Based on Behavioral Cues and Audiovisual Data

1
Graduate School of Information Science, Nara Institute of Science and Technology, Nara 630-0192, Japan
2
Fellow of Japan Society for the Promotion of Science, Tokyo 102-0083, Japan
3
RIKEN, Center for Advanced Intelligence Project AIP, Tokyo 103-0027, Japan
4
Institute of Communications Engineering, Ulm University, 89081 Ulm, Germany
5
ITMO University, Saint Petersburg 197101, Russia
6
JST Presto, Tokyo 102-0076, Japan
*
Author to whom correspondence should be addressed.
Received: 16 October 2018 / Revised: 7 November 2018 / Accepted: 12 November 2018 / Published: 15 November 2018
(This article belongs to the Special Issue Exploiting the IoT within Cyber Physical Social System)
Full-Text   |   PDF [12335 KB, uploaded 21 November 2018]   |  
  |   Review Reports

Abstract

With the spread of smart devices, people may obtain a variety of information on their surrounding environment thanks to sensing technologies. To design more context-aware systems, psychological user context (e.g., emotional status) is a substantial factor for providing useful information in an appropriate timing. As a typical use case that has a high demand for context awareness but is not tackled widely yet, we focus on the tourism domain. In this study, we aim to estimate the emotional status and satisfaction level of tourists during sightseeing by using unconscious and natural tourist actions. As tourist actions, behavioral cues (eye and head/body movement) and audiovisual data (facial/vocal expressions) were collected during sightseeing using an eye-gaze tracker, physical-activity sensors, and a smartphone. Then, we derived high-level features, e.g., head tilt and footsteps, from behavioral cues. We also used existing databases of emotionally rich interactions to train emotion-recognition models and apply them in a cross-corpus fashion to generate emotional-state prediction for the audiovisual data. Finally, the features from several modalities are fused to estimate the emotion of tourists during sightseeing. To evaluate our system, we conducted experiments with 22 tourists in two different touristic areas located in Germany and Japan. As a result, we confirmed the feasibility of estimating both the emotional status and satisfaction level of tourists. In addition, we found that effective features used for emotion and satisfaction estimation are different among tourists with different cultural backgrounds. View Full-Text
Keywords: ubiquitous computing; emotion recognition; satisfaction estimation; wearable computing; dialogue systems; smart tourism; smart cities ubiquitous computing; emotion recognition; satisfaction estimation; wearable computing; dialogue systems; smart tourism; smart cities
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Matsuda, Y.; Fedotov, D.; Takahashi, Y.; Arakawa, Y.; Yasumoto, K.; Minker, W. EmoTour: Estimating Emotion and Satisfaction of Users Based on Behavioral Cues and Audiovisual Data. Sensors 2018, 18, 3978.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Sensors EISSN 1424-8220 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top