Next Article in Journal
Exploring User Perception Challenges in Vibrotactile Haptic Display Using Resonant Microbeams under Contact with Skin
Previous Article in Journal
Better Sleep Experience for the Critically Ill: A Comprehensive Strategy for Designing Hospital Soundscapes
Previous Article in Special Issue
Semantic Fusion for Natural Multimodal Interfaces using Concurrent Augmented Transition Networks
Open AccessArticle

A Survey on Psycho-Physiological Analysis & Measurement Methods in Multimodal Systems

Department of Computing, Faculty of Science and Engineering, Macquaire University, Sydney 2109, Australia
*
Author to whom correspondence should be addressed.
Multimodal Technologies Interact. 2019, 3(2), 37; https://doi.org/10.3390/mti3020037
Received: 23 April 2019 / Revised: 8 May 2019 / Accepted: 14 May 2019 / Published: 28 May 2019
(This article belongs to the Special Issue Multimodal User Interfaces Modelling and Development)
Psycho-physiological analysis has gained greater attention in the last few decades in various fields including multimodal systems. Researchers use psychophysiological feedback devices such as skin conductance (SC), Electroencephalography (EEG) and Electrocardiography (ECG) to detect the affective states of the users during task performance. Psycho-physiological feedback has been successful in detection of the cognitive states of users in human-computer interaction (HCI). Recently, in game studies, psycho-physiological feedback has been used to capture the user experience and the effect of interaction on human psychology. This paper reviews several psycho-physiological, cognitive, and affective assessment studies and focuses on the use of psychophysiological signals in estimating the user’s cognitive and emotional states in multimodal systems. In this paper, we review the measurement techniques and methods that have been used to record psycho-physiological signals as well as the cognitive and emotional states in a variety of conditions. The aim of this review is to conduct a detailed study to identify, describe and analyze the key psycho-physiological parameters that relate to different mental and emotional states in order to provide an insight into key approaches. Furthermore, the advantages and limitations of these approaches are also highlighted in this paper. The findings state that the classification accuracy of >90% has been achieved in classifying emotions with EEG signals. A strong correlation between self-reported data, HCI experience, and psychophysiological data has been observed in a wide range of domains including games, human-robot interaction, mobile interaction, and simulations. An increase in β and γ -band activity have been observed in high intense games and simulations. View Full-Text
Keywords: psycho-physiological analysis; multimodal interaction; mental load; cognitive activity; emotion recognition; EEG; gaming systems psycho-physiological analysis; multimodal interaction; mental load; cognitive activity; emotion recognition; EEG; gaming systems
Show Figures

Figure 1

MDPI and ACS Style

Baig, M.Z.; Kavakli, M. A Survey on Psycho-Physiological Analysis & Measurement Methods in Multimodal Systems. Multimodal Technologies Interact. 2019, 3, 37.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop