sensors-logo

Journal Browser

Journal Browser

Special Issue "Deep Learning Methods for Human Activity Recognition and Emotion Detection"

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Sensor Networks".

Deadline for manuscript submissions: 1 February 2021.

Special Issue Editor

Prof. Dr. Mario Munoz-Organero
Website
Guest Editor
Telematics Engineering Department, University Carlos III de Madrid, Av. Universidad, 30, Leganes 28911, Madrid, Spain
Interests: wearable technologies for health and wellbeing applications; mobile and pervasive computing for assistive living; Internet of Things and assistive technologies; machine learning algorithms for physiological; inertial and location sensors; personal assistants and coaching for health self-management; activity detection and prediction methods
Special Issues and Collections in MDPI journals

Special Issue Information

Dear Colleagues,

Detecting and characterizing human movements and activities is the base for providing contextual information while solving more complex challenges such as health self-management, personal recommender systems, object detection and manipulation, behavioral pattern recognition, and professional sport training. Human activities provide information about what the user does. Combining human activity recognition with emotion recognition enhances the contextual information to how the user feels while doing something and provides rich knowledge of context that is able to characterize both the physical and psychological wellbeing aspects of a person.

A wide range of machine learning methods have been applied over the last 20 years to try to automatically characterize human activities and emotions either based on visual information from environment cameras, embedded sensors in different tools and appliances, or wearable non-intrusive sensor devices. The proliferation of data together with the recent deep-learning-based methods have allowed the research community to achieve high-accuracy algorithms to detect human movements and emotions. This Special Issue is focused on papers that provide up-to-date information on either human activity and emotion detection or the combination of both using machine learning methods in different types of sensors. Both research and survey papers are welcome.

Prof. Dr. Mario Munoz-Organero
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2000 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Human activity recognition
  • Emotion recognition
  • Machine learning
  • Deep learning
  • Wearable sensors

Published Papers (4 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Open AccessArticle
An Innovative Multi-Model Neural Network Approach for Feature Selection in Emotion Recognition Using Deep Feature Clustering
Sensors 2020, 20(13), 3765; https://doi.org/10.3390/s20133765 - 05 Jul 2020
Abstract
Emotional awareness perception is a largely growing field that allows for more natural interactions between people and machines. Electroencephalography (EEG) has emerged as a convenient way to measure and track a user’s emotional state. The non-linear characteristic of the EEG signal produces a [...] Read more.
Emotional awareness perception is a largely growing field that allows for more natural interactions between people and machines. Electroencephalography (EEG) has emerged as a convenient way to measure and track a user’s emotional state. The non-linear characteristic of the EEG signal produces a high-dimensional feature vector resulting in high computational cost. In this paper, characteristics of multiple neural networks are combined using Deep Feature Clustering (DFC) to select high-quality attributes as opposed to traditional feature selection methods. The DFC method shortens the training time on the network by omitting unusable attributes. First, Empirical Mode Decomposition (EMD) is applied as a series of frequencies to decompose the raw EEG signal. The spatiotemporal component of the decomposed EEG signal is expressed as a two-dimensional spectrogram before the feature extraction process using Analytic Wavelet Transform (AWT). Four pre-trained Deep Neural Networks (DNN) are used to extract deep features. Dimensional reduction and feature selection are achieved utilising the differential entropy-based EEG channel selection and the DFC technique, which calculates a range of vocabularies using k-means clustering. The histogram characteristic is then determined from a series of visual vocabulary items. The classification performance of the SEED, DEAP and MAHNOB datasets combined with the capabilities of DFC show that the proposed method improves the performance of emotion recognition in short processing time and is more competitive than the latest emotion recognition methods. Full article
Show Figures

Graphical abstract

Open AccessArticle
Keys for Action: An Efficient Keyframe-Based Approach for 3D Action Recognition Using a Deep Neural Network
Sensors 2020, 20(8), 2226; https://doi.org/10.3390/s20082226 - 15 Apr 2020
Abstract
In this paper, we propose a novel and efficient framework for 3D action recognition using a deep learning architecture. First, we develop a 3D normalized pose space that consists of only 3D normalized poses, which are generated by discarding translation and orientation information. [...] Read more.
In this paper, we propose a novel and efficient framework for 3D action recognition using a deep learning architecture. First, we develop a 3D normalized pose space that consists of only 3D normalized poses, which are generated by discarding translation and orientation information. From these poses, we extract joint features and employ them further in a Deep Neural Network (DNN) in order to learn the action model. The architecture of our DNN consists of two hidden layers with the sigmoid activation function and an output layer with the softmax function. Furthermore, we propose a keyframe extraction methodology through which, from a motion sequence of 3D frames, we efficiently extract the keyframes that contribute substantially to the performance of the action. In this way, we eliminate redundant frames and reduce the length of the motion. More precisely, we ultimately summarize the motion sequence, while preserving the original motion semantics. We only consider the remaining essential informative frames in the process of action recognition, and the proposed pipeline is sufficiently fast and robust as a result. Finally, we evaluate our proposed framework intensively on publicly available benchmark Motion Capture (MoCap) datasets, namely HDM05 and CMU. From our experiments, we reveal that our proposed scheme significantly outperforms other state-of-the-art approaches. Full article
Show Figures

Figure 1

Open AccessArticle
Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition
Sensors 2020, 20(7), 1936; https://doi.org/10.3390/s20071936 - 30 Mar 2020
Cited by 1
Abstract
Understanding a person’s feelings is a very important process for the affective computing. People express their emotions in various ways. Among them, facial expression is the most effective way to present human emotional status. We propose efficient deep joint spatiotemporal features for facial [...] Read more.
Understanding a person’s feelings is a very important process for the affective computing. People express their emotions in various ways. Among them, facial expression is the most effective way to present human emotional status. We propose efficient deep joint spatiotemporal features for facial expression recognition based on the deep appearance and geometric neural networks. We apply three-dimensional (3D) convolution to extract spatial and temporal features at the same time. For the geometric network, 23 dominant facial landmarks are selected to express the movement of facial muscle through the analysis of energy distribution of whole facial landmarks.We combine these features by the designed joint fusion classifier to complement each other. From the experimental results, we verify the recognition accuracy of 99.21%, 87.88%, and 91.83% for CK+, MMI, and FERA datasets, respectively. Through the comparative analysis, we show that the proposed scheme is able to improve the recognition accuracy by 4% at least. Full article
Show Figures

Figure 1

Open AccessArticle
eXnet: An Efficient Approach for Emotion Recognition in the Wild
Sensors 2020, 20(4), 1087; https://doi.org/10.3390/s20041087 - 17 Feb 2020
Abstract
Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have [...] Read more.
Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk. Full article
Show Figures

Figure 1

Back to TopTop