Next Article in Journal
Pallet Recognition with Multi-Task Learning for Automated Guided Vehicles
Next Article in Special Issue
Human Activity Signatures Captured under Different Directions Using SISO and MIMO Radar Systems
Previous Article in Journal
Development of a Novel Design Strategy for Moving Mechanisms Used in Multi-Material Plastic Injection Molds
Previous Article in Special Issue
Using Homemade Pressure Device to Improve Plantar Pressure—A Case Study on the Patient with Lower Limb Lymphedema
Article

A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors

1
Faculty of Software and Information Science, Iwate Prefectural University, Takizawa City 020-0693, Japan
2
Faculty of Systems Science and Technology, Akita Prefectural University, Yurihonjo City 015-0055, Japan
3
Institute of Engineering Innovation, Graduate School of Engineering, The University of Tokyo, Tokyo 113-8656, Japan
*
Author to whom correspondence should be addressed.
Academic Editor: Juan Ye
Appl. Sci. 2021, 11(24), 11807; https://doi.org/10.3390/app112411807
Received: 15 November 2021 / Revised: 8 December 2021 / Accepted: 10 December 2021 / Published: 12 December 2021
(This article belongs to the Special Issue Sensor-Based Human Activity Recognition in Real-World Scenarios)
Numerous methods and applications have been proposed in human activity recognition (HAR). This paper presents a mini-survey of recent HAR studies and our originally developed benchmark datasets of two types using environmental sensors. For the first dataset, we specifically examine human pose estimation and slight motion recognition related to activities of daily living (ADL). Our proposed method employs OpenPose. It describes feature vectors without effects of objects or scene features, but with a convolutional neural network (CNN) with the VGG-16 backbone, which recognizes behavior patterns after classifying the obtained images into learning and verification subsets. The first dataset comprises time-series panoramic images obtained using a fisheye lens monocular camera with a wide field of view. We attempted to recognize five behavior patterns: eating, reading, operating a smartphone, operating a laptop computer, and sitting. Even when using panoramic images including distortions, results demonstrate the capability of recognizing properties and characteristics of slight motions and pose-based behavioral patterns. The second dataset was obtained using five environmental sensors: a thermopile sensor, a CO2 sensor, and air pressure, humidity, and temperature sensors. Our proposed sensor system obviates the need for constraint; it also preserves each subject’s privacy. Using a long short-term memory (LSTM) network combined with CNN, which is a deep-learning model dealing with time-series features, we recognized eight behavior patterns: eating, operating a laptop computer, operating a smartphone, playing a game, reading, exiting, taking a nap, and sitting. The recognition accuracy for the second dataset was lower than for the first dataset consisting of images, but we demonstrated recognition of behavior patterns from time-series of weak sensor signals. The recognition results for the first dataset, after accuracy evaluation, can be reused for automatically annotated labels applied to the second dataset. Our proposed method actualizes semi-automatic annotation, false recognized category detection, and sensor calibration. Feasibility study results show the new possibility of HAR used for ADL based on unique sensors of two types. View Full-Text
Keywords: activities of daily living; behavior pattern; deep learning; human activity recognition; long short-term memory activities of daily living; behavior pattern; deep learning; human activity recognition; long short-term memory
Show Figures

Figure 1

MDPI and ACS Style

Madokoro, H.; Nix, S.; Woo, H.; Sato, K. A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors. Appl. Sci. 2021, 11, 11807. https://doi.org/10.3390/app112411807

AMA Style

Madokoro H, Nix S, Woo H, Sato K. A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors. Applied Sciences. 2021; 11(24):11807. https://doi.org/10.3390/app112411807

Chicago/Turabian Style

Madokoro, Hirokazu, Stephanie Nix, Hanwool Woo, and Kazuhito Sato. 2021. "A Mini-Survey and Feasibility Study of Deep-Learning-Based Human Activity Recognition from Slight Feature Signals Obtained Using Privacy-Aware Environmental Sensors" Applied Sciences 11, no. 24: 11807. https://doi.org/10.3390/app112411807

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop