Next Article in Journal
Wearable Sensors for Assessing the Role of Olfactory Training on the Autonomic Response to Olfactory Stimulation
Previous Article in Journal
Positional Differences in Pre-Season Scrimmage Performance of Division I Collegiate Football Players
Article

Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors

1
Institute of Mathematical and Computer Sciences, University of Sao Paulo, Sao Carlos, SP 13566-590, Brazil
2
Edinburgh Centre for Robotics, Heriot-Watt University, Edinburgh, EH14 4AS, UK
*
Author to whom correspondence should be addressed.
Academic Editor: Susanna Spinsante
Sensors 2021, 21(3), 768; https://doi.org/10.3390/s21030768
Received: 15 December 2020 / Revised: 8 January 2021 / Accepted: 21 January 2021 / Published: 24 January 2021
(This article belongs to the Section Intelligent Sensors)
Worldwide demographic projections point to a progressively older population. This fact has fostered research on Ambient Assisted Living, which includes developments on smart homes and social robots. To endow such environments with truly autonomous behaviours, algorithms must extract semantically meaningful information from whichever sensor data is available. Human activity recognition is one of the most active fields of research within this context. Proposed approaches vary according to the input modality and the environments considered. Different from others, this paper addresses the problem of recognising heterogeneous activities of daily living centred in home environments considering simultaneously data from videos, wearable IMUs and ambient sensors. For this, two contributions are presented. The first is the creation of the Heriot-Watt University/University of Sao Paulo (HWU-USP) activities dataset, which was recorded at the Robotic Assisted Living Testbed at Heriot-Watt University. This dataset differs from other multimodal datasets due to the fact that it consists of daily living activities with either periodical patterns or long-term dependencies, which are captured in a very rich and heterogeneous sensing environment. In particular, this dataset combines data from a humanoid robot’s RGBD (RGB + depth) camera, with inertial sensors from wearable devices, and ambient sensors from a smart home. The second contribution is the proposal of a Deep Learning (DL) framework, which provides multimodal activity recognition based on videos, inertial sensors and ambient sensors from the smart home, on their own or fused to each other. The classification DL framework has also validated on our dataset and on the University of Texas at Dallas Multimodal Human Activities Dataset (UTD-MHAD), a widely used benchmark for activity recognition based on videos and inertial sensors, providing a comparative analysis between the results on the two datasets considered. Results demonstrate that the introduction of data from ambient sensors expressively improved the accuracy results. View Full-Text
Keywords: human activity recognition; multimodal datasets; deep learning; video classification; inertial sensors; human–robot interaction human activity recognition; multimodal datasets; deep learning; video classification; inertial sensors; human–robot interaction
Show Figures

Figure 1

MDPI and ACS Style

Ranieri, C.M.; MacLeod, S.; Dragone, M.; Vargas, P.A.; Romero , R.F. Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors. Sensors 2021, 21, 768. https://doi.org/10.3390/s21030768

AMA Style

Ranieri CM, MacLeod S, Dragone M, Vargas PA, Romero  RF. Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors. Sensors. 2021; 21(3):768. https://doi.org/10.3390/s21030768

Chicago/Turabian Style

Ranieri, Caetano M., Scott MacLeod, Mauro Dragone, Patricia A. Vargas, and Roseli Aparecida F. Romero . 2021. "Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors" Sensors 21, no. 3: 768. https://doi.org/10.3390/s21030768

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop