Submit to Sensors Review for Sensors Propose a Special Issue

Journal Browser

► Journal Browser

Signal, Image Processing and Computer Vision in Smart Living Applications

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Related Special Issue
Published Papers

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Intelligent Sensors".

Deadline for manuscript submissions: closed (20 July 2022) | Viewed by 40912

Share This Special Issue

Special Issue Editor

Dr. Alessandro Leone

E-Mail Website
Guest Editor

National Research Council of Italy, Institute for Microelectronics and Microsystems, 73100 Lecce, Italy
Interests: ambient assisted living; active&healthy ageing technologies; signal processing; image processing; computer vision
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

Smart spaces and ubiquitous computing extend pervasive computing capabilities to everyday objects, providing context-aware services in smart living environments. One of the main aspects is to build smart environments integrating information from independent multi-sensor systems, including cameras and ranging devices. “Smart Living Technologies" aims to make all the environments in which people spend their time (at home, at work, in mobility, etc.) more adapted to the needs of those persons, whether they are in good physical condition in terms of frailty and disability, disease and social exclusion, in different age groups (children, adults or elderly people, in poor health, etc.).

The Special Issue refers to the use of key enabling technologies and smart system integration for the development of advanced technological solutions for the realization of products (sensors, devices, etc.) and services, which include Ambient Assisted Living, Ambient Intelligence and IoT paradigms, and reframing the sense of “Smart Living” to ensure inclusion, safety, comfort, care, health care and environmental sustainability. The creation of smart devices and services pass through innovation in signal processing, image processing and computer vision techniques. The Special Issue aims to cover technological issues related to the integration of processing aspects in smart living environments. We invite papers that include but are not exclusive to the following topics:

Artificial Intelligence
Pattern Recognition/Analysis
Biometrics
Human Analysis
Behavior Understanding
Computer Vision
Robotics and Intelligent Systems
Document and Media Analysis
Image Processing
Signal Processing
Soft Computing Techniques
Ambient Intelligence
Context-aware Computing
Machine Learning
Deep Learning
Embedded Systems and Devices
Human-Computer Interfaces
Innovative Sensing Devices and Applications
Sensor Networks and Mobile Ad-Hoc Networks
Security and Privacy Techniques

The sequel Special Issue “Signal, Image Processing and Computer Vision in Smart Living Applications: Part II” has been announced. We look forward to receiving your submission for the new Special Issue.
https://www.mdpi.com/journal/sensors/special_issues/C0A868YEO3
Deadline for manuscript submissions: 20 March 2023.

Dr. Alessandro Leone
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Signal processing
image processing
computer vision
embedded systems
ubiquitous computing
multi-sensor systems
artificial intelligence
pattern recognition
deep learning
ambient assisted living
active and healthy ageing
health-care applications
human computer interaction

Related Special Issue

Signal, Image Processing and Computer Vision in Smart Living Applications: Part II in Sensors (20 articles)

Published Papers (13 papers)

Download All Papers

Research

18 pages, 1976 KiB

Open AccessArticle

On the Use of Assistive Technology during the COVID-19 Outbreak: Results and Lessons Learned from Pilot Studies

by Laura Fiorini, Erika Rovini, Sergio Russo, Lara Toccafondi, Grazia D’Onofrio, Federica Gabriella Cornacchia Loizzo, Manuele Bonaccorsi, Francesco Giuliani, Gianna Vignani, Daniele Sancarlo, Antonio Greco and Filippo Cavallo

Sensors 2022, 22(17), 6631; https://doi.org/10.3390/s22176631 - 2 Sep 2022

Cited by 10 | Viewed by 2342

Abstract

As a consequence of the COVID-19 emergency, frail citizens felt isolated because of social isolation, suspended and/or strongly reduced home assistance, and limited access to hospitals. In this sense, assistive technology could play a pivotal role in empowering frail older adults reducing their isolation, as well as in reinforcing the work of formal caregivers and professionals. In this context, the goal of this paper is to present four pilot studies—conducted from March 2020 to April 2021—to promptly react to COVID-19 by providing assistive technology solutions, aiming to (1) guarantee high-quality service to older adults in-home or in residential facility contexts, (2) promote social inclusion, and (3) reduce the virus transmission. In particular, four services, namely, telepresence service, remote monitoring service, virtual visit, and environmental disinfection, were designed, implemented, and tested in real environments involving 85 end-users to assess the user experience and/or preliminary assess the technical feasibility. The results underlined that all the proposed services were generally accepted by older adults and professionals. Additionally, the results remarked that the use of telepresence robots in private homes and residential facilities increased enjoyment reducing anxiety, whereas the monitoring service supported the clinicians in monitoring the discharged COVID-19 patients. It is also worth mentioning that two new services/products were developed to disinfect the environment and to allow virtual visits within the framework of a hospital information system. The virtual visits service offered the opportunity to expand the portfolio of hospital services. The main barriers were found in education, technology interoperability, and ethical/legal/privacy compliance. It is also worth mentioning the key role played by an appropriate design and customer needs analysis since not all assistive devices were designed for older persons. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

14 pages, 3028 KiB

Open AccessArticle

Tunable White Light for Elders (TWLITE): A Protocol Demonstrating Feasibility and Acceptability for Deployment, Remote Data Collection, and Analysis of a Home-Based Lighting Intervention in Older Adults

by Jonathan E. Elliott, Carolyn E. Tinsley, Christina Reynolds, Randall J. Olson, Kristianna B. Weymann, Wan-Tai M. Au-Yeung, Andrea Wilkerson, Jeffrey A. Kaye and Miranda M. Lim

Sensors 2022, 22(14), 5372; https://doi.org/10.3390/s22145372 - 19 Jul 2022

Cited by 3 | Viewed by 2174

Abstract

Sleep disturbances are common in older adults and may contribute to disease progression in certain populations (e.g., Alzheimer’s disease). Light therapy is a simple and cost-effective intervention to improve sleep. Primary barriers to light therapy are: (1) poor acceptability of the use of devices, and (2) inflexibility of current devices to deliver beyond a fixed light spectrum and throughout the entirety of the day. However, dynamic, tunable lighting integrated into the native home lighting system can potentially overcome these limitations. Herein, we describe our protocol to implement a whole-home tunable lighting system installed throughout the homes of healthy older adults already enrolled in an existing study with embedded home assessment platforms (Oregon Center for Aging & Technology—ORCATECH). Within ORCATECH, continuous data on room location, activity, sleep, and general health parameters are collected at a minute-to-minute resolution over years of participation. This single-arm longitudinal protocol collected participants’ light usage in addition to ORCATECH outcome measures over a several month period before and after light installation. The protocol was implemented with four subjects living in three ORCATECH homes. Technical/usability challenges and feasibility/acceptability outcomes were explored. The successful implementation of our protocol supports the feasibility of implementing and integrating tunable whole-home lighting systems into an automated home-based assessment platform for continuous data collection of outcome variables, including long-term sleep measures. Challenges and iterative approaches are discussed. This protocol will inform the implementation of future clinical intervention trials using light therapy in patients at risk for developing Alzheimer’s disease and related conditions. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

14 pages, 3931 KiB

Open AccessArticle

A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution

by Hesen Feng, Lihong Ma and Jing Tian

Sensors 2022, 22(11), 4231; https://doi.org/10.3390/s22114231 - 1 Jun 2022

Cited by 2 | Viewed by 1865

Abstract

Image super-resolution aims to reconstruct a high-resolution image from its low-resolution counterparts. Conventional image super-resolution approaches share the same spatial convolution kernel for the whole image in the upscaling modules, which neglect the specificity of content information in different positions of the image. In view of this, this paper proposes a regularized pattern method to represent spatially variant structural features in an image and further exploits a dynamic convolution kernel generation method to match the regularized pattern and improve image reconstruction performance. To be more specific, first, the proposed approach extracts features from low-resolution images using a self-organizing feature mapping network to construct regularized patterns (RP), which describe different contents at different locations. Second, the meta-learning mechanism based on the regularized pattern predicts the weights of the convolution kernels that match the regularized pattern for each different location; therefore, it generates different upscaling functions for images with different content. Extensive experiments are conducted using the benchmark datasets Set5, Set14, B100, Urban100, and Manga109 to demonstrate that the proposed approach outperforms the state-of-the-art super-resolution approaches in terms of both PSNR and SSIM performance. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

20 pages, 8837 KiB

Open AccessArticle

LEMON: A Lightweight Facial Emotion Recognition System for Assistive Robotics Based on Dilated Residual Convolutional Neural Networks

by Rami Reddy Devaram, Gloria Beraldo, Riccardo De Benedictis, Misael Mongiovì and Amedeo Cesta

Sensors 2022, 22(9), 3366; https://doi.org/10.3390/s22093366 - 28 Apr 2022

Cited by 15 | Viewed by 2652

Abstract

The development of a Social Intelligence System based on artificial intelligence is one of the cutting edge technologies in Assistive Robotics. Such systems need to create an empathic interaction with the users; therefore, it os required to include an Emotion Recognition (ER) framework which has to run, in near real-time, together with several other intelligent services. Most of the low-cost commercial robots, however, although more accessible by users and healthcare facilities, have to balance costs and effectiveness, resulting in under-performing hardware in terms of memory and processing unit. This aspect makes the design of the systems challenging, requiring a trade-off between the accuracy and the complexity of the adopted models. This paper proposes a compact and robust service for Assistive Robotics, called Lightweight EMotion recognitiON (LEMON), which uses image processing, Computer Vision and Deep Learning (DL) algorithms to recognize facial expressions. Specifically, the proposed DL model is based on Residual Convolutional Neural Networks with the combination of Dilated and Standard Convolution Layers. The first remarkable result is the few numbers (i.e., 1.6 Million) of parameters characterizing our model. In addition, Dilated Convolutions expand receptive fields exponentially with preserving resolution, less computation and memory cost to recognize the distinction among facial expressions by capturing the displacement of the pixels. Finally, to reduce the dying ReLU problem and improve the stability of the model, we apply an Exponential Linear Unit (ELU) activation function in the initial layers of the model. We have performed training and evaluation (via one- and five-fold cross validation) of the model with five datasets available in the community and one mixed dataset created by taking samples from all of them. With respect to the other approaches, our model achieves comparable results with a significant reduction in terms of the number of parameters. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

17 pages, 13932 KiB

Open AccessArticle

Comparative Analysis of Supervised Classifiers for the Evaluation of Sarcopenia Using a sEMG-Based Platform

by Alessandro Leone, Gabriele Rescio, Andrea Manni, Pietro Siciliano and Andrea Caroppo

Sensors 2022, 22(7), 2721; https://doi.org/10.3390/s22072721 - 1 Apr 2022

Cited by 14 | Viewed by 2789

Abstract

Sarcopenia is a geriatric condition characterized by a loss of strength and muscle mass, with a high impact on health status, functional independence and quality of life in older adults. [d=TT, ]To reduce the effects of the disease, just the diagnostic is not enough, it is necessary more than recognition.To reduce the effects of the disease, it is important to recognize the level and progression of sarcopenia early. Surface electromyography is becoming increasingly relevant for the prevention and diagnosis of sarcopenia, also due to a wide diffusion of smart and minimally invasive wearable devices suitable for electromyographic monitoring. The purpose of this work is manifold. The first aim is the design and implementation of a hardware/software platform. It is based on the elaboration of surface electromyographic signals extracted from the Gastrocnemius Lateralis and Tibialis Anterior muscles, useful to analyze the strength of the muscles with the purpose of distinguishing three different “confidence” levels of sarcopenia. The second aim is to compare the efficiency of state of the art supervised classifiers in the evaluation of sarcopenia. The experimentation stage was performed on an “augmented” dataset starting from data acquired from 32 patients. The latter were distributed in an unbalanced manner on 3 “confidence” levels of sarcopenia. The obtained results in terms of classification accuracy demonstrated the ability of the proposed platform to distinguish different sarcopenia “confidence” levels, with highest accuracy value given by Support Vector Machine classifier, outperforming the other classifiers by an average of 7.7%. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

10 pages, 584 KiB

Open AccessArticle

Gender Identification in a Two-Level Hierarchical Speech Emotion Recognition System for an Italian Social Robot

by Antonio Guerrieri, Eleonora Braccili, Federica Sgrò and Giulio Nicolò Meldolesi

Sensors 2022, 22(5), 1714; https://doi.org/10.3390/s22051714 - 22 Feb 2022

Cited by 7 | Viewed by 1967

Abstract

The real challenge in Human-Robot Interaction (HRI) is to build machines capable of perceiving human emotions so that robots can interact with humans in a proper manner. Emotion varies accordingly to many factors, and gender represents one of the most influential ones: an appropriate gender-dependent emotion recognition system is recommended indeed. In this article, we propose a Gender Recognition (GR) module for the gender identification of the speaker, as a preliminary step for the final development of a Speech Emotion Recognition (SER) system. The system was designed to be installed on social robots for hospitalized and living at home patients monitoring. Hence, the importance of reducing the software computational effort of the architecture also minimizing the hardware bulkiness, in order for the system to be suitable for social robots. The algorithm was executed on the Raspberry Pi hardware. For the training, the Italian emotional database EMOVO was used. Results show a GR accuracy value of 97.8%, comparable with the ones found in the literature. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

15 pages, 3723 KiB

Open AccessCommunication

Generating High-Quality Panorama by View Synthesis Based on Optical Flow Estimation

by Wenxin Zhang, Yumei Wang and Yu Liu

Sensors 2022, 22(2), 470; https://doi.org/10.3390/s22020470 - 8 Jan 2022

Cited by 4 | Viewed by 2505

Abstract

Generating high-quality panorama is a key element in promoting the development of VR content. The panoramas generated by the traditional image stitching algorithm have some limitations, such as artifacts and irregular shapes. We consider solving this problem from the perspective of view synthesis. We propose a view synthesis approach based on optical flow to generate a high-quality omnidirectional panorama. In the first stage, we present a novel optical flow estimation algorithm to establish a dense correspondence between the overlapping areas of the left and right views. The result obtained can be approximated as the parallax of the scene. In the second stage, the reconstructed version of the left and the right views is generated by warping the pixels under the guidance of optical flow, and the alpha blending algorithm is used to synthesize the final novel view. Experimental results demonstrate that the subjective experience obtained by our approach is better than the comparison algorithm without cracks or artifacts. Besides the commonly used image quality assessment PSNR and SSIM, we also calculate MP-PSNR, which can provide accurate high-quality predictions for synthesized views. Our approach can achieve an improvement of about 1 dB in MP-PSNR and PSNR and 25% in SSIM, respectively. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

27 pages, 3232 KiB

Open AccessArticle

COLAEVA: Visual Analytics and Data Mining Web-Based Tool for Virtual Coaching of Older Adult Populations

by Jon Kerexeta Sarriegi, Andoni Beristain Iraola, Roberto Álvarez Sánchez, Manuel Graña, Kristin May Rebescher, Gorka Epelde, Louise Hopper, Joanne Carroll, Patrizia Gabriella Ianes, Barbara Gasperini, Francesco Pilla, Walter Mattei, Francesco Tessarolo, Despoina Petsani, Panagiotis D. Bamidis and Evdokimos I. Konstantinidis

Sensors 2021, 21(23), 7991; https://doi.org/10.3390/s21237991 - 30 Nov 2021

Cited by 1 | Viewed by 2713

Abstract

The global population is aging in an unprecedented manner and the challenges for improving the lives of older adults are currently both a strong priority in the political and healthcare arena. In this sense, preventive measures and telemedicine have the potential to play an important role in improving the number of healthy years older adults may experience and virtual coaching is a promising research area to support this process. This paper presents COLAEVA, an interactive web application for older adult population clustering and evolution analysis. Its objective is to support caregivers in the design, validation and refinement of coaching plans adapted to specific population groups. COLAEVA enables coaching caregivers to interactively group similar older adults based on preliminary assessment data, using AI features, and to evaluate the influence of coaching plans once the final assessment is carried out for a baseline comparison. To evaluate COLAEVA, a usability test was carried out with 9 test participants obtaining an average SUS score of 71.1. Moreover, COLAEVA is available online to use and explore. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

19 pages, 2169 KiB

Open AccessArticle

Age and Gender Recognition Using a Convolutional Neural Network with a Specially Designed Multi-Attention Module through Speech Spectrograms

by Anvarjon Tursunov, Mustaqeem, Joon Yeon Choeh and Soonil Kwon

Sensors 2021, 21(17), 5892; https://doi.org/10.3390/s21175892 - 1 Sep 2021

Cited by 44 | Viewed by 6710

Abstract

Speech signals are being used as a primary input source in human–computer interaction (HCI) to develop several applications, such as automatic speech recognition (ASR), speech emotion recognition (SER), gender, and age recognition. Classifying speakers according to their age and gender is a challenging task in speech processing owing to the disability of the current methods of extracting salient high-level speech features and classification models. To address these problems, we introduce a novel end-to-end age and gender recognition convolutional neural network (CNN) with a specially designed multi-attention module (MAM) from speech signals. Our proposed model uses MAM to extract spatial and temporal salient features from the input data effectively. The MAM mechanism uses a rectangular shape filter as a kernel in convolution layers and comprises two separate time and frequency attention mechanisms. The time attention branch learns to detect temporal cues, whereas the frequency attention module extracts the most relevant features to the target by focusing on the spatial frequency features. The combination of the two extracted spatial and temporal features complements one another and provide high performance in terms of age and gender classification. The proposed age and gender classification system was tested using the Common Voice and locally developed Korean speech recognition datasets. Our suggested model achieved 96%, 73%, and 76% accuracy scores for gender, age, and age-gender classification, respectively, using the Common Voice dataset. The Korean speech recognition dataset results were 97%, 97%, and 90% for gender, age, and age-gender recognition, respectively. The prediction performance of our proposed model, which was obtained in the experiments, demonstrated the superiority and robustness of the tasks regarding age, gender, and age-gender recognition from speech signals. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

13 pages, 1172 KiB

Open AccessArticle

Physical Training In-Game Metrics for Cognitive Assessment: Evidence from Extended Trials with the Fitforall Exergaming Platform

by Evdokimos I. Konstantinidis, Panagiotis D. Bamidis, Antonis Billis, Panagiotis Kartsidis, Despoina Petsani and Sokratis G. Papageorgiou

Sensors 2021, 21(17), 5756; https://doi.org/10.3390/s21175756 - 26 Aug 2021

Cited by 5 | Viewed by 3513

Abstract

Conventional clinical cognitive assessment has its limitations, as evidenced by the environmental shortcomings of various neuropsychological tests conducted away from an older person’s everyday environment. Recent research activities have focused on transferring screening tests to computerized forms, as well as on developing short screening tests for screening large populations for cognitive impairment. The purpose of this study was to present an exergaming platform, which was widely trialed (116 participants) to collect in-game metrics (built-in game performance measures). The potential correlation between in-game metrics and cognition was investigated in-depth by scrutinizing different in-game metrics. The predictive value of high-resolution monitoring games was assessed by correlating it with classical neuropsychological tests; the area under the curve (AUC) in the receiver operating characteristic (ROC) analysis was calculated to determine the sensitivity and specificity of the method for detecting mild cognitive impairment (MCI). Classification accuracy was calculated to be 73.53% when distinguishing between MCI and normal subjects, and 70.69% when subjects with mild dementia were also involved. The results revealed evidence that careful design of serious games, with respect to in-game metrics, could potentially contribute to the early and unobtrusive detection of cognitive decline. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

0 pages, 3581 KiB

Open AccessArticle

Evaluation of Abstraction Capabilities and Detection of Discomfort with a Newscaster Chatbot for Entertaining Elderly Users

by Francisco de Arriba-Pérez, Silvia García-Méndez, Francisco J. González-Castaño and Enrique Costa-Montenegro

Sensors 2021, 21(16), 5515; https://doi.org/10.3390/s21165515 - 17 Aug 2021

Cited by 6 | Viewed by 3054

Abstract

We recently proposed a novel intelligent newscaster chatbot for digital inclusion. Its controlled dialogue stages (consisting of sequences of questions that are generated with hybrid Natural Language Generation techniques based on the content) support entertaining personalisation, where user interest is estimated by analysing the sentiment of his/her answers. A differential feature of our approach is its automatic and transparent monitoring of the abstraction skills of the target users. In this work we improve the chatbot by introducing enhanced monitoring metrics based on the distance of the user responses to an accurate characterisation of the news content. We then evaluate abstraction capabilities depending on user sentiment about the news and propose a Machine Learning model to detect users that experience discomfort with precision, recall, F1 and accuracy levels over 80%. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

20 pages, 1502 KiB

Open AccessArticle

Vision-Based Road Rage Detection Framework in Automotive Safety Applications

by Alessandro Leone, Andrea Caroppo, Andrea Manni and Pietro Siciliano

Sensors 2021, 21(9), 2942; https://doi.org/10.3390/s21092942 - 22 Apr 2021

Cited by 10 | Viewed by 3554

Abstract

Drivers’ road rage is among the main causes of road accidents. Each year, it contributes to more deaths and injuries globally. In this context, it is important to implement systems that can supervise drivers by monitoring their level of concentration during the entire driving process. In this paper, a module for Advanced Driver Assistance System is used to minimise the accidents caused by road rage, alerting the driver when a predetermined level of rage is reached, thus increasing the transportation safety. To create a system that is independent of both the orientation of the driver’s face and the lighting conditions of the cabin, the proposed algorithmic pipeline integrates face detection and facial expression classification algorithms capable of handling such non-ideal situations. Moreover, road rage of the driver is estimated through a decision-making strategy based on the temporal consistency of facial expressions classified as “anger” and “disgust”. Several experiments were executed to assess the performance on both a real context and three standard benchmark datasets, two of which containing non-frontal-view facial expression and one which includes facial expression recorded from participants during driving. Results obtained show that the proposed module is competent for road rage estimation through facial expression recognition on the condition of multi-pose and changing in lighting conditions, with the recognition rates that achieve state-of-art results on the selected datasets. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures

Figure 1

14 pages, 1428 KiB

Open AccessArticle

Improved Action Recognition with Separable Spatio-Temporal Attention Using Alternative Skeletal and Video Pre-Processing

by Pau Climent-Pérez and Francisco Florez-Revuelta

Sensors 2021, 21(3), 1005; https://doi.org/10.3390/s21031005 - 2 Feb 2021

Cited by 9 | Viewed by 2611

Abstract

The potential benefits of recognising activities of daily living from video for active and assisted living have yet to be fully untapped. These technologies can be used for behaviour understanding, and lifelogging for caregivers and end users alike. The recent publication of realistic datasets for this purpose, such as the Toyota Smarthomes dataset, calls for pushing forward the efforts to improve action recognition. Using the separable spatio-temporal attention network proposed in the literature, this paper introduces a view-invariant normalisation of skeletal pose data and full activity crops for RGB data, which improve the baseline results by 9.5% (on the cross-subject experiments), outperforming state-of-the-art techniques in this field when using the original unmodified skeletal data in dataset. Our code and data are available online. Full article

(This article belongs to the Special Issue Signal, Image Processing and Computer Vision in Smart Living Applications)

► Show Figures