Sensors

Research

Jump to: Review

25 pages, 9781 KiB

Open AccessArticle

Building Segmentation in Urban and Rural Areas with MFA-Net: A Multidimensional Feature Adjustment Approach

by Zijie Han, Xue Li, Xianteng Wang, Zihao Wu and Jian Liu

Sensors 2025, 25(8), 2589; https://doi.org/10.3390/s25082589 - 19 Apr 2025

Viewed by 230

Abstract

Deep-learning-based methods are crucial for building extraction from high-resolution remote sensing images, playing a key role in applications like natural disaster response, land resource management, and smart city development. However, extracting precise building from complex urban and rural environments remains challenging due to [...] Read more.

Deep-learning-based methods are crucial for building extraction from high-resolution remote sensing images, playing a key role in applications like natural disaster response, land resource management, and smart city development. However, extracting precise building from complex urban and rural environments remains challenging due to spectral variability and intricate background interference, particularly in densely packed and small buildings. To address these issues, we propose an enhanced U²-Net architecture, MFA-Net, which incorporates two key innovations: a Multidimensional Feature Adjustment (MFA) module that refines feature representations through Cascaded Channel, Spatial, and Multiscale Weighting Mechanisms and a Dynamic Fusion Loss function that enhances edge geometric fidelity. Evaluation on three datasets (Urban, Rural, and WHU) reveals that MFA-Net outperforms existing methods, with average improvements of 6% in F1-score and 7.3% in IoU and an average increase of 9.9% in training time. These advancements significantly improve edge delineation and the segmentation of dense building clusters, making MFA-Net especially beneficial for urban planning and land resource management. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

21 pages, 2488 KiB

Open AccessArticle

Classification of Mycena and Marasmius Species Using Deep Learning Models: An Ecological and Taxonomic Approach

by Fatih Ekinci, Guney Ugurlu, Giray Sercan Ozcan, Koray Acici, Tunc Asuroglu, Eda Kumru, Mehmet Serdar Guzel and Ilgaz Akata

Sensors 2025, 25(6), 1642; https://doi.org/10.3390/s25061642 - 7 Mar 2025

Viewed by 751

Abstract

Fungi play a critical role in ecosystems, contributing to biodiversity and providing economic and biotechnological value. In this study, we developed a novel deep learning-based framework for the classification of seven macrofungi species from the genera Mycena and Marasmius, leveraging their unique [...] Read more.

Fungi play a critical role in ecosystems, contributing to biodiversity and providing economic and biotechnological value. In this study, we developed a novel deep learning-based framework for the classification of seven macrofungi species from the genera Mycena and Marasmius, leveraging their unique ecological and morphological characteristics. The proposed approach integrates a custom convolutional neural network (CNN) with a self-organizing map (SOM) adapted for supervised learning and a Kolmogorov–Arnold Network (KAN) layer to enhance classification performance. The experimental results demonstrate significant improvements in classification metrics when using the CNN-SOM and CNN-KAN architectures. Additionally, advanced pretrained models such as MaxViT-S and ResNetV2-50 achieved high accuracy rates, with MaxViT-S achieving 98.9% accuracy. Statistical analyses using the chi-square test confirmed the reliability of the results, emphasizing the importance of validating evaluation metrics statistically. This research represents the first application of SOM in fungal classification and highlights the potential of deep learning in advancing fungal taxonomy. Future work will focus on optimizing the KAN architecture and expanding the dataset to include more fungal classes, further enhancing classification accuracy and ecological understanding. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

19 pages, 15983 KiB

Open AccessArticle

Advanced Deep Learning Models for Melanoma Diagnosis in Computer-Aided Skin Cancer Detection

by Ranpreet Kaur, Hamid GholamHosseini and Maria Lindén

Sensors 2025, 25(3), 594; https://doi.org/10.3390/s25030594 - 21 Jan 2025

Cited by 1 | Viewed by 1325

Abstract

The most deadly type of skin cancer is melanoma. A visual examination does not provide an accurate diagnosis of melanoma during its early to middle stages. Therefore, an automated model could be developed that assists with early skin cancer detection. It is possible [...] Read more.

The most deadly type of skin cancer is melanoma. A visual examination does not provide an accurate diagnosis of melanoma during its early to middle stages. Therefore, an automated model could be developed that assists with early skin cancer detection. It is possible to limit the severity of melanoma by detecting it early and treating it promptly. This study aims to develop efficient approaches for various phases of melanoma computer-aided diagnosis (CAD), such as preprocessing, segmentation, and classification. The first step of the CAD pipeline includes the proposed hybrid method, which uses morphological operations and context aggregation-based deep neural networks to remove hairlines and improve poor contrast in dermoscopic skin cancer images. An image segmentation network based on deep learning is then used to extract lesion regions for detailed analysis and calculate the optimized classification features. Lastly, a deep neural network is used to distinguish melanoma from benign lesions. The proposed approaches use a benchmark dataset named International Skin Imaging Collaboration (ISIC) 2020. In this work, two forms of evaluations are performed with the classification model. The first experiment involves the incorporation of the results from the preprocessing and segmentation stages into the classification model. The second experiment involves the evaluation of the classifier without employing these stages i.e., using raw images. From the study results, it can be concluded that a classification model using segmented and cleaned images contributes more to achieving an accurate classification rate of 93.40% with a 1.3 s test time on a single image. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

18 pages, 5444 KiB

Open AccessArticle

Bird Species Detection Net: Bird Species Detection Based on the Extraction of Local Details and Global Information Using a Dual-Feature Mixer

by Chaoyang Li, Zhipeng He, Kai Lu and Chaoyang Fang

Sensors 2025, 25(1), 291; https://doi.org/10.3390/s25010291 - 6 Jan 2025

Viewed by 1499

Abstract

Bird species detection is critical for applications such as the analysis of bird population dynamics and species diversity. However, this task remains challenging due to local structural similarities and class imbalances among bird species. Currently, most deep learning algorithms focus on designing local [...] Read more.

Bird species detection is critical for applications such as the analysis of bird population dynamics and species diversity. However, this task remains challenging due to local structural similarities and class imbalances among bird species. Currently, most deep learning algorithms focus on designing local feature extraction modules while ignoring the importance of global information. However, this global information is essential for accurate bird species detection. To address this limitation, we propose BSD-Net, a bird species detection network. BSD-Net efficiently learns local and global information in pixels to accurately detect bird species. BSD-Net consists of two main components: a dual-branch feature mixer (DBFM) and a prediction balancing module (PBM). The dual-branch feature mixer extracts features from dichotomous feature segments using global attention and deep convolution, expanding the network’s receptive field and achieving a strong inductive bias, allowing the network to distinguish between similar local details. The prediction balance module balances the difference in feature space based on the pixel values of each category, thereby resolving category imbalances and improving the network’s detection accuracy. The experimental results using two public benchmarks and a self-constructed Poyang Lake Bird dataset demonstrate that BSD-Net outperforms existing methods, achieving 45.71% and 80.00% mAP50 with the CUB-200-2011 and Poyang Lake Bird datasets, respectively, and 66.03% AP with FBD-SV-2024, allowing for more accurate location and species information for bird detection tasks in video surveillance. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

25 pages, 7532 KiB

Open AccessArticle

A Novel Approach to Detect Drones Using Deep Convolutional Neural Network Architecture

by Hrishi Rakshit and Pooneh Bagheri Zadeh

Sensors 2024, 24(14), 4550; https://doi.org/10.3390/s24144550 - 13 Jul 2024

Cited by 1 | Viewed by 2111

Abstract

Over the past decades, drones have become more attainable by the public due to their widespread availability at affordable prices. Nevertheless, this situation sparks serious concerns in both the cyber and physical security domains, as drones can be employed for malicious activities with [...] Read more.

Over the past decades, drones have become more attainable by the public due to their widespread availability at affordable prices. Nevertheless, this situation sparks serious concerns in both the cyber and physical security domains, as drones can be employed for malicious activities with public safety threats. However, detecting drones instantly and efficiently is a very difficult task due to their tiny size and swift flights. This paper presents a novel drone detection method using deep convolutional learning and deep transfer learning. The proposed algorithm employs a new feature extraction network, which is added to the modified YOU ONLY LOOK ONCE version2 (YOLOv2) network. The feature extraction model uses bypass connections to learn features from the training sets and solves the “vanishing gradient” problem caused by the increasing depth of the network. The structure of YOLOv2 is modified by replacing the rectified linear unit (relu) with a leaky-relu activation function and adding an extra convolutional layer with a stride of 2 to improve the small object detection accuracy. Using leaky-relu solves the “dying relu” problem. The additional convolution layer with a stride of 2 reduces the spatial dimensions of the feature maps and helps the network to focus on larger contextual information while still preserving the ability to detect small objects. The model is trained with a custom dataset that contains various types of drones, airplanes, birds, and helicopters under various weather conditions. The proposed model demonstrates a notable performance, achieving an accuracy of 77% on the test images with only 5 million learnable parameters in contrast to the Darknet53 + YOLOv3 model, which exhibits a 54% accuracy on the same test set despite employing 62 million learnable parameters. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

14 pages, 6859 KiB

Open AccessCommunication

Addressing Challenges in Port Depth Analysis: Integrating Machine Learning and Spatial Information for Accurate Remote Sensing of Turbid Waters

by Xin Li, Zhongqiang Wu and Wei Shen

Sensors 2024, 24(12), 3802; https://doi.org/10.3390/s24123802 - 12 Jun 2024

Cited by 4 | Viewed by 1292

Abstract

Bathymetry estimation is essential for various applications in port management, navigation safety, marine engineering, and environmental monitoring. Satellite remote sensing data can rapidly acquire the bathymetry of the target shallow waters, and researchers have developed various models to invert the water depth from [...] Read more.

Bathymetry estimation is essential for various applications in port management, navigation safety, marine engineering, and environmental monitoring. Satellite remote sensing data can rapidly acquire the bathymetry of the target shallow waters, and researchers have developed various models to invert the water depth from the satellite data. Geographically weighted regression (GWR) is a common method for satellite-based bathymetry estimation. However, in sediment-laden water environments, especially ports, the suspended materials significantly affect the performance of GWR for depth inversion. This study proposes a novel approach that integrates GWR with Random Forest (RF) techniques, using longitude, latitude, and multispectral remote sensing reflectance as input variables. This approach effectively addresses the challenge of estimating bathymetry in turbid waters by considering the strong correlation between water depth and geographical location. The proposed method not only overcomes the limitations of turbid waters but also improves the accuracy of depth inversion results in such complex aquatic settings. This breakthrough in modeling has significant implications for turbid waters, enhancing port management, navigational safety, and environmental monitoring in sediment-laden maritime zones. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

24 pages, 8032 KiB

Open AccessArticle

GM-DETR: Research on a Defect Detection Method Based on Improved DETR

by Xin Liu, Xudong Yang, Lianhe Shao, Xihan Wang, Quanli Gao and Hongbo Shi

Sensors 2024, 24(11), 3610; https://doi.org/10.3390/s24113610 - 3 Jun 2024

Cited by 6 | Viewed by 2421

Abstract

Defect detection is an indispensable part of the industrial intelligence process. The introduction of the DETR model marked the successful application of a transformer for defect detection, achieving true end-to-end detection. However, due to the complexity of defective backgrounds, low resolutions can lead [...] Read more.

Defect detection is an indispensable part of the industrial intelligence process. The introduction of the DETR model marked the successful application of a transformer for defect detection, achieving true end-to-end detection. However, due to the complexity of defective backgrounds, low resolutions can lead to a lack of image detail control and slow convergence of the DETR model. To address these issues, we proposed a defect detection method based on an improved DETR model, called the GM-DETR. We optimized the DETR model by integrating GAM global attention with CNN feature extraction and matching features. This optimization process reduces the defect information diffusion and enhances the global feature interaction, improving the neural network’s performance and ability to recognize target defects in complex backgrounds. Next, to filter out unnecessary model parameters, we proposed a layer pruning strategy to optimize the decoding layer, thereby reducing the model’s parameter count. In addition, to address the issue of poor sensitivity of the original loss function to small differences in defect targets, we replaced the L1 loss in the original loss function with MSE loss to accelerate the network’s convergence speed and improve the model’s recognition accuracy. We conducted experiments on a dataset of road pothole defects to further validate the effectiveness of the GM-DETR model. The results demonstrate that the improved model exhibits better performance, with an increase in average precision of 4.9% (mAP@0.5), while reducing the parameter count by 12.9%. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

18 pages, 1567 KiB

Open AccessArticle

Image Classifier for an Online Footwear Marketplace to Distinguish between Counterfeit and Real Sneakers for Resale

by Joshua Onalaja, Essa Q. Shahra, Shadi Basurra and Waheb A. Jabbar

Sensors 2024, 24(10), 3030; https://doi.org/10.3390/s24103030 - 10 May 2024

Cited by 3 | Viewed by 2299

Abstract

The sneaker industry is continuing to expand at a fast rate and will be worth over USD 120 billion in the next few years. This is, in part due to social media and online retailers building hype around releases of limited-edition sneakers, which [...] Read more.

The sneaker industry is continuing to expand at a fast rate and will be worth over USD 120 billion in the next few years. This is, in part due to social media and online retailers building hype around releases of limited-edition sneakers, which are usually collaborations between well-known global icons and footwear companies. These limited-edition sneakers are typically released in low quantities using an online raffle system, meaning only a few people can get their hands on them. As expected, this causes their value to skyrocket and has created an extremely lucrative resale market for sneakers. This has given rise to numerous counterfeit sneakers flooding the resale market, resulting in online platforms having to hand-verify a sneaker’s authenticity, which is an important but time-consuming procedure that slows the selling and buying process. To speed up the authentication process, Support Vector Machines and a convolutional neural network were used to classify images of fake and real sneakers and then their accuracies were compared to see which performed better. The results showed that the CNNs performed much better at this task than the SVMs with some accuracies over 95%. Therefore, a CNN is well equipped to be a sneaker authenticator and will be of great benefit to the reselling industry. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

20 pages, 32970 KiB

Open AccessArticle

Faces in Event Streams (FES): An Annotated Face Dataset for Event Cameras

by Ulzhan Bissarinova, Tomiris Rakhimzhanova, Daulet Kenzhebalin and Huseyin Atakan Varol

Sensors 2024, 24(5), 1409; https://doi.org/10.3390/s24051409 - 22 Feb 2024

Cited by 5 | Viewed by 3115

Abstract

The use of event-based cameras in computer vision is a growing research direction. However, despite the existing research on face detection using the event camera, a substantial gap persists in the availability of a large dataset featuring annotations for faces and facial landmarks [...] Read more.

The use of event-based cameras in computer vision is a growing research direction. However, despite the existing research on face detection using the event camera, a substantial gap persists in the availability of a large dataset featuring annotations for faces and facial landmarks on event streams, thus hampering the development of applications in this direction. In this work, we address this issue by publishing the first large and varied dataset (Faces in Event Streams) with a duration of 689 min for face and facial landmark detection in direct event-based camera outputs. In addition, this article presents 12 models trained on our dataset to predict bounding box and facial landmark coordinates with an mAP₅₀ score of more than 90%. We also performed a demonstration of real-time detection with an event-based camera using our models. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

15 pages, 1792 KiB

Open AccessArticle

Rethinking Attention Mechanisms in Vision Transformers with Graph Structures

by Hyeongjin Kim and Byoung Chul Ko

Sensors 2024, 24(4), 1111; https://doi.org/10.3390/s24041111 - 8 Feb 2024

Cited by 2 | Viewed by 3044

Abstract

In this paper, we propose a new type of vision transformer (ViT) based on graph head attention (GHA). Because the multi-head attention (MHA) of a pure ViT requires multiple parameters and tends to lose the locality of an image, we replaced MHA with [...] Read more.

In this paper, we propose a new type of vision transformer (ViT) based on graph head attention (GHA). Because the multi-head attention (MHA) of a pure ViT requires multiple parameters and tends to lose the locality of an image, we replaced MHA with GHA by applying a graph to the attention head of the transformer. Consequently, the proposed GHA maintains both the locality and globality of the input patches and guarantees the diversity of the attention. The proposed GHA-ViT commonly outperforms pure ViT-based models using small-sized CIFAR-10/100, MNIST, and MNIST-F datasets and a medium-sized ImageNet-1K dataset in scratch training. A Top-1 accuracy of 81.7% was achieved for ImageNet-1K using GHA-B, which is a base model with approximately 29 M parameters. In addition, with CIFAR-10/100, the existing ViT and parameters are reduced 17-fold and the performance increased by 0.4/4.3%, respectively. The proposed GHA-ViT shows promising results in terms of the number of parameters and operations and the level of accuracy in comparison with other state-of-the-art ViT-lightweight models. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

21 pages, 2798 KiB

Open AccessArticle

An Improved YOLOv5-Based Underwater Object-Detection Framework

by Jian Zhang, Jinshuai Zhang, Kexin Zhou, Yonghui Zhang, Hongda Chen and Xinyue Yan

Sensors 2023, 23(7), 3693; https://doi.org/10.3390/s23073693 - 3 Apr 2023

Cited by 64 | Viewed by 11217

Abstract

To date, general-purpose object-detection methods have achieved a great deal. However, challenges such as degraded image quality, complex backgrounds, and the detection of marine organisms at different scales arise when identifying underwater organisms. To solve such problems and further improve the accuracy of [...] Read more.

To date, general-purpose object-detection methods have achieved a great deal. However, challenges such as degraded image quality, complex backgrounds, and the detection of marine organisms at different scales arise when identifying underwater organisms. To solve such problems and further improve the accuracy of relevant models, this study proposes a marine biological object-detection architecture based on an improved YOLOv5 framework. First, the backbone framework of Real-Time Models for object Detection (RTMDet) is introduced. The core module, Cross-Stage Partial Layer (CSPLayer), includes a large convolution kernel, which allows the detection network to precisely capture contextual information more comprehensively. Furthermore, a common convolution layer is added to the stem layer, to extract more valuable information from the images efficiently. Then, the BoT3 module with the multi-head self-attention (MHSA) mechanism is added into the neck module of YOLOv5, such that the detection network has a better effect in scenes with dense targets and the detection accuracy is further improved. The introduction of the BoT3 module represents a key innovation of this paper. Finally, union dataset augmentation (UDA) is performed on the training set using the Minimal Color Loss and Locally Adaptive Contrast Enhancement (MLLE) image augmentation method, and the result is used as the input to the improved YOLOv5 framework. Experiments on the underwater datasets URPC2019 and URPC2020 show that the proposed framework not only alleviates the interference of underwater image degradation, but also makes the mAP@0.5 reach 79.8% and 79.4% and improves the mAP@0.5 by 3.8% and 1.1%, respectively, when compared with the original YOLOv8 on URPC2019 and URPC2020, demonstrating that the proposed framework presents superior performance for the high-precision detection of marine organisms. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

15 pages, 5805 KiB

Open AccessArticle

Real-Time Forest Fire Detection by Ensemble Lightweight YOLOX-L and Defogging Method

by Jiarun Huang, Zhili He, Yuwei Guan and Hongguo Zhang

Sensors 2023, 23(4), 1894; https://doi.org/10.3390/s23041894 - 8 Feb 2023

Cited by 34 | Viewed by 4030

Abstract

Forest fires can destroy forest and inflict great damage to the ecosystem. Fortunately, forest fire detection with video has achieved remarkable results in enabling timely and accurate fire warnings. However, the traditional forest fire detection method relies heavily on artificially designed features; CNN-based [...] Read more.

Forest fires can destroy forest and inflict great damage to the ecosystem. Fortunately, forest fire detection with video has achieved remarkable results in enabling timely and accurate fire warnings. However, the traditional forest fire detection method relies heavily on artificially designed features; CNN-based methods require a large number of parameters. In addition, forest fire detection is easily disturbed by fog. To solve these issues, a lightweight YOLOX-L and defogging algorithm-based forest fire detection method, GXLD, is proposed. GXLD uses the dark channel prior to defog the image to obtain a fog-free image. After the lightweight improvement of YOLOX-L by GhostNet, depth separable convolution, and SENet, we obtain the YOLOX-L-Light and use it to detect the forest fire in the fog-free image. To evaluate the performance of YOLOX-L-Light and GXLD, mean average precision (mAP) was used to evaluate the detection accuracy, and network parameters were used to evaluate the lightweight effect. Experiments on our forest fire dataset show that the number of the parameters of YOLOX-L-Light decreased by 92.6%, and the mAP increased by 1.96%. The mAP of GXLD is 87.47%, which is 2.46% higher than that of YOLOX-L; and the average fps of GXLD is 26.33 when the input image size is 1280 × 720. Even in a foggy environment, the GXLD can detect a forest fire in real time with a high accuracy, target confidence, and target integrity. This research proposes a lightweight forest fire detection method (GXLD) with fog removal. Therefore, GXLD can detect a forest fire with a high accuracy in real time. The proposed GXLD has the advantages of defogging, a high target confidence, and a high target integrity, which makes it more suitable for the development of a modern forest fire video detection system. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

Review

Jump to: Research

23 pages, 497 KiB

Open AccessReview

The Influence of Ambient Factors on Emotional Wellbeing of Older Adults: A Review

by Arturo Peralta, José A. Olivas, Francisco P. Romero and Pedro Navarro

Sensors 2025, 25(4), 1071; https://doi.org/10.3390/s25041071 - 11 Feb 2025

Viewed by 865

Abstract

This work conducts a systematic review following PRISMA guidelines and using software tools like Covidence^® 2024 and Nvivo^® 15 for thematic analysis, aiming to examine significant studies on the impact of external factors on the emotional wellbeing of older adults and [...] Read more.

This work conducts a systematic review following PRISMA guidelines and using software tools like Covidence^® 2024 and Nvivo^® 15 for thematic analysis, aiming to examine significant studies on the impact of external factors on the emotional wellbeing of older adults and propose new conclusions and future research directions. In this context, studies using sensors to measure factors such as ambient temperature or lighting are key to understanding their impact on the emotional wellbeing of older individuals. These technologies offer opportunities to monitor and adapt environments in real-time, enabling targeted interventions. It is widely recognised that aspects like noise levels, ambient temperature, or lighting can influence an individual’s mood and overall wellbeing; however, it is crucial to further explore the effect of less studied factors. This review not only validates and questions popular beliefs about these factors but also highlights how the results can be useful for designing living environments that enhance the emotional wellbeing of the elderly and for establishing new directions in related research. By addressing these factors, this review provides actionable insights for policymakers, urban planners, and care providers to design environments that enhance the emotional wellbeing of older adults. Furthermore, this study not only validates previous knowledge but also highlights the need for future interdisciplinary interventions that integrate these factors holistically. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

33 pages, 18843 KiB

Open AccessReview

Deep Learning for Human Activity Recognition on 3D Human Skeleton: Survey and Comparative Study

by Hung-Cuong Nguyen, Thi-Hao Nguyen, Rafał Scherer and Van-Hung Le

Sensors 2023, 23(11), 5121; https://doi.org/10.3390/s23115121 - 27 May 2023

Cited by 25 | Viewed by 8370

Abstract

Human activity recognition (HAR) is an important research problem in computer vision. This problem is widely applied to building applications in human–machine interactions, monitoring, etc. Especially, HAR based on the human skeleton creates intuitive applications. Therefore, determining the current results of these studies [...] Read more.

Human activity recognition (HAR) is an important research problem in computer vision. This problem is widely applied to building applications in human–machine interactions, monitoring, etc. Especially, HAR based on the human skeleton creates intuitive applications. Therefore, determining the current results of these studies is very important in selecting solutions and developing commercial products. In this paper, we perform a full survey on using deep learning to recognize human activity based on three-dimensional (3D) human skeleton data as input. Our research is based on four types of deep learning networks for activity recognition based on extracted feature vectors: Recurrent Neural Network (RNN) using extracted activity sequence features; Convolutional Neural Network (CNN) uses feature vectors extracted based on the projection of the skeleton into the image space; Graph Convolution Network (GCN) uses features extracted from the skeleton graph and the temporal–spatial function of the skeleton; Hybrid Deep Neural Network (Hybrid–DNN) uses many other types of features in combination. Our survey research is fully implemented from models, databases, metrics, and results from 2019 to March 2023, and they are presented in ascending order of time. In particular, we also carried out a comparative study on HAR based on a 3D human skeleton on the KLHA3D 102 and KLYOGA3D datasets. At the same time, we performed analysis and discussed the obtained results when applying CNN-based, GCN-based, and Hybrid–DNN-based deep learning networks. Full article

(This article belongs to the Special Issue Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Computer Vision and Machine Learning for Intelligent Sensing Systems—2nd Edition

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (14 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI