Human Posture Estimation and Sustainable Events Classification via Pseudo-2D Stick Model and K-ary Tree Hashing
Abstract
:1. Introduction
- Due to the complex movement of the human body and complex events, 19 human body key points were detected using frame size, human silhouette size, and a change detection approach.
- We propose a pseudo-2D stick model using the information from detected key points, 2D stick model, volumetric data, degree of freedom, and kinematics. This produced much better accuracy for sustainable event classification.
- Context-aware feature extraction based upon the full human body and human body key points was applied. Energy features, sine features, distinct motion body parts, and a 3D Cartesian view of smoothing gradients were extracted from the full human silhouette. Rich 2D appearance features, angular point features, and multi-point autocorrelation features were extracted using human body key points information. With the help of the extracted feature vector, we produced more accurate sustainable event classification results.
- The hierarchical optimization approach is adopted in this paper where ray optimization was used for data optimization and K-ary tree hashing was applied for sustainable event classification.
2. Related Work
2.1. Sustainable Event Classification via 2D/3D Images
2.2. Sustainable Event Classification via Body-Marker Sensors
3. Designed System Methodology
3.1. Pre-Processing of Data and Human Detection
3.2. Human Posture Estimation: Human Key Points Detection
Algorithm 1. Human body key points detection. |
Input: HES: Extracted human silhouette |
Output: 19 body parts, specifically, head, neck, shoulders, elbows, wrists, hands, mid, hips, knees, ankles, and feet. |
HES = human silhouette, HS = human shape, HI = height, WI = width, LI = left, RI = right, IH = head, IN = neck |
Repeat |
For i = 1 to N do |
Search (HES) |
IH = Human_head_point_Area; |
HImg_HI = UpperPoint (IH) |
EHI = Human _End_Head_point (IH) |
HImg_Feet = Bottom (HS) |
HImg_Mid = mid (HI,WI)/2 |
HImg_Foot = Bottom(HS)& earch(LI,RI) |
HImg_K = mid(HImg_Mid, HImg Foot) |
HImg_H = HR &search(LI,RI) |
HImg_S = search(IH, IN)&search(RI,LI) |
HImg_E = mid(HImg_H, HImg_S) |
HImg_W = mid (HImg_H, Img_S)/2 |
HImg_Hp = HImg_Mid &search(RI,LI) |
HImg_A = mid (HImg_K, HImg Foot)/4 |
End |
Until largest regions of extracted human silhouette are searched. |
return 19 body parts: head, neck, shoulders, elbows, wrists, hands, mid, hips, knees, ankles, and feet. HES = human silhouette, HS = human shape, HI = height, WI = width, LI = left, RI = right, IH = head, IN = neck |
3.3. Pseudo 2D Stick Model
Algorithm 2 Pseudo 2D stick model. |
Input: Human body key point and 2D stick model |
Output: Pseudo 2D stick model (p1, p2, p3, …, pn) |
HD = human key points detection, SN = self-connection with each node, SS = scaling of sticks, FG = fix undirected skeleton graph, VD = volumetric data, HK = human body key points tracking and kinematic dependency, KE = key points and edges information, DF = degree of freedom, LG = local and global coordinate system, CP = Cartesian product of skeleton graph. |
% initiating pseudo 2D % |
Pseudo 2D stick model ← [] |
P2DSM_Size ← Get P2DSM_Size () |
% for loop on segmented silhouettes frames of all interaction classes % |
For I = 1:N |
P2DSM_interactions ← GetP2DSM(interactions) |
%Extracting HD, SN, SS, FG, VD, HK, KE, DF, LG, CP% |
Human key points ← HD(P2DSM_interactions) |
Self-connection with each node ← SN(P2DSM_interactions) |
Scaling of sticks and key points ← SS(P2DSM_interactions) |
Fix undirected skeleton graph ← FG(P2DSM_interactions) |
Volumetric data ← VD(P2DSM_interactions) |
Key points tracking ← HK(P2DSM_interactions) |
Key points and edges information ← KEP2DSM_interactions) |
Degree of freedom with root position ← DF (P2DSM_interactions) |
Local and global coordinate system ← LG(P2DSM_interactions) |
Cartesian product of skeleton graph ← CP(P2DSM_interactions) |
Pseudo 2D stick model ← Get P2DSM |
Pseudo 2D stick model.append (P2DSM) |
End |
Pseudo 2D stick model ← Normalize (pseudo 2D stick model) |
return Pseudo 2D stick model (p1, p2, p3, …, pn) |
3.4. Context-Aware Features
Algorithm 3 Context-aware feature extraction. |
Input: N: Segmented silhouettes frames of RGB images |
Output: context- awarefeature vectors(f1,f2,f3,…,fn) |
% initiating feature vector for sustainable event classification % |
context-awarefeature-vectors ← [] |
Fearurevectorsize ← GetVectorsize () |
%for loop on segmented silhouettes frames of all interaction classes % |
For i = 1:N |
Featuresvectors_ interactions ← Getfeaturesvectors(interactions) |
% extracting energy features, disting motion body parts flow, 3D cartesian view smoothing gradient, sine features , multi points auto correlation, rich 2D appearance feature% |
Energy Features ← ExtractEnergyFeatures(Featuresvectors _interactions) |
Disting Motion Body Parts Flow←ExtractdistinctMotionBodyPartsFlowFeatures (Featuresvectors _interactions) |
3D Cartesian View Smoothing Gradient ← |
Extract3DCartesianViewSmoothingGradientFeatures(Featuresvectors _interactions) |
Sine Features ← ExtractSineFeatures(Featuresvectors _interactions) |
Multi Points Auto correlation ← ExtractMultiPointsAutocorrelation(Featuresvectors _interactions) |
Rich 2D Appearance Feature ← ExtractRich2DAppearanceFeatures(Featuresvectors _interactions) |
Vectors Angle Point features ← ExtractVectorsAnglePointFeatures(Featuresvectors _interactions) |
Feature-vectors ← GetFeaturevector |
Context-aware Feature-vectors.append (Feature-vectors) |
End |
Context-aware Feature-vectors ← Normalize (context-aware Feature-vectors) |
return context-awarefeature-vectors(f1,f2,f3,…,fn) |
3.4.1. Full Body: Energy Feature
3.4.2. Full Body: Distinct Motion Body Parts Flow Features
3.4.3. Full Body: 3D Cartesian View Smoothing Gradient Features
3.4.4. Full Body: Sine Features
3.4.5. Key Body Points: Angle Point Feature
3.4.6. Key Body Points: Multi-Points Autocorrelation Features
3.4.7. Key Body Points: Rich 2D Appearance Feature
3.5. Sustainable Event Optimization: Ray Optimization
3.6. Sustainable Event Classification: K-ary Tree Hashing Algorithm
Algorithm 4 Naïve approach. |
Require: L, Ni |
Ensure: T(v) |
% N is neighbor, L is Data, and T is size fixing approach% |
1. Temp ← sort (L(Ni)) 2. j ← min(j,|Ni|) 3. t(i) ← [i, index(temp(1 : j)] |
4. Experimental Results and Analysis
4.1. Datasets Description
4.1.1. UCF50
4.1.2. hmdb51
4.1.3. Olympic Sports
4.2. Experimental Analysis
4.2.1. Experiment 1: Human Body Key Points Detection
4.2.2. Experiment 2: Event Classification over the UCF50 Dataset
4.2.3. Experiment 3: Event Classification over the Hmdb51 Dataset
4.2.4. Experiment 4: Event Classification over the Olympic Sports Dataset
4.3. Comparison Analysis
4.3.1. Experiment 5: Comparison Using Various Classifiers
4.3.2. Experiment 6: Comparison of Various Features Combinations
4.3.3. Experiment 7: Event Classification Comparison with State-of-the-Art Methods
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Tzelepis, C.; Ma, Z.; Mezaris, V.; Ionescu, B.; Kompatsiaris, I.; Boato, G.; Yan, S. Event-based media processing and analysis: A survey of the literature. I&VC 2016, 53, 3–19. [Google Scholar]
- Susan, S.; Agrawal, P.; Mittal, M.; Bansal, S. New shape descriptor in the context of edge continuity. CAAI Trans. Intell. Technol. 2019, 4, 101–109. [Google Scholar] [CrossRef]
- Tingting, Y.; Junqian, W.; Lintai, W.; Yong, X. Three-stage network for age estimation. CAAI Trans. Intell. Technol. 2019, 4, 122–126. [Google Scholar] [CrossRef]
- Zhu, C.; Miao, D. Influence of kernel clustering on an RBFN. CAAI Trans. Intell. Technol. 2019, 4, 255–260. [Google Scholar] [CrossRef]
- Jalal, A.; Sharif, N.; Kim, J.T.; Kim, T.-S. Human activity recognition via recognized body parts of human depth silhouettes for residents monitoring services at smart homes. Indoor Built Environment. 2013, 22, 271–279. [Google Scholar] [CrossRef]
- Jalal, A.; Kamal, S.; Kim, D. Individual Detection-Tracking-Recognition using depth activity images. In Proceedings of the 12th IEEE International Conference on Ubiquitous Robots and Ambient Intelligence, KINTEX, Goyang City, Korea, 28–30 October 2015; pp. 450–455. [Google Scholar]
- Jalal, A.; Majid, A.; Quaid, K.; Hasan, A.S. Wearable Sensor-Based Human Behavior Understanding and Recognition in Daily Life for Smart Environments. In Proceedings of the IEEE Conference on International Conference on Frontiers of Information Technology, Islamabad, Pakistan, 17–19 December 2018; pp. 105–110. [Google Scholar]
- Jalal, A.; Mahmood, M.; Sidduqi, M.A. Robust spatio-temporal features for human interaction recognition via artificial neural network. In Proceedings of the IEEE International Conference on Frontiers of Information Technology, Islamabad, Pakistan, 17–19 December 2018; pp. 218–223. [Google Scholar] [CrossRef]
- Jalal, A.; Mahmood, M.; Hasan, A.S. Multi-features descriptors for human activity tracking and recognition in Indoor-outdoor environments. In Proceedings of the 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, Pakistan, 8–12 January 2019; pp. 371–376. [Google Scholar] [CrossRef]
- Jalal, A.; Nadeem, A.; Bobasu, S. Human body parts estimation and detection for physical sports movements. In Proceedings of the 2nd International Conference on Communication, Computing and Digital Systems (C-CODE), Islamabad, Pakistan, 6–7 March 2019; pp. 104–109. [Google Scholar] [CrossRef]
- Ahmed, A.; Jalal, A.; Rafique, A.A. Salient Segmentation based Object Detection and Recognition using Hybrid Genetic Transform. In Proceedings of the 2019 International Conference on Applied and Engineering Mathematics (ICAEM), Taxila, Pakistan, 27–29 August 2019; pp. 203–208. [Google Scholar] [CrossRef]
- Nadeem, A.; Jalal, A.; Kim, K. Human actions tracking and recognition based on body parts detection via Artificial neural network. In Proceedings of the 3rd International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan, 17–19 February 2020; pp. 1–6. [Google Scholar] [CrossRef]
- Badar, S.; Jalal, A.; Batool, M. Wearable Sensors for Activity Analysis using SMO-based Random Forest over Smart home and Sports Datasets. In Proceedings of the 3rd International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan, 17–19 February 2020; pp. 1–6. [Google Scholar] [CrossRef]
- Badar, S.; Jalal, A.; Kim, K. Wearable Inertial Sensors for Daily Activity Analysis Based on Adam Optimization and the Maximum Entropy Markov Model. Entropy 2020, 22, 1–19. [Google Scholar]
- Rehman, M.A.; Raza, H.; Akhter, I. Security enhancement of hill cipher by using non-square matrix approach. In Proceedings of the 4th International Conference on Knowledge and Innovation in Engineering Science and Technology, Berlin, Germany, 21–23 December 2018. [Google Scholar] [CrossRef]
- Wiens, T. Engine speed reduction for hydraulic machinery using predictive algorithms. Int. J. Hydromech. 2019, 2, 16–31. [Google Scholar] [CrossRef]
- Li, Z.; Miao, Y.; Cen, X.-P.; Zhang, L.; Chen, S. Abnormal event detection in surveillance videos based on low-rank and compact coefficient dictionary learning. Pattern Recognit. 2020, 108, 107355. [Google Scholar] [CrossRef]
- Einfalt, M.; Dampeyrou, C.; Zecha, D.; Lienhart, R. Frame-level event detection in athletics videos with pose-based convolutional sequence networks. In Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports, New York, NY, USA, 6–8 October 2019; pp. 42–50. [Google Scholar] [CrossRef] [Green Version]
- Yu, J.; Lei, A.; Hu, Y. Soccer video event detection based on deep learning. In Proceedings of the International Conference on Multimedia Modeling, Thessaloniki, Greece, 8–11 January 2019; pp. 377–389. [Google Scholar] [CrossRef]
- Franklin, R.J.; Dabbagol, V. Anomaly Detection in Videos for Video Surveillance Applications Using Neural Networks. In Proceedings of the 2020 Fourth International Conference on Inventive Systems and Control (ICISC), Coimbatore, India, 8–10 January 2020; pp. 632–637. [Google Scholar] [CrossRef]
- Lohithashva, B.H.; Aradhya, V.N.M.; Guru, D.S. Violent video event detection based on integrated LBP and GLCM texture features. Revue Intell. Artif. 2020, 34, 179–187. [Google Scholar] [CrossRef]
- Feng, Q.; Gao, C.; Wang, L.; Zhao, Y.; Song, T.; Li, Q. Spatio-temporal fall event detection in complex scenes using attention guided LSTM. Pattern Recognit. Lett. 2020, 130, 242–249. [Google Scholar] [CrossRef]
- Rado, D.; Sankaran, A.; Plasek, J.; Nuckley, D.; Keefe, D.F. A Real-Time Physical Therapy Visualization Strategy to Improve Unsupervised Patient Rehabilitation. In Proceedings of the IEEE Transactions on Visualization and Computer Graphics, Atlantic City, NJ, USA, 11–16 October 2009; Volume 15, pp. 1–2. [Google Scholar]
- Khan, M.H.; Zöller, M.; Farid, M.S.; Grzegorzek, M. Marker-Based Movement Analysis of Human Body Parts in Therapeutic Procedure. Sensors 2020, 20, 3312. [Google Scholar] [CrossRef] [PubMed]
- Mokhlespour Esfahani, M.I.; Zobeiri, O.; Moshiri, B.; Narimani, R.; Mehravar, M.; Rashedi, E.; Parnianpour, M. Trunk Motion System (TMS) Using Printed Body Worn Sensor (BWS) via Data Fusion Approach. Sensors 2017, 17, 112. [Google Scholar] [CrossRef] [PubMed]
- Golestani, N.; Moghaddam, M. Human activity recognition using magnetic induction-based motion signals and deep recurrent neural networks. Nat. Commun. 2020, 11, 1551. [Google Scholar] [CrossRef] [PubMed]
- Jalal, A.; Kamal, S.; Kim, D. Depth Silhouettes Context: A new robust feature for human tracking and activity recognition based on embedded HMMs. In Proceedings of the 12th International Conference on Ubiquitous Robots and Ambient Intelligence, KINTEX, Goyang City, Korea, 28–30 October 2015; pp. 294–299. [Google Scholar]
- Zhang, J.; Hu, J. Image segmentation based on 2D Otsu method with histogram analysis. In Proceedings of the 2008 International Conference on Computer Science and Software Engineering, Wuhan, China, 12–14 December 2008; pp. 105–108. [Google Scholar] [CrossRef]
- Moschini, D.; Fusiello, A. Tracking human motion with multiple cameras using an articulated model. In Proceedings of the International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications, Rocquencourt, France, 4–6 May 2009; pp. 1–12. [Google Scholar] [CrossRef]
- Li, H.; Lu, H.; Lin, Z.; Shen, X.; Price, B. Inner and inter label propagation: Salient object detection in the wild. IEEE Trans. Image Process. 2015, 24, 3176–3186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Jalal, A.; Kim, Y.H.; Kim, Y.J.; Kamal, S.; Kim, D. Robust human activity recognition from depth video using spatiotemporal multi-fused features. Pattern Recognit. 2017, 61, 295–308. [Google Scholar] [CrossRef]
- Kaveh, A.; Khayatazad, M. A new meta-heuristic method: Ray optimization. Comput. Struct. 2012, 112, 283–294. [Google Scholar] [CrossRef]
- Wu, W.; Li, B.; Chen, L.; Zhu, X.; Zhang, C. K -Ary Tree Hashing for Fast Graph Classification. IEEE Trans. Knowl. Data Eng. 2017, 30, 936–949. [Google Scholar] [CrossRef]
- Reddy, K.K.; Shah, M. Recognizing 50 human action categories of web videos. Mach. Vis. Appl. 2013, 24, 971–981. [Google Scholar] [CrossRef] [Green Version]
- Kuehne, H.; Jhuang, H.; Garrote, E.; Poggio, T.; Serre, T. HMDB: A large video database for human motion recognition. In Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 2556–2563. [Google Scholar] [CrossRef] [Green Version]
- Niebles, J.C.; Chen, C.W.; Fei-Fei, L. Modeling temporal structure of decomposable motion segments for activity classification. In Proceedings of the European Conference on Computer Vision, Heraklion, Crete, Greece, 5–11 September 2010; pp. 392–405. [Google Scholar]
- Wang, L.; Zhang, Y.; Feng, J. On the Euclidean distance of images. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27, 1334–1339. [Google Scholar] [CrossRef]
- Jain, M.; Jegou, H.; Bouthemy, P. Better exploiting motion for better action recognition. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 2555–2562. [Google Scholar] [CrossRef] [Green Version]
- Shi, F.; Petriu, E.; Laganiere, R. Sampling strategies for real-time action recognition. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 2595–2602. [Google Scholar] [CrossRef]
- Uijlings, J.; Duta, I.C.; Sangineto, E.; Sebe, N. Video classification with densely extracted hog/hof/mbh features: An evaluation of the accuracy/computational efficiency trade-off. Int. J. Multimed. Inf. Retr. 2015, 4, 33–44. [Google Scholar] [CrossRef]
- Wang, H.; Oneata, D.; Verbeek, J.; Schmid, C. A robust and efficient video representation for action recognition. Int. J. Comput. Vis. 2016, 119, 219–238. [Google Scholar] [CrossRef] [Green Version]
- Hara, K.; Kataoka, H.; Satoh, Y. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 6546–6555. [Google Scholar] [CrossRef] [Green Version]
- Li, Y.; Liu, C.; Ji, Y.; Gong, S.; Xu, H. Spatio-Temporal Deep Residual Network with Hierarchical Attentions for Video Event Recognition. ACM Trans. MCCA 2020, 16, 1–21. [Google Scholar] [CrossRef]
- Meng, Q.; Zhu, H.; Zhang, W.; Piao, X.; Zhang, A. Action Recognition Using Form and Motion Modalities. ACM Trans. MCCA 2020, 16, 1–16. [Google Scholar] [CrossRef]
- Sun, S.; Kuang, Z.; Sheng, L.; Ouyang, W.; Zhang, W. Optical flow guided feature: A fast and robust motion representation for video action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 1390–1399. [Google Scholar] [CrossRef] [Green Version]
- Park, E.; Han, X.; Berg, T.L.; Berg, A.C. Combining multiple sources of knowledge in deep cnns for action recognition. In Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), New York, NY, USA, 7–9 March 2016; pp. 1–8. [Google Scholar] [CrossRef]
- Torpey, D.; Celik, T. Human Action Recognition using Local Two-Stream Convolution Neural Network Features and Support Vector Machines. arXiv 2020, arXiv:2002.09423. Available online: https://arxiv.org/abs/2002.09423 (accessed on 19 February 2020).
- Zhu, Y.; Zhou, K.; Wang, M.; Zhao, Y.; Zhao, Z. A comprehensive solution for detecting events in complex surveillance videos. Multimed. Tools. Appl. 2019, 78, 817–838. [Google Scholar] [CrossRef]
- Zhang, L.; Xiang, X. Video event classification based on two-stage neural network. Multimed. Tools. Appl. 2020, 1–16. [Google Scholar] [CrossRef]
- Nadeem, A.; Jalal, A.; Kim, K. Accurate Physical Activity Recognition using Multidimensional Features and Markov Model for Smart Health Fitness. Symmetry 2020, 12, 1766. [Google Scholar] [CrossRef]
Body Key Points | Distance | UCF50 | Distance | Hmdb51 | Distance | Olympic Sports |
---|---|---|---|---|---|---|
HP | 9.3 | 91 | 11.3 | 85 | 9.9 | 90 |
NP | 9.5 | 84 | 9.7 | 87 | 10.8 | 83 |
RSP | 9.7 | 81 | 9.3 | 83 | 10.7 | 82 |
REP | 11.0 | 76 | 10.2 | 78 | 12.1 | 80 |
RWP | 9.4 | 72 | 10.7 | 75 | 9.7 | 75 |
RHP | 12.3 | 83 | 11.3 | 84 | 11.4 | 80 |
LSP | 11.2 | 82 | 12.7 | 84 | 13.2 | 81 |
LEP | 10.3 | 77 | 12.9 | 79 | 11.1 | 80 |
LWP | 11.5 | 71 | 10.8 | 77 | 12.5 | 74 |
LHP | 9.3 | 82 | 10.3 | 86 | 8.8 | 85 |
MP | 10.3 | 92 | 9.3 | 92 | 11.0 | 91 |
RHP | 11.7 | 74 | 10.6 | 80 | 10.9 | 79 |
LHP | 13.0 | 74 | 11.5 | 76 | 13.4 | 80 |
LKP | 12.1 | 85 | 13.2 | 83 | 11.3 | 80 |
RKP | 11.9 | 87 | 9.8 | 86 | 12.9 | 82 |
RAP | 10.2 | 78 | 11.5 | 78 | 9.7 | 79 |
LAP | 10.5 | 74 | 13.5 | 76 | 12.7 | 70 |
LFP | 9.9 | 85 | 10.8 | 91 | 11.3 | 90 |
RFP | 8.2 | 90 | 9.3 | 80 | 10.2 | 92 |
Mean Accuracy Rate | 80.9% | 82.10% | 81.7% |
Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
BP | 0.9 | BB | 1.0 | BE | 0.9 | BK | 0.8 | CJ | 0.7 | DI | 0.8 | FE | 1.0 |
GS | 1.0 | HJ | 0.9 | HR | 0.9 | HO | 0.8 | HH | 0.9 | JT | 0.9 | JB | 0.9 |
JJ | 1.0 | JR | 0.9 | KK | 0.9 | LU | 0.8 | NC | 0.9 | PT | 0.9 | PV | 0.9 |
PH | 0.9 | PU | 1.0 | PU | 0.9 | PU | 0.9 | RD | 1.0 | RC | 1.0 | RO | 0.9 |
SS | 0.9 | SB | 0.9 | SK | 0.9 | SK | 0.9 | SJ | 1.0 | SW | 0.9 | TA | 1.0 |
TS | 0.8 | TD | 0.9 | TJ | 0.9 | VS | 1.0 | WD | 0.9 | YY | 0.8 | ||
Mean event classification accuracy = 90.48% |
Events | Precision | Recall | F1-Score | Events | Precision | Recall | F1-Score |
---|---|---|---|---|---|---|---|
Baseball pitch | 0.429 | 0.900 | 0.581 | Pull ups | 1.000 | 0.769 | 0.857 |
Basketball | 0.714 | 0.714 | 0.714 | Punch | 0.900 | 0.750 | 0.870 |
Bench press | 1.000 | 0.818 | 0.900 | Push-ups | 0.900 | 0.900 | 0.818 |
Biking | 0.727 | 0.800 | 0.762 | Rock climbing indoor | 1.000 | 0.833 | 0.900 |
Clean and jerk | 0.778 | 0.636 | 0.700 | Rope climbing | 0.769 | 0.769 | 0.909 |
Diving | 0.320 | 0.800 | 0.457 | Rowing | 0.900 | 0.900 | 0.769 |
Fencing | 1.000 | 0.833 | 0.909 | Salsa spin | 0.818 | 0.692 | 0.900 |
Golf swing | 1.000 | 0.769 | 0.870 | Skateboarding | 0.750 | 0.900 | 0.750 |
High jump | 0.692 | 0.750 | 0.720 | Skiing | 1.000 | 0.900 | 0.818 |
Horse race | 0.750 | 0.750 | 0.750 | Skijet | 0.750 | 1.000 | 0.947 |
Horse riding | 0.400 | 0.800 | 0.533 | Soccer juggling | 1.000 | 0.909 | 0.857 |
Hula hoop | 1.000 | 0.818 | 0.900 | Swing | 0.900 | 0.818 | 0.952 |
Javelin throw | 0.563 | 0.750 | 0.643 | TaiChi | 1.000 | 0.833 | 0.857 |
Juggling balls | 1.000 | 0.900 | 0.643 | Tennis swing | 0.800 | 0.818 | 0.909 |
Jumping jack | 0.909 | 0.833 | 0.947 | Throw discus | 0.750 | 0.818 | 0.809 |
Jump rope | 0.900 | 10.000 | 0.870 | Trampoline jumping | 0.692 | 0.750 | 0.783 |
Kayaking | 0.900 | 10.000 | 1.651 | Volleyball spiking | 0.909 | 0.769 | 0.720 |
Lunges | 0.889 | 0.727 | 1.651 | Walking with dog | 1.000 | 0.818 | 0.833 |
Nunchucks | 0.818 | 0.750 | 0.800 | Yo-yo | 1.000 | 0.800 | 0.900 |
Pizza tossing | 1.000 | 0.900 | 0.783 | Pommel horse | 1.000 | 0.750 | 0.857 |
Pole vault | 1.000 | 0.750 | 0.947 |
Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CW | 1.0 | CA | 0.9 | DA | 1.0 | DI | 0.9 | DS | 0.8 | DI | 1.0 | DS | 0.8 |
DB | 0.9 | FF | 0.9 | FC | 0.8 | FF | 0.9 | GO | 1.0 | HS | 0.8 | HI | 0.9 |
HU | 0.9 | JU | 1.0 | KI | 0.8 | KB | 0.9 | PI | 0.8 | PU | 1.0 | PU | 0.9 |
PU | 0.8 | PU | 1.0 | RB | 0.8 | RH | 0.9 | RU | 1.0 | SH | 0.8 | SB | 0.9 |
SB | 0.8 | SG | 0.9 | SU | 1.0 | SS | 0.8 | SB | 0.8 | SW | 1.0 | SE | 0.9 |
TH | 0.9 | TU | 0.8 | WA | 0.9 | ||||||||
Mean event classification accuracy = 89.21% |
Events | Precision | Recall | F1-Score | Events | Precision | Recall | F1-Score |
---|---|---|---|---|---|---|---|
Cartwheel | 0.909 | 0.833 | 0.870 | Punch | 0.900 | 0.900 | 0.900 |
Catch | 0.900 | 0.750 | 0.818 | Push | 0.889 | 0.889 | 0.889 |
Dap | 0.909 | 0.833 | 0.870 | Pushup | 0.909 | 0.909 | 0.909 |
Dimb | 0.750 | 0.900 | 0.818 | ride_bike | 0.889 | 0.889 | 0.889 |
dimb_stairs | 0.800 | 0.889 | 0.842 | ride_horse | 1.000 | 0.750 | 0.857 |
Dive | 0.909 | 0.909 | 0.909 | Run | 0.909 | 0.909 | 0.909 |
draw_sword | 0.727 | 0.889 | 0.800 | shake_hands | 0.727 | 0.889 | 0.800 |
Dribble | 1.000 | 0.818 | 0.900 | shoot_ball | 0.900 | 0.900 | 0.900 |
fall_floor | 0.900 | 0.900 | 0.900 | shoot_bow | 1.000 | 0.889 | 0.941 |
Fencing | 0.889 | 0.889 | 0.889 | shoot_gun | 0.818 | 0.818 | 0.818 |
flic_flac | 0.750 | 0.900 | 0.818 | Situp | 1.000 | 0.833 | 0.909 |
Golf | 0.769 | 0.833 | 0.800 | Somersault | 0.889 | 0.889 | 0.889 |
handstand | 0.800 | 0.889 | 0.842 | swing_baseball | 0.889 | 0.800 | 0.842 |
Hit | 0.900 | 0.818 | 0.857 | Sword | 0.909 | 0.909 | 0.909 |
Hug | 1.000 | 0.900 | 0.947 | sword_exercise | 1.000 | 0.900 | 0.947 |
jump | 0.909 | 1.000 | 0.952 | Throw | 0.818 | 0.818 | 0.818 |
Kick | 0.889 | 0.889 | 0.889 | Turn | 0.800 | 0.889 | 0.842 |
kick_ball | 0.900 | 0.900 | 0.900 | Walk | 1.000 | 0.900 | 0.947 |
Pick | 0.727 | 0.800 | 0.762 | Pullup | 0.667 | 0.833 | 0.741 |
Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc | Event | Acc |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
BO | 1.0 | DT | 0.9 | DP | 0.9 | HT | 0.8 | JT | 0.9 | LJ | 0.9 | PV | 1.0 |
SP | 0.9 | SN | 0.8 | TM | 0.9 | TJ | 1.0 | VA | 0.9 | ||||
Mean Event Classification Accuracy = 90.83% |
Events | Precision | Recall | F1-Score | Events | Precision | Recall | F1-Score |
---|---|---|---|---|---|---|---|
Bowling | 0.769 | 0.833 | 0.800 | pole_vault | 1.000 | 0.909 | 0.952 |
discus_throw | 0.692 | 0.900 | 0.783 | shot_put | 1.000 | 0.818 | 0.900 |
diving_platform_10m | 1.000 | 0.750 | 0.857 | snatch | 1.000 | 0.800 | 0.889 |
hammer_throw | 0.889 | 0.727 | 0.800 | toolbox-master | 0.750 | 0.818 | 0.783 |
javelin_throw | 0.900 | 0.900 | 0.900 | triple_jump | 0.769 | 0.909 | 0.833 |
long_jump | 0.643 | 0.900 | 0.750 | vault | 1.000 | 0.900 | 0.947 |
Classifiers | Dataset | Accuracy | Dataset | Accuracy | Dataset | Accuracy |
---|---|---|---|---|---|---|
ANN | UCF50 | 84.63 | hmdb51 | 83.94 | Olympic sports | 85.8 |
G.A | UCF50 | 86.34 | hmdb51 | 86.31 | Olympic sports | 83.3 |
Adaboost | UCF50 | 85.36 | hmdb51 | 88.42 | Olympic sports | 86.6 |
K-ary Tree | UCF50 | 90.48 | hmdb51 | 89.21 | Olympic sports | 90.83 |
Features Name | UCF50/Accuracy | hmdb51/Accuracy | Olympic Sports/Accuracy |
---|---|---|---|
EF,DMBF, SF | 76.09 | 74.21 | 75.83 |
3D-CVM, APF, MPA, R-2DA, SF | 82.68 | 80.78 | 81.66 |
EF,DMBF, 3D-CVM, APF, MPA, R-2DA, SF | 90.48 | 89.21 | 90.83 |
Methods | UCF50 | Methods | hmdb51 | Methods | Olympic Sports |
---|---|---|---|---|---|
J. Uijlings [40] | 81.8% | M. Jain et al [38] | 52.10% | L. Zhang [49] | 59.1% |
F. Shi [39] | 83.3% | H. Wang [41] | 60.10% | S. Sun [45] | 74.2% |
Y. Zhu [48] | 83.1% | D. Torpey [47] | 62.80% | M. Jain et al. [38] | 83.2% |
D. Torpey [47] | 86.4% | Y. Li [43] | 70.69% | E. Park [46] | 89.1% |
L. Zhang [49] | 88.0% | K. Hara [42] | 70.20% | H. Wang [41] | 89.6% |
H. Wang [41] | 89.1% | Y. Zhu [48] | 76.30% | A. Nadeem [50] | 88.26% |
Q. Meng [44] | 89.3% | A. Nadeem [50] | 89.09% | --- | --- |
Ours | 90.48 | 89.21 | 90.83 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jalal, A.; Akhtar, I.; Kim, K. Human Posture Estimation and Sustainable Events Classification via Pseudo-2D Stick Model and K-ary Tree Hashing. Sustainability 2020, 12, 9814. https://doi.org/10.3390/su12239814
Jalal A, Akhtar I, Kim K. Human Posture Estimation and Sustainable Events Classification via Pseudo-2D Stick Model and K-ary Tree Hashing. Sustainability. 2020; 12(23):9814. https://doi.org/10.3390/su12239814
Chicago/Turabian StyleJalal, Ahmad, Israr Akhtar, and Kibum Kim. 2020. "Human Posture Estimation and Sustainable Events Classification via Pseudo-2D Stick Model and K-ary Tree Hashing" Sustainability 12, no. 23: 9814. https://doi.org/10.3390/su12239814