Toward Improving Human Training by Combining Wearable Full-Body IoT Sensors and Machine Learning
Abstract
:1. Introduction
- Digital twins of workers that continuously synthesise avatar-like kinematic models of the activities for each worker being trained. In doing so, it uses wearable sensors that observe how individual workers perform physical work activities;
- A hybrid GAN-ML work activity recognition model for recognising the types of work activities each worker performs;
- Skill proficiency recognition analysis for evaluating how well each trainee performs specific work activities and the overall task he/she is responsible for;
- An industry study that illustrates how highly accurate work activity recognition GAN-ML models can detect complex meat processing activities using body movement data from full-body wearable IoT sensors.
2. Related Work
3. DigitalUpSkilling Framework for Digital Personalised Training
3.1. Data Collection Using Wearable IoT Sensors and Generation of Workers’ Digital Twins
3.2. Work Activity Recognition
3.3. Skill Proficiency ML
4. Study of Work Activity Recognition for Meat Processing Activities
4.1. Data Labelling
- n represents the total number of sensors;
- , , and represent the sensor readings for the ith sensor for , and , respectively.
4.2. Data Preprocessing
- Handling Missing Values
- Train–Test Split
4.3. Generating Synthetic Data Using GAN
- Generator: The generator takes a random noise vector, and outputs synthetic data, , in the same feature space as real data. The generator uses Leaky ReLU activations for hidden layers.
- Leaky ReLU activation
- Discriminator: The discriminator, , takes input, and outputs a probability, that the input is real. It is trained to distinguish between real data, , and synthetic data, .
- Discriminator loss for real data
- Discriminator loss for fake data
- Training GAN
- Synthetic data generation
4.4. Handling Class Imbalance with SMOTE
4.5. Data Cleaning with ENN
4.6. Classification with RF Model
- True positives: Data correctly predicted as positive;
- False positives: Data incorrectly predicted as positive;
- False negatives: Data incorrectly predicted as negative;
- True negatives: Data correctly predicted as negative.
5. Performance of the Work Activity Recognition Model
6. Discussion of Skill Proficiency
7. Conclusions and Future Directions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Bosse, H.M.; Mohr, J.; Buss, B.; Krautter, M.; Weyrich, P.; Herzog, W.; Jünger, J.; Nikendei, C. The Benefit of Repetitive Skills Training and Frequency of Expert Feedback in the Early Acquisition of Procedural Skills. BMC Med. Educ. 2015, 15, 22. [Google Scholar] [CrossRef] [PubMed]
- Guthrie, J.P. High-Involvement Work Practices, Turnover, and Productivity: Evidence from New Zealand. Acad. Manag. J. 2001, 44, 180–190. [Google Scholar] [CrossRef]
- Salas, E.; Tannenbaum, S.I.; Kraiger, K.; Smith-Jentsch, K.A. The Science of Training and Development in Organizations. Psychol. Sci. Public Interest 2012, 13, 74–101. [Google Scholar] [CrossRef] [PubMed]
- Schonewille, M. Does Training Generally Work? Explaining labour productivity effects from schooling and training. Int. J. Manpow. 2001, 22, 158–173. [Google Scholar] [CrossRef]
- Anders Ericsson, K. Deliberate Practice and Acquisition of Expert Performance: A General Overview. Acad. Emerg. Med. 2008, 15, 988–994. [Google Scholar] [CrossRef]
- Analoui, F. Training and Development. J. Manag. Dev. 1994, 13, 61–72. [Google Scholar] [CrossRef]
- Urbancová, H.; Vrabcová, P.; Hudáková, M.; Petrů, G.J. Effective Training Evaluation: The Role of Factors Influencing the Evaluation of Effectiveness of Employee Training and Development. Sustainability 2021, 13, 2721. [Google Scholar] [CrossRef]
- Pashler, H.; McDaniel, M.; Rohrer, D.; Bjork, R. Learning Styles. Psychol. Sci. Public Interest 2008, 9, 105–119. [Google Scholar] [CrossRef]
- Jenkins, J.M. Strategies for Personalizing Instruction, Part Two. Int. J. Educ. Reform 1999, 8, 83–88. [Google Scholar] [CrossRef]
- Morgan, P.J.; Barnett, L.M.; Cliff, D.P.; Okely, A.D.; Scott, H.A.; Cohen, K.E.; Lubans, D.R. Fundamental Movement Skill Interventions in Youth: A Systematic Review and Meta-Analysis. Pediatrics 2013, 132, e1361–e1383. [Google Scholar] [CrossRef]
- Arthur, W.; Bennett, W.; Edens, P.S.; Bell, S.T. Effectiveness of Training in Organizations: A Meta-Analysis of Design and Evaluation Features. J. Appl. Psychol. 2003, 88, 234–245. [Google Scholar] [CrossRef] [PubMed]
- Zhang, L.; Diraneyya, M.M.; Ryu, J.; Haas, C.T.; Abdel-Rahman, E.M. Jerk as an Indicator of Physical Exertion and Fatigue. Autom. Constr. 2019, 104, 120–128. [Google Scholar] [CrossRef]
- Georgakopoulos, D.; Jayaraman, P.P. Internet of Things: From Internet Scale Sensing to Smart Services. Computing 2016, 98, 1041–1058. [Google Scholar] [CrossRef]
- Hidaka, I.; Inoue, S.; Zin, T.T. A Study on Worker Tracking Using Camera to Improve Work Efficiency in Factories. In Proceedings of the 2022 IEEE 4th Global Conference on Life Sciences and Technologies (LifeTech), Osaka, Japan, 7–9 March 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 568–569. [Google Scholar]
- Guan, A.C.K.; Mamat, M.J. Bin Allocation of Thermal Imaging Camera: Promoting Efficient Movement in Entrepreneur Workplace. ARTEKS J. Tek. Arsit. 2022, 7, 67–76. [Google Scholar] [CrossRef]
- Chintada, A.; V, U. Improvement of Productivity by Implementing Occupational Ergonomics. J. Ind. Prod. Eng. 2022, 39, 59–72. [Google Scholar] [CrossRef]
- Jegham, I.; Ben Khalifa, A.; Alouani, I.; Mahjoub, M.A. Vision-Based Human Action Recognition: An Overview and Real World Challenges. Forensic Sci. Int. Digit. Investig. 2020, 32, 200901. [Google Scholar] [CrossRef]
- Nawal, Y.; Oussalah, M.; Fergani, B.; Fleury, A. New Incremental SVM Algorithms for Human Activity recognition in Smart Homes. J. Ambient Intell. Humaniz. Comput. 2022, 14, 13433–13450. [Google Scholar] [CrossRef]
- Bock, M.; Hölzemann, A.; Moeller, M.; Van Laerhoven, K. Improving Deep Learning for HAR with Shallow LSTMs. In Proceedings of the 2021 International Symposium on Wearable Computers, Online, 21–26 September 2021; ACM: New York, NY, USA, 2021; pp. 7–12. [Google Scholar]
- Kim, Y.-W.; Joa, K.-L.; Jeong, H.-Y.; Lee, S. Wearable IMU-Based Human Activity Recognition Algorithm for Clinical Balance Assessment Using 1D-CNN and GRU Ensemble Model. Sensors 2021, 21, 7628. [Google Scholar] [CrossRef]
- Fu, D. Research on Intelligent Recognition Technology of Gymnastics Posture Based on KNN Fusion DTW Algorithm Based on Sensor Technology. Int. J. Wirel. Mob. Comput. 2023, 25, 58–67. [Google Scholar] [CrossRef]
- Su, F.; Chen, M. Basketball Players’ Score Prediction Using Artificial Intelligence Technology via the Internet of Things. J. Supercomput. 2022, 78, 19138–19166. [Google Scholar] [CrossRef]
- Meżyk, E.; Unold, O. Machine Learning Approach to Model Sport Training. Comput. Hum. Behav. 2011, 27, 1499–1506. [Google Scholar] [CrossRef]
- Fernandes, C.; Matos, L.M.; Folgado, D.; Nunes, M.L.; Pereira, J.R.; Pilastri, A.; Cortez, P. A Deep Learning Approach to Prevent Problematic Movements of Industrial Workers Based on Inertial Sensors. In Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 18–23 July 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1–8. [Google Scholar]
- Georgakopoulos, D.; Jayaraman, P.P. Digital Twins and Dependency/Constraint-Aware AI for Digital Manufacturing. Commun. ACM 2023, 66, 87–88. [Google Scholar] [CrossRef]
- Luo, X.; Qin, M.S.; Fang, Z.; Qu, Z. Artificial Intelligence Coaches for Sales Agents: Caveats and Solutions. J. Mark. 2021, 85, 14–32. [Google Scholar] [CrossRef]
- Beef and Veal—DAFF. Available online: https://www.agriculture.gov.au/abares/research-topics/agricultural-outlook/beef-and-veal (accessed on 14 March 2024).
- Ampc Cost to Operate and Processing Cost Competitiveness. Available online: https://www.ampc.com.au/getmedia/cddf3a65-fac3-49a9-a0db-986bd25dfd1b/AMPC_CostToOperateAndProcessingCostCompetetitivness_FinalReport.pdf?ext=.pdf (accessed on 14 March 2024).
- Taborri, J.; Keogh, J.; Kos, A.; Santuz, A.; Umek, A.; Urbanczyk, C.; van der Kruk, E.; Rossi, S. Sport Biomechanics Applications Using Inertial, Force, and EMG Sensors: A Literature Overview. Appl. Bionics Biomech. 2020, 2020, 2041549. [Google Scholar] [CrossRef]
- Ahmed, K.; Saini, M. FCML-Gait: Fog Computing and Machine Learning Inspired Human identity and Gender Recognition Using Gait Sequences. Signal Image Video Process. 2023, 17, 925–936. [Google Scholar] [CrossRef]
- Rana, M.; Mittal, V. Wearable Sensors for Real-Time Kinematics Analysis in Sports: A Review. IEEE Sens. J. 2021, 21, 1187–1207. [Google Scholar] [CrossRef]
- Hamidi Rad, M.; Gremeaux, V.; Massé, F.; Dadashi, F.; Aminian, K. SmartSwim, a Novel IMU-Based Coaching Assistance. Sensors 2022, 22, 3356. [Google Scholar] [CrossRef]
- Qin, Z.; Zhang, Y.; Meng, S.; Qin, Z.; Choo, K.-K.R. Imaging and Fusing Time Series for Wearable Sensor-Based Human Activity Recognition. Inf. Fusion 2020, 53, 80–87. [Google Scholar] [CrossRef]
- Yuan, B.; Kamruzzaman, M.M.; Shan, S. Application of Motion Sensor Based on Neural Network in Basketball Technology and Physical Fitness Evaluation System. Wirel. Commun. Mob. Comput. 2021, 2021, 5562954. [Google Scholar] [CrossRef]
- Wiesener, C.; Seel, T.; Axelgaard, J.; Horton, R.; Niedeggen, A.; Schauer, T. An Inertial Sensor-Based Trigger Algorithm for Functional Electrical Stimulation-Assisted Swimming in Paraplegics. IFAC-PapersOnLine 2019, 51, 278–283. [Google Scholar] [CrossRef]
- Chen, K.; Zhang, D.; Yao, L.; Guo, B.; Yu, Z.; Liu, Y. Deep Learning for Sensor-Based Human Activity Recognition: Overview, Challenges, and Opportunities. ACM Comput. Surv. 2022, 54, 77. [Google Scholar] [CrossRef]
- Cescon, M.; Choudhary, D.; Pinsker, J.E.; Dadlani, V.; Church, M.M.; Kudva, Y.C.; Doyle III, F.J.; Dassau, E. Activity Detection and Classification from Wristband Accelerometer Data Collected on People with Type 1 Diabetes in Free-Living Conditions. Comput. Biol. Med. 2021, 135, 104633. [Google Scholar] [CrossRef] [PubMed]
- Straczkiewicz, M.; James, P.; Onnela, J.-P. A Systematic Review of Smartphone-Based Human Activity Recognition Methods for Health Research. NPJ Digit. Med. 2021, 4, 148. [Google Scholar] [CrossRef] [PubMed]
- Wong, J.C.Y.; Wang, J.; Fu, E.Y.; Leong, H.V.; Ngai, G. Activity Recognition and Stress Detection via Wristband. In Proceedings of the 17th International Conference on Advances in Mobile Computing & Multimedia, Munich, Germany, 2–4 December 2019; ACM: New York, NY, USA, 2019; pp. 102–106. [Google Scholar]
- Song, X.; van de Ven, S.S.; Chen, S.; Kang, P.; Gao, Q.; Jia, J.; Shull, P.B. Proposal of a Wearable Multimodal Sensing-Based Serious Games Approach for Hand Movement Training After Stroke. Front. Physiol. 2022, 13, 811950. [Google Scholar] [CrossRef] [PubMed]
- Albert, J.A.; Zhou, L.; Glöckner, P.; Trautmann, J.; Ihde, L.; Eilers, J.; Kamal, M.; Arnrich, B. Will You Be My Quarantine. In Proceedings of the 14th EAI International Conference on Pervasive Computing Technologies for Healthcare, Atlanta, GA, USA, 18–20 May 2020; ACM: New York, NY, USA, 2020; pp. 380–383. [Google Scholar]
- Caserman, P.; Krug, C.; Göbel, S. Recognizing Full-Body Exercise Execution Errors Using the Teslasuit. Sensors 2021, 21, 8389. [Google Scholar] [CrossRef]
- Demircan, E. A Pilot Study on Locomotion Training via Biomechanical Models and a Wearable Haptic Feedback System. Robomech J. 2020, 7, 19. [Google Scholar] [CrossRef]
- Chartomatsidis, M.; Goumopoulos, C. A Balance Training Game Tool for Seniors Using Microsoft Kinect and 3D Worlds. In Proceedings of the 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health, Heraklion, Greece, 2–4 May 2019; Ziefle, M., Maciaszek, L., Eds.; SCITEPRESS—Science and Technology Publications: Setúbal, Portugal, 2019; pp. 135–145. [Google Scholar]
- Bianchi, V.; Bassoli, M.; Lombardo, G.; Fornacciari, P.; Mordonini, M.; De Munari, I. IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment. IEEE Internet Things J. 2019, 6, 8553–8562. [Google Scholar] [CrossRef]
- Ding, W.; Abdel-Basset, M.; Mohamed, R. HAR-DeepConvLG: Hybrid Deep Learning-Based Model for Human Activity Recognition in IoT Applications. Inf. Sci. 2023, 646, 119394. [Google Scholar] [CrossRef]
- Mekruksavanich, S.; Jitpattanakul, A. LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes. Sensors 2021, 21, 1636. [Google Scholar] [CrossRef]
- Subedi, S.; Pradhananga, N. Sensor-Based Computational Approach to Preventing Back Injuries in Construction Workers. Autom. Constr. 2021, 131, 103920. [Google Scholar] [CrossRef]
- Tao, F.; Zhang, H.; Liu, A.; Nee, A.Y.C. Digital Twin in Industry: State-of-the-Art. IEEE Trans. Ind. Inform. 2019, 15, 2405–2415. [Google Scholar] [CrossRef]
- Uhlemann, T.H.-J.; Lehmann, C.; Steinhilper, R. The Digital Twin: Realizing the Cyber-Physical Production System for Industry 4.0. Procedia CIRP 2017, 61, 335–340. [Google Scholar] [CrossRef]
- Kritzinger, W.; Karner, M.; Traar, G.; Henjes, J.; Sihn, W. Digital Twin in Manufacturing: A Categorical Literature Review and Classification. IFAC-PapersOnline 2018, 51, 1016–1022. [Google Scholar] [CrossRef]
- Ryselis, K.; Petkus, T.; Blažauskas, T.; Maskeliūnas, R.; Damaševičius, R. Multiple Kinect Based System to Monitor and Analyze Key Performance Indicators of Physical Training. Hum.-Cent. Comput. Inf. Sci. 2020, 10, 51. [Google Scholar] [CrossRef]
- Upadhyay, A.K.; Khandelwal, K. Artificial Intelligence-Based Training Learning from Application. Dev. Learn. Organ. Int. J. 2019, 33, 20–23. [Google Scholar] [CrossRef]
- Alo, U.R.; Nweke, H.F.; Teh, Y.W.; Murtaza, G. Smartphone Motion Sensor-Based Complex Human Activity Identification Using Deep Stacked Autoencoder Algorithm for Enhanced Smart Healthcare System. Sensors 2020, 20, 6300. [Google Scholar] [CrossRef]
- Yadav, S.K.; Tiwari, K.; Pandey, H.M.; Akbar, S.A. A Review of Multimodal Human Activity Recognition with Special Emphasis on Classification, Applications, Challenges and Future Directions. Knowl. Based Syst. 2021, 223, 106970. [Google Scholar] [CrossRef]
- Forkan, A.R.M.; Montori, F.; Georgakopoulos, D.; Jayaraman, P.P.; Yavari, A.; Morshed, A. An Industrial IoT Solution for Evaluating Workers’ Performance Via Activity Recognition. In Proceedings of the 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), Dallas, TX, USA, 7–9 July 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1393–1403. [Google Scholar]
- Choi, J.; Gu, B.; Chin, S.; Lee, J.-S. Machine Learning Predictive Model Based on National Data for Fatal Accidents of Construction Workers. Autom. Constr. 2020, 110, 102974. [Google Scholar] [CrossRef]
- Sopidis, G.; Ahmad, A.; Haslgruebler, M.; Ferscha, A.; Baresch, M. Micro Activities Recognition and Macro Worksteps Classification for Industrial IoT Processes. In Proceedings of the 11th International Conference on the Internet of Things, St. Gallen, Switzerland, 8–12 November 2021; ACM: New York, NY, USA, 2021; pp. 185–188. [Google Scholar]
- Chen, W.-H.; Cho, P.-C. A GAN-Based Data Augmentation Approach for Sensor-Based Human Activity Recognition. Int. J. Comput. Commun. Eng. 2021, 10, 75–84. [Google Scholar] [CrossRef]
- Jimale, A.O.; Mohd Noor, M.H. Fully Connected Generative Adversarial Network for Human Activity Recognition. IEEE Access 2022, 10, 100257–100266. [Google Scholar] [CrossRef]
- Shafqat, W.; Byun, Y.-C. A Hybrid GAN-Based Approach to Solve Imbalanced Data Problem in Recommendation Systems. IEEE Access 2022, 10, 11036–11047. [Google Scholar] [CrossRef]
- Veigas, K.C.; Regulagadda, D.S.; Kokatnoor, S.A. Optimized Stacking Ensemble (OSE) for Credit Card Fraud Detection Using Synthetic Minority Oversampling Model. Indian J. Sci. Technol. 2021, 14, 2607–2615. [Google Scholar] [CrossRef]
- He, H.; Yang, B.; Garcia, E.A.; Li, S.T. ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning. In Proceedings of the 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), Hong Kong, China, 1–8 June 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 1322–1328. [Google Scholar]
- Sauber-Cole, R.; Khoshgoftaar, T.M. The Use of Generative Adversarial Networks to Alleviate Class Imbalance in Tabular Data: A Survey. J. Big Data 2022, 9, 98. [Google Scholar] [CrossRef]
- Elhassan, A.T.; Aljourf, M.; Al-Mohanna, F.; Shoukri, M. Classification of Imbalance Data Using Tomek Link (T-Link) Combined with Random Under-Sampling (RUS) as a Data Reduction Method. Glob. J. Technol. Optim. 2016, 1, 111. [Google Scholar] [CrossRef]
- Xu, Z.; Shen, D.; Nie, T.; Kou, Y. A Hybrid Sampling Algorithm Combining M-SMOTE and ENN Based on Random Forest for Medical Imbalanced Data. J. Biomed. Inform. 2020, 107, 103465. [Google Scholar] [CrossRef]
- Oytun, M.; Tinazci, C.; Sekeroglu, B.; Acikada, C.; Yavuz, H.U. Performance Prediction and Evaluation in Female Handball Players Using Machine Learning Models. IEEE Access 2020, 8, 116321–116335. [Google Scholar] [CrossRef]
- Pappalardo, L.; Cintia, P.; Ferragina, P.; Massucco, E.; Pedreschi, D.; Giannotti, F. PlayeRank: Data-driven Performance Evaluation and Player Ranking in Soccer via a Machine Learning Approach. ACM Trans. Intell. Syst. Technol. 2019, 10, 59. [Google Scholar] [CrossRef]
- Lam, K.; Chen, J.; Wang, Z.; Iqbal, F.M.; Darzi, A.; Lo, B.; Purkayastha, S.; Kinross, J.M. Machine Learning for Technical Skill Assessment in Surgery: A Systematic Review. NPJ Digit. Med. 2022, 5, 24. [Google Scholar] [CrossRef]
- Matijevich, E.S.; Scott, L.R.; Volgyesi, P.; Derry, K.H.; Zelik, K.E. Combining Wearable Sensor Signals, Machine Learning and Biomechanics to Estimate Tibial Bone Force and Damage during Running. Hum. Mov. Sci. 2020, 74, 102690. [Google Scholar] [CrossRef]
- Patalas-Maliszewska, J.; Pajak, I.; Krutz, P.; Pajak, G.; Rehm, M.; Schlegel, H.; Dix, M. Inertial Sensor-Based Sport Activity Advisory System Using Machine Learning Algorithms. Sensors 2023, 23, 1137. [Google Scholar] [CrossRef]
- Senanayake, D.; Halgamuge, S.; Ackland, D.C. Real-Time Conversion of Inertial Measurement Unit Data to Ankle Joint Angles Using Deep Neural Networks. J. Biomech. 2021, 125, 110552. [Google Scholar] [CrossRef] [PubMed]
- Fraile, F.; Psarommatis, F.; Alarcón, F.; Joan, J. A Methodological Framework for Designing Personalised Training Programs to Support Personnel Upskilling in Industry 5.0. Computers 2023, 12, 224. [Google Scholar] [CrossRef]
- Patki, S.; Sankhe, V.; Jawwad, M.; Mulla, N. Personalised Employee Training. In Proceedings of the 2021 International Conference on Communication information and Computing Technology (ICCICT), Mumbai, India, 25–27 June 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1–6. [Google Scholar]
- Butean, A.; Olescu, M.L.; Tocu, N.A.; Florea, A. Improving Training Methods for Industry Workers Though AI Assisted Multi-Stage Virtual Reality Simulations. Elearning Softw. Educ. 2019, 1, 61–67. [Google Scholar]
- MVN Analyze|Movella.Com. Available online: https://www.movella.com/products/motion-capture/mvn-analyze?utm_feeditemid=&utm_device=c&utm_term=xsens%20mvn%20analyze&utm_source=google&utm_medium=ppc&utm_campaign=&hsa_cam=15264151125&hsa_grp=132748033194&hsa_mt=e&hsa_src=g&hsa_ad=561667410925&hsa_acc=1306794700&hsa_net=adwords&hsa_kw=xsens%20mvn%20analyze&hsa_tgt=aud-1297463063737:kwd-928333037508&hsa_ver=3&utm_feeditemid=&utm_device=c&utm_term=xsens%20mvn%20analyze&utm_source=google&utm_medium=ppc&utm_campaign=HMM+%7C+Asia+%7C+Search&hsa_cam=15264151125&hsa_grp=132748033194&hsa_mt=e&hsa_src=g&hsa_ad=561667410925&hsa_acc=1306794700&hsa_net=adwords&hsa_kw=xsens%20mvn%20analyze&hsa_tgt=aud-1297463063737:kwd-928333037508&hsa_ver=3&gad_source=1&gclid=CjwKCAiA8NKtBhBtEiwAq5aX2Be-eBaXkvEk2ctESXuZnoGwfKpW2j2hAjrwPmjMZKQJww9fYd2l3RoCfRIQAvD_BwE (accessed on 28 January 2024).
- Morshed, M.G.; Sultana, T.; Alam, A.; Lee, Y.-K. Human Action Recognition: A Taxonomy-Based Survey, Updates, and Opportunities. Sensors 2023, 23, 2182. [Google Scholar] [CrossRef] [PubMed]
- Hulshof, C.T.J.; Pega, F.; Neupane, S.; van der Molen, H.F.; Colosio, C.; Daams, J.G.; Descatha, A.; Kc, P.; Kuijer, P.P.F.M.; Mandic-Rajcevic, S.; et al. The Prevalence of Occupational Exposure to Ergonomic Risk Factors: A Systematic Review and Meta-Analysis from the WHO/ILO Joint Estimates of the Work-Related Burden of Disease and Injury. Environ. Int. 2021, 146, 106157. [Google Scholar] [CrossRef]
- He, Z.; Liu, T.; Yi, J. A Wearable Sensing and Training System: Towards Gait Rehabilitation for Elderly Patients With Knee Osteoarthritis. IEEE Sens. J. 2019, 19, 5936–5945. [Google Scholar] [CrossRef]
- Zhao, Y.; You, Y. Design and Data Analysis of Wearable Sports Posture Measurement System Based on Internet of Things. Alex. Eng. J. 2021, 60, 691–701. [Google Scholar] [CrossRef]
Actions and Activities | Names |
---|---|
Work Activities | Idleness, walking, cutting, reaching, slicing, dropping |
Single Actions | body locomotion: sit, stand, walk, bend left-hand actions: spread, reach, open, close, move, unlock, hold, cut, spread, release, drop, pick, throw right-hand actions: spread, reach, open, close, move, unlock, hold, cut, spread, release, drop, pick, throw left-leg actions: still, move, spread, straighten, bend, lift right-leg actions: still, move, spread, straighten, bend, lift |
Activity | Label | Observation |
---|---|---|
Idle | 0 | The participant keeps a hand stationary and waits for the next piece to arrive in the carcass. |
Walking | 1 | The participant walks to get the next piece of meat in the carcass or to move around. |
Steeling | 2 | The participant sharpens the knife with sharpening tools; to do this, they use both hands. |
Reaching | 3 | The participant reaches for a new piece of meat from the carcass. |
Cutting | 4 | Using the knife, the participant turns a large piece of meat into smaller pieces. |
Dropping | 5 | The participant grabs a small piece of separated meat and throws it away on the conveyor belt. |
Activity | Label | Actions Description |
---|---|---|
Idle | 0 | The participant keeps a hand stationary and waits for the next piece to arrive in the carcass. |
Walking | 1 | The participant walks to get the next piece of meat in the carcass or to move around. |
Steeling | 2 | The participant sharpens the knife with sharpening tools; to do this, they use both hands. |
Reaching | 3 | The participant reaches for a new piece of meat on the belt or table. |
Cutting | 4 | The participant turns a large piece of meat into a smaller piece with the help of a knife. |
Slicing | 5 | The participant cuts fats from a meat piece. |
Pulling | 6 | The participant rips away fat/meat from the meat piece. |
Placing/Manipulating | 7 | The participant manipulates the meat placement or pinches the meat. |
Dropping | 8 | The participant grabs a small piece of separated meat and throws it away. |
Sensors | Activity | Velocity | Magnitude | Pitch and Roll | Velocity + Magnitude + Pitch and Roll |
---|---|---|---|---|---|
Right Hand (4 Sensors) | Boning | 0.7371 | 0.8372 | 0.9936 | 0.9921 |
Slicing | 0.4972 | 0.5648 | 0.9779 | 0.9737 | |
Both Hand (8 Sensors) | Boning | 0.8062 | 0.9158 | 0.9984 | 0.9992 |
Slicing | 0.5275 | 0.7359 | 0.9923 | 0.9897 | |
Full Body (17 Sensors) | Boning | 0.8372 | 0.8713 | 0.9984 | 0.9984 |
Slicing | 0.5563 | 0.5988 | 0.9962 | 0.9962 |
Data | Activity | Accuracy | Precision | Recall | F-Score |
---|---|---|---|---|---|
3D acceleration | Boning | 0.8362 ± 0.0011 | 0.8516 ± 0.0048 | 0.8362 ± 0.0011 | 0.8032 ± 0.0101 |
Slicing | 0.596 ± 0.0397 | 0.69 ± 0.0346 | 0.596 ± 0.0397 | 0.5188 ± 0.0604 | |
Magnitude | Boning | 0.8682 ± 0.0031 | 0.8651 ± 0.0056 | 0.8682 ± 0.0031 | 0.8531 ± 0.0087 |
Slicing | 0.6213 ± 0.0225 | 0.6854 ± 0.0082 | 0.6213 ± 0.0225 | 0.566 ± 0.0356 | |
Pitch and roll | Boning | 0.9979 ± 0.0005 | 0.998 ± 0.0004 | 0.9979 ± 0.0005 | 0.9979 ± 0.0005 |
Slicing | 0.9961 ± 0.0001 | 0.9962 ± 0.0001 | 0.9961 ± 0.0001 | 0.9961 ± 0.0001 | |
3D Acceleration, magnitude, pitch and roll | Boning | 0.9982 ± 0.0002 | 0.9982 ± 0.0002 | 0.9982 ± 0.0002 | 0.9982 ± 0.0002 |
Slicing | 0.9963 ± 0.0001 | 0.9964 ± 0.0000 | 0.9963 ± 0.0001 | 0.9963 ± 0.0001 | |
3D Acceleration, magnitude, pitch and roll, centre of mass | Boning | 0.9978 ± 0.0006 | 0.9978 ± 0.0006 | 0.9978 ± 0.0006 | 0.9978 ± 0.0006 |
Slicing | 0.9962 ± 0.0009 | 0.9962 ± 0.0009 | 0.9962 ± 0.0009 | 0.9962 ± 0.0009 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Akter, N.; Molnar, A.; Georgakopoulos, D. Toward Improving Human Training by Combining Wearable Full-Body IoT Sensors and Machine Learning. Sensors 2024, 24, 7351. https://doi.org/10.3390/s24227351
Akter N, Molnar A, Georgakopoulos D. Toward Improving Human Training by Combining Wearable Full-Body IoT Sensors and Machine Learning. Sensors. 2024; 24(22):7351. https://doi.org/10.3390/s24227351
Chicago/Turabian StyleAkter, Nazia, Andreea Molnar, and Dimitrios Georgakopoulos. 2024. "Toward Improving Human Training by Combining Wearable Full-Body IoT Sensors and Machine Learning" Sensors 24, no. 22: 7351. https://doi.org/10.3390/s24227351
APA StyleAkter, N., Molnar, A., & Georgakopoulos, D. (2024). Toward Improving Human Training by Combining Wearable Full-Body IoT Sensors and Machine Learning. Sensors, 24(22), 7351. https://doi.org/10.3390/s24227351