Skip to Content
Applied SciencesApplied Sciences
  • Article
  • Open Access

10 July 2025

33 Pages

Multimodal Hidden Markov Models for Real-Time Human Proficiency Assessment in Industry 5.0: Integrating Physiological, Behavioral, and Subjective Metrics

and
1
Industrial and Manufacturing Engineering Department, Pennsylvania State University, State College, PA 16802, USA
2
Department of Architectural Engineering and Construction Management, King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia
*
Author to whom correspondence should be addressed.

Abstract

This paper presents a Multimodal Hidden Markov Model (MHMM) framework specifically designed for real-time human proficiency assessment, integrating physiological (Heart Rate Variability (HRV)), behavioral (Task Completion Time (TCT)), and subjective (NASA Task Load Index (NASA-TLX)) data streams to infer latent human proficiency states in industrial settings. Using published empirical data from the surgical training literature, a comprehensive simulation study was conducted, with the MHMM (Trained) achieving 92.5% classification accuracy, significantly outperforming unimodal Hidden Markov Model (HMM) variants 61–63.9% and demonstrating competitive performance with advanced models such as Long Short-Term Memory (LSTM) networks 90%, and Conditional Random Field (CRF) 88.5%. The framework exhibited robustness across stress-test scenarios, including sensor noise, missing data, and imbalanced class distributions. A key advantage of the MHMM over black-box approaches is its interpretability by providing quantifiable transition probabilities that reveal learning rates, forgetting patterns, and contextual influences on proficiency dynamics. The model successfully captures context-dependent effects, including task complexity and cumulative fatigue, through dynamic transition matrices. When demonstrated through simulation, this framework establishes a foundation for developing adaptive operator-AI collaboration systems in Industry 5.0 environments. The MHMM’s combination of high accuracy, robustness, and interpretability makes it a promising candidate for future empirical validation in real-world industrial, healthcare, and training applications in which it is critical to understand and support human proficiency development.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.