Abstract
With the rapid advancement of aviation technology, modern aircraft cockpits are evolving toward high automation and intelligence, making pilot-cockpit interaction a critical factor influencing flight safety and efficiency. Pilot pose estimation and behavior recognition are critical for monitoring pilot state, preventing operational errors, and enabling adaptive human–machine interaction, thus playing an essential role in aviation safety assurance and intelligent cockpit development. However, existing methods face challenges in real-time performance, reliability, and computational complexity in practical applications. Traditional approaches, such as wearable sensors and image-processing-based algorithms, demonstrate certain effectiveness but still exhibit limitations in aviation environments. To address these issues, this paper proposes a lightweight pilot pose estimation and behavior recognition framework, integrating Vision Transformer with depth-wise separable convolution to optimize the accuracy and efficiency of keypoint detection. Additionally, a novel multimodal data fusion technique is introduced, along with a scientifically designed evaluation system, to enhance the robustness and security of the system in complex environments. Experimental results on a pilot keypoint detection dataset captured in a simulated cockpit environment show that the proposed method achieves 81.9 AP, while substantially reducing model parameters and notably improving inference efficiency compared with HRNet. This study provides new insights and methodologies for the design and evaluation of aviation human-machine interaction systems.