Pilot Intent State Recognition Based on Eye-Movement Behavior Characteristics

Abstract

This paper aims to uncover the generation patterns of pilot intentions during complex flight missions and to identify pilot intention states, thereby enabling airborne warning systems to understand and predict pilot intentions for early warning strategies. To achieve this, we designed a simulated flight experiment incorporating various risk scenarios to induce pilot intentions, collected eye-tracking data reflecting pilot intention states, and proposed a method for identifying the persistence of pilot intentions based on eye-tracking data. We then constructed a pilot intention dataset and analyzed the time–frequency characteristics of eye-tracking data in the intention persistence state, revealing key behavioral features following the generation of pilot intentions. Furthermore, we examined eye-tracking features that enhance the performance of pilot intention recognition models. Finally, we developed a deep learning model integrating recurrent neural networks (RNN) and bidirectional long short-term memory (BiLSTM) networks to recognize pilot intentions. The results demonstrate that the model achieved a recognition accuracy of 97.8% on the test set, and its performance in identifying pilot intention states was validated through comparison with a baseline model. This study confirms that eye-tracking data can effectively identify pilot intention states and offers new insights into aircraft safety early warning and intelligent control systems.

1. Introduce

With the rapid advancement of the air transport industry, aviation accidents have steadily declined. However, civil aviation safety continues to be a top priority. According to IATA statistics on aviation accidents from 2017 to 2022, over 50% of flight accidents are attributed to pilot control errors, with pilot decision-making issues contributing to 11% of these incidents [1]. To enhance flight safety, Automatic Flight Control Systems (AFCS) are now widely integrated into civil aircraft. Nonetheless, current autopilot systems primarily control the aircraft’s motion by adjusting the control surfaces via the Flight Management Computer (FMC), ensuring that the aircraft follows the route input by the pilot. While these systems maintain stable flight operations, they do not account for the pilot’s cognitive processes, such as decision-making [2,3,4,5]. Intent, which precedes decision-making, plays a crucial role in pilot actions [6,7]. Following the Boeing 737 MAX 8 accident, in which the MCAS automatic anti-stall system overrode the pilot’s maneuvering intent—leading to a crash—there has been growing attention on identifying pilot maneuvering intent [8,9,10,11].

1.1. Literature Review

Pilot intention refers to a pilot’s anticipatory cognitive activity regarding a series of future aircraft control actions. It emerges from the complex interaction of cognitive processes such as perception, memory, and decision-making. Research on pilot intention is still in its early stages. However, although aircraft operate in three-dimensional space and therefore involve one more dimension than ground vehicles, the classification of driver intention can still serve as a useful reference. Accordingly, pilot intention can be divided into strategic, tactical, and operational (control) levels [12,13,14]. Strategic maneuvering intention refers to the pilot’s overall objective during task execution [15], such as conducting pre-flight preparation before takeoff or giving a briefing before entering a specific flight phase. Tactical maneuvering intention refers to a sequence of coordinated actions aimed at achieving a short-term goal [16], such as climb, descent, or turn maneuvers during takeoff and landing through the manipulation of devices such as the control column, rudder, altitude knob, and heading knob. Control-level maneuvering intention refers to the specific control actions performed to fulfill the strategic maneuvering intention [17,18], such as throttle and control column operations. In addition, a special type of control-level maneuvering intention may arise when the pilot encounters unsafe events. This type of intention may involve sudden climb, descent, or turn actions to avoid hazardous situations [19,20,21], such as emergency evasive maneuvers in response to runway incursions or wind shear recovery maneuvers.

Research on pilot intention has mainly focused on tactical maneuvering intention, which is commonly identified using external environmental information [16], flight data [22], physiological data [23], and eye-tracking data [24]. During flight, aircraft interact with the external environment in a highly complex manner, which requires pilots to perceive hazardous flight situations rapidly and accurately. Therefore, some scholars have recognized situation awareness as an important approach for examining the evolution of pilot intention [25]. In the field of aviation human factors, the theory of Situation Awareness (SA) proposed by Endsley has been widely used to explain pilots’ cognitive processes in dynamic environments. According to this theory, pilots’ situation awareness consists of three levels: perception of environmental elements, comprehension of their meaning, and projection of future states. In flight tasks, pilots continuously acquire environmental information and form situation awareness, which provides the cognitive basis for subsequent decision-making and control behavior. Within this framework, pilot intention can be regarded as an important transitional cognitive state linking situation assessment and specific control actions. Therefore, investigating pilots’ behavioral characteristics during information acquisition can provide important clues for understanding the process of intention formation. Because eye-movement behavior can reflect pilots’ visual attention allocation and information acquisition patterns during task execution, eye-tracking data provide an important objective means for exploring the cognitive mechanisms underlying pilot intention formation [26]. Utku Kale et al. emphasized the important role of situation analysis in pilots’ subjective decision-making and argued that it constitutes the first stage of the pilot decision-making process, followed by decision formation and action execution [27]. Katie J. Parnell et al. used three different decision models to investigate pilots’ decision-making process in a dual-engine failure scenario during takeoff [28].

Flight data provide the most direct reflection of the interaction between the aircraft and the external environment, as well as the pilot’s control behavior. Zhili Tang et al. combined flight data with an expert system database to infer the pilot’s most likely maneuvering intentions and behaviors [29]. Stefan Suck et al. used a hidden Markov model (HMM) to train and evaluate interaction data between pilots and a simulated cockpit system for pilot maneuvering intention recognition [30]. Lei Dong et al. used data from a digital experimental cockpit (DECO) to model and analyze the behavioral intentions of single-pilot crews and further explored pilots’ behavioral tendencies [31]. Akio Kato et al. input simulated flight data into an output-error model to investigate the relationship between pilot dynamics and the dynamic stability characteristics of an aircraft [32]. Jayaprakash et al. modeled pilot intention as a finite-state machine based on the continuous variation in flight data across different flight phases [33], and further developed a probabilistic validation framework to quantitatively describe the degree of pilot mode confusion [34].

Physiological data can directly reflect changes in pilots’ psychophysiological states during different cognitive processes. Yining Zeng et al. proposed a spatial attention network (SA-Net)-based method to recognize pilots’ control intentions from electroencephalogram (EEG) signals, and introduced receptive field attention (RFA) and squeeze-and-excitation (SE) modules to improve feature extraction and classification performance [35]. Qiyou Wu et al. improved the conventional motor imagery paradigm and used two visual guidance modes, namely pictures and videos, to recognize pilots’ operational intentions. They compared the recognition performance under different paradigms and further conducted classification studies using multiple machine learning algorithms [36]. Chenyang Zhang et al. combined flight simulator experiments with functional near-infrared spectroscopy (fNIRS) to measure changes in oxyhemoglobin concentration in 25 student pilots during different turning behaviors, and found that differences in turning behavior were associated with the frontopolar cortex (BA10) [37]. Yuhan Li et al. selected 19 features related to pilot behavior, including electrocardiogram, electrodermal activity, electromyography, respiration, and skin temperature, and established an ensemble learning classifier by comparing the performance of multiple machine learning models, achieving a pilot behavior prediction accuracy of up to 98.15% [38].

Eye-tracking data can reveal pilots’ visual attention distribution and information search strategies under different intention patterns. Jiaao Cheng et al. identified average fixation duration, pupil size, and blink duration as eye-movement parameters strongly correlated with intention through significance testing, and used a support vector machine to classify intention, achieving a maximum accuracy of over 89% [39]. Hongyan Wang et al. proposed an intention classification method integrating a Transformer model with a bidirectional long short-term memory network (Bi-LSTM). By inputting long-term multifeature eye-tracking time-series data into the model, they achieved a pilot intention recognition accuracy of up to 94.84% [40]. In this study, pilot intention was modeled as a binary state indicating whether the pilot was preparing to execute a control action, rather than classifying the specific type of maneuver.

1.2. Contribution

In summary, scholars have begun to model pilot intention, decision-making, and related processes using different types of data to achieve higher intention recognition accuracy. However, from the perspective of data sources, flight data exhibit an inherent lag in intention research. The data corresponding to a pilot’s control behavior can only be recorded in the QAR after the pilot has interacted with the control devices, which is clearly later than the moment when the behavioral intention is generated. Physiological data can reflect the pilot’s real-time psychophysiological state, but the complex cockpit environment often introduces substantial noise during data acquisition, resulting in high requirements for data integrity, cleaning, and filtering. In contrast, eye-tracking data can reveal early information search patterns and attention allocation strategies during intention formation, making them particularly sensitive to the cognitive state at the early stage of intention emergence. However, studies on the relationship between eye-tracking data and pilot intention states remain limited. Therefore, this study first proposes a method for recognizing the persistence state of pilot intention based on experimentally collected eye-tracking data. Second, by analyzing the time-frequency characteristics of eye-tracking data under intention-present states, the study identifies the eye-tracking features that effectively improve the performance of pilot intention recognition models. Finally, a deep learning method combining a recurrent neural network (RNN) and a bidirectional long short-term memory network (Bi-LSTM) is established to recognize the persistence state of pilot intention. This study focuses only on the intention state in which pilots take action to achieve a specific flight goal, without considering the specific type of intention.

2. Methods

This section introduces the methods used for pilot intention recognition. Section 2.1 describes the experimental design for inducing pilot intention. Section 2.2 presents the data collected in the experiment. Section 2.3 introduces the data processing procedure and dataset construction process. Section 2.4 describes the eye-tracking feature extraction methods from both the time domain and frequency domain perspectives. Section 2.5 and Section 2.6 present the pilot intention recognition models developed in this study.

2.1. Experiment

2.1.1. Experimental Equipment

To simulate the evolution of pilot intention in real flight, this study identified pilot intention based on eye-tracking data collected while pilots performed tasks in a Level D Boeing B787 full-flight simulator, as shown in Figure 1. Before the formal experiment, the experimenters provided a brief introduction and asked the participants to sign an informed consent form. Subsequently, the participants wore Tobii Pro Glasses 3 (Tobii AB, Danderyd, Sweden) wearable eye trackers and completed the calibration procedure. The sampling frequency of the eye tracker was set to 50 Hz. Once the participants were ready, eye-tracking recording was initiated through the Tobii glasses, and their eye-movement trajectories were continuously recorded throughout the entire experiment.

Figure 1. B787 Full Motion Level D Flight Simulator (CAE Inc., Montreal, QC, Canada).

2.1.2. Participants

A total of 11 flight crews, including 22 pilots, participated in the simulated flight experiments in this study. Each crew consisted of one captain and one first officer. All pilots were aged between 28 and 45 years and held line operation qualifications for the B787 aircraft.

2.1.3. Experimental Scenario

The experimental route was set as Fuzhou Changle International Airport (ZSFZ) to Tokyo Narita International Airport (RJAA). The aircraft was scheduled to take off from Runway 21 at Fuzhou Changle International Airport and land on Runway 16L at Tokyo Narita International Airport. Before takeoff, the flight crew had completed all pre-flight, engine start, taxi, and before-takeoff procedures, as well as the taxi and takeoff briefings. The aircraft had already established communication with the tower and was waiting for takeoff clearance. To induce pilot intention, three risk scenarios, namely wind shear, thunderstorm, and single-engine failure, were introduced during the execution of the flight task. The flight tasks corresponding to these risk scenarios are shown in Table 1. In addition, a senior flight instructor participated in each simulated flight experiment by acting as the air traffic controller and was responsible for triggering the risk scenarios to induce pilot intention.

Table 1. Experimental flight task settings for pilot maneuvering intention induction.

Flight Task	Description
Task 1: Wind shear escape maneuver	Position: Takeoff position, Runway 21, ZSFZ
	Weather: Thunderstorm conditions were set on one side of the runway
	Failure: None
	Description: Starting from the takeoff position, wind shear was introduced before 400 ft AGL. The task ended after the pilot retracted the flaps to the clean configuration and completed the wind shear escape maneuver.
Task 2: Thunderstorm avoidance	Position: Takeoff position, Runway 21, ZSFZ
	Weather: Thunderstorm conditions were set on one side of the runway
	Failure: None
	Description: After completing the wind shear escape maneuver, the pilot climbed and turned away from the thunderstorm area. The task ended when the aircraft reached the next waypoint.
Task 3: Single-engine VOR approach and go-around	Position: 12 NM from the fifth approach segment of Runway 16L, RJAA
	Weather: Visibility 1 km, cloud base 500 ft, crosswind 10 m/s
	Failure: ENG ICA SYS and ENG LIMIT EXCEED were preset before the FAF
	Description: After completing the approach briefing and engine failure checklist, the pilot continued the approach to DA (decision altitude). If the runway was not in sight at DA, a go-around was executed. The task ended after the go-around procedure was completed and the aircraft had established a stable climb.

During the experiment, pilot intention was induced by the experimental conditions and air traffic controller instructions, while the pilots independently made decisions and handled the flight tasks. Therefore, no fixed experimental duration was specified. The experiment ended when the pilots completed the entire route flight task and landed safely. In this study, intention was defined as a preparatory state before the pilot executed a clear control action. Specifically, when performing flight tasks, pilots needed to assess risks, select a response strategy, and prepare to execute control actions. The cognitive activity during this stage was regarded as the intention-present state. In contrast, during stable flight phases without new control actions and mainly maintained by the autopilot, the corresponding samples were labeled as the intention-absent state.

2.2. Data Acquisition

The experiment collected seven eye-tracking parameters that reflect the pilot’s gaze position, eye state, and pupil diameter, as shown in Table 2. These parameters served as reference data for identifying the pilot’s maneuver intentions. Since the objective of this study was to identify the pilot’s intention state prior to maneuver execution, the eye-tracking data were collected only from the pilot flying (left seat). The right-seat pilot mainly assisted the pilot flying in completing the flight task and was not involved in the key flight decision-making process.

Table 2. Pilot maneuvering intention parameters obtained in the experiment.

NO.	Parameter Name	Notes
1	Fixation point X percentage position ${\mathit{f}}_X$	The gaze sample’s horizontal (X) position is represented in normalized video coordinates ranging from 0 to 1. In this system, (0, 0) corresponds to the top-left corner of the normalized video, while (1, 1) corresponds to the bottom-right corner.
2	Fixation point Y percentage position ${\mathit{f}}_Y$	The vertical (Y) position of the gaze sample is represented in normalized video coordinates ranging from 0 to 1.
3	Eye condition type	fixation type and saccade type
4	Eye movement angular velocity $\mathit{v}$	The angle of eye movement per unit time
5	Left pupil diameter ${\mathit{d}}_L$	The diameter of the left pupil
6	Right pupil diameter ${\mathit{d}}_R$	The diameter of the right pupil
7	Average pupil diameter ${\mathit{d}}_M$	Average pupil diameter of the left and right eyes

As shown in Table 2, the percentage X position of the fixation point ${\mathit{f}}_X$, percentage Y position of the fixation point ${\mathit{f}}_Y$, angular velocity of eye movement $\mathit{v}$, left pupil diameter ${\mathit{d}}_L$, right pupil diameter ${\mathit{d}}_R$, and average pupil diameter ${\mathit{d}}_M$ were used for eye-tracking feature analysis under the intention-present and intention-absent states, and were also used as model input features. The eye status type was used to identify blinks and invalid samples and was removed or interpolated during the preprocessing stage; therefore, it was not directly used as an input feature for the model.

2.3. Data Processing

2.3.1. Data Preprocessing

Eye-tracking data processing primarily involves gaze data processing, gaze classification, and the recognition and cleaning of pupil diameter and blink behavior. Gaze data is denoised using a sliding median filter, with a maximum gap length set to 75 ms for interpolation. Gaze classification employs the I-VT (Identification by Velocity Threshold) algorithm, which categorizes eye movement behavior into gaze, saccades (eye saccades), and invalid points by applying an angular velocity threshold of 30°/s. Pupil data is also smoothed using a sliding median filter, and linear interpolation is utilized to address pupil diameters less than 2 mm. Blink recognition is based on the duration of eye closure, with a time domain threshold of 350 ms and 75 ms established for judgment. As illustrated in Figure 2, data on blink durations, gaze durations, and saccade durations are obtained through this eye-tracking data preprocessing.

Figure 2. Diagram of eye-tracking data.

2.3.2. Pilot Intention Dataset Construction

Based on the pilots’ eye-tracking video data, this study proposed a method for determining pilot intention samples using eye-tracking information. Before a pilot exhibited a specific control behavior, the moment when the fixation point first entered the area of the Navigation Display (ND), Primary Flight Display (PFD), or Control Display Unit (CDU) was defined as the start time of the intention, and the moment when the pilot began to execute the corresponding control action was defined as the end time of the intention. Specifically, the ND mainly presents trajectory, navigation, weather, and traffic information; the PFD integrates attitude, altitude, speed, heading, and flight guidance information; and the CDU serves as the main human–machine interface of the flight management system for inputting and modifying routes, altitude/speed constraints, and approach procedures. According to the B787 cockpit design and standard operating procedures (SOP), these three panels constitute the core information channels for pilots during trajectory planning and modification. When pilots perform maneuvers that change the aircraft motion state, such as climb, descent, left turn, and right turn, they typically obtain situational information from the ND/PFD and complete command inputs through the CDU or the mode control panel. For the end time of intention, the control wheel and control column deflection signals were used as the primary criteria. Within a 2 s time window before the occurrence of maneuvers such as climb, descent, left turn, and right turn, the baseline mean and fluctuation level of the corresponding control channel were calculated. Then, scanning forward from the end of the baseline period, if the deflection amplitude of that channel exceeded a preset threshold relative to the baseline and remained in the same direction for at least 0.5 s, the corresponding sampling point was defined as the end time of the maneuvering intention. Using the timestamps of the eye-tracking data as the main timeline, intention-present samples were labeled as “Intention = 1”, and intention-absent samples were labeled as “Non-Intention = 0”. Finally, an intention dataset was constructed, containing fixation point positions, angular velocity of eye movement, pupil diameter, and pilot intention labels.

2.4. Feature Extraction

2.4.1. Time Domain Feature Extraction

The time domain features of eye-tracking data refer to the instantaneous characteristics of eye-tracking signals along the time axis, obtained by analyzing the time series of these signals. Time domain features can intuitively reflect the behavioral patterns inherent in eye-tracking signals and are frequently used to characterize their time domain attributes and stability. The mean, standard deviation, and root mean square in the time domain are utilized as feature indicators.

2.4.2. Frequency Domain Feature Extraction

To capture the energy distribution and dynamic transfer characteristics of eye movement in each frequency band under the pilot’s intention state, this study further employs power spectral density (PSD) to characterize the preprocessed eye movement signal in the frequency domain, as shown in Equation (1).

$$\left\{\begin{array}{l}P=\underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}{\displaystyle {\int }_0^T{\left|x\left(t\right)\right|}^{2}dt}\\ \widehat{x}\left(\omega \right)={\displaystyle \frac{1}{T}}{\displaystyle {\int }_0^{T}x\left(t\right)e^{-iwt}dt}\\ \mathbf{E}\left[{\left|\widehat{x}\left(\omega \right)\right|}^2\right]={\displaystyle \frac{1}{T}}{\displaystyle {\int }_0^T{\displaystyle {\int }_0^T\mathbf{E}\left[{\mathit{x}}^{*}\left(t\right)\mathit{x}\left(t^{\prime }\right)\right]e^{-iw\left(t-t^{\prime }\right)}dt}d{t}^{\prime }}\\ psd\left(\omega \right)=\underset{T\to \infty }{\lim }\mathbf{E}\left[{\left|\widehat{x}\left(\omega \right)\right|}^2\right]\end{array}\right.$$

(1)

This study divides the power spectrum into four bandwidths: very low frequency (ULF, 0.05–0.20 Hz), low frequency (LF, 0.20–0.60 Hz), mid frequency (MF, 0.60–1.60 Hz), and high frequency (HF, 1.60–6.00 Hz). Under a sampling frequency of 50 Hz, the fixation point position, angular velocity of eye movement, and average pupil diameter of each participant were normalized using z-score standardization. For each continuous sequence under the intention-present or intention-absent state, the one-sided power spectral density was estimated using the Welch method. Specifically, for a given normalized sequence, a 10% cosine window was first applied at both ends of the sequence for endpoint smoothing. The sequence was then transformed using the discrete Fourier transform at the sampling frequency, and one-sided power spectral density estimates were obtained at a series of discrete frequency points within the range of 0 to f_s/2 (0–25 Hz).

Considering that the main energy and behavior-related components of eye-movement and pupil signals are concentrated in the 0.05–6.00 Hz range, whereas the very slow variations below 0.05 Hz mainly reflect baseline drift and the components above 6.00 Hz are mostly high-frequency noise or are unrelated to the intention process considered in this study, frequency domain features were extracted only within the 0.05–6.00 Hz range. The power in each frequency band was then accumulated and normalized relative to the total power within 0.05–6.00 Hz to obtain the band-specific indices, as shown in Equation (2).

$$\left\{\begin{array}{l}\mathrm{ULF}={\displaystyle \frac{{\displaystyle {\int }_{0.05}^{0.20}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}{{\displaystyle {\int }_{0.05}^{6.00}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}}\\ \mathrm{LF}={\displaystyle \frac{{\displaystyle {\int }_{0.20}^{0.60}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}{{\displaystyle {\int }_{0.05}^{6.00}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}}\\ \mathrm{MF}={\displaystyle \frac{{\displaystyle {\int }_{0.60}^{1.60}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}{{\displaystyle {\int }_{0.05}^{6.00}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}}\\ \mathrm{HF}={\displaystyle \frac{{\displaystyle {\int }_{1.60}^{6.00}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}{{\displaystyle {\int }_{0.05}^{6.00}{\widehat{S}}_x\left(f\right)\mathrm{d}f}}}\end{array}\right.$$

(2)

where $f={\displaystyle \frac{\omega }{2\pi }}$ denotes the frequency variable, ${\widehat{S}}_x\left(f\right)$ represents the one-sided PSD estimate of a flight parameter, $p_i={\displaystyle \frac{{\widehat{S}}_x\left(f_i\right)}{{\displaystyle \sum _j{\widehat{S}}_x\left(f_j\right)}}}$ denotes the normalized probability distribution of the discrete PSD, and $I$ represents the number of frequency points.

2.5. Problem Formulation

Pilot intention recognition can be formulated as a supervised classification task in which intention states are labeled as intentional or unintentional. The input temporal eye-tracking data is expressed in Equation (3).

$$\mathbb{C}:{\left\{E_i,I_i\right\}}_{i=1}^N$$

(3)

where $E_i$ represents the eye-tracking time-series data of length $k$, $I_i$ denotes the corresponding intention state, and $N$ is the number of samples. Each eye-tracking data sequence contains six channels, each of length $k$, as shown in Equation (4).

$$E_i=\left[{\mathit{f}}_X,{\mathit{f}}_Y,\mathit{v},{\mathit{d}}_L,{\mathit{d}}_R,{\mathit{d}}_M\right]\in {\mathbb{R}}^{k\times 6}$$

(4)

where ${\mathit{f}}_X$ and ${\mathit{f}}_Y$ represent the horizontal and vertical coordinates of the fixation point on the normalized video plane of the scene camera, respectively. Their values range from 0 to 1, with the origin located at the upper-left corner of the normalized video frame, and they are dimensionless proportions. $\mathit{v}$ denotes the angular displacement velocity of the gaze direction per unit time, in degrees per second. ${\mathit{d}}_L$ and ${\mathit{d}}_R$ denote the left and right pupil diameters, respectively, in millimeters. ${\mathit{d}}_M$ represents the average pupil diameter. When data from both eyes are valid, ${\mathit{d}}_M$ is calculated by $d_M\left(t\right)={\displaystyle \frac{d_L\left(t\right)+d_R\left(t\right)}{2}}$; when only one eye measurement is valid at a given moment, ${\mathit{d}}_M\left(t\right)$ is set equal to the pupil diameter of the valid eye. To characterize both the overall arousal level and the potential asymmetry between the two eyes, and to improve robustness in the presence of brief signal loss from one eye, the three pupil-related channels, ${\mathit{d}}_L$, ${\mathit{d}}_M$, and ${\mathit{d}}_M$, were retained in the model input. The model used the time-series information from the above six eye-tracking channels to recognize the pilot’s intention state, as shown in Equation (5).

$${\widehat{I}}_i=\arg \max p\left(I_i\left|E_i\right.\right),E_i\in \xi$$

(5)

where $\xi =\left\{0,1\right\}$ denotes the pilot’s intention state. $I_i=0$ indicates the intention-absent state, whereas $I_i=1$ indicates the intention-present state. In this study, the moment when the pilot executed a specific control action, such as changing heading, altitude, or thrust, was taken as the reference point. Tracing backward from that moment, the onset of the corresponding maneuvering intention was defined as the first time the fixation point entered a task-critical instrument area, such as the ND, PFD, or CDU. The eye-tracking interval from that moment to the actual onset of the corresponding control action was labeled as the intention-present state, whereas other eye-tracking segments outside this interval and without obvious control behavior were labeled as the intention-absent state.

2.6. Model Construction

This study proposes a hybrid neural network based on an RNN-BiLSTM architecture for pilot intention recognition, as illustrated in Figure 3. First, an RNN layer is employed to dynamically encode the input multivariate eye-tracking time series, capturing short-term dependencies and local smoothing patterns. Next, two stacked bidirectional LSTM layers jointly model longer-term fixation–saccade transitions and pupil-dynamics features in both forward and backward temporal contexts, thereby enhancing the network’s ability to represent strategic sustained fixations and rapid gaze shifts. Dropout and recurrent dropout are applied between the recurrent layers to mitigate overfitting, and two fully connected layers are added at the end of the network to perform nonlinear feature mapping. Finally, a Softmax layer outputs the posterior probability of each intention category. Overall, the model effectively captures historical information in the time series and extracts nonlinear and dynamic features from the eye-tracking data.

Figure 3. Architecture of the RNN-BiLSTM Model for Pilot Intention Recognition.

2.6.1. Model Training Strategy

The dataset was randomly divided into training and test sets for model training and evaluation. This random split ensured a balanced distribution of intention data for the four maneuver types in the training and test sets, so that the proportion of each maneuver type remained consistent across the two datasets.

During training, class weights for Intention and Non-Intention samples were calculated to address class imbalance within the dataset. To mitigate distribution drift across subjects, the input features were robustly standardized using median centering and MAD (Median Absolute Deviation) scaling. The training process employed cross-entropy loss with the Adam optimizer, while early stopping and adaptive learning rate decay were implemented to ensure convergence stability. Furthermore, the learning rate and gradient clipping values were set to 0.001 and 1.0, respectively. The model underwent 80 training epochs, and the training and optimization settings are summarized in Table 3.

Table 3. Model Training Parameters.

Item	Setting
Loss	Categorical cross-entropy (label smoothing = 0.05), class-weighted
Optimizer	Adam, learning rate = 0.001, clip = 1.0
Regularization	Dropout/Recurrent Dropout = 0.20; Dense L2 = 0.00001
Batch size/Epochs	128/80
LR schedule	factor = 0.5, patience = 4, min lr = 1 × 10−5

2.6.2. Evaluation Metrics

To evaluate model performance, four commonly used classification metrics were adopted, namely accuracy, precision, recall, and F1-score, as defined in Equation (6).

$$\left\{\begin{array}{l}\mathrm{Acc}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}\\ \mathrm{Pr}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}\\ \mathrm{Re}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}\\ \mathrm{F}1=2\cdot {\displaystyle \frac{\mathrm{Pr}\cdot \mathrm{Re}}{\mathrm{Pr}+\mathrm{Re}}}\end{array}\right.$$

(6)

where $\mathrm{Acc}$ represents accuracy, $\mathrm{Pr}$ represents precision, $\mathrm{Re}$ represents recall, $\mathrm{F}1$ represents the F1 score, and $\mathrm{TP}$, $\mathrm{TN}$, $\mathrm{FP}$, and $\mathrm{FN}$ represent true positives, true negatives, false positives, and false negatives, respectively. Since the number of Non-Intention samples is higher than that of Intention samples in this study, down sampling is performed exclusively on Non-Intention samples within the training set to enhance the model’s classification ability.

2.6.3. Baseline Models and Parameter Settings

To assess the effectiveness of the proposed RNN-BiLSTM model, several baseline models were introduced for comparison, including Logistic Regression, SVM, XGBoost, Random Forest, and LSTM. The LSTM model adopted a training strategy similar to that of the proposed RNN-BiLSTM model, including the same optimizer and learning rate schedule. In contrast, the traditional machine learning models, i.e., linear regression, SVM, random forest, and XGBoost, used grid search to determine the optimal parameters. The detailed parameter settings of all baseline models are listed in Table 4. In addition, to avoid confusion arising from different input formats across models, this study consistently used the raw eye-tracking time series composed of six channels as the input. For the traditional machine learning models, each fixed-length multivariate time-series sample was flattened along the temporal dimension into a one-dimensional fixed-length vector before being fed into the model. For the deep learning models, the original temporal structure of the time series was preserved, and the data were directly input into the model in the form of multivariate sequences.

Table 4. Baseline Model.

Model	Parameter Settings	Training Settings
Logistic Regression	As a baseline model, the standard implementation in the Scikit-learn library was adopted in this study, without introducing additional complex regularization methods.	Utilized the Adam optimizer. The initial learning rate was set to 0.001. The learning rate was dynamically adjusted using a cosine annealing strategy. Batch size was set to 128. Models were trained for a total of 80 epochs.
SVM	The SVC implementation in the Scikit-learn library was adopted with the radial basis function (RBF) kernel, a penalty parameter (C) = 10, and a kernel width parameter (gamma) = 0.01
XGBoost	The XGBoost library was used, and the main parameters were optimized by grid search. The final configuration was as follows: number of base learners (n estimators) = 200, maximum depth (max depth) = 6, learning rate = 0.1, and subsample ratio(subsample) = 0.8
Random Forest	The parameters were optimized by grid search, and the final configuration was as follows: number of trees (n estimators) = 100, maximum depth (max depth) = 15, and minimum number of samples per leaf (min samples leaf) = 5.
LSTM	In this study, a two-layer LSTM network was constructed, with the number of hidden units set to 128.

3. Results

This section presents the analysis of the eye-movement features of the pilot primarily responsible for controlling the aircraft, as well as the corresponding intention recognition results. Section 3.1 and Section 3.2 analyze the differences in the time domain and frequency domain eye-tracking features between the two intention states, respectively. Section 3.3 and Section 3.4 describe the model parameters and training process for intention recognition. Section 3.5 presents the results of the intention recognition models through performance comparison across multiple models.

3.1. Intention Sample Acquisition

A total of 650 intention events were detected from the experimental videos, including 114 climb maneuvers, 362 descent maneuvers, 108 left-turn maneuvers, and 66 right-turn maneuvers. Statistical analysis showed that the duration of most intention event segments was within 4 s, as shown in Figure 4. Therefore, all intention event samples were uniformly resampled into 4 s time segments preceding maneuver initiation. In addition, 600 non-intention events with a duration of 4 s were extracted from the intervals between the completion of one maneuver and the start of the next maneuver. The sequence data were then labeled according to the final maneuver action in the video sequence. For the pilot intention corresponding to each maneuver category, 80% of the data were randomly selected for model training, and the remaining 20% were used for model testing. This random split ensured a balanced distribution of intention data for the four maneuver types across the training and test sets, thereby maintaining a consistent proportion of each maneuver type in both datasets.

Figure 4. Statistics of the duration of intention event segments.

3.2. Time Domain Feature Analysis Results

To verify the differences in the distribution of eye movement time domain features between pilots in intentional and unintentional states, this study conducted a statistical analysis of the time domain feature distribution for fixation point position, eye movement angular velocity, and pupil diameter, as illustrated in Figure 5.

Figure 5. Time Domain Distribution of Gaze Point Positions.

According to Figure 5, considering the three time-domain indicators comprehensively, the mean, variance, and RMS of Fixation Position X did not show significant differences between the two types of samples (p > 0.05), whereas the variance of Fixation Position Y reached a significant level between the two groups (p < 0.05). From the distributions shown in the figure, the Intention samples exhibited a more concentrated and stable distribution pattern, whereas the Non-Intention samples were relatively more dispersed spatially and showed greater fluctuations. This suggests that, before entering the maneuver execution stage, pilots had already gradually narrowed their gaze toward task-critical areas during the intention formation stage. In contrast, during the non-intention monitoring stage, gaze distribution is more dispersed in the vertical direction. In the horizontal direction (X), the mean and median of the Intention group are generally slightly lower, and the distribution is more concentrated, with a markedly narrower interquartile range. At the same time, the variance and RMS are generally smaller than those of the Non-Intention group, and the long tail is compressed. This indicates that, under the intention-driven state, the horizontal gaze position is more stable and exhibits smaller fluctuations, reflecting stronger target orientation and gaze control. In the vertical direction (Y), the mean and median of the Intention group also show a slightly lower trend, and the variance and RMS are generally smaller. However, the distribution width in the Y direction is relatively larger than that in the X direction, indicating that this direction is more susceptible to task scenarios and individual differences.

According to the distribution statistics, no significant differences were observed in the mean, variance, or RMS of fixation position X between the two states (p > 0.05). This indicates that, under the present experimental setting, the overall horizontal gaze center and horizontal coverage range remain largely consistent before and after intention formation. In the vertical direction, no significant differences were found in the mean or RMS of fixation position Y (p > 0.05); however, its variance was significantly higher in the Non-Intention state than in the Intention state (p < 0.05). This result suggests that, during the Non-Intention monitoring stage, gaze distribution is more dispersed in the vertical direction, whereas during the intention stage preceding a specific maneuver, gaze is more concentrated within a certain vertical band of the instrument panel, with a noticeably reduced vertical scanning range. This is consistent with the mechanism by which pilots focus on critical information in the ND/PFD area to support decision-making.

According to Figure 6, the median and interquartile ranges of the mean for both the Intention and Non-Intention categories exhibit a high degree of overlap, with only slight shifts. This suggests that the average movement velocity has limited ability to distinguish between intentional states. In contrast, the variance and RMS reveal more significant differences between the two categories. The distribution of the intentional category displays a pronounced thickening of the upper tail and a multimodal structure, indicating that, while the overall central position is similar, high-fluctuation, high-energy segments occur more frequently. The distribution of the non-intentional category is more concentrated, featuring a lighter heavy tail and a more stable overall distribution. This indicates that intention-driven fixation and saccadic behavior are more prone to phased “bursts,” exhibiting rapid shifts and short-term accelerations, thus increasing the upper tail of the variance and RMS. In the non-intentional state, eye movement is closer to a slow, uniform follow-up pattern.

Figure 6. Time Domain Distribution of Eye Movement Angular Velocity.

According to the distribution statistics, no significant differences were observed between the Intention and Non-Intention states in the mean, variance, or RMS of angular velocity of eye movement (p > 0.05). This indicates that, at the time scale considered in this study, pilots did not adjust their visual search pattern by significantly increasing or decreasing saccadic velocity when entering the intention stage. Instead, the differences were mainly reflected in the spatial distribution of fixation, rather than in the amplitude or velocity of eye movements themselves.

According to Figure 7, the Intention and Non-Intention categories still overlap in terms of their means; however, the variance and RMS indices are more discriminative than the mean. This suggests that the combined characterization of pupil diameter and stability is more effective in identifying the pilot’s intentional and unintentional states. The intentional type exhibits a higher yet more concentrated RMS compared to the unintentional type, while its variance is significantly smaller and its distribution is narrower. The non-Intention sample, on the other hand, displays a wider variance distribution and a heavier upper tail. This indicates that when there is a clear intention, the pupil demonstrates continuous slight dilation and remains relatively stable, whereas in the unintentional state, it experiences greater fluctuations and is more significantly affected by external disturbances.

Figure 7. Time Domain Distribution of Pupil.

According to the distribution statistics, no significant differences were found in the mean or RMS of average pupil diameter between the intention-present and intention-absent states. This suggests that the overall arousal level reflected by pupil diameter did not show a marked increase or decrease during the intention-present state. However, the variance of average pupil diameter was significantly greater in the intention-absent state than in the intention-present state (p < 0.05). In other words, pupil size fluctuated more strongly during intention-absent monitoring, whereas pupil changes became more stable during the stage of maneuver intention formation. Combined with the above results on fixation distribution, this finding suggests that, during the intention-present stage, pilots relied on a more concentrated gaze area and more stable pupil responses to support the extraction of critical instrument information

3.3. Frequency Domain Feature Analysis Results

For statistical analysis, the mean PSD of the Intention and Non-Intention samples was compared for each variable and each frequency band. Histogram inspection showed that none of the features followed a normal distribution; however, at the current sample size, all features generally exhibited a unimodal distribution with limited skewness, as shown in Figure 8. Therefore, this study adopted the null hypothesis that the overall means of the eye-tracking indicators were equal between the two intention states, and Welch’s t-test, which is robust to unequal variances, was used to analyze the between-group differences with intention state (Intention/Non-Intention) as the independent variable. For indicators with relatively strong skewness, the Mann–Whitney U test was further used for verification, and the two methods yielded consistent significance conclusions. To control the false discovery rate caused by multiple comparisons, the p-values were corrected using the Benjamini–Hochberg method. The results are presented in Table 3.

Figure 8. Distribution statistics of eye-tracking indicators.

According to Table 5, the power spectrum of pupil diameter under intentional conditions was significantly higher than that under unintentional conditions across all four frequency bands, with the most pronounced increases observed in the low frequency (LF, 0.20–0.60 Hz) and mid frequency (MF, 0.60–1.60 Hz) bands. This indicates that when pilots form clear intentions, both autonomic and cognitive load-related pupillary oscillations exhibit broad-spectrum enhancement, particularly in the low-to-mid-frequency range, reflecting higher levels of arousal and sustained attention.

Table 5. Mean PSD of Eye Movement Characteristics in Each Frequency Band Under Different Intention States.

Variable	Frequency Band	Unintentional (Mean ± SD)	Intentional (Mean ± SD)	Mean Difference	Cohen’s d	p
Fixation Position X (%)	ULF	0.15 ± 0.02	0.13 ± 0.06	−0.02	0.47	0.252
	LF	0.21 ± 0.03	0.23 ± 0.08	0.03	−0.43	0.296
	MF	0.21 ± 0.01	0.28 ± 0.06	0.07	−1.69	<0.001
	HF	0.22 ± 0.04	0.26 ± 0.10	0.04	−0.56	0.177
Fixation Position Y (%)	ULF	0.15 ± 0.03	0.12 ± 0.06	−0.03	0.59	0.152
	LF	0.17 ± 0.02	0.21 ± 0.06	0.04	−0.80	0.060
	MF	0.18 ± 0.02	0.24 ± 0.05	0.06	−1.64	<0.001
	HF	0.25 ± 0.05	0.29 ± 0.08	0.04	−0.58	0.154
Velocity (degree/s)	ULF	0.03 ± 0.02	0.03 ± 0.01	−0.003	0.23	0.571
	LF	0.05 ± 0.02	0.06 ± 0.03	0.004	−0.13	0.737
	MF	0.10 ± 0.02	0.10 ± 0.02	0.01	−0.22	0.576
	HF	0.27 ± 0.03	0.30 ± 0.04	0.02	−0.76	0.067
Average Pupil (mm)	ULF	0.14 ± 0.03	0.22 ± 0.12	0.08	−0.90	0.039
	LF	0.12 ± 0.03	0.20 ± 0.04	0.08	−2.19	<0.001
	MF	0.12 ± 0.05	0.19 ± 0.06	0.07	−1.29	0.003
	HF	0.11 ± 0.06	0.18 ± 0.10	0.07	−0.90	0.032

Regarding gaze position, the power in the mid frequency (MF) bands of the X and Y directions significantly increased during intentional gaze, while the low frequency (LF) band of the Y direction exhibited notable edge-related fluctuations. This indicates that pilots demonstrate stronger gaze position fluctuations in the mid-frequency rhythm of 0.6–1.6 Hz during intentional gaze, suggesting more frequent target searches and instrument monitoring behaviors. In contrast, angular velocity power was not significant across any frequency band, indicating that velocity-related power is more susceptible to subtle saccades and individual strategies.

Overall, based on the analysis results of frequency domain features, pupil diameter and fixation point position represent the most discriminative combination of variables for distinguishing between intentional and unintentional fixations. In the model recognition process, the low-to-mid frequency power of the pupil and the mid-frequency power of the fixation point should be prioritized, while event-level extraction of velocity features should be incorporated at the model level to further enhance recognition performance.

3.4. Model Training

The model was trained and evaluated using the strategy described in Section 2.6. Figure 9 illustrates the accuracy and loss of the RNN-BiLSTM model. Its training and test set accuracies are 98.0% and 97.8%, respectively. After only a small number of iterations, the accuracy on both the training and test sets exceeds 95%, demonstrating strong short-term learning capabilities. Furthermore, as the number of iterations increases, the accuracy on the test set gradually stabilizes, with oscillations controlled within 3%, and the loss value does not exhibit an upward trend. This indicates that the model can effectively identify pilot intentions during long-term learning.

Figure 9. Training Results of Model Accuracy on the Training and Test Sets.

3.5. Model Performance

The performance comparison between the proposed RNN-BiLSTM model and the baseline models was conducted using the experimental settings described in Section 2.6.

Table 6 presents the performance of the six models. It is evident that the LSTM model exhibits significantly higher recognition accuracy compared to the other models. By dynamically encoding the eye-tracking time series using RNN layers and subsequently stacking two bidirectional LSTM layers, there was a marked improvement in model performance. This indicates that the neural network architecture of deep learning models has a considerable advantage in processing large-scale data and recognizing feature patterns. The combination of RNN and LSTM models demonstrates enhanced performance in capturing effective features and processing time series data.

Table 6. Comparison of Evaluation Metrics for Pilot Intention Recognition Models.

Model	Accuracy	Precision	Recall	F1 Score
Logistic Regression	0.497	0.427	0.584	0.493
SVM	0.672	0.654	0.419	0.511
XGBoost	0.839	0.814	0.850	0.832
Random Forest	0.712	0.741	0.638	0.686
LSTM	0.913	0.822	0.905	0.862
RNN-BiLSTM	0.978	0.932	0.965	0.948

To provide a more intuitive comparison of the models’ ability to recognize Intention samples, the F1-score and recall values of each model listed in Table 6 were plotted in a bar chart, as shown in Figure 10. It can be seen that, for pilot intention state recognition, the RNN-BiLSTM model achieved the best performance in terms of both F1-score and recall.

Figure 10. Bar Chart of F1 Score vs. Recall for the “Intention” Category.

4. Discussion

This section discusses the results of using eye-tracking data to recognize the intention of the pilot primarily responsible for controlling the aircraft. Section 4.1 discusses the changes in pilots’ eye-movement patterns during intention formation based on the feature analysis results of eye-tracking data under different intention samples. Section 4.2 discusses the applicable modeling paradigms of eye-tracking data in the pilot intention recognition task. Section 4.3 discusses the implications of pilot intention recognition for airborne warning systems from the perspective of flight safety. Section 4.4 discusses the limitations of this study and the corresponding future plans.

4.1. Pilots’ Eye-Movement Patterns During Intention Formation

In this study, the primary objective was to investigate the relationship between pilots’ eye-movement patterns and their intention states. These eye-movement patterns were reflected in fixation position, angular velocity of eye movement, and pupil variation. The results showed that, although the significance levels of different indicators were not entirely consistent, the overall trend was clear. After entering the Intention state, pilots’ fixation distribution became more concentrated in task-critical areas, the fluctuation of fixation position in the vertical direction was significantly reduced for some indicators, and the relative power of average pupil diameter increased significantly in several frequency bands. These findings suggest that, before executing a maneuver, the pilot primarily responsible for aircraft control reallocated visual resources to display areas highly relevant to the current task, accompanied by a certain increase in arousal level and cognitive workload.

To investigate the task-critical areas in which pilots’ gaze positions were mainly concentrated after entering the Intention state, we calculated the proportions of the pilots’ primary areas of interest in both the Intention and Non-Intention samples, as shown in Figure 11. It can be seen that, after the pilot entered the Intention state, the proportions of fixation on the PFD, ND, and CDU instruments increased markedly, whereas the observation of other areas decreased substantially. Notably, the proportion of fixation on the ND instrument increased significantly in the Intention state, which may indicate that pilots paid greater attention to navigation and surrounding environmental information during the process of intention formation.

Figure 11. Proportions of pilots’ primary fixation areas in the Intention and Non-Intention samples.

From the perspective of the frequency domain indicators of eye-tracking data, the energy proportion of average pupil diameter in the low-frequency and mid-frequency components increased under the Intention state, whereas the high-frequency noise components did not increase significantly. This suggests that, during maneuver intention formation, pilots did not exhibit disordered rapid fluctuations, but rather showed relatively stable pupil regulation with greater amplitude. This eye-movement pattern is consistent with the multi-stage decision-making process of “perception–judgment–execution”: information is first perceived and integrated through relatively slow eye-movement scanning, followed by maneuver decision-making and execution preparation within a short period of time.

4.2. Modeling Paradigms Applicable to Pilot Intention Recognition Based on Eye-Tracking Data

All models discussed in this study were developed for intention recognition based on the same six-channel raw eye-tracking data. The differences among the models lie in how the same raw sequence was represented and modeled. Traditional machine learning models performed static pattern learning by flattening the time series into fixed-length vectors, whereas deep learning models preserved the temporal information of the eye-tracking data. According to the model performance comparison shown in Table 6, all models were able to recognize the pilot’s intention state, but their performance levels differed substantially. This indicates that the eye-tracking sequences contain exploitable predictive information related to pilot intention, but that such information can be more fully utilized through temporal modeling.

The performance differences among the models reflect the capability of eye-tracking data to support intention recognition under different modeling paradigms. The Logistic Regression model achieved an accuracy of 0.497 and an F1 score of 0.493, whereas the SVM model achieved an accuracy of 0.672 and an F1 score of 0.511. These results indicate that, when each time window is compressed into a static feature vector and the samples are assumed to be independent and identically distributed, the models can capture only limited linear or kernel-space separability, which is insufficient to characterize the more complex nonlinear and temporal dependencies involved in intention formation. Tree-based ensemble models alleviated this limitation to some extent. The Random Forest model achieved an accuracy of 0.712 and an F1 score of 0.686, while the accuracy and F1 score of XGBoost further increased to 0.839 and 0.832, respectively. This suggests that ensemble learning can more effectively exploit the nonlinear interactions among eye-tracking statistical features. However, such models still mainly operate in an “unordered feature” space and cannot explicitly represent temporal structures such as gaze sequence and rhythm.

The results of the sequence models show that the predictive ability of eye-tracking data is significantly enhanced when temporal dependencies can be explicitly encoded by the model. The unidirectional LSTM achieved an accuracy of 0.913 and an F1 score of 0.862, representing an improvement of approximately 0.03 in F1 over the best-performing tree-based model, while the recall increased from 0.850 to 0.905. This indicates that, compared with models based only on static features, sequence models that can encode the temporal dependencies of eye-tracking data have clear advantages in the intention recognition task. Furthermore, the RNN-BiLSTM introduced a bidirectional structure on this basis, allowing the model to make simultaneous use of short-term “past” and “future” context within a local time window. Its accuracy and F1 score reached 0.978 and 0.948, respectively, and the recall further increased to 0.965 while maintaining high precision compared with the unidirectional LSTM. These results suggest that the predictive advantage of eye-tracking sequences arises not only from differences in spatial distribution, but also from the transition sequence of fixation among task-critical areas and its temporal evolution pattern. Such information can be fully exploited only by temporal models with memory mechanisms.

In summary, for pilot intention recognition, eye-tracking data are more suitable for use within a temporal modeling framework. In future work, when multimodal information such as flight parameters, control inputs, and physiological signals is introduced, priority should also be given to integrating them within a unified sequence modeling paradigm in order to preserve and further enhance the predictive advantage brought by temporal features.

4.3. Implications of Pilot Intention Recognition for Airborne Warning Systems

From the perspective of flight safety, recognizing pilot intention through eye-tracking data can provide advance information about an upcoming maneuver, thereby offering an additional dimension for airborne warning systems in a variety of scenarios. Most existing airborne warnings, such as terrain proximity warnings, stall warnings, and airspeed/altitude deviation warnings, are mainly based on aircraft states and environmental measurements, without considering the pilot’s control intention. Within this framework, warning systems can only be triggered after the deviation has already become apparent. This may lead not only to delayed warnings, but also to frequent warnings during intended maneuvers, thereby reducing crew trust in the system. For example, when the aircraft is already in landing configuration and aligned with the runway, but the altitude is still too high and the pilot initiates a missed approach, pressing the Level Change (LVL CHG) switch to command a climb may cause the autopilot system to rapidly establish a climb, resulting in a sudden reduction in airspeed that may not be consistent with the pilot’s intention.

In this context, pilot intention recognition based on eye-tracking data can predict the pilot’s intention before the control action begins. On the one hand, when the pilot intentionally changes altitude or heading, certain warnings can be adaptively suppressed or delayed, thereby reducing frequent alerts during expected maneuvers. On the other hand, when the intention predicted by the system is clearly inconsistent with the current flight procedure, autopilot mode, or standard operating procedures (SOP), the warning system can issue an earlier alert or prompt indicating an intention–flight-state mismatch, thus providing the crew with additional time to assess the reasonableness of the intention and the potential human-factor risks. A similar concept may also be applied to flight training and evaluation systems by monitoring trainee intention in real time during simulator training and analyzing the timing and causes of incorrect intention formation.

However, from the perspective of practical application, an intention recognition model should not exist as an isolated function. Instead, this method needs to work in coordination with other flight safety warning systems to support integrated prediction. In future research, it will be necessary to further explore the mechanisms underlying the generation of pilot intention in risk scenarios and to identify pilot intention types in a more fine-grained manner, so that flight safety warning systems can take pilot intention into account in a more interpretable way. Therefore, to achieve effective pilot assistance, a comprehensive system integrating flight safety warning and pilot intention recognition is required.

4.4. Limitations and Future Prospects

This study has several limitations that should be fully considered in future research. First, this study was based on data obtained from controlled simulated flight experiments, in which the flight scenarios, risk events, and task pace were preset by the experimental design, and the number of pilot participants and aircraft types was also limited. Although this setting was beneficial for controlling interfering factors and verifying the feasibility of the proposed method, it still differs from the more complex flight situations, task workload, and operational habits encountered in real operations. Therefore, the current results mainly reflect the predictability of pilot intention based on eye-tracking data under a specific simulated environment and participant group, and are not yet sufficient to be directly generalized to multiple aircraft types, multiple crews, and complex operational environments. Future studies should further validate the robustness and transferability of the model using larger-scale simulated flight data and real-flight operational data from multiple aircraft types, and should also examine the effects of different experience levels and task types on intention recognition performance.

Second, to reveal the relationship between pilots’ eye-movement behavior before control execution and their intention states, this study intentionally restricted the input features to eye-tracking-related variables. Therefore, the model did not incorporate other modal information such as flight parameters, control inputs, or physiological signals. It also did not perform finer-grained multiclass prediction for different maneuver types, such as climb, descent, left turn, and right turn, but instead focused first on the binary classification task of Intention/Non-Intention. This setting helps highlight the independent contribution of eye-tracking signals, but it also means that the current model cannot directly distinguish which specific maneuver the intention corresponds to, and it is likewise unable to characterize more complex combined maneuver intentions. To extend the proposed method to the recognition of specific types of pilot maneuver intention, it is first necessary to define different intention types within a unified framework. A hierarchical strategy can then be adopted, in which a binary classifier is used to detect whether an intention exists, followed by a second-stage classifier to further identify the specific maneuver type within the time windows classified as Intention.

Finally, although the RNN-BiLSTM achieved the best predictive performance in this study, its internal state representation and decision-making process remain essentially black-box for pilots. The current analysis mainly demonstrates the importance of temporal dependencies in pilot intention recognition based on eye-tracking data indirectly through overall performance and comparisons among different models. However, the key time segments and feature dimensions attended to by the model have not yet been systematically linked to pilots’ cognitive stages. In future work, methods such as attention mechanisms, time-step sensitivity analysis, and feature contribution measurement can be introduced to improve model interpretability. Combined with behavioral and cognitive theories, these methods may help establish a consistent interpretive framework linking eye-movement patterns during intention formation with model outputs.

5. Conclusions

This study analyzed the time domain and frequency domain characteristics of eye-tracking data under Intention and Non-Intention states, and established a deep learning model to recognize pilots’ intention states. Based on the results of this study, the following main conclusions can be drawn:

(1)

According to the time domain analysis results of the eye-tracking features, during intention formation, pilots tend to focus visual resources on task-critical instrument areas in advance and support information acquisition with more stable pupil responses.

(2)

According to the frequency domain analysis results of the eye-tracking features, the more discriminative indicators during intention formation are the rhythmic oscillations of pupil diameter in the low- to mid-frequency range and the enhanced fluctuations of fixation position in the mid-frequency band. This suggests that, in modeling, priority should be given to the low- and mid-frequency power of pupil diameter and the mid-frequency power of fixation position, while velocity-related features should be transformed into higher-level event-based features.

(3)

The results of pilot intention state recognition based on eye-tracking data indicate that eye-movement behavior itself contains exploitable predictive information about pilot intention, and that such information can be fully utilized only by temporal models with memory mechanisms.

(4)

This study is still limited by the sample size, the use of a single eye-tracking modality, and the performance obtained under the event-level cross-validation framework. Future work is therefore needed to further evaluate the engineering application boundaries of this method in airborne warning systems on the basis of multimodal fusion and larger-scale validation.

Compared with existing pilot intention recognition methods, this study analyzed the time-frequency characteristics of eye-tracking data when pilot intention was present at the data level, revealed the eye-tracking features that effectively improve the performance of pilot intention recognition models, and discussed the modeling paradigms applicable to pilot intention recognition based on eye-tracking data. The findings of this study provide researchers and policymakers with a new perspective for improving the accuracy of pilot intention recognition. At present, the proposed method is only applicable to recognizing whether pilot intention is present. Future work will build on the conclusions of this study and focus on classification methods for recognizing different types of pilot intention.