# Multimodal Analysis of Eye Movements and Fatigue in a Simulated Glass Cockpit Environment

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background

#### 2.1. PVT Measures and Fatigue Assessment

^{−1}.

#### 2.2. Eye Movement Measures and Fatigue Evaluation

## 3. Methods

- Step 1:
- Assess fatigue level through PVT after each task. Measures are (a) reaction times (RT), (b) number of false starts (FS), and (c) number of lapses (L).
- Step 2:
- Collect eye tracking data, analyzing the data using the context-specific areas of interest (AOIs). Measures are:
- (a)
- Mean eye fixation number on AOIs;
- (b)
- Mean eye fixation duration on AOIs;
- (c)
- Mean pupil size on AOIs;
- (d)
- Visual entropy (calculation process explained below).

- Step 3:
- Plot the relationships of the variables and investigate the correlations between the PVT measures and the eye tracking measures. The measures were all those provided in Steps 1 and 2. Step 3 was needed to first see whether linear correlations could be observed prior to applying multiple regression. In other words, different regression models should be applied based on the relationships. For example, if the relationship among the variables were quadratic, then a quadratic regression should be applied.
- Step 4:
- Create a “unified” PVT measure by combining the PVT measures of RT, FA, and L. The unified measure (S) is expressed as follows:$$S={W}_{1}\times RT+{W}_{2}\times FA+{W}_{3}\times L$$
- Step 5:
- Discover an optimal regression model that can predict fatigue using one or more eye-tracking measures. Stepwise regression approach was applied (both forward and backward) to discover the optimal regression model. We assumed that the unified PVT measure accurately represented one’s fatigue level, and we found eye tracking measures that could predict fatigue level. All eye tracking measures were normalized, meaning that the minimum and maximum values obtained from all the experiment participants were mapped to 0 and 1. The full model and associated variable for the backward regression is:$$S={\beta}_{0}+{\beta}_{1}\times FN+{\beta}_{2}\times FD+{\beta}_{3}\times PS+{\beta}_{4}\times TE+{\beta}_{5}\times SE$$

## 4. Experiment

#### 4.1. Participants

#### 4.2. Apparatus

#### 4.3. Tasks and Procedures

#### 4.4. Measures

#### 4.5. Data analysis

## 5. Results

#### 5.1. PVT Measures

#### 5.2. Eye Movement Measures

#### 5.3. Correlation Results

#### 5.4. Regression Models

- (1)
- Multiple linear regression results: The multiple linear regression analysis using the unified PVT measure ($S$) and all eye movement measures resulted in regression models provided in Equations (5) and (6). The full model of the novice pilots resulted in the overall model fit of adjusted ${R}^{2}=0.85$ and AIC = −177.04. Whereas, the full model of the expert pilots’ group, resulted in the overall model fit of adjusted ${R}^{2}=0.66$ and AIC = −145.12.

- (2)
- Stepwise regression results: The results of the stepwise regressions are provided in Equations (7) and (8). For the novice pilots, to predict $S$, eye fixation duration ($\beta =0.45$, p < 0.05), eye fixation number ($\beta =-0.31$, p < 0.05), transition entropy ($\beta =0.42$, p < 0.05), and stationary entropy ($\beta =0.38$, p < 0.05) were found to be significant with an overall model fit of ${R}^{2}=0.84$ and AIC= −178.76. For expert pilots, only eye fixation duration ($\beta =0.65$, p < 0.05) was found significant with a lower model fit ${R}^{2}=0.64$ and AIC= −151.15.

## 6. Discussion

## 7. Limitations and Future Research

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Goode, J.H. Are pilots at risk of accidents due to fatigue? J. Saf. Res.
**2003**, 34, 309–313. [Google Scholar] [CrossRef] - Wiegmann, D.A.; Shappell, S.A. The Human Factors Analysis and Classification System (HFACS). In A Human Error Approach to Aviation Accident Analysis; Routledge: Burlington, VT, USA, 2003; pp. 45–71. [Google Scholar]
- Li, G.; Baker, S.P.; Grabowski, J.G.; Rebok, G.W. Factors associated with pilot error in aviation crashes. Aviat. Space Environ. Med.
**2001**, 72, 52–58. [Google Scholar] [PubMed] - Oster, C.V.; Strong, J.S.; Zorn, C.K. Analyzing aviation safety: Problems, challenges, opportunities. Res. Transp. Econ.
**2013**, 43, 148–164. [Google Scholar] [CrossRef] - Hartzler, B.M. Fatigue on the flight deck: The consequences of sleep loss and the benefits of napping. Accid. Anal. Prev.
**2014**, 62, 309–318. [Google Scholar] [CrossRef] [PubMed] - Dismukes, R.K. Effects of Acute Stress on Aircrew Performance: Literature Review and Analysis of Operational Aspects. Available online: https://human-factors.arc.nasa.gov/publications/NASA_TM_2015_218930-2.pdf (accessed on 11 July 2021).
- Lee, S.; Kim, J.K. Factors contributing to the risk of airline pilot fatigue. J. Air Transp. Manag.
**2018**, 67, 197–207. [Google Scholar] [CrossRef] - Bourgeois-Bougrine, S.; Carbon, P.; Gounelle, C.; Mollard, R.; Coblentz, A. Perceived fatigue for short- and long-haul flights: A survey of 739 airline pilots. Aviat. Space Environ. Med.
**2003**, 74, 1072–1077. [Google Scholar] [PubMed] - Powell, D.M.C.; Spencer, M.B.; Holland, D.; Broadbent, E.; Petrie, K.J. Pilot fatigue in short-haul operations: Effects of num-ber of sectors, duty length, and time of day. Aviat. Space Environ. Med.
**2007**, 78, 698–701. [Google Scholar] - Powell, D.; Spencer, M.B.; Holland, D.; Petrie, K.J. Fatigue in two-pilot operations: Implications for flight and duty time limitations. Aviat. Space Environ. Med.
**2008**, 79, 1047–1050. [Google Scholar] [CrossRef] - Petrilli, R.M.; Roach, G.; Dawson, D.; Lamond, N. The Sleep, Subjective Fatigue, and Sustained Attention of Commercial Airline Pilots during an International Pattern. Chrono-Int.
**2006**, 23, 1357–1362. [Google Scholar] [CrossRef] - Samel, A.; Wegmann, H.M.; Vejvoda, M. Jet lag and sleepiness in aircrew. J. Sleep Res.
**1995**, 4, 30–36. [Google Scholar] [CrossRef] - Honn, K.A.; Satterfield, B.C.; McCauley, P.; Caldwell, J.L.; Van Dongen, H.P. Fatiguing effect of multiple take-offs and landings in regional airline operations. Accid. Anal. Prev.
**2016**, 86, 199–208. [Google Scholar] [CrossRef][Green Version] - Van Drongelen, A.; Van Der Beek, A.J.; Hlobil, H.; Smid, T.; Boot, C.R. Development and evaluation of an intervention aiming to reduce fatigue in airline pilots: Design of a randomised controlled trial. BMC Public Heal
**2013**, 13, 776. [Google Scholar] [CrossRef][Green Version] - Arsintescu, L.; Chachad, R.; Gregory, K.B.; Mulligan, J.B.; Flynn-Evans, E.E. The Relationship between Workload, Perfor-mance and Fatigue in a Short-Haul Airline. Chronobiol. Int.
**2020**, 37, 1492–1494. [Google Scholar] [CrossRef] - Binias, B.; Myszor, D.; Palus, H.; Cyran, K.A. Prediction of Pilot’s Reaction Time Based on EEG Signals. Front. Neuroinforma.
**2020**, 14, 6. [Google Scholar] [CrossRef] - Borghini, G.; Astolfi, L.; Vecchiato, G.; Mattia, D.; Babiloni, F. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev.
**2014**, 44, 58–75. [Google Scholar] [CrossRef] [PubMed] - Dehais, F.; Dupres, A.; Di Flumeri, G.; Verdiere, K.; Borghini, G.; Babiloni, F.; Roy, R. Monitoring Pilot’s Cognitive Fatigue with Engagement Features in Simulated and Actual Flight Conditions Using an Hybrid FNIRS-EEG Passive BCI. In Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan, 7–10 October 2018. [Google Scholar]
- Di Stasi, L.L.; McCamy, M.B.; Martinez-Conde, S.; Gayles, E.; Hoare, C.; Foster, M.; Catena, A.; Macknik, S.L. Effects of long and short simulated flights on the saccadic eye movement velocity of aviators. Physiol. Behav.
**2016**, 153, 91–96. [Google Scholar] [CrossRef][Green Version] - Diaz-Piedra, C.; Rieiro, H.; Suárez, J.; Rios-Tejada, F.; Catena, A.; Di Stasi, L.L. Fatigue in the military: Towards a fatigue detection test based on the saccadic velocity. Physiol. Meas.
**2016**, 37, N62–N75. [Google Scholar] [CrossRef] - Wu, X.; Wanyan, X.; Zhuang, D. Pilot’s visual attention allocation modeling under fatigue. Technol. Heal. Care
**2015**, 23 (Suppl. S2), S373–S381. [Google Scholar] [CrossRef] [PubMed][Green Version] - Naeeri, S.; Kang, Z. Exploring the Relationship between Pilot’s Performance and Fatigue When Interacting with Cockpit In-terfaces. In Proceedings of the 2018 IISE Annual Conference, Orlando, FL, USA, 19–22 May 2018. [Google Scholar]
- Naeeri, S.; Mandal, S.; Kang, Z. Analyzing Pilots’ Fatigue for Prolonged Flight Missions: Multimodal Analysis Approach Us-ing Vigilance Test and Eye Tracking. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Seattle, WA, USA, 28 October–1 November 2019; Volume 63, pp. 111–115. [Google Scholar]
- Gander, P.H.; Signal, T.L.; Berg, M.V.D.; Mulrine, H.M.; Jay, S.M.; Mangie, C.J. In-flight sleep, pilot fatigue and Psychomotor Vigilance Task performance on ultra-long range versus long range flights. J. Sleep Res.
**2013**, 22, 697–706. [Google Scholar] [CrossRef] [PubMed] - Gander, P.H.; Mulrine, H.M.; Berg, M.V.D.; Smith, A.A.T.; Signal, T.L.; Wu, L.J.; Belenky, G. Pilot Fatigue: Relationships with Departure and Arrival Times, Flight Duration, and Direction. Aviat. Space Environ. Med.
**2014**, 85, 833–840. [Google Scholar] [CrossRef] - Thomas, L.C.; Gast, C.; Grube, R.; Craig, K. Fatigue Detection in Commercial Flight Operations: Results Using Physiological Measures. Procedia Manuf.
**2015**, 3, 2357–2364. [Google Scholar] [CrossRef][Green Version] - Caldwell, J.A.; Hall, K.K.; Erickson, B.S. EEG Data Collected from Helicopter Pilots in Flight Are Sufficiently Sensitive to De-tect Increased Fatigue from Sleep Deprivation. Int. J. Aviat. Psychol.
**2002**, 12, 19–32. [Google Scholar] [CrossRef] - Naeeri, S.; Kang, Z.; Mandal, S. Exploring the effect of fatigue on pilot performance during single and multi-takeoffs and landings flight missions. In Proceedings of the 7th Annual World Conference of the Society for Industrial and Systems Engineering, Binghamton, NY, USA, 11–12 October 2018; Volume 7, pp. 174–181. [Google Scholar]
- Millar, M. Measuring Fatigue Overview. In Proceedings of the Asia-Pacific, ICAO/IATA/IFALPA FRMS Seminar, Bangkok, Thailand, 1–2 November 2012. [Google Scholar]
- Bleichner, M.G.; Debener, S. Concealed, Unobtrusive Ear-Centered EEG Acquisition: cEEGrids for Transparent EEG. Front. Hum. Neurosci.
**2017**, 11, 163. [Google Scholar] [CrossRef][Green Version] - Bodala, I.P.; Li, J.; Thakor, N.V.; Al-Nashash, H. EEG and Eye Tracking Demonstrate Vigilance Enhancement with Challenge Integration. Front. Hum. Neurosci.
**2016**, 10, 273. [Google Scholar] [CrossRef][Green Version] - Lu, T.; Lou, Z.; Shao, F.; Li, Y.; You, X. Attention and Entropy in Simulated Flight with Varying Cognitive Loads. Aerosp. Med. Hum. Perform.
**2020**, 91, 489–495. [Google Scholar] [CrossRef] - Diaz-Piedra, C.; Rieiro, H.; Cherino, A.; Fuentes, L.J.; Catena, A.; Stasi, L.L.D. The Effects of Flight Complexity on Gaze En-tropy: An Experimental Study with Fighter Pilots. Appl. Ergon.
**2019**, 77, 92–99. [Google Scholar] [CrossRef] [PubMed] - Bellenkes, A.H.; Wickens, C.D.; Kramer, A. Visual scanning and pilot expertise: The role of attentional flexibility and mental model development. Aviat. Space Environ. Med.
**1997**, 68, 569–579. [Google Scholar] [PubMed] - Thorne, D.R.; Johnson, D.E.; Redmond, D.P.; Sing, H.C.; Belenky, G.; Shapiro, J.M. The Walter Reed palm-held psychomotor vigilance test. Behav. Res. Methods
**2005**, 37, 111–118. [Google Scholar] [CrossRef] - Lee, I.-S.; Bardwell, W.A.; Ancoli-Israel, S.; Dimsdale, J.E. Number of Lapses during the Psychomotor Vigilance Task as an Objective Measure of Fatigue. J. Clin. Sleep Med.
**2010**, 6, 163–168. [Google Scholar] [CrossRef][Green Version] - Dinges, D.F.; Mallis, M.M.; Maislin, G.; Powell, J.W. Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and as the Basis for Alertness Management; National Highway Traffic Safety Administration: Washington, DC, USA, 1998. [Google Scholar]
- Dinges, D.F.; Maislin, G.; Brewster, R.M.; Krueger, G.P.; Carroll, R.J. Pilot test of fatigue management technologies. Transp. Res. Rec.
**2005**, 1922, 175–182. [Google Scholar] [CrossRef] - LeDuc, P.A.; Greig, J.L.; Dumond, S.L. Self-Report and Ocular Measures of Fatigue in U.S. Army Apache Aviators Following Flight; Army Aeromedical Research Lab: Fort Rucker, AL, USA, 2005. [Google Scholar]
- Shannon, C.E. A Mathematical Theory of Communication. Bell Syst. Tech. J.
**1948**, 27, 379–423. [Google Scholar] [CrossRef][Green Version] - Shiferaw, B.; Downey, L.; Westlake, J.; Stevens, B.; Rajaratnam, S.; Berlowitz, D.J.; Swann, P.; Howard, M.E. Stationary gaze entropy predicts lane departure events in sleep-deprived drivers. Sci. Rep.
**2018**, 8, 1–10. [Google Scholar] [CrossRef][Green Version] - Wu, C.; Cha, J.; Sulek, J.; Zhou, T.; Sundaram, C.P.; Wachs, J.; Yu, D. Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training. Hum. Factors: J. Hum. Factors Ergon. Soc.
**2020**, 62, 1365–1386. [Google Scholar] [CrossRef][Green Version] - Krejtz, K.; Szmidt, T.; Duchowski, A.T.; Krejtz, I. Entropy-Based Statistical Analysis of Eye Movement Transitions. In Proceedings of the Symposium on Eye Tracking Research and Applications, Safety Harbor, FL, 26–28 March 2014. [Google Scholar]
- Cover, T.M.; Thomas, J.A. Elements of Information Theory; John Wiley & Sons, Inc.: New York, NY, USA, 2006. [Google Scholar]
- Han, X.; Shao, Y.; Yang, S.; Yu, P. Entropy-Based Effect Evaluation of Delineators in Tunnels on Drivers’ Gaze Behavior. Entropy
**2020**, 22, 113. [Google Scholar] [CrossRef] [PubMed][Green Version] - Gateau, T.; Durantin, G.; Lancelot, F.; Scannella, S.; Dehais, F. Real-Time State Estimation in a Flight Simulator Using fNIRS. PLoS ONE
**2015**, 10, e0121279. [Google Scholar] [CrossRef] [PubMed] - Kirby, C.E.; Kennedy, Q.; Yang, J.H. Helicopter pilot scan techniques during low-altitude high-speed flight. Aviat. Space Environ. Med.
**2014**, 85, 740–744. [Google Scholar] [CrossRef][Green Version] - Mueller, S.T.; Piper, B.J. The Psychology Experiment Building Language (PEBL) and PEBL Test Battery. J. Neurosci. Methods
**2014**, 222, 250–259. [Google Scholar] [CrossRef] [PubMed][Green Version] - Lim, J.; Dinges, D.F. Sleep Deprivation and Vigilant Attention. Ann. N. Y. Acad. Sci.
**2008**, 1129, 305–322. [Google Scholar] [CrossRef] - Falk, R.F.; Miller, N.B. A Primer for Soft Modeling; University of Akron: Akron, OH, USA, 1992. [Google Scholar]
- Bradley, M.M.; Miccoli, L.; Escrig, M.A.; Lang, P.J. The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology
**2008**, 45, 602–607. [Google Scholar] [CrossRef][Green Version] - Kang, Z.; Landry, S.J. An Eye Movement Analysis Algorithm for a Multielement Target Tracking Task: Maximum Transition-Based Agglomerative Hierarchical Clustering. IEEE Trans. Hum. Mach. Syst.
**2015**, 45, 13–24. [Google Scholar] [CrossRef] - Mandal, S.; Kang, Z. Using Eye Movement Data Visualization to Enhance Training of Air Traffic Controllers: A Dynamic Network Approach. J. Eye Mov. Res.
**2018**, 11, 1–20. [Google Scholar] [CrossRef] - Gao, X.-Y.; Zhang, Y.-F.; Zheng, W.-L.; Lu, B.-L. Evaluating Driving Fatigue Detection Algorithms Using Eye Tracking Glass-es. In Proceedings of the 7th International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, France, 22–24 April 2015. [Google Scholar]
- Powell, D.M.C.; Spencer, M.B.; Petrie, K.J. Comparison of In-Flight Measures with Predictions of a Bio-Mathematical Fatigue Model. Aviat. Space Environ. Med.
**2014**, 85, 1177–1184. [Google Scholar] [CrossRef] [PubMed] - Williamson, A.; Lombardi, D.A.; Folkard, S.; Stutts, J.; Courtney, T.; Connor, J. The link between fatigue and safety. Accid. Anal. Prev.
**2011**, 43, 498–515. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Measures used to investigate fatigue for a multiphase flight task: FN is eye fixation numbers, FD is eye fixation durations, PS is pupil size, Ht is transition visual entropy, Hs is stationary visual entropy, RT is reaction times, FS is number of false starts, and L is number of lapses.

**Figure 2.**Four consecutive tasks (without any rest) labeled as tasks 1 through 4. Each task lasted approximately 1 h. The total duration was approximately 4 h.

**Figure 3.**Context-specific AOIs that were defined based on the instrument fight rules (IFRs). The AOI names were as follows: engine oil pressure: EOP; engine indicators: EIs; enhanced visual screen: EVS; attitude indicator: ATT; horizontal situation indicator: HS; flight command indicator: FC; altimeter: ALT; airspeed indicator: AS; true airspeed indicator: TS; heading indicator: HI; vertical velocity indicator: VV; radar altimeter: RA; Mach indicator: MI; standby horizon indicator: SHS. Most of the eye fixations occurred on these AOIs during an IFR flight when we observed the recorded data after the experiments. The response variables related to the eye movements were eye fixation number on the AOIs, eye fixation duration on the AOIs, pupil size, and visual entropy (both transition and stationary entropy).

**Figure 5.**Examples of visual scanpaths of an expert and a novice pilot: The yellow circles represent the eye fixations where the numbers represent its index. The yellow lines represent the saccades. The size of the eye fixation circles have been kept at a fixed size for visual clarity. In addition, only 40 s of data are provided for each sample. FN is the eye fixation number, and FD is the eye fixation duration.

**Table 1.**Classification of existing research in pilot fatigue: Classifications are mostly based on the fatigue evaluation method. The last three listed in the table are studies not related to fatigue but worth mentioning.

Research Related to Pilots’ Fatigue | Research Topic | Fatigue Evaluation Method | Expertise | Single or Multiple Take-off Landings | Short (1~3 h) vs. Long Duration (3+ h) Flight | Statistical Method |
---|---|---|---|---|---|---|

[8] | Fatigue | Subjective | Experts | Single | Short and long | Multiple regression and ANOVA |

[14] | Fatigue | Subjective | Experts | Single | Long and Short | linear mixed-model |

[15] | Workload and fatigue | PVTand Subjective | Experts | Single | Short | Stepwise Regression and correlation |

[16] | Reaction time | EEG | Novice | None | Short | Robust linear model |

[17] | Mental workload and fatigue | EEG | - | - | - | Literature review |

[18] | Fatigue | EEG | Novices | Single | Short | Classification model |

[19] | Fatigue | Eye tracking | Experts | Single | Short and Long | ANOVA and linear regression |

[20] | Fatigue | Eye tracking | Experts | Single | Short | Pre/Post-Test design |

[21] | Fatigue | Eye tracking and PVT | Novices | Single | Short | ANOVA and regression |

[22] | Fatigue | Eye tracking and PVT | Novices and experts | Single | Short | Mann-Whitney-Wilcoxon tests |

[23] | Fatigue | Eye tracking and PVT | Novices | Multiple | Long | Kruskal-Wallis test |

[11] | Fatigue and sustained attention | PVT and Subjective | Experts | Single | Long | Linear mixed model regression |

[24] | Fatigue and Performance | PVT and Subjective | Experts | Single | Long and ultra-long | Mixed-model ANOVA |

[25] | Fatigue and performance | PVT and Subjective | Experts | Single | Long and ultra-long | ANOVA |

[26] | Fatigue | PVT and Subjective | Experts | Single | Short and Long | Statistical/Machine learning model |

[27] | Fatigue | EEG | Novice | Single | Long | ANOVA |

[28] | Fatigue | Eye tracking and PVT | Experts | Multiple | Long | ANOVA |

[32] | Cognitive load | Eye tracking | Experts | Single | Short | Paired t-tests |

[33] | Workload | EEG, Eye tracking, and Subjective | Experts | Single | Short | ANOVA and correlation |

[34] | Performance | Eye tracking | Novices and experts | Single | Short | ANOVA |

(a) | ||||

${H}_{t}=1.6;\text{}{H}_{s}=2.0$ | ||||

TO | ||||

From | A | B | C | D |

A | 0 | 0.33 | 0.33 | 0.34 |

B | 0.33 | 0 | 0.33 | 0.34 |

C | 0.33 | 0.34 | 0 | 0.33 |

D | 0.33 | 0.33 | 0.34 | 0 |

(b) | ||||

${H}_{t}=0.1;\text{}{H}_{s}=1.6$ | ||||

TO | ||||

From | A | B | C | D |

A | 0 | 0.99 | 0.01 | 0 |

B | 0 | 0 | 0.99 | 0.01 |

C | 0.99 | 0 | 0 | 0.01 |

D | 0 | 0.99 | 0.01 | 0 |

**Table 3.**Results of the mixed model analysis of variance on PVT measures: Exp is expertise factor (experts vs. novices) related to the between-subjects design, and task is the task factor (tasks 1, 2, 3, and 4) related to the within-subjects design.

Between-Subjects | Within-Subjects | |||||
---|---|---|---|---|---|---|

F (1,18) | p | ${\mathit{\eta}}_{\mathit{p}}^{2}$ | F (3,54) | p | ${\mathit{\eta}}_{\mathit{p}}^{2}$ | |

Reaction time (RT) | ||||||

Exp | 82.45 | <0.001 | 0.8 | |||

Task | 177 | <0.001 | 0.91 | |||

Exp × Task | 5.26 | <0.003 | 0.23 | |||

Lapse (L) | ||||||

Exp | 104.7 | <0.001 | 5.26 | |||

Task | 35.11 | <0.001 | 0.66 | |||

Exp × Task | 4.67 | <0.001 | 0.21 | |||

False start (FS) | ||||||

Exp | 90.72 | <0.001 | 0.83 | |||

Task | 39.87 | <0.001 | 0.69 | |||

Exp × Task | 6.99 | <0.001 | 0.28 |

Between-Subjects | Within-Subjects | |||||
---|---|---|---|---|---|---|

F (1,18) | p | ${\mathit{\eta}}_{\mathit{p}}^{2}$ | F (3,54) | p | ${\mathit{\eta}}_{\mathit{p}}^{2}$ | |

Eye fixation number (FN) | ||||||

Exp | 72.41 | <0.001 | 0.80 | |||

Task # | 157.12 | <0.001 | 0.89 | |||

Exp $\times $ Task # | 5.25 | <0.003 | 0.22 | |||

Eye fixation duration (FD) | ||||||

Exp | 459.9 | <0.001 | 0.96 | |||

Task # | 168.75 | <0.001 | 0.90 | |||

Exp $\times $ Task # | 7.51 | <0.001 | 0.29 | |||

Pupil size (PS) | ||||||

Exp | 101.89 | <0.001 | 0.85 | |||

Task # | 408.79 | <0.001 | 0.96 | |||

Exp $\times $ Task # | 21.24 | <0.001 | 0.54 | |||

Transition entropy (${H}_{t}$) | ||||||

Exp | 210.88 | <0.001 | 0.92 | |||

Task # | 200.75 | <0.001 | 0.92 | |||

Exp $\times $ Task # | 9.15 | <0.001 | 0.34 | |||

Stationary entropy (${H}_{s}$) | ||||||

Exp | 119.11 | <0.001 | 0.87 | |||

Task # | 75.99 | <0.001 | 0.81 | |||

Exp $\times $ Task # | 3.85 | <0.014 | 0.18 |

**Table 5.**Results of the one-way repeated measures analysis of variance on eye movements measures in which the task number (tasks 1, 2, 3, and 4) is the factor.

DV | Experts | Novices | ||
---|---|---|---|---|

F (3,27) | p | F (3,27) | p | |

Eye fixation number (FN) | 64.66 | <0.001 | 107.72 | <0.001 |

Eye fixation duration (FD) | 151.37 | <0.001 | 71.98 | <0.001 |

Pupil size (PS) | 160.11 | <0.001 | 264.57 | <0.001 |

Transition entropy (${H}_{t}$) | 125.02 | <0.001 | 98.08 | <0.001 |

Stationary entropy (${H}_{s}$) | 42.39 | <0.001 | 38.59 | <0.001 |

Expert | Novice | |||||
---|---|---|---|---|---|---|

RT | L | FS | RT | L | FS | |

FN | −0.69 | −0.61 | −0.49 | −0.84 | −0.73 | −0.76 |

FD | 0.76 | 0.68 | 0.61 | 0.88 | 0.78 | 0.74 |

PS | −0.81 | −0.56 | −0.53 | −0.86 | −0.78 | −0.70 |

${H}_{t}$ | 0.75 | 0.63 | 0.63 | 0.79 | 0.68 | 0.70 |

${H}_{s}$ | 0.65 | 0.56 | 0.59 | 0.73 | 0.64 | 0.62 |

**Table 7.**Stepwise regression (backward) results with unified PVT measure as response and eye movement measures as predictors for both expert and novice pilots.

Variables | Expert | Novice | |||||
---|---|---|---|---|---|---|---|

Step I | Step II | Step III | Step IV | Step V | Step I | Step II | |

(Constant) | 0.31 | 0.31 | 0.29 | 0.20 | 0.23 | 0.46 | 0.43 |

FD | 0.33 | 0.33 | 0.35 | 0.40 | 0.65 | 0.42 | 0.45 |

FN | −0.05 | −0.05 | −0.27 | −0.31 | |||

PS | −0.07 | −0.09 | −0.08 | 0.42 | |||

HT | 0.25 | −0.07 | 0.25 | 0.29 | 0.42 | −0.38 | |

HS | −0.007 | 0.25 | −0.40 | ||||

Adjusted R | 0.61 | 0.62 | 0.63 | 0.64 | 0.64 | 0.83 | 0.84 |

F | 13.26 * | 17.06 * | 23.32 * | 35.61 * | 68.72 * | 39.17 * | 50.0 * |

AIC | −145.12 | −147.1 | −149.02 | −150.78 | −151.15 | −177.03 | −178.76 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Naeeri, S.; Kang, Z.; Mandal, S.; Kim, K. Multimodal Analysis of Eye Movements and Fatigue in a Simulated Glass Cockpit Environment. *Aerospace* **2021**, *8*, 283.
https://doi.org/10.3390/aerospace8100283

**AMA Style**

Naeeri S, Kang Z, Mandal S, Kim K. Multimodal Analysis of Eye Movements and Fatigue in a Simulated Glass Cockpit Environment. *Aerospace*. 2021; 8(10):283.
https://doi.org/10.3390/aerospace8100283

**Chicago/Turabian Style**

Naeeri, Salem, Ziho Kang, Saptarshi Mandal, and Kwangtaek Kim. 2021. "Multimodal Analysis of Eye Movements and Fatigue in a Simulated Glass Cockpit Environment" *Aerospace* 8, no. 10: 283.
https://doi.org/10.3390/aerospace8100283