1. Introduction
Driven by rapid advancements in information and sensor technologies, modern aviation operations are increasingly reliant on the electromagnetic (EM) spectrum. Consequently, the complexity of aviation EM environment data has grown exponentially, characterized by highly coupled, multivariable, and large-scale spatiotemporal data. This inherent complexity renders the effective visualization of EM fields a formidable technical challenge [
1].
To manage this complexity, modern human–machine interfaces are required to simultaneously present fundamental flight parameters alongside real-time EM data that accurately reflects congested and dynamic airspaces [
2]. However, the composite display of this multi-source information imposes severe cognitive loads on human operators, placing high demands on their visual attention and rapid information processing capabilities under extreme time constraints [
3,
4]. Therefore, scientific EM visualization strategies that inherently account for human cognitive capability are critical to enhancing human–machine collaborative decision-making in the aviation industry.
To effectively mitigate operator cognitive load, user-centered visualization is founded upon two primary design pillars: visual coding strategies [
5] and structural interface layout [
6]. In the context of the aviation EM environment, visual coding requires synthesizing an immense volume of multi-source data into an intuitive format. To prevent cognitive overload, an effective interface design distills this complexity into several fundamental dimensions [
7]. The spatial dimension, anchored by radar detection ranges, delineates the geometric boundaries indispensable for tactical path planning. The energy dimension conveys instantaneous operational risks and interference intensity through EM field strength. Concurrently, the frequency dimension monitors spectrum occupancy to track resource allocation and pinpoint potential signal conflicts, while the temporal dimension reflects the continuous dynamic evolution of EM signals.
Furthermore, the attribute dimension provides critical contextual cues by classifying the discrete threat levels of detected targets. Beyond the distinct visual encoding of these parameters, the spatial layout of the interface is also paramount. For instance, Shao et al. [
8] demonstrated that the dynamic information layout within spatial interfaces fundamentally dictates users’ visual search performance and reaction times. Building on this, Zhang et al. [
9] established that optimizing layout order, specifically the strategic positioning of core visual components, significantly mitigates the adverse cognitive effects of high interface complexity.
However, there is a lack of empirical research providing a comprehensive cognitive evaluation of these multidimensional EM visualization strategies under dynamic operational conditions. Future interface development would benefit from a rigorous, objective perspective on this area. To address this gap, this study proposes a human-centered visualization approach tailored for complex aviation EM environments. The primary objective of this research is to empirically evaluate how multidimensional visual encodings and spatial layouts affect operator information acquisition under dynamic, concurrent task demands. Specifically, we hypothesized that variations in visual encoding strategies and the spatial layout of EM indicators would significantly influence operators’ visual search efficiency, task performance, and subjective mental workload.
To test these hypotheses, we first designed multi-dimensional visual encodings and spatial layouts to translate abstract EM data into intuitive graphical representations. To examine information acquisition under dynamic, concurrent task demands, an aviation task simulation was constructed utilizing the Unity 3D engine. Within this simulated operational environment, we conducted a simulation experiment to investigate how different visual strategies impact operator performance. By analyzing behavioral data (reaction time and accuracy), visual attention patterns, and subjective cognitive load assessments via the NASA Task Load Index (NASA-TLX), we empirically identified the most effective EM visualization design. Ultimately, this research bridges the gap between technical EM data presentation and human cognitive ergonomics, providing experimental evidence for visualization strategies that support efficient recognition and monitoring of aviation EM information while reducing subjective mental workload.
The remainder of this paper is organized as follows.
Section 2 reviews related work.
Section 3 details the experimental design, including the visual encoding strategies and the Unity 3D task simulation.
Section 4 presents the behavioral and eye-tracking results.
Section 5 discusses the main findings and design implications. Lastly,
Section 6 offers concluding remarks.
3. Materials and Methods
3.1. Participants
Thirty-two participants (16 males and 16 females, aged 19 to 27 years) were recruited for the experiment. None of the participants had professional piloting or electronic-warfare experience. All participants reported normal or corrected-to-normal vision and no history of color vision deficiency. Prior to the experiment, written informed consent was obtained from each participant. This study was conducted in accordance with the ethical guidelines of the institution. Upon completion, participants received monetary compensation for their time and sustained engagement throughout the study. An a priori power analysis using G*Power 3.1.9.7 (F-test, repeated measures within factors, effect size f = 0.25, α = 0.05, power = 0.80, 8 measurements) indicated a minimum required sample size of 16. Thus, our recruited sample of 32 participants was adequately powered.
3.2. Experimental Design
A 2 × 2 × 2 repeated-measures design was employed for this study. The independent variables were radar-range encoding (2 levels: grid and color-overlay), target-threat encoding (2 levels: color-background and border-thickness), and spatial layout (2 levels: bottom-left and center), all manipulated within participants, resulting in a total of 8 experimental blocks. The dependent variables included objective task performance, subjective evaluations and qualitative eye-tracking patterns.
3.2.1. EM Visual Encoding and Spatial Layout Design
Based on the results of several pilot tests, the spatial and attribute dimensions of the EM environment were encoded as follows. For the spatial dimension, the radar detection ranges were encoded in two alternative formats: (1) a two-dimensional grid sector designed to minimize obstruction of the situational map and reduce visual clutter (
Figure 1a), and (2) a color-overlay sector utilizing semi-transparent gradient blocks to intuitively distinguish between scanned and non-scanned areas (
Figure 1b).
Regarding the attribute dimension, detected targets were represented using one of two design formats. Format 1 employed a semi-transparent background overlaid on the target icon, where red, orange, and yellow denoted high, medium, and low threat levels, respectively (
Figure 2a). Format 2 utilized the border thickness of the target icon to indicate the threat level: a thick border for high threat, a thin border for medium threat, and no border for low threat (
Figure 2b). Additionally, target identity and status were standardized across conditions: solid red icons represented enemy targets, whereas empty yellow icons denoted unknown targets. Solid lines trailing the target indicated historical flight paths, and dashed lines represented predicted paths based on motion models. Finally, two concentric arcs surrounding the target demarcated the maximum and minimum firing ranges (
Figure 2c). The attribute encoding of a target appeared immediately upon its detection by the radar.
Based on these design dimensions, the four specific visual encoding combinations evaluated in this study are defined as follows: Encoding 1 utilizes the grid radar-range with the color-background target-threat; Encoding 2 utilizes the color-overlay radar-range with the color-background target-threat; Encoding 3 utilizes the grid radar-range with the border-thickness target-threat; and Encoding 4 utilizes the color-overlay radar-range with the border-thickness target-threat.
To represent the energy and frequency dimensions, a synthetic EM situational indicator was developed. This indicator was derived through a weighted fusion of parameters, including frequency band occupancy, key frequency band overlap, interference signal power density, and the degree of impact on the ego-aircraft’s operational frequency band. Visually, this indicator utilized color changes to convey the state of the EM environment: green indicated a normal state, while yellow and red indicated varying degrees of anomalies (
Figure 3a). Specifically, this indicator was computed as a normalized linear combination of four variables: frequency band occupancy (30%), key frequency band overlap (30%), interference signal power density (20%), and operational impact (20%). The resulting value, ranging from 0 to 1, dictated the indicator’s visual state: values below 0.35 maintained a normal green state, values between 0.35 and 0.75 triggered a yellow warning state, and values exceeding 0.75 triggered a red danger state. Upon detecting an environmental anomaly, an interactive decision window popped up at a fixed location near the indicator (
Figure 3b). This window contained three action buttons and a “Send” button. The “Automatic Frequency Hopping” button corresponded to a red anomaly, advising the system or operator to actively evade the interfering frequency bands due to a high-risk EM environment. The “Switch Relay” button corresponded to a yellow anomaly, indicating a moderate interference risk and suggesting a switch to the relay early-warning unit’s perspective for more stable scanning. The “Maintain Frequency Band” button corresponded to the normal green state, indicating a controllable EM environment; this button was primarily included as a baseline option and was rarely required unless the window was triggered unexpectedly. Because this synthetic indicator conveyed high-priority information reflecting the overall EM environment, it was overlaid directly onto the situational map. Two spatial layouts were compared for this indicator: positioned in the bottom-left corner of the interface, or positioned centrally beneath the ego-aircraft (
Figure 3c).
In addition to the aforementioned EM elements, the interface incorporated standard aviation components to ensure ecological validity. These included an aviation situational map, a primary flight display, engine indicating and crew alerting system components, and an electronic centralized aircraft monitor. A complete, representative interface is depicted in
Figure 4.
3.2.2. Experimental Task Design
To investigate how visualization strategies support information recognition and monitoring in a dynamic aviation context, the study simulated an aviation detection scenario requiring participants to attend to target information and changing EM conditions concurrently. The experimental task comprised two concurrent sub-tasks.
Sub-task 1 was a target recognition task. During the simulated flight, targets appeared at random locations on the situational map. Once a target entered the ego-aircraft’s radar detection range, its attribute information, including its flight trajectory, firepower envelope, and threat-level visual encodings, became visible. Simultaneously, a task prompt window appeared, requiring participants to identify the target’s threat level from the displayed visual encodings, select the appropriate option, and submit their response.
Sub-task 2 was an EM situation monitoring task. The synthetic indicator, located in a fixed area of the interface (either the bottom-left corner or center), continuously updated to reflect the overall state of the EM environment. Under normal conditions, the indicator remained green, signaling a stable and reliable system state. If the EM environment degraded, the indicator turned yellow or red, signaling a system anomaly. When this occurred, an interactive selection window appeared, prompting the participant to execute and transmit the appropriate command. Crucially, this indicator monitoring task was assigned a higher priority than the target assessment task. Because a compromised spectrum (indicated by a yellow or red state) degraded the system’s detection capabilities and rendered target information potentially unreliable, participants were explicitly instructed to prioritize resolving EM situational anomalies to restore system stability before proceeding with target interpretations.
Each 1 min trial contained exactly 10 predefined airborne targets, with threat levels uniformly distributed. Similarly, EM situational anomalies were introduced at controlled frequencies. Each trial contained 3 yellow warning anomalies and 2 red danger anomalies, with the indicator remaining in a normal green state during the rest periods. To prevent severe learning and anticipation effects, the exact appearance timing of both the targets and the EM anomalies was completely randomized within each trial. Prior to the formal experiment, a pilot test was conducted to verify that all UI elements, visual angles, and color encodings were easily discriminable under these timing constraints.
3.3. Experimental Setup
The experimental environment was developed using the Unity 3D engine (version 2022.3.53f1c1) and hosted on a desktop PC equipped with a 4K resolution monitor (U28R550UQC; Tianjin Samsung Electronics Co., Ltd., Tianjin, China). Raw experimental data, including task performance metrics, were encapsulated in real time by the backend code. The system automatically mapped the behavioral data to the corresponding encoding scheme identifiers and exported the results in a structured format (.csv). Eye-tracking data were collected at a sampling rate of 250 Hz using a Tobii Pro Fusion eye tracker (Tobii AB, Stockholm, Sweden) mounted directly beneath the bottom bezel of the monitor. Subjective workload measures (e.g., NASA-TLX) were recorded on a separate laptop. Participants were seated at a standard viewing distance of approximately 60 cm from the monitor. Eye-tracking data were captured using the default Tobii I-VT (Identification by Velocity Threshold) fixation filter parameters provided by the manufacturer. Trials with gaze data loss exceeding 15% due to blinks or track loss were excluded from the analysis to maintain data quality. The overall experimental setup is illustrated in
Figure 5.
3.4. Experimental Procedure
Before the formal experiment commenced, the experimenter briefed the participants on the research objectives and standard operating procedures. Once participants understood the task requirements, they completed a series of practice trials to familiarize themselves with the interface layouts and interactive controls. The formal experiment began only after the experimenter confirmed that a participant had fully mastered the procedures.
The formal experiment consisted of 8 conditional blocks. To mitigate potential learning and order effects, the presentation sequence of these blocks was counterbalanced across participants using a Latin square design. Each block comprised 3 trials, with each trial lasting exactly 1 min, simulating a continuous and dynamic flight scenario. Between consecutive trials, participants were required to double-click the spacebar to proceed. This self-paced transition was designed to prevent fatigue accumulation and afford participants a brief period to adjust their cognitive state.
Prior to initiating each experimental block, a standardized calibration was performed using the Tobii Pro Fusion eye tracker (Tobii AB, Stockholm, Sweden) to ensure the accuracy and reliability of the gaze data. During the trials, the system continuously logged participants’ operational behaviors and eye-movement data. After completing each block, participants filled out the NASA-TLX questionnaire on the laptop to evaluate their subjective cognitive workload under that specific condition. Following the standard NASA-TLX procedure, the overall workload score was calculated as a weighted average based on participants’ pairwise comparisons of the six subscales. Upon finishing all 8 blocks, the experimenter conducted a brief interview, asking participants about their most preferred interface and the rationale for their selection.
3.5. Data Analysis
Task performance was evaluated separately for the EM situation monitoring task and the target recognition task. Reaction time was defined as the interval between the onset of a task event and submission of the participant’s response. Prior to analysis, reaction time outliers falling more than three standard deviations from the participant’s mean were excluded. Response accuracy was coded as 1 for a correct response and 0 for an incorrect response. Responses not completed within 5 s were classified as misses and coded as incorrect. To form the dependent variables for both tasks, the trial-level accuracy and reaction time data were aggregated. Specifically, responses across the 30 targets (10 targets per trial × 3 trials) for the recognition task, and the 15 anomalies (5 anomalies per trial × 3 trials) for the monitoring task, were averaged to yield a single continuous accuracy proportion and mean reaction time per participant per experimental condition.
Eye-tracking analysis was conducted to evaluate the effects of interface layout on attentional guidance. The interface was divided into two distinct areas of interest (AOIs): the region displaying the synthetic EM situational indicator, and the remaining area for displaying EM information. The critical event, defined as an EM situation anomaly, served as the temporal reference point. The analysis window was established as the interval from the onset of the anomaly to the completion of the participant’s motor response. Fixation data were subsequently extracted at two critical time points: stimulus onset (the exact moment the anomaly occurred) and 0.25 s post-onset. This 0.25 s threshold was selected because it reflects the standard cognitive baseline for human visual processing time, representing the typical latency required to execute a reactive saccade and establish a fixation following an abrupt visual onset [
23,
24]. To visualize these patterns, spatial aggregation of the fixation data at these time points was performed to generate eye-tracking heatmaps, mapping the participants’ attention distribution across different interface layout conditions.
To supplement the qualitative heatmaps with quantitative evaluation, three metrics were extracted: time to first fixation (TTFF) on the synthetic indicator, total fixation count, and gaze entropy. Gaze entropy, reflecting the spatial dispersion of visual attention across the two AOIs, was calculated as , where represents the proportion of fixations falling within each AOI. Due to technical recording errors, eye-tracking data from 3 participants were excluded, resulting in a final sample of 29 participants for this analysis. Paired t-tests were conducted to compare these metrics between the bottom-left and center layouts.
Reaction time, accuracy, and NASA-TLX scores were each analyzed using a 2 × 2 × 2 repeated-measures analysis of variance (ANOVA), with radar-range encoding, target-threat encoding, and spatial layout, each with two levels, as within-participant factors. Because all factors had two levels, the assumption of sphericity was inherently met, and sphericity corrections were not required. Effect sizes were reported as partial eta squared (
ηp2). For descriptive interpretation, values of 0.01, 0.06, and 0.14 were considered indicative of small, medium, and large effects, respectively, following conventional benchmarks [
25]. Interview responses were summarized as selection frequencies. Statistical significance was set at
p < 0.05. Statistical analyses were performed using IBM SPSS Statistics for Windows, Version 27.0 (IBM Corp., Armonk, NY, USA).
5. Discussion
This study evaluated two dimensions of visual encoding (radar-range and target-threat) and two spatial layouts for presenting aviation electromagnetic (EM) information within a dynamic task simulation. The results showed that visual encoding and spatial layout influenced task performance, subjective workload, and gaze behavior. Although the experiment used a desktop display, the observed encoding and layout effects may serve as preliminary design implications for helmet-mounted and other wearable displays, where rapid information interpretation and efficient use of visual attention are important.
The visual encoding strategies demonstrated distinct effects across the experimental tasks. Target-threat encoding mainly affected the target recognition task; border-thickness coding yielded higher accuracy than color-background coding, although reaction time did not differ significantly. The reason might be color already encoded target identity, using a different visual channel for threat level likely reduced competition between meanings and improved discrimination [
26]. Lazaro et al. [
27] likewise found that greater visual complexity in cockpit displays increased search time and target-detection errors. The absence of a reaction-time effect in the present study suggests that the border-thickness encoding primarily reduced classification errors rather than accelerating the complete perception–decision–response sequence. Conversely, radar-range encoding significantly influenced the EM situation monitoring task, with the grid encoding yielding faster reaction times than the color-overlay. The workload and interview results provided complementary evidence: border-thickness coding and the center layout significantly reduced overall NASA-TLX scores, and their combination with the grid-based radar sector was most frequently preferred. The stronger preference for this combination may reflect the lower visual obstruction of its grid-based radar sector, although this interpretation requires direct testing. Furthermore, because subjective preference primarily reflects perceived visual comfort and does not guarantee objective performance advantages, these preference outcomes are interpreted mainly as supplementary indicators of user acceptance.
Spatial layout affected gaze behavior during EM anomaly responses and overall subjective workload across the concurrent tasks. The center layout reduced NASA-TLX scores, but spatial layout did not significantly affect target-recognition accuracy or reaction time in either task. A significant radar-range encoding and spatial layout interaction further showed that the decrease in workload from the bottom-left to the center layout was greater with color-overlay encoding than with grid encoding. Zhou et al. [
28] examined recognition responses to HUD information presented at 77 positions within a 50° × 50° field of view. They found that recognition performance and subjective ratings declined as the information moved farther from the visual center, supporting the placement of time-critical information near the operator’s focal region while avoiding obstruction of primary task content. Consistent with this finding, the present quantitative eye-tracking metrics and heatmaps demonstrated that gaze converged more rapidly on the indicator when it was positioned beneath the ego-aircraft. Specifically, the center layout resulted in a significantly shorter time to first fixation (TTFF), fewer total fixations, and lower gaze entropy. This indicates a more efficient, direct focal capture, whereas the bottom-left indicator required a longer, more dispersed peripheral attentional shift. Interview responses similarly indicated that the bottom-left indicator was easier to overlook and demanded deliberate searching. Yuan et al. [
29] found that an integrated cockpit AR-HUD reduced gaze shifts toward separate instruments while supporting lower workload and better situation awareness. The absence of a layout effect on target recognition is not contradictory because the manipulation changed only the indicator position; target symbols continued to appear at variable locations within the radar map. Thus, the benefit of central placement was task-specific and should not be interpreted as evidence that all display elements should be centralized.
A further contribution is the evaluation of EM visualization in a continuously changing, time-constrained environment rather than through isolated or static interface stimuli. The simulation required participants to divide attention between randomly appearing targets and changing EM states while following explicit task priorities. Bennett et al. [
30] showed that dynamic naturalistic scenes can reveal visual-search behavior with greater behavioral relevance than conventional static tests. Cheng et al. [
31] examined experienced pilots during simulated helicopter emergencies and found that changes in task demands substantially altered workload and the allocation of visual attention across critical cockpit areas. The synthetic indicator was introduced for the same task-oriented reason. Pilots are not dedicated EM-spectrum monitors. They should judge system state rapidly while simultaneously managing flight and target information [
32,
33]. Therefore, frequency occupancy, band overlap, interference power density, and effects on the aircraft operating band were fused into a readily interpretable state cue. Havlíková et al. [
34] presented task-relevant virtual information through an augmented-reality interface for industrial quality inspection and found that this approach improved task efficiency and reduced perceived workload compared with conventional paper-based support. Nylin et al. [
35] developed a compact glyph for initiating real-time human-automation communication in safety-critical traffic-management domains, emphasizing the need to communicate why attention is required and when action is needed. Similarly, the present indicator converted several technical parameters into an actionable overview. It should nevertheless supplement, rather than replace, access to detailed EM information for diagnosis and non-routine decisions.
The findings suggest several preliminary design implications for aviation EM interfaces. First, target identity and threat level could be represented using perceptually separable visual variables. When color already conveys identity, border thickness is a promising option for indicating threat, based on the observed accuracy advantage. Second, positioning high-priority EM states near the focal task region may support earlier visual orientation and lower overall subjective workload, while avoiding obstruction of targets, trajectories, or flight parameters. Third, a hierarchical organization of EM information could be explored in future studies, in which a compact synthetic overview is supplemented by detailed frequency- and energy-related data available on demand. Friedrich and Vollrath [
36] found that matching visual salience to the urgency of unmanned-aircraft system states improved safety-critical visual search, supporting consistent mappings among indicator state, urgency, and recommended action. King et al. [
37] applied ecological interface design to an aviation risk-management display, illustrating the value of making operationally meaningful system relationships directly visible. Finally, interface evaluation would benefit from dynamic multitask scenarios and converging measures of accuracy, response time, workload, preference, and visual attention.
Several limitations should be acknowledged. First, the participants were young adults without professional piloting or electronic-warfare experience, limiting generalization to trained pilots with different domain knowledge and scanning strategies [
38]. These differences may influence both the magnitude and relative advantages of the observed encoding and layout effects. The findings provide an empirical basis for aviation EM interface design, but further evaluation with professional pilots in representative flight scenarios is needed to establish their operational applicability. Second, the desktop-based experiment and one-minute trials did not reproduce prolonged vigilance, fatigue, aircraft motion, communications, or other operational demands of a full cockpit environment [
39]. Third, the tasks assessed recognition of displayed threat information and responses to EM state changes, capturing perceptual and attentional demands relevant to aviation information monitoring. Although these outcomes inform visualization design, they do not directly measure overall pilot situation awareness or the quality of independent tactical decisions. Further studies with trained pilots and operationally representative tasks are needed to establish whether the observed benefits extend to these broader outcomes. Finally, the weights and thresholds of the synthetic EM indicator were simplified for experimental purposes and were not operationally validated; their applicability requires evaluation using operational data and expert input.
6. Conclusions
This study evaluated multidimensional visual encoding conditions (radar-range and target-threat) and two spatial layouts for aviation electromagnetic (EM) information in a dynamic task simulation. A 2 × 2 × 2 repeated-measures experiment with 32 participants examined target recognition and EM situation monitoring using task performance, NASA-TLX scores, interface preferences, and eye-tracking metrics. The results showed that target-threat encoding mainly affected target-recognition accuracy, with border-thickness coding achieving better performance than color-background coding while maintaining comparable reaction times. Conversely, radar-range encoding significantly influenced EM monitoring, where the grid encoding produced faster responses than the color-overlay. In terms of subjective workload, both target-threat encoding and spatial layout demonstrated significant effects. Overall, the combination of a grid-based radar sector, border-thickness threat coding, and a center layout was preferred most frequently. Eye-tracking metrics and heatmaps further demonstrated that the center layout promoted faster visual attention to the EM indicator, requiring fewer total fixations and yielding lower gaze entropy after anomaly onset. Taken together, the findings identify promising encoding and layout choices for supporting information acquisition and reducing workload in the tested desktop simulation. These results provide preliminary design implications for aviation EM interfaces. Their applicability should be tested with professional pilots in cockpit simulators, and proposed extensions to helmet-mounted or other wearable displays require evaluation on the corresponding hardware.