Immersive and Multimodal Interfaces for Radar and Spatial Data Visualization in Critical Operational Environments: A Scoping Review

Abstract

In safety-critical domains such as aviation, autonomous driving, and defense, operators must process complex spatial and radar data under severe time pressure. Traditional two-dimensional interfaces often force a “head-down” posture, increasing cognitive workload and impairing situational awareness. Extended reality and multimodal interfaces—incorporating gesture, voice, and haptic feedback—offer a promising paradigm to mitigate these limitations by enabling natural, egocentric data visualization. This scoping review systematically maps the empirical evidence on immersive and multimodal interfaces designed for radar and spatial data visualization in critical operational environments. Following the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews guidelines, a systematic search was conducted across five major databases for articles published between 2015 and 2025. Out of 538 unique records screened, 54 studies met the eligibility criteria and underwent structured data charting. The findings reveal a technological ecosystem heavily dominated by augmented reality and virtual reality, supplemented by non-extended reality multimodal baselines (n = 8) to evaluate sensory load distribution. While subjective metrics such as the NASA Task Load Index (n = 17, 31.4%) dominate current evaluation practices, there is a notable scarcity of objective real-time physiological biosensors (n = 7, 13%). Crucially, the synthesized data challenges uncritical technological optimism: while multimodal extended reality effectively mitigates visual bottlenecks, certain modalities like mid-air gestures frequently induce physical fatigue and a documented speed–accuracy trade-off. To fully realize the potential of immersive decision support systems, future research must prioritize standardized, ecologically valid evaluation frameworks and explore artificial intelligence-driven adaptive interfaces capable of dynamically modulating information density based on operator workload.

1. Introduction

1.1. Context and Importance of the Field

In safety-critical domains such as aviation, air traffic control (ATC), and security operations, professionals are required to perceive, analyze, and filter large amounts of real-time information under severe time pressure to ensure safety and make effective decisions [1,2]. For example, pilots and air traffic controllers must integrate data from multiple auxiliary tools, such as ground and air surveillance radars, to build a coherent mental model of the operational environment.

While traditional radar surveillance relies on active radio-frequency detection primarily used in aerospace and defense, the contemporary operational landscape has expanded significantly. Today, critical decision-making increasingly depends on homologous spatial sensor arrays across diverse fields. This includes LiDAR and environmental mapping in autonomous driving [3,4], 3D volumetric tracking in advanced medical simulation [5], and real-time proximity sensors in human–robot collaboration [6]. Therefore, in this review, the concept of “spatial operational data” adopts a holistic approach, encompassing both traditional radar systems and advanced real-time sensor mapping across multiple safety-critical domains.

1.2. Limitations of Traditional 2D Interfaces

Despite the increasing volume and complexity of available sensor data, traditional 2D graphical interfaces frequently force operators into “head-down” monitoring postures, dividing their visual attention between the instrument panels and the external physical environment [7]. Consequently, this division of attention imposes substantial cognitive demands, elevating mental workload and severely limiting the operator’s ability to maintain high situational awareness (SA) in dynamic, fast-paced scenarios [1]. As recently highlighted in comprehensive aerospace reviews, the continuous mental effort required to synthesize disparate 2D variables limits the cognitive resources available for anticipating critical future states [8].

Furthermore, traditional 2D viewing lacks the binocular depth discrimination provided by stereoscopic 3D cues, which limits the operator’s accuracy and scenery comprehension when assessing complex spatial relationships [9]. The continuous mental effort required to perform cognitive mapping between 2D screen representations and the 3D real-world operational space increases the cognitive distance, demanding an extraneous workload that jeopardizes task performance and overall system efficiency [10,11].

1.3. Emergence of Immersive and Multimodal Technologies

To overcome the inherent constraints of traditional 2D displays, there has been a rapid emergence of Extended Reality (XR) technologies, encompassing Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR). XR represents a paradigm shift toward egocentric, “head-up” visualization, allowing operators to perceive spatial data as conformal, three-dimensional elements registered directly within their physical or simulated environment [12].

Simultaneously, the scientific community has recognized that relying solely on the visual channel inevitably leads to sensory overload [13]. As a result, immersive technologies are increasingly integrated with multimodal interaction mechanisms—such as natural language voice commands, mid-air gesture recognition, and vibrotactile haptic feedback. This convergence aims to distribute the information processing across independent sensory channels, creating highly intuitive, human-centric systems that alleviate visual bottlenecks and improve reaction times [14,15].

1.4. Operational Contexts

The application of immersive and multimodal interfaces has expanded across a multitude of safety-critical domains. In traditional radar systems and aerial surveillance, MR has been deployed to project shared holographic tactical tabletops, enabling collaborative monitoring [16,17]. Furthermore, AR head-up displays are utilized to reduce the “head-down” time of pilots and air traffic controllers, superimposing spatial flight paths directly into their field of view [7,12].

Similarly, in the automotive and autonomous driving domain, drivers face analogous spatial challenges. Immersive interfaces and multimodal alerts (e.g., vibrotactile seats) are employed to deliver critical take-over requests (TORs) based on spatial hazards [18,19], as well as to mitigate motion sickness when transitioning from passive passengers to active controllers [3,20].

The teleoperation of robotics and unmanned systems also relies on spatial data visualization. Head-mounted displays (HMDs) equipped with in-device haptic feedback restore essential depth cues lost in standard 2D video feeds [21,22], while XR projects machine intent into the shared physical space [6,23]. Finally, in security and healthcare, AR is utilized for virtual co-location to project spatial hazard symbols [2,24], and VR transforms 2D physiological data into immersive 3D environments for high-fidelity critical care training [5,25].

1.5. Research Gap and Contribution of This Study

While the individual benefits of immersive displays and multimodal interactions have been explored, the current literature remains highly fragmented across isolated academic disciplines. Existing reviews often focus on specific niches, such as exclusively automotive contexts [3] or solely aerospace applications [8]. There is a critical lack of cross-domain synthesis investigating how multimodal interactions (voice, gestures, haptics) specifically interact with spatial data visualization to mitigate cognitive load. Furthermore, while XR technologies are at the forefront, the integration of multimodal feedback in non-immersive spatial displays remains a critical comparative baseline that is often overlooked in overarching reviews.

To address this notable research gap, this article presents a scoping review analyzing the literature published between 2015 and 2025. Following the rigorous PRISMA-ScR methodological framework, this review synthesizes the current research landscape of immersive and multimodal interfaces designed explicitly for radar and spatial operational data. By systematically charting the empirical evidence, this study contributes a structured overview of technological ecosystems, interaction modalities, comparators, and human factors evaluation practices, thereby identifying precise knowledge gaps for future human-centered decision support systems.

1.6. Research Questions

Guided by the Population, Intervention, Comparator, Outcomes, and Context (PICOC) framework, this scoping review addresses the following research questions:

• RQ1 (Population): What professional populations and user groups have been targeted in studies involving interfaces for spatial operational data?
• RQ2 (Intervention): What types of immersive technologies (AR, VR, MR) and multimodal interaction mechanisms (voice, gesture, haptics) are predominantly utilized?
• RQ3 (Comparator): How are these systems evaluated in comparison with traditional visualization approaches or non-immersive spatial displays?
• RQ4 (Outcomes): What evaluation metrics and physiological assessment methods are used to quantify cognitive workload, SA, usability, and task performance?
• RQ5 (Context): In which specific operational domains and critical environments have these interfaces been successfully applied?

2. Materials and Methods

2.1. Protocol and Registration

The methodology for this scoping review was designed and executed in strict accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines [26]. A scoping review approach was selected as the most appropriate methodological strategy to systematically map the interdisciplinary and rapidly evolving research landscape of immersive and multimodal interfaces, where conceptual boundaries are still developing and evidence is highly heterogeneous. Figure 1 provides a schematic overview of the methodological workflow, illustrating how the PICOC framework guided the screening, charting, and synthesis phases.

Figure 1. Schema of the Methodological Approach.

While prospective registration is standard for systematic reviews, scoping reviews in such rapidly evolving human–computer interaction domains often require iterative refinement of search parameters during initial screening. Therefore, this protocol was registered retrospectively in the Open Science Framework (OSF) after the completion of the review process to ensure full transparency, provide a verifiable, time-stamped record of the exact methodology executed, and prevent protocol drift. All eligibility criteria, search strings, and exclusion rationales were finalized prior to data synthesis. The complete audit trail is publicly accessible on OSF to ensure reproducibility and mitigate selective reporting bias. Registration URL: https://osf.io/yacfu/ (accessed on 21 May 2026).

2.2. Eligibility Criteria

To systematically identify relevant literature, eligibility criteria were established prior to the literature search following the PICOC [27]. Only peer-reviewed journal articles and conference proceedings written in English and published between 2015 and 2025 were included. This timeframe reflects the rapid technological maturation and commercial availability of modern XR hardware, such as optical see-through head-mounted display (OST-HMDs) and advanced VR headsets, which have fundamentally transformed human–computer interaction in recent years [8,28].

• Population (P): Professionals, trained operators, or human-in-the-loop participants (including university students serving as proxies) operating in complex or safety-critical environments.
• Intervention (I): Studies deploying immersive technologies (AR, VR, or MR). Crucially, the systems had to implement multimodal or advanced interaction mechanisms, such as mid-air gesture recognition, voice commands, spatial audio, or haptic feedback.
• Comparator (C): Studies comparing the XR intervention against traditional 2D graphical interfaces, legacy control systems, unimodal interfaces, or an absence of intervention. A small subset of non-XR multimodal spatial systems was also eligible if utilized as a direct comparative baseline to assess sensory load distribution.
• Outcomes (O): Empirical user-centered evaluations reporting measurements of cognitive workload (e.g., NASA-TLX), SA (e.g., SART, SAGAT), usability scales, or objective task performance. Purely technical or algorithmic system architectures lacking human evaluation were excluded.
• Context (C): Safety-critical or complex operational environments demanding the visualization of spatial, radar-like, or sensor-based data. Eligible domains included aviation, ATC, autonomous driving, robotics teleoperation, healthcare, and security operations.

2.3. Information Sources and Search Strategy

A comprehensive and systematic literature search was conducted across five major scientific databases selected for their broad coverage of engineering, computer science, and human factors research: Scopus, Web of Science Core Collection, IEEE Xplore, ACM Digital Library, and SpringerLink. The final execution of the search string across all databases was performed in September 2025.

The search strategy was structured around the conceptual components of the study. The following Boolean search string was adapted and applied to the title, abstract, and keyword fields of the respective databases:

$$\begin{array}{c}\mathrm{TITLE}\text{-}\mathrm{ABS}\text{-}\mathrm{KEY}(radar \mathrm{OR} “air traffic control” \mathrm{OR} ATC \mathrm{OR} “3D radar”\hfill \\ \mathrm{OR} “situational awareness” \mathrm{OR} “surveillance radar”)\hfill \\ \mathrm{AND}\hfill \\ \mathrm{TITLE}\text{-}\mathrm{ABS}\text{-}\mathrm{KEY}(immersive \mathrm{OR} “augmented reality” \mathrm{OR} “virtual reality”\hfill \\ \mathrm{OR} “mixed reality” \mathrm{OR} XR)\hfill \\ \mathrm{AND}\hfill \\ \mathrm{TITLE}\text{-}\mathrm{ABS}\text{-}\mathrm{KEY}(multimodal \mathrm{OR} gesture \mathrm{OR} voice \mathrm{OR} haptic)\hfill \\ \mathrm{AND}\hfill \\ \mathrm{TITLE}\text{-}\mathrm{ABS}\text{-}\mathrm{KEY}(“cognitive load” \mathrm{OR} “NASA-TLX” \mathrm{OR} usability\hfill \\ \mathrm{OR} SUS \mathrm{OR} CSUQ \mathrm{OR} accuracy \mathrm{OR} “response time”)\hfill \\ \mathrm{AND}\hfill \\ \mathrm{PUBYEAR}>2015\hfill \end{array}$$

During the query validation phase, the isolated acronym “MR” was intentionally excluded from the final search strings. Pilot searches revealed that including “MR” introduced an unmanageable volume of irrelevant database noise, predominantly retrieving “Magnetic Resonance” imaging studies from the medical field. Relying on the full term “mixed reality” allowed the search to maintain optimal sensitivity without sacrificing precision. Additionally, grey literature (such as academic theses, dissertations, and unreviewed technical reports) was explicitly excluded to preserve empirical rigor. Because the primary objective of this review—guided by the PICOC framework—was to map the rigorous empirical assessment of human factors in safety-critical domains, limiting the inclusion criteria to peer-reviewed journal articles and conference proceedings was strictly necessary. This criteria ensures a high baseline of methodological validity, particularly regarding the use of standardized cognitive workload and SA metrics (e.g., NASA-TLX, SART, SAGAT) requiring peer-reviewed experimental designs.

Minor adjustments to the search syntax were made when necessary to adapt the query to the specific search interfaces of each database.The overall strategy was designed to balance sensitivity and specificity, ensuring broad coverage of relevant studies while minimizing the inclusion of unrelated publications.

2.4. Selection of Sources of Evidence

The study selection process followed a structured screening procedure. The initial database search yielded a total of 684 records (SpringerLink: 482, ACM Digital Library: 107, Scopus: 44, IEEE Xplore: 31, and Web of Science: 20). All identified records were exported to a reference management system, where 146 duplicates were removed, leaving 538 unique records.

During the primary screening phase, the titles and abstracts of these 538 records were independently reviewed against the predefined eligibility criteria by two primary reviewers. Any discrepancies regarding study eligibility were resolved through discussion and consensus, or by consulting a third senior reviewer when necessary. While formal inter-rater reliability metrics, such as Cohen’s Kappa index, were not calculated, the iterative consensus process ensured high agreement and a rigorous application of the inclusion criteria. This primary screening resulted in the exclusion of 453 irrelevant records.

In the secondary phase, 85 reports were retrieved and assessed for full-text eligibility by the same two independent reviewers. During this rigorous full-text review, 31 studies were excluded for the following reasons: purely technical studies without human evaluation (n = 10), non-compliance with PICOC intervention/outcome criteria (n = 8), non-critical operational contexts (n = 6), lack of empirical user evaluation (n = 5), and review articles without primary data (n = 2). Disagreements during this phase were also resolved through the aforementioned consensus mechanism involving the third reviewer. Consequently, 54 studies met all eligibility criteria and were included in the final qualitative synthesis.

2.5. Data Charting Process

A structured data charting process was executed to systematically extract and organize relevant information from the 54 included studies. Following practices established in similar human–computer interaction systematic reviews [3], a standardized extraction form was developed in a spreadsheet environment. Each study was assigned a unique identifier to ensure traceability.

The extracted data items were categorized into four main dimensions to directly address the research questions and the PICOC framework:

• Bibliographic and Population Data: Publication year, venue, participant profile (e.g., experts vs. novices), and operational application domain.
• Technological Ecosystem: Type of immersive technology (e.g., AR, VR, MR) and specific hardware devices deployed.
• Interaction Modalities and Comparators: Modalities utilized by the operators to interact with the system (e.g., gesture, voice, haptics) and the type of baseline comparator used in the evaluation (e.g., traditional 2D interfaces, absence of cues).
• Evaluation Metrics and Outcomes: Subjective and objective instruments used to assess human factors, such as cognitive workload (e.g., NASA-TLX), SA (e.g., SART, SAGAT), physiological bio-sensing devices, and objective task performance.

To ensure the reliability of the charted data while maintaining methodological feasibility, a partial validation approach was employed. The primary data extraction for all 54 studies was conducted comprehensively by the lead reviewer. Subsequently, a second independent reviewer verified a randomly selected sample of the extracted data (approximately 20% of the included studies). Any minor inconsistencies identified during this cross-checking phase were resolved through discussion and consensus among the research team, ensuring that the data extraction framework was applied uniformly and accurately across the entire dataset.

2.6. Synthesis of Results

The synthesized data were analyzed using a descriptive and thematic approach. Descriptive statistics were utilized to characterize the distribution of studies across publication years, target populations, and operational domains. Furthermore, a classification framework was applied to map the technological ecosystem and summarize the relationships between the deployed multimodal interactions and the resulting human factors outcomes (cognitive workload and SA). The findings are presented using structured tables and descriptive summaries to highlight current trends, methodological patterns, and critical research gaps within the domain of immersive spatial data visualization.

3. Results

3.1. Study Selection

The systematic literature search and study selection process is summarized in the PRISMA flow diagram as illustrated in Figure 2. The initial search across the five selected databases yielded a total of 684 records. After removing duplicates, 538 unique records were screened based on their titles and abstracts. Following the full-text assessment of 85 retrieved reports, 31 studies were excluded due to strict non-compliance with the predefined inclusion criteria. Ultimately, 54 studies met all eligibility criteria and were included in the final qualitative synthesis.

Figure 2. PRISMA flow diagram illustrating the study selection process.

3.2. Overview of Included Studies

A comprehensive summary of representative included studies—detailing their respective application domains, target populations, implemented immersive technologies, interaction modalities, and primary evaluation metrics—is presented in Table 1. The full data extraction matrix containing the complete charting of all 54 studies is provided in the Supplementary Material.

Table 1. Summary of Representative Included Studies (2015–2025).

Study ID	Application Domain	Participants Type	Immersive Tech	Interaction Modalities	Evaluation Metrics
S01 [29]	Aviation & ATC	Aviation professionals	AR—Augmented Reality	Gesture, Voice	NASA-TLX, Performance
S02 [1]	Aviation & ATC	Aviation professionals	AR—Augmented Reality	Gesture, Voice, Audio	NASA-TLX, SART-10D
S03 [6]	Human–Robot Collab.	Students	MR—Mixed Reality	Spatial Display	NASA-TLX, SART, TRUST
S04 [7]	Aviation & ATC	Aviation professionals	XR—Extended Reality	Gesture, Visual Cues	NASA-TLX, Performance
S11 [5]	Healthcare Simulation	Students	VR—Virtual Reality	Audio Systems	NASA-TLX, SART, SUS
S12 [20]	Automotive Driving	Mixed	VR—Virtual Reality	Visual, Auditory Cues	NASA-TLX, SART, SSQ
S14 [2]	Military & Security	Security experts	AR—Augmented Reality	Spatial Annotations	Usability, Questionnaires
S47 [30]	Industrial Teleoperation	Students	VR—Virtual Reality	Gesture, Haptic	NASA-TLX, Performance

The included literature predominantly consists of empirical, human-in-the-loop experimental studies (n = 46, 85.1%), which evaluated functional prototypes or simulated environments. A smaller subset of the selected literature encompasses comprehensive and systematic reviews (n = 2, 3.7%) [3,8], as well as conceptual frameworks and design requirement analyses (n = 6, 11.1%) [11,14].

3.3. Distribution by Year and Publication Trends

An analysis of the publication years (2015–2025) reveals a clear upward trend in research interest regarding immersive and multimodal interfaces for critical operations (Figure 3). Between 2015 and 2019, 15 studies (28%) were published, primarily exploring foundational concepts of VR flight simulators, early Head-Up Displays (HUDs), and preliminary teleoperation interfaces. However, a significant acceleration occurred from 2020 onwards, with 39 studies (72%) published in the last five years of the analyzed timeframe. The years 2022 to 2024 were particularly prolific, accounting for 22 studies (41%). This inflection point strongly correlates with the commercial availability and technological maturation of advanced OST-HMD and high-resolution VR headsets equipped with native tracking capabilities.

Figure 3. Publication Trend Analysis.

3.4. Application Domains

The selected studies applied immersive and multimodal interfaces across a variety of safety-critical and complex operational contexts, categorized into the following domains:

• Aviation and ATC (n = 16, 29.6%): The largest application domain. Studies focused on Remote and Virtual Towers (RVT), flight deck interface design, and augmented visual aids for pilots [1,7,8,11,12,16,17,29,31,32,33,34,35,36,37,38].
• Automotive and Autonomous Driving (n = 10, 18.5%): Research primarily addressed semi-autonomous driving, employing multimodal Take-Over Requests (TOR) and visual-auditory cues to mitigate motion sickness in intelligent vehicles [3,4,10,18,19,20,21,39,40,41].
• General Immersive Analytics and Industrial Systems (n = 10, 18.5%): These studies focused on the fundamental human factors of interacting with abstract spatial data (e.g., 3D radar charts) and evaluating advanced human–machine interfaces in manufacturing and general XR contexts [13,14,15,30,42,43,44,45,46,47].
• Human-Robot Collaboration and Teleoperation (n = 9, 16.6%): Studies explored the teleoperation of Unmanned Ground Vehicles (UGVs) and Unmanned Aerial Vehicles (UAVs) using XR to project machine intent and provide spatial navigational views [6,9,22,23,48,49,50,51,52].
• Military, Security, and First Responders (n = 5, 9.2%): Applications included virtual co-location for forensic investigation, surveillance radar monitoring, and tactical alignment in security-sensitive areas [2,24,53,54,55].
• Healthcare and Medical Simulation (n = 4, 7.4%): VR and XR environments were utilized for high-fidelity critical care learning and surgical training involving spatial coordination [5,25,56,57].

As shown in Figure 4, Aviation and ATC was the most represented application domain, followed by Automotive and Autonomous Driving and General Immersive Analytics and Industrial Systems.

Figure 4. Distribution of application domains across the 54 included studies.

3.5. Hardware and Technological Ecosystem

The hardware ecosystem reported in the extracted data is heavily dominated by immersive systems, categorized into AR (n = 15, 27.7%), studies leveraging AR primarily focused on superimposing computer-generated 2D and 3D digital objects onto the real-world operational environment, allowing operators to maintain their out-of-the-window (OTW) view while receiving critical flight, navigation, or tactical data [1,12]. VR (n = 15, 27.7%), these systems were predominantly used for high-fidelity training simulations, psychological examinations, and the evaluation of complex spatial data (e.g., 3D radar charts) where isolating the user from physical distractions was beneficial [38,40,42]. MR (n = 7, 13.0%), These were highly prevalent in collaborative decision-making scenarios, such as shared holographic tabletops for monitoring aerial missions [16,17], and generalized XR platforms (n = 9, 16.6%). OST-HMD (e.g., Microsoft HoloLens series) and fully immersive VR headsets (e.g., HTC Vive, Oculus/Meta Quest) were the primary devices utilized [8,50]. The hardware ecosystem reported in the extracted data is heavily dominated by commercial off-the-shelf HMDs.

• AR/MR Hardware: Microsoft HoloLens (versions 1 and 2) was the most prominent device. Its native spatial mapping, optical see-through waveguides, and integrated gesture/voice recognition made it the primary choice for industrial and ATC applications. Other lightweight smart glasses included the Epson Moverio BT-300 (Seiko Epson Corporation, Suwa, Japan) and META SpaceGlasses (Meta Company, San Mateo, CA, USA) [2,12].
• VR Hardware: Fully immersive setups predominantly utilized the HTC Vive ecosystem (Vive, Vive Pro, Vive Pro Eye), noted for its integrated eye-tracking capabilities, and the Oculus/Meta ecosystem (Rift, Quest 2) [20,21].
• Haptic Systems: Beyond standard handheld controllers and optical sensors (e.g., Leap Motion for hand tracking), specialized haptic hardware included vibrotactile car seats, massage cushions (e.g., Beurer MG 145), Thrustmaster steering wheels with force feedback, and chest-worn tactile devices [19,40,47].

It is noteworthy that a small subset of the included studies (n = 8, 14.8%) investigated non-XR multimodal interfaces (e.g., spatial audio arrays, haptic seats, and jaw-gesture radar sensors in conventional simulator setups) [32,33,52]. These were retained in the review due to their highly relevant empirical insights into sensory load distribution, serving as critical baselines for spatial data environments.

As shown in Figure 5, AR and VR were the most frequently reported hardware ecosystem categories, while non-XR multimodal interfaces also represented a relevant subset of the reviewed studies.

Figure 5. Distribution of hardware ecosystem categories across the included studies.

3.6. Interaction Modalities and Comparators

In accordance with the PICOC framework, this review analyzed the multimodal interaction strategies deployed to alleviate visual sensory overload. The extracted data reveals a strong integration of non-visual modalities across the operational domains:

• Gestural Interaction: Mid-air gesture tracking and spatial interactions were highly prevalent, allowing for the touchless manipulation of spatial data without breaking visual focus [6,42,51].
• Haptic Feedback: Force-feedback joysticks, vibrotactile vests, and augmented steering wheels were predominantly applied in the automotive and teleoperation domains to provide non-visual proximity alerts and depth perception [19,21,46].
• Voice and Auditory Systems: Voice commands and spatial auditory alerts were favored to maintain hands-free operations while guiding overt visual attention, particularly in ATC and aviation [29,33].

Regarding the study design (Comparators), empirical evaluations predominantly compared these novel multimodal XR interfaces against traditional 2D baseline displays (e.g., standard desktop simulators [39,49] or paper checklists [1,29]), legacy control systems, or unimodal sensory baselines (absence of intervention) [3,18] to quantitatively measure human performance shifts. As shown in Figure 6, gesture-based interaction dominates across most domains, particularly in aviation and general XR contexts, while multimodal interaction remains less frequently implemented.

Figure 6. Distribution of interaction modalities across application domains.

3.7. Evaluation Methods and Metrics

A structured analysis of the evaluation methodologies reveals that researchers relied heavily on established subjective questionnaires, supplemented increasingly by objective task performance and physiological metrics to evaluate the human-in-the-loop systems.

• Subjective Cognitive Workload: The NASA Task Load Index (NASA-TLX) was the undisputed standard, utilized in 17 studies (31.4%) to quantify mental demand, physical demand, and frustration [6,15,29].
• Objective Task Performance: Response times, task completion times, and error rates were formally logged in 25 studies (46.2%) to quantitatively assess the efficacy of the interaction modalities [21,29,48].
• Situational Awareness: SA was formally assessed using validated instruments such as the Situation Awareness Rating Technique (SART) and the Situation Awareness Global Assessment Technique (SAGAT) in 8 studies (14.8%) [1,5], while other studies evaluated SA observationally.
• Physiological and Bio-sensing Metrics: To overcome the limitations of post-trial subjective reporting, 7 studies (13.0%) incorporated real-time physiological sensors (e.g., ECG, EEG, HRV, and pupillometry) to objectively quantify stress and cognitive load during task execution [10,25,40].
• Simulator Sickness: Due to the immersive nature of the interventions, instruments like the Simulator Sickness Questionnaire (SSQ) were administered in 5 empirical studies to measure nausea and oculomotor disturbances [20,56].

4. Discussion

4.1. Multimodality and Sensory Load Distribution

The integration of multimodal interactions presents a highly nuanced relationship with cognitive load, reflecting a paradigm shift in how operators interact with spatial data. The foundational premise, grounded in the Multiple Resource Theory, suggests that distributing information processing across independent sensory channels—visual, auditory, and haptic—prevents the overtaxing of any single cognitive resource [13]. Our extracted data validates this theoretical framework across different operational contexts. For instance, haptic feedback (such as force-feedback joysticks and vibrotactile car seats) has proven highly effective in automotive and teleoperation domains because it provides crucial spatial proximity alerts without requiring the operator to divert their gaze from the dynamic environment [19,21].

Conversely, voice commands and spatial audio are heavily prioritized in aviation and ATC to maintain hands-free operations while guiding overt visual attention [1,33]. Importantly, the inclusion of non-XR multimodal interfaces in this review—such as conventional simulators utilizing spatial audio arrays and haptic seats—serves as a critical baseline [32,52]. These studies demonstrate that while immersive visual displays are advantageous, the strategic distribution of sensory alerts is the actual catalyst for mitigating visual bottlenecks in complex data environments.

4.2. Usability Barriers and Cognitive Trade-Offs

While this review corroborates the hypothesis that spatial interfaces enhance environmental comprehension, it also challenges the prevailing, often uncritical optimism surrounding immersive technologies. By bringing persistent usability barriers to the forefront, the synthesized data reveals significant cognitive trade-offs. While prior isolated experiments have championed the absolute superiority of 3D visualizations, our analysis indicates that simply porting legacy 2D symbology into an immersive space often exacerbates visual clutter and sensory overload [11,45].

Furthermore, earlier studies often presumed that immersive technologies would universally accelerate reaction times. Our synthesis reveals contradictions to this assumption, illustrating a clear speed–accuracy trade-off. For example, Hayden et al. (2020) demonstrated that while novice operators using an AR procedure assistance system achieved significantly higher decision accuracy, their overall task completion times increased (M = 310.52 s) compared to traditional baselines (M = 239.56 s) [36]. This trade-off is most pronounced during static procedural tasks and initial learning phases (e.g., preflight checklists), where AR-assisted users were on average 32.3 s slower due to the complex visual search required in visually dense environments. However, in highly dynamic scenarios (e.g., landing procedures), this trade-off is minimized to a 1.7-s difference as users rapidly adapt to the spatial cues [36]. Furthermore, specific alert modalities can also impact this trade-off; Grandi et al. (2023) found that auditory alerts yielded the slowest reaction times compared to visual or haptic feedback, indicating higher cognitive processing demands [15]. Similarly, relying extensively on mid-air gestural commands to manipulate spatial data can induce physical fatigue (the “gorilla arm” effect) and increase temporal demands when compared to traditional baselines or voice commands [29,45].

4.3. Barriers to the Adoption of Objective Real-Time Metrics

Despite the technological advancements in physiological bio-sensing and continuous objective measurements, their widespread adoption in industrial and critical operational environments remains limited compared to subjective self-reporting tools. A rigorous analysis of the included literature reveals several methodological and interpretational barriers that hinder the implementation of real-time objective metrics.

Firstly, objective evaluation methods—including the analysis of completion times, error rates, and continuous physiological data—frequently require a large pool of test participants to generate statistically viable results [38]. In critical domains such as aviation or robotics, securing a large sample of highly trained Subject Matter Experts (SMEs) is practically unfeasible, making these objective metrics difficult to implement pragmatically compared to the valuable and immediate feedback obtained through subjective appraisals [38].

Secondly, relying on visual process indices like eye-tracking is complicated by severe perceptual phenomena induced by augmented displays, such as “attentional tunneling” [1] and “inattentional blindness” [11]. Studies have shown that users can become visually fixated on augmented spatial data, meaning that objective gaze data alone might indicate visual focus but completely fails to guarantee cognitive processing or SA of the broader operational environment [1,11].

Besides, interpretational challenges heavily complicate the viability of real-time bio-sensors. The literature underscores the inherent difficulty in disentangling emotional stress from actual cognitive workload [10]. Differentiating between these two distinct psychological states often requires complex, multi-modal physiological sensing approaches (e.g., fusing EEG, ECG, and EDA) to accurately distinguish high-workload events from stress-induced arousal, adding immense analytical burden to the evaluation process [10,25].

Consequently, standardized subjective instruments like the NASA-TLX remain the pragmatic industry standard. They offer a validated, multidimensional assessment of mental, physical, and temporal demands [8] without the intrusive hardware setup, complex data synchronization, and massive sample size requirements of real-time objective metrics.

4.4. Theoretical and Operational Implications

The implications of this scoping review resonate deeply across theoretical and practical dimensions. Theoretically, this research contributes to cognitive load and SA models by demonstrating how immersive computing fundamentally reshapes the perception-action loop. It suggests that traditional human factors models must be updated to account for the phenomenon of embodied cognition and spatial presence, recognizing that cognitive load in virtual environments is inextricably linked to physical interaction fidelity [43,57]. The evidence demonstrates that conformal AR graphics successfully bridge the cognitive distance between the operator and the physical environment [10].

From an operational standpoint, the deployment of these interfaces holds the potential to profoundly alter decision-making processes. The findings align with the historical success of early “highway-in-the-sky” concepts, proving that three-dimensional trajectory guidance outperforms discrete 2D instruments [8,11]. By providing intuitive, spatially registered alerts, operators in defense, aviation, and emergency response can maintain higher levels of sustained vigilance and respond more effectively to dynamic threats, particularly in degraded visual environments (DVE) [8,55].

Furthermore, the design of immersive interfaces for safety-critical domains can greatly benefit from universal design and accessibility frameworks established in broader XR applications [13]. As seen in domains such as cultural heritage, where VR and AR are increasingly used to create interactive and personalized exhibitions, inclusive design principles ensure that systems remain accessible to a diverse public with varying physical, sensory, or cognitive capabilities. For instance, multisensory approaches incorporating haptic feedback and spatial audio have been successfully applied to make spatial data and cultural artifacts accessible to visually impaired individuals [13]. Adopting these accessibility frameworks in critical operational environments could facilitate the creation of adaptive interfaces that dynamically adjust their sensory outputs (e.g., enhancing auditory warnings or scaling haptic cues) to match the specific ergonomic needs, age-related perceptual changes, or temporary cognitive impairments of the operator [4,18]. This cross-pollination of inclusive design practices ultimately fosters more resilient, accessible, and human-centric human–machine teaming.

4.5. Limitations of the Review

Despite the rigorous inclusion criteria employed, this scoping review is subject to several methodological limitations. The relatively small dataset size and the highly specific scope of the included literature may constrain the broad generalizability of the findings. Furthermore, the heterogeneity of the evaluation methods utilized across the primary studies poses a significant challenge. While subjective metrics like the NASA-TLX and System Usability Scale (SUS) are prevalent, their application varies widely, and studies rarely standardize their baseline comparators. Additionally, the frequent reliance on convenience samples, such as novice university students rather than highly trained subject matter experts, may misrepresent the true cognitive load experienced by seasoned professionals in high-stakes environments [6,31]. Finally, because our systematic database search concluded in September 2025, the extensive time required for manual, human-led screening and data synthesis means that very recent studies published in early 2026 are not captured in this review.

4.6. Research Gaps and Future Directions

Identifying these limitations illuminates several critical research gaps and establishes clear directions for future scientific inquiry. Foremost, there is an urgent need for the development of standardized evaluation frameworks specifically tailored for XR interfaces in operational settings. Such frameworks should mandate a rigorous combination of subjective questionnaires and objective, real-time physiological measures—such as Electrocardiography (ECG), Heart Rate Variability (HRV), and Electroencephalography (EEG)—to provide a holistic assessment of the user’s cognitive state [25,40].

Technologically, the integration of artificial intelligence with immersive spatial visualization presents a highly compelling frontier. Machine learning algorithms could enable real-time, adaptive multimodal interactions that dynamically modulate information density based on the operator’s immediate cognitive workload and fatigue levels [4]. Ultimately, future research must prioritize longitudinal studies conducted in ecologically valid environments to fully understand the long-term effects of immersive technologies on operational safety and skill retention in critical domains [50].

5. Conclusions

This scoping review provides a comprehensive and empirical synthesis of the rapidly evolving landscape of immersive and multimodal interfaces for spatial and radar data visualization in safety-critical environments. Based on the systematic charting of 54 studies, the findings reveal a definitive technological trajectory: a critical transition from visually demanding, “head-down” two-dimensional legacy displays toward conformal, “head-up” XR (AR, VR, MR) ecosystems. Crucially, this review underscores that simply transplanting 2D data into 3D space is insufficient to improve human performance. Instead, an industry-wide paradigm shift toward multimodality—integrating mid-air gestures, voice commands, and haptic feedback—is essential to effectively distribute sensory processing and mitigate severe visual bottlenecks.

Across diverse domains, from aviation and autonomous driving to robotics and healthcare, multimodal interventions demonstrated distinct operational benefits tailored to specific environmental constraints. Haptic alerts proved highly effective for proximity awareness in automotive and teleoperation contexts, while voice and spatial audio were heavily favored in aviation to maintain hands-free control. However, the empirical data also highlights critical usability trade-offs that challenge the uncritical optimism surrounding immersive tech. For instance, while mid-air gestures facilitate intuitive, touchless interaction with spatial data, they frequently induce physical fatigue and increase temporal demands, resulting in a documented speed–accuracy trade-off. Furthermore, the inclusion of non-XR multimodal baselines in this review proved that the strategic allocation of sensory alerts is often as critical as the visual immersion itself for preventing sensory overload.

Methodologically, this review identified a heavy reliance on subjective self-reporting instruments to assess these interventions. The NASA-TLX remains the undisputed standard (utilized in nearly a third of the included studies), whereas only a small minority of the research (13%) employed real-time physiological biosensors (e.g., EEG, ECG, HRV).

To advance the field beyond early-stage prototypes, future research must prioritize standardized, ecologically valid evaluation frameworks that mandate objective cognitive state monitoring. Ultimately, to fully realize the potential of immersive spatial data visualization, the next generation of interfaces must leverage artificial intelligence to create adaptive systems capable of dynamically modulating information density and interaction modalities based on the operator’s real-time cognitive workload and fatigue levels.