Spatial Sound Modeling and Optimization of Flight Simulator with Multiple Off-Center Listening Positions

Abstract

Accurate spatial sound localization is critical in flight simulators for enhancing situational awareness and pilot training effectiveness, particularly for diagnosing severe faults like engine surge which emit directional sound cues. However, existing spatial audio systems are primarily optimized for a single central listening position, failing to provide consistent localization accuracy for pilots seated in naturally off-center positions within the cockpit. To bridge this gap, this paper proposes a novel compensation method incorporating near-field loudspeakers. A comprehensive mathematical model for multiple off-center listening points is established based on acoustic velocity and energy vector theory. We further formulate a dual-phase optimization framework: a multi-objective model employing the NSGA-II algorithm to Pareto-optimize the trade-off between minimizing localization error and maximizing spatial stability, followed by a maximin model that guarantees robustness during head movements. A formal listening experiment demonstrates that the proposed optimized design significantly improves both localization accuracy and stability over conventional uniform layouts, thereby enhancing the fidelity and safety of flight simulation training.

1. Introduction

In flight simulators, accurate sound source localization is crucial for pilot training, as it provides essential auditory cues that enhance situational awareness, spatial orientation, and immersion within the cockpit environment. Such cues, including azimuth and distance information, aid pilots in detecting potential threats and mitigating the risk of spatial disorientation under visually degraded conditions. Our analysis from a Boeing 737 Full Flight Simulator (FFS) indicates that, among 78 typical engine fault simulations, at least 20 faults produce sound signatures with discernible spatial information, many of which correspond to critical failures such as engine surge. Misidentification of the direction of such auditory alerts could lead to delayed responses and simulated accidents, with potentially serious operational consequences.

In recent years, research on the use of spatial audio technology to enhance pilots’ situational awareness has steadily increased. Veltman [1] demonstrates that spatial audio cues enable rapid threat and target localization, providing direct, intuitive indications of danger locations without diverting visual attention from primary flight tasks. Further studies by Oving [2], Niermann [3], and Jones [4] corroborate that spatially localized auditory warnings offer intuitive location information, significantly reducing response times compared to conventional monaural or stereo alerts. These findings underscore the importance of high-fidelity spatial audio reproduction in aviation environments to mitigate cognitive load, improve reaction times, and ultimately enhance flight safety.

Despite these advancements, conventional flight simulator sound systems, predominantly reliant on directional loudspeakers [5,6], are limited in their ability to provide the high-fidelity spatial auditory cues essential for effective pilot training. These systems may fail to accurately reproduce psychoacoustic cues such as interaural time differences (ITDs) and interaural level differences (ILDs), leading to significant sound localization errors. In multi-crew aircraft, this limitation is exacerbated by listener displacement from the optimal listening position (’sweet spot’), resulting in multi-speaker cross-talk, degraded spatial fusion, and an unstable auditory image [7].

To address these challenges, advanced spatial audio technologies offer promising solutions. Recent approaches, including Wave Field Synthesis [8], Ambisonics [9], and binaural reproduction base on Head-Related Transfer Function (HRTF) [10], have significantly advanced spatial audio reproduction. Concurrent developments in head-tracking technology techniques have enabled personalized spatial audio experiences [11]. However, these methods face limitations in cockpit-scale environments, such as restricted listening zones, sensitivity to seat displacement, restriction on the number of speakers, and incompatibility with multi-user training. Although directional loudspeakers like for constraining sound propagation have gained popularity [12], their application in flight simulators requires further validation due to unconfirmed physiological effects of ultrasonic carriers on pilots. Consequently, psychoacoustics-based approximations, which leverage perceptual models to optimize sound rendering, emerge as arguably the most viable current approach for multi-position setups.

This paper proposes a novel mathematical modeling of multiple off-center listening positions in a flight simulator loudspeaker array system, utilizing psychoacoustic acoustic velocity and energy vector synthesis methods. It addresses two key problems: optimizing loudspeaker gains for fixed positions and determining optimal loudspeaker layouts. A dual-metric optimization framework is introduced, focusing on localization accuracy and stability. For a representative configuration of 12 far-field and 2 near-field loudspeakers, a multi-objective optimization is performed using the NSGA-II method, resulting in an enhanced loudspeaker arrangement.NSGA-II [13] is a fast elitist multi-objective genetic algorithm that ranks solutions by non-dominated layers and maintains diversity via crowding distance; it iteratively evolves the population toward the Pareto front in $\mathcal{O}\left(M{N}^2\right)$ time. Finally, a listening experiment evaluates the localization performance and stability of the optimized layout compared to a uniform distribution across multiple off-center positions.

2. Materials and Methods

2.1. Problem Statement

2.1.1. Spatial Positioning Approximation Based on Psychoacoustics

The psychoacoustic approach to spatial positioning leverages the characteristics of human auditory perception by adjusting the delay and gain within a loudspeaker array to simplify the sound field, creating the illusion of sound spatial orientation. In 1957, Leakey [14] introduced a sine law that governs virtual source control using two speakers. The method employs gains $A_L$ and $A_R$ from two speakers, at an angular deviation ${\theta }_0$, to synthesize a virtual source direction, denoted as ${\theta }_I$, at low frequencies below 700 Hz.

$$sin{\theta }_I={\displaystyle \frac{A_L-A_R}{A_L+A_R}}sin{\theta }_0={\displaystyle \frac{A_L/A_R-1}{A_L/A_R+1}}sin{\theta }_0,$$

(1)

To improve the stability of the virtual source during head turning, Bernfeld [15] proposed the tangent theorem, which is applicable to low frequencies, specifically below 700 Hz.

$$tan{\theta }_I={\displaystyle \frac{A_L-A_R}{A_L+A_R}}tan{\theta }_0={\displaystyle \frac{A_L/A_R-1}{A_L/A_R+1}}tan{\theta }_0$$

(2)

Gerzon derived an acoustic velocity vector (frequencies below 700 Hz) and energy vector (700 Hz–5 kHz) sound direction synthesis method [16], from the perspective of the velocity and energy components of the superimposed sound field. He also introduced the velocity coefficient and energy coefficient as metrics for evaluating the stability of head movements.

The velocity vector method assumes that the pressure gradient is in the direction of the virtual sound source. The relationship between sound pressure p and particle velocity $\mathbf{v}$ in a free field can be described in the frequency domain as

$$\mathbf{v}={\displaystyle \frac{1}{j\omega {\rho }_0}}\nabla p$$

(3)

where $\omega$ is the angular frequency, and ${\rho }_0$ is the medium density.

The synthesized particle velocity ${\mathbf{v}}_m$ at observation point ${\mathbf{r}}_m$ is given by

$${\mathbf{v}}_m={\displaystyle \frac{1}{j\omega {\rho }_0}}\sum _{n=1}^N\nabla \left(p_n{\displaystyle \frac{e^{j(\omega t-k{\mathbf{r}}_{mn})}}{{\mathbf{r}}_{mn}}}\right)$$

(4)

After all, for loudspeaker i in the loudspeaker array,

$${\mathbf{r}}_v^i={\left[\begin{array}{c}{V}_{\mathrm{norm},x}\\ V_{\mathrm{norm},y}\end{array}\right]}^T={\left[\begin{array}{c}cos{\theta }_S\\ sin{\theta }_S\end{array}\right]}^T$$

(5)

For a loudspeaker i in a loudspeaker array, with position vector ${\mathbf{r}}_{\mathbf{i}}=(v_x^i,v_y^i)$ and gain $A_i$, the normalized velocity vector is defined as

$${\mathbf{r}}_{\mathbf{v}}^{\mathbf{i}}=A_i{\mathbf{r}}_{\mathbf{i}}$$

(6)

When employing an array of N speakers, the composite velocity vector for sound localization is given by

$${\mathbf{r}}_{\mathbf{v}}={\displaystyle \frac{{\sum }_{i=1}^N{A}_i{\mathbf{r}}_{\mathbf{i}}}{{\sum }_{j=1}^N{A}_i}}$$

(7)

$$r_v=\left|{\mathbf{r}}_{\mathbf{v}}\right|$$

(8)

The velocity coefficient $r_v=\left|{\mathbf{r}}_{\mathbf{v}}\right|$ can be used to assess the stability of the synthesized sound source from the loudspeaker array. A velocity coefficient closer to 1 indicates greater stability of the sound localization against head turning. Specifically, when $r_v=1$, the sound localization is stable against rotation, which typically corresponds to the case of a single loudspeaker.

For audio signals with a frequency range of 700 Hz to 5 kHz, Gerzon proposed using vector-based energy localization.

$$I={\displaystyle \frac{1}{T}}{\int }_0^T\mathrm{Re}\left(p\right)\mathrm{Re}\left(v\right)dt={\displaystyle \frac{1}{2}}{P}_A{\mathbf{V}}_{\mathbf{A}}=-{\displaystyle \frac{P_A^2}{2{\rho }_{0}c}}\mathbf{r}$$

(9)

where

• I is the acoustic intensity.
• $P_A$ is the peak acoustic pressure.
• ${\mathbf{V}}_{\mathbf{A}}$ is the particle velocity amplitude.
• ${\rho }_0$ is the ambient density of the medium.
• c is the speed of sound in the medium.
• $\mathbf{r}$ is the directional vector.

For the loudspeaker array

$$I^{\prime }=\sum _{i=0}^{M-1}{I}_i=-{\displaystyle \frac{1}{2{\rho }_{0}c}}\sum _{i=0}^{M-1}{A}_i^2{\mathbf{r}}_i$$

(10)

Similarly, for the synthesis direction of the loudspeaker array

$${\mathbf{r}}_{\mathbf{e}}={\displaystyle \frac{{\sum }_{i=0}^{M-1}{A}_j^2{\mathbf{r}}_j}{{\sum }_{j=0}^{M-1}{A}_j^2}}$$

(11)

Stability coefficient for the head turning of the energy vector

$$r_e=\left|{\mathbf{r}}_{\mathbf{e}}\right|$$

(12)

2.1.2. The Challenge of Sound Spatial Localization in Flight Simulation

Flight simulators commonly employ loudspeaker arrays to reproduce spatial sound localization. However, due to the limited number of speakers available for spatial audio reproduction, psychoacoustic methods are adopted to achieve this. Since both the pilot and co-pilot are off-center positioned, compensation is required to ensure accurate spatial sound reproduction. This section addresses the issue by incorporating near-field speakers to enhance localization accuracy.

The directional synthesis of sound in multi-loudspeaker arrays can be analyzed through velocity and energy vectors. Velocity vectors are effective for low frequencies below 700 Hz, while energy vectors are more suited for higher frequencies up to 5 kHz. We don’t consider localization for frequencies beyond this range, as psychophysical research suggests that low-frequency components play a primary role in influencing sound localization, even in signals containing both low and high frequencies [17].

To achieve optimal spatial localization at both listening positions, it is crucial to determine the ideal loudspeaker gains and layout. Two primary evaluation criteria for spatial positioning are defined:

• The accuracy of directional localization, quantified by the error between the virtual source direction and the target direction;
• Stability during head-turning, assessed by the stability coefficients—velocity and energy coefficients—where values closer to 1 indicate higher stability.

Here, it is assumed that several loudspeakers are distributed in the outer ring and inner ring for formula derivation. The loudspeaker layout used for modeling is as illustrated in Figure 1, m loudspeakers are distributed along a circle with a radius of 4 m, centered at the origin in a right-handed coordinate system. The positions of the two key pilots, denoted as a and b, are fixed at coordinates (0, −1) and (0, 1), respectively. Additionally, n near-field loudspeakers are positioned within a circle of radius 1 surrounding each pilot. For example, in configurations where each pilot employs three near-field loudspeakers, the loudspeakers are classified based on their positions: those located in front and to the sides. The coordinates for the left pilot are (−1, 1), (−2, 0), and (0, 0), while for the right pilot they are (1, 1), (0, 0), and (2, 0).

Figure 1. Loudspeaker distribution for optimized model, a and b denote two off-center listening positions; $s_1$–$s_m$ are the outer-ring loudspeakers, and $s_{11}$–$s_{1n}$, $s_{21}$–$s_{2n}$ are the near-field loudspeakers.

At point a, considering the quadratic decay of sound pressure with respect to the distance $d_{ai}$ from the reference position, the synthesized virtual source can be expressed as

$${\mathbf{r}}_{\mathbf{a}}={\left[\begin{array}{c}{\mathbf{r}}_{a1}\\ {\mathbf{r}}_{a2}\\ ⋮\\ {\mathbf{r}}_{am}\end{array}\right]}^T\left\{\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_m\end{array}\right]\odot \left[\begin{array}{c}\frac{1}{d_{a1}^2}\\ \frac{1}{d_{a2}^2}\\ ⋮\\ \frac{1}{d_{am}^2}\end{array}\right]\right\}+{\left[\begin{array}{c}{\mathbf{r}}_{a1}^{\prime }\\ {\mathbf{r}}_{a2}^{\prime }\\ ⋮& \\ {\mathbf{r}}_{an}^{\prime }\end{array}\right]}^T\left[\begin{array}{c}{x}_{a1}\\ x_{a2}\\ ⋮\\ x_{an}\end{array}\right]$$

(13)

where:

• ${\mathbf{r}}_{ai}$ represents the unit vector pointing from listening position a to the i-th loudspeaker.
• ${\mathbf{r}}_{ai}^{\prime }$ represents the unit vector pointing from listening position a to the i-th near-field loudspeaker.
• $x_{ai}$ denotes the weighting coefficient of the i-th near-field loudspeaker for listening position a.
• $x_i$ denotes the weighting coefficient of the i-th loudspeaker. ⊙ denotes element-wise multiplication.

According to Equations (7) and (11). if we define the loudspeaker gain as $A_i$, then

$${\mathbf{x}}_{\mathbf{i}}=\left\{\begin{array}{cc}\hfill A_i,& \mathrm{for}\mathrm{Low}\mathrm{frequency}\hfill \\ \hfill A_i^2,& \mathrm{for}\mathrm{High}\mathrm{frequency}\hfill \end{array}\right.$$

(14)

Based on the previous definitions, the relationship can be reformulated as

$${\mathbf{r}}_a={\mathbf{R}}_a^T(\mathbf{x}\odot {\mathbf{d}}_a)+{\mathbf{R}}_n^T{\mathbf{x}}_a$$

(15)

where ${\mathbf{R}}_a$ and ${\mathbf{R}}_n$ are matrices representing the orientation of the loudspeakers relative to listening position a, $\mathbf{x}$ and ${\mathbf{x}}_{\mathbf{a}}$ are the weighting coefficients vector for the loudspeakers.

Likewise, at listening position b, the direction vector of the virtual source, ${\mathbf{r}}_{\mathbf{b}}$, can be defined as

$${\mathbf{r}}_b={\mathbf{R}}_b^T(\mathbf{x}\odot {\mathbf{d}}_b)+{\mathbf{R}}_n^T{\mathbf{x}}_b$$

(16)

Thus, there are two optimization problems:

• Given a specified loudspeaker layout $\mathbf{u}({\varphi }_1,{\varphi }_2,\dots )$, and for a virtual target source position $s(x_t,y_t)$, determine the optimal loudspeaker weight vector ${\mathbf{x}}_{opt}(x_1,x_2,\dots )$ for two listening points. The objective is to minimize direction error and maximize head-turning stability coefficients for both positions.

  $${\mathbf{x}}_{opt}=f_1(\mathbf{u},\mathbf{s})$$

  (17)
• Given a constraint on the number of loudspeakers M, determine the optimal layout of the loudspeakers to maximize sound source localization accuracy for two listening points within the target zone $\mathcal{A}$. The objective is to position the loudspeakers such that localization performance is optimized across the entire zone.

$${\mathbf{u}}_{opt}=f_2(M,\mathcal{A})$$

(18)

2.2. Problem 1: Determine the Best Loudspeaker Gain for Two Off-Center Listening Positions

For a single listener, virtual sound sources originating from arbitrary directions can be projected onto the circumference defined by the loudspeaker array. In such cases, any virtual directional source can effectively be reduced to a two-loudspeaker configuration, where the localization performance depends primarily on the angular separation between the two loudspeakers. The accuracy of this virtual source reproduction is influenced by the relative positions of the loudspeakers.

When multiple listeners are present, virtual sources that deviate from the circular arrangement of loudspeakers introduce additional complexity and cannot be reduced to a two-loudspeaker problem. Unless the virtual source lies on the circumference defined by the loudspeakers, it is generally impossible to achieve simultaneous optimization of both the stability coefficient and directional angular error for all listeners. Therefore, optimization must consider the trade-off between minimizing angular error and maintaining stable reproduction, represented by the stability coefficient.

The summary angular error $\Delta \theta$ between the target angle and the synthetic sound source angle at positions a and b is

$$\Delta \theta =|\Delta {\theta }_a|+|\Delta {\theta }_b|$$

(19)

where

$$cos(\Delta {\theta }_a)={\displaystyle \frac{{\mathbf{r}}_a·{\mathbf{r}}_{at}}{|{\mathbf{r}}_a\left|\right|{\mathbf{r}}_{at}|}}$$

$$cos(\Delta {\theta }_b)={\displaystyle \frac{{\mathbf{r}}_b·{\mathbf{r}}_{bt}}{|{\mathbf{r}}_b\left|\right|{\mathbf{r}}_{bt}|}}$$

${\mathbf{r}}_{at},{\mathbf{r}}_{bt}$ are vectors from target source to listen position a, b.

On the other hand, for the loudspeaker weight vector $\mathbf{x}$, the positioning stability coefficient $r_s$ can be expressed as

$$r_s=\left|r_{sa}\right|+\left|r_{sb}\right|={\displaystyle \frac{|{\mathbf{R}}_a^T(\mathbf{x}\odot {\mathbf{d}}_a)+{\mathbf{R}}_n^T{\mathbf{x}}_a|}{|\mathbf{x}\odot {\mathbf{d}}_a|}}+{\displaystyle \frac{|{\mathbf{R}}_b^T(\mathbf{x}\odot {\mathbf{d}}_b)+{\mathbf{R}}_n^T{\mathbf{x}}_b|}{|\mathbf{x}\odot {\mathbf{d}}_b|}}$$

(20)

This leads to the optimization problem 1:

$$\begin{array}{cc}\hfill & \underset{\mathbf{x}}{min}[\Delta \theta ,-r_s]\hfill \\ \hfill & \mathrm{Subject}\mathrm{to}\hfill \\ \hfill & 0\le x_i\le 1,i=1,2,\dots ,m\hfill \\ \hfill & 0\le x_{ai}\le 1,i=1,2,\dots ,n\hfill \\ \hfill & 0\le x_{bi}\le 1,i=1,2,\dots ,n\hfill \\ \hfill & \Delta \theta =|\Delta {\theta }_a|+|\Delta {\theta }_b|\hfill \\ \hfill & r_s=|r_{sa}|+|r_{sb}|\hfill \end{array}$$

This problem is evidently a multi-objective optimization challenge. By utilizing the NSGA-II algorithm, the Pareto frontier is effectively obtained. The key steps and mechanisms of the algorithm are detailed as in Algorithm 1:

Algorithm 1 Best solution of $\Delta \theta ,-r_s$ by NSGA-II Algorithm

Require: loudspeaker angles vector $\mathbf{u}$, listen position $a,b$

Ensure: Set of Pareto Front $\Delta \theta ,-r_s$,

1: Calculate ${\mathbf{r}}_{\mathbf{a}}$,${\mathbf{r}}_{\mathbf{b}}$ according to Equations (15) and (16)   2: Initialize population   3: while termination condition not met do   4:    for Each individual in the population do   5:      Evaluate objectives $\Delta \theta ,r_s$ according to Equations (19) and (20)   6:      Apply constraints $\mathbf{x}$   7:      Update individual based on dominance relationship   8:    end for   9:    Perform genetic operators (crossover and mutation) 10: end while 11: Return the set of non-dominated solutions

2.3. Problem 2: Determining the Best Loudspeaker Layout for Improved Listening Experience

In a uniform loudspeaker layout, angular errors $\Delta \theta$ and stability coefficients $r_s$ exhibit non-uniform distribution across different target angles $\theta$. This section focuses on optimizing head-turning stability across all directions by utilizing a fixed number of speakers, with an emphasis on enhancing stability in the worst-case scenarios. Additionally, Table 1 shows that the aircraft noise we aim to simulate originates from a limited range of angles, suggesting that the optimization can be concentrated within this specific scope.

Table 1. Aircraft Noise Sources in Different Degrees.

Degree Range (deg)	Noise Sources
−30 to 30	Equipment Noise, Rain
75 to 115	Airframe Noise
135 to 170	Fan Noise, Jet Noise, Main Gear Noise, Wing Noise

To minimize computational efforts and simplify the analysis, we will restrict our focus to the angular range from 0 to 180 degrees, and mirror the results symmetrically to the opposite side. For the loudspeaker layout $\mathbf{u}({\varphi }_1,\dots ,{\varphi }_m)$, the target virtual source angle $\theta$ is distributed on a circle with radius d. The optimization objective is to maximize the minimum stability coefficient $r_s$ across all angles $\theta \in \mathcal{A}$, where $\mathcal{A}$ represents the set of angles of interest. For loudspeaker i,

$${\mathbf{r}}_{ai}={\displaystyle \frac{\left[\begin{array}{c}{x}_i-cos{\varphi }_i\\ y_i-sin{\varphi }_i\end{array}\right]}{\left|\begin{array}{c}{x}_a-cos{\varphi }_i\\ y_a-sin{\varphi }_i\end{array}\right|}},{\mathbf{r}}_{bi}={\displaystyle \frac{\left[\begin{array}{c}{x}_b-cos{\varphi }_i\\ y_b-sin{\varphi }_i\end{array}\right]}{\left|\begin{array}{c}{x}_b-cos{\varphi }_i\\ y_b-sin{\varphi }_i\end{array}\right|}}$$

(21)

The optimization problem 2 can be described mathematically as

$$\begin{array}{cc}\hfill & \underset{\varphi }{min}\underset{d,\theta }{max}f(d,\theta )\hfill \\ \hfill & \mathrm{Subject}\mathrm{to}\hfill \\ \hfill & \theta \in \mathcal{A}\hfill \\ \hfill & 0\le {\varphi }_i\le \pi \hfill \\ \hfill & 0\le d\hfill \end{array}$$

The function $f(d,\theta )$ represents the optimal stability coefficient for a given target angle $\theta$ when the distribution of the loudspeakers is known. As described by optimization problem 1, two optimization objectives, the stability coefficient $r_s$ and the angular error $\Delta \theta$, are addressed. However, solving this iteratively using the Algorithm 1 is computationally inefficient due to the multiple iterations of NSGA-II algorithm.

To streamline the computation, we simplify the problem by focusing on scenarios where the target point lies on a circular path with a radius of 4 m, while ignoring near-field loudspeakers. Additionally, in this simplified model, any given angle can correspond to at most two loudspeakers. Achieving zero angular error ($\Delta \theta =0$) yields a unique solution, and the corresponding stability coefficient $r_s$ at this optimal configuration is then used in subsequent optimization tasks. Therefore, $f(d,\theta )$ can be simplified as sub-optimize problem 3:

$$\begin{array}{cc}\hfill & \underset{\mathbf{x}}{min}-r_s\hfill \\ \hfill & \mathrm{Subject}\mathrm{to}\hfill \\ \hfill & \theta \in \mathcal{A}\hfill \\ \hfill & {\mathbf{r}}_a={\mathbf{r}}_{at},{\mathbf{r}}_b={\mathbf{r}}_{bt}\hfill \\ \hfill & r_s={\displaystyle \frac{1}{2}}\left({\displaystyle \frac{|{\mathbf{r}}_{a}x|}{\left|\mathbf{x}\right|}}+{\displaystyle \frac{|{\mathbf{r}}_{b}x|}{\left|\mathbf{x}\right|}}\right)\hfill \\ \hfill & 0\le x_i\le 1,i=1,2,\dots ,m\hfill \\ \hfill \end{array}$$

The optimization problem 2 can be addressed using the fmincon function in Matlab, as detailed shown in Algorithm 2:

Algorithm 2 Best loudspeaker layout

Require: loudspeaker number M, radius d, target area set $\mathcal{A}$

Ensure: loudspeaker best distribution vector $\mathbf{u}({\varphi }_1,\dots ,{\varphi }_m)$

1: Initial $\mathbf{u}$ with Speakers are evenly distributed from 0–180   2: Initial Target angle set $\mathcal{A}$   3: while fmincon termination condition not met do   4:    for Each individual $\theta$ in the $\mathcal{A}$ do   5:      Solution sub-optimize problem 3   6:      max $f(d,\theta )$ is the maximize value of $-r_s$ for all $\theta \in \mathcal{A}$.   7:    end for   8:    Update $\mathbf{u}$ Use fmincon method   9: end while 10: Return $\mathbf{u}$

2.4. Experimental Design

The experimental setup for the sound array was performed in a realistic operational environment, with the equipment positioned in a spacious room measuring approximately 6 × 6 × 3 m, reverberation time is approximately 0.54 s. The loudspeaker array was meticulously arranged in the right-hand sector (co-pilot side) of the Flight Training Device, as shown in Figure 2.

Figure 2. Experimental Setup (a) The schematic diagram of the experimental design. (b) Schematic diagram of the experimental setup.

2.4.1. Experimental Setup

The experimental system comprised a computer, 16-channel sound card, digital audio processor, power amplifier, and loudspeakers. A Simulink model implemented the synthesis algorithm, while the digital audio processor converted digital signals from the audio interface into analog signals. These signals were amplified and routed to corresponding loudspeaker channels. Frequency response calibration was achieved through channel-specific filter adjustments in the digital audio processor. Table 2 provides the specific hardware and software details.

Table 2. Hardware and Software Setup.

Name	Value/Description
Loudspeakers	Yamaha NS-B20
Amplifiers	Yamaha XMV 8140 D
Computer	Dell Precision R7610
Software Platform	Python 3.11 + Simulink 2016

Loudspeakers were deployed on the right side of the simulated cabin along a 4-m radius circular arc. Three distinct configurations were tested across experimental iterations:

• Uniform angular distribution (base line), using the velocity vector model;
• Omnidirectional optimization, using Algorithm 1;
• Aircraft sound position-optimized configuration, using Algorithm 1.

As illustrated in Figure 2. Digital markers were placed at 5-degree intervals to facilitate subjects’ spatial orientation of virtual sound sources.

2.4.2. Participant Protocol

8 flight simulation practitioners (2 females, aged 25–35 years) with normal hearing were recruited. Each participant contributed two datasets per configuration, yielding 16 valid datasets per configuration (48 total). The experimental procedure comprised

• A male voice counting from 1 to 7 served as the auditory stimulus, delivered through 5-s interval loop playback.
• Participants were instructed to maintain forward-facing head orientation while permitted minor positional adjustments to optimize localization accuracy.
• Prior to testing, individual loudspeakers were sequentially activated without participant awareness of marker-speaker correspondence. Data were discarded if participants failed to correctly identify all baseline speaker positions.
• Synthesized auditory stimuli were presented at 5-degree target angles. Participants recorded perceived source locations using the marker system. Each participant completed testing at both near position (b) and far position (a) listening positions, with two datasets collected per position.

3. Results

3.1. Optimized Result for the Problem 1: The Best Loudspeaker Gain for Two Off-Center Listening Positions

This model employs an analysis based on uniformly distributed speakers from 0 to 180 degrees. To minimize computational effort and simplify the analysis, only the results on the right side of the target are considered. Near-field loudspeakers are placed at the front and on the inner side relative to the listening position, as illustrated in Figure 1.

Solutions were obtained for a target source positioned at 0 degrees with respect to the x-axis, at distances of 3.5 m, 4 m, and 4.5 m from the origin. The results presented is shown in Figure 3, the minimum values for $\Delta \theta$ and the $r_v$ occur at 4.5 m, followed by those at 4 m. The parameter settings for Algorithm 1 are as shown in Table 3. Figure 4 presents the algorithm iteration details when the distance is 4.5. To verify the robustness of the algorithm, comparing of the multi-objective optimization results of NSGA-II with those of other algorithms using the statistical values from 10 calculations, as shown in Figure 5. The NSGA-II algorithm demonstrates consistent and superior performance on both the HV and Spacing metrics compared to the SPEA2 and IBEA. In contrast, the MOEA/D algorithm yields notably inferior results for these two indicators.

Figure 3. Pareto front with different distances, the left side (a) shows comparisons at 3.5 m, 4 m, and 4.5 m; the right side (b) is the enlarged detail view.

Table 3. The parameter settings for Algorithm 1.

Hyper-Parameter	Value/Description
Crossover operator	Simulated Binary Crossover (SBX, ${\eta }_c=20$)
Population size	200
Generations	2000
Pareto fraction	0.3
Stall generation limit	2500
Function tolerance	$1\times {10}^{-10}$

Figure 4. Pareto front iterate, on the left (a) is the evolution of the Pareto front across generations; on the right (b) are the GD and IGD metrics vs. generations.

Figure 5. Comparison with other multi-objective models, (a) shows the comparison of the Pareto fronts; (b) is the enlarged detail. (c,d) present the HV and spacing comparisons after 10 runs.

Figure 6 presents the median values of $r_s$ and $\Delta \theta$ for the target source, calculated at 5-degree intervals at a distance of 4 m from the origin. The results indicate that accuracy is relatively high at specific loudspeaker positions (0° 180°), while significantly lower at intermediate angles. Additionally, the result shows that positioning accuracy fluctuates depending on the target direction $\theta$, $\Delta \theta$ decreased as the value approaches 90°.

Figure 6. $\Delta \theta$, $r_s$ vs. target angle $\theta$ under the specific object, (a) shows the Pareto front at $\theta$ = 15, with the dashed line indicating the object. (b) displays $r_s$ vs. $\theta$, and (c) shows $\Delta \theta$ vs $\theta$.

3.2. Optimized Result for the Problem 2: The Best Loudspeaker Layout for Improved Listening Experience

In a horizontal plane, we obtain a set of solutions for the distribution of 12 outer ring speakers and 4 near-field speakers.

The results show a significant improvement in the minimum stability coefficient $r_s$, as shown in Figure 7a, which increased from 0.9144 to 0.9528 after optimizing for omnidirectional performance. This improvement was further enhanced by incorporating the directional requirements of aircraft noise distribution, resulting in a final stability coefficient of 0.9627. Figure 7b illustrates the change in the stability coefficient $r_s$ after optimization.

Figure 7. Comparison of the optimized (a) Comparison of the optimized average $r_s$ with the circumference of the target Angle (* denotes the speaker position), the dashed circles denote the minimum $r_s$ values under different configurations. (b) $\Delta \theta =0$, $r_s$ vs. $\theta$ after optimization of the loudspeaker layout.

3.3. Experiment Result

A total of 16 sets of experimental data were collected, and average localization angles and their corresponding standard deviations were calculated. Figure 8 demonstrates the comparative listening test results between optimized and baseline configurations. The localization accuracy $\Delta \theta$ is tested by calculating the angular deviation between the target sound source’s direction and the perceived direction. The stability is assessed by the standard deviation across multiple measurements. The results are shown as

Figure 8. Speaker Distribution Optimized, (a) Listening test results at three positions for the uniform layout, used as the baseline; (b) mean angular error comparison at the far position (a); (c) mean angular error comparison at the near position (b).

• Far-position (Point a): Maximum $\Delta \theta$ increased from ${6.3}^{\circ }$ to ${9.3}^{\circ }$, indicating reduced positional accuracy.
• Near-position (Point b): Maximum $\Delta \theta$ decreased from ${14.3}^{\circ }$ to ${10.5}^{\circ }$, showing enhanced reproduction fidelity.
• Peak error variation at point b decreased significantly:

  $$\mathrm{Optimized}:{10.5}^{\circ }-{7.7}^{\circ }={2.8}^{\circ }\mathrm{vs}.\mathrm{Baseline}:{14.3}^{\circ }-{7.1}^{\circ }={7.2}^{\circ }$$

The results of implementing directional restrictions on aircraft noise are presented in Figure 9. The average values of the $\Delta \theta$ and standard deviation are summarized in Table 4, indicating a slight improvement in localization stability (standard deviation), consistent with the simulation results.

Figure 9. Optimization of directional restrictions, (a) shows the mean angular at the far position a, (b) shows the mean angular at the near position b; (c,d) compare the standard deviations of the angular errors at positions a and b, respectively.

Table 4. Performance Comparison in Optimized Regions.

Metric	Far Position (b)	Near Position (a)
Average $\Delta \theta$	${4.37}^{\circ }\to {3.43}^{\circ }$	${4.66}^{\circ }\to {3.77}^{\circ }$
Average STD	$4.45\to 3.06$	$5.52\to 3.22$

4. Discussion

The results demonstrate that optimizing both directional accuracy ($\Delta \theta$) and head-turning stability ($-r_s$) simultaneously is impossible for off-center listening positions in aircraft, as shown in the Pareto frontier (Figure 3). When the virtual sound source is positioned outside the loudspeaker, the $\Delta \theta$ and $-r_s$ are lower, indicating superior localization performance for distant sound sources. The NSGA-II algorithm effectively identified optimal loudspeaker configurations, achieving high convergence and reliability compared to other methods (Figure 5). Optimization quality metrics, such as hypervolume and spacing, confirm the robustness of the NSGA-II approach, making it a reliable tool for multi-objective optimization in sound localization.

After optimization, the loudspeaker layout demonstrated a significant enhancement in localization performance. As illustrated in Table 4, both the average values and the standard deviation observed of the $\Delta \theta$ in the experiment showed a marked reduction. Furthermore, the error distribution became more uniform across all listening positions, as depicted in Figure 7. Notably, when directional constraints were applied to aircraft noise, localization accuracy was further refined, showcasing the effectiveness of the implemented restrictions.

It should be noted that the current model is analyzed under the ideal condition of evenly distributed loudspeakers. In practical applications, since the model incorporates the coordinates of each loudspeaker, it can theoretically accommodate variations in loudspeaker arrangement. However, if the pilot’s seat is too close to a loudspeaker, the masking effect may lead to failure in synthesizing the intended sound direction. In such cases, the loudspeaker layout would evidently deviate significantly from the optimal configuration. Theoretically, this risk can be foreseen through analysis of the stability index $r_s$.

5. Conclusions

5.1. Summary of Contributions

This study addressed the fundamental challenge of sound source localization for multiple off-center listening positions in flight simulators. By integrating near-field loudspeakers with a psychoacoustics-inspired methodology based on velocity and energy vectors, we developed a comprehensive framework for optimizing both loudspeaker gains and physical layout. The main contributions and findings are summarized as follows.

First, we formulated a multi-objective optimization problem to simultaneously minimize localization angular error ($\Delta \theta$) and maximize head-turning stability ($r_s$) for two off-center listening positions. The NSGA-II algorithm successfully generated Pareto-optimal solutions, revealing an inherent trade-off between these competing objectives as listeners deviate from the central sweet spot. This finding highlights a fundamental design constraint in multi-listener spatial audio systems.

Second, we implemented a loudspeaker layout optimization strategy focused on maximizing the minimum stability coefficient within the operational area. By constraining the optimization to the circular array region, we transformed the problem into a tractable single-objective formulation. The resulting layout demonstrated significantly improved stability in worst-case directions, with additional refinements achievable by targeting specific acoustic zones such as aircraft noise sources.

The proposed methodology offers a practical compromise between computationally intensive wavefield synthesis and Higher-Order Ambisonics approaches, proving particularly valuable for cockpit environments where multi-user compatibility is essential.

5.2. Limitations and Future Work

This study has several limitations that warrant attention. First, the current model operates in two-dimensional space; extension to full three-dimensional analysis would enhance practical applicability. Second, the constraint of optimizing only within the loudspeaker circumference may limit performance; future algorithmic improvements could enable more comprehensive search spaces. Last, there is a lack of analysis regarding disturbances such as background noise and variations in loudspeaker placement.

Future research will focus on experimental validation using realistic scenarios with azimuth-dependent acoustic signatures, such as aircraft collisions and engine faults, to quantify the impact of improved localization on pilot performance. Additionally, we will explore the simulation of dynamic sound fields from moving objects. The proposed loudspeaker-based approach also shows significant promise for virtual reality (VR) spatial audio rendering, especially in multi-user training environments where individualized Head-Related Transfer Function (HRTF) calibration presents practical challenges. Furthermore, future research should consider optimizing parameter sensitivity analysis to enhance the algorithm’s universality and effectiveness. The use of a weighted cost function that balances localization accuracy and stability depending on task importance is also a highly promising approach.