Validating Aircraft Noise Models

: Aircraft noise, especially at takeoffs and landings, became a major environmental nuisance and a health hazard for the population around metropolitan airports. In the battle for a better quality of life, wellbeing, and health, aircraft noise models are essential for noise abatement, control, enforcement, evaluation, policy-making, and shaping the entire aviation industry. Aircraft noise models calculate noise and exposure levels based on aircraft types, engines and airframes, aircraft flight paths, environment factors, and more. Validating the aircraft noise model is a mandatory step towards the model credibility, especially when these models play such a key role with a huge impact on society, economy, and public health. Yet, no validation procedure was offered, and it turns out to be a challenging task. The actual, measured, aircraft noise level is known to be subject to statistical variation, even for the same aircraft type at the same situation and flight phase, executing the same flight procedure, with similar environmental factors and at the same place. This study tries to validate the FAA’s AEDT aircraft noise model, by trying to correlate the specific flight path of an aircraft with its measured noise level. The results show that the AEDT noise model underestimates the actual noise level, and four validation steps should be performed to correct or tune aircraft noise databases and flight profiles.


Introduction
Noise pollution originating from aircraft landings and takeoffs creates an environmental nuisance and a health hazard. Various technologies, legislations, procedures, and methodologies were introduced to confront this issue, ranging from reducing the noise sources (aircraft engine and airframe) to airport usage policies, and to means for evaluation, control, and enforcement of aircraft noise. A major method for noise abatement is based on aircraft noise modelling, which is used for airport, traffic, and land-use planning.
Models that calculate aircraft noise levels consider engine and airframe types, aircraft path, flight profile, environment factors, and more. The most common aircraft noise model is the USA Federal Aviation Authority's (FAA) Aviation Environmental Design Tool (AEDT) [1]. It uses ICAO's/Eurocontrol's Aircraft Noise and Performance Database (ANP) and Base of Aircraft Data (BADA) for data (noise-power-distance (NPD) tables), and calculate the aircraft noise level according to the methodologies offered by ECAC doc 29 [2] or ICAO doc 9911 [3]. Like any other model, these noise models need to be verified (during their development) and validated (to be trusted and utilized). Although EDAC doc 29 Vol. 3 part 1 addresses verification procedures, validation is expected to be addressed in part 2 of Vol. 3, and it will be based on using noise measurements of actual aircraft events.
Validation of aircraft noise models, and the importance of validation, were described even decades ago in the research literature. These studies found in general that the model calculated noise levels correlate well with measurements under controlled conditions, when using the industry supplied NPD data, despite 5 dB(A) differences or more between the calculated and the measured aircraft noise levels [4][5][6][7][8]. Recent studies point to the implications of using the default NPD data and the procedure profiles and show the variance in aircraft flight paths, power settings, and measured noise levels [9,10]. A significant problem in the validation process is that measuring aircraft noise level is subject to statistical variation, even for the same type of aircraft, at a certain situation and flight phase, executing the same flight procedure or maneuver, with similar environmental factors and at the same place [10,11]. In other words, what should be then the correct actual noise level, to which the calculated one should be compared? The statistical variation in measured noise levels results from compound factors, due to differences in atmospheric conditions, fluctuations in the emitted noise, variations in operational settings, different configurations, diverse pilots' practices and procedures, or specific adjustments [12][13][14]. Noise measurements confirmed that such variations can be responsible for differences of as much as 12 dB [15,16].
This statistical variation of noise measurements explains the possible aggregation of different types of aircraft in noise models, of hundreds of airframes and engines, down to four to nine classes, with a ±6% difference in the calculated areas surrounded by a certain noise level contour [17,18].
One of the possible reasons for the statistical distribution in aircraft noise measurements is associated with variations in the exact path and altitude of the aircraft while its noise level is measured. AEDT model provides definition of "disperse tracks", allowing the creation of multiple sub-tracks from the original, "backbone" track, to capture deviations from the nominal flight path [1]. However, this feature is useful for calculating the aggregated noise of multiple aircraft using the backbone track, and it can be used properly only if the sub-track dispersion distribution is defined, and if we assume that each sub-track noise calculation is valid.
AEDT model enables also the definition of "point-type" tracks, created from track nodes [1], which can be defined by the realistic position of aircraft, using the OpenSky Network database for example. These point-type tracks can be used in conjunction with aircraft noise measurements to validate the noise calculation in each actual flight path. Furthermore, the OpenSky Network database can also be used to define the distribution of the sub-tracks around the backbone tracks, in a subsequent stage.
In this paper, we correlate measurements taken at a specific flight path and altitude of an aircraft, with AEDT model results, which uses the same flight path and altitude of the aircraft being sampled, in order to validate the accuracy of AEDT model. The impact of flight path and altitude variations on the measured aircraft noise level, and on the model calculation, is analyzed.

Methodology
It is widely accepted that validating aircraft noise models means a comparison of the noise models calculated results with the measured aircraft noise levels for the same monitor locations, the same aircraft types, and the same aircraft operations (e.g., landing, taking off, or traversing). This concept of validating aircraft noise models is explicitly expressed in ECAC doc 29 Vol. 3 part 1 [2], as mentioned above.
In this study, we used the FAA's recent aircraft noise model, the AEDT version 3c [1]. We used four sets of published noise measurements in different locations, representing landing and takeoff operations, and we focused on Airbus 320 aircraft. For the aircraft's flight paths we used either the "default" published flight paths in the departing or approaching charts used by the pilots as "average" flight path, or the precise, actual flight paths, taken from Automatic Dependent Surveillance-Broadcast (ADS-B) messages (received from OpenSky data or our ADS-B receivers). In one case, that of San Francisco International Airport, we used the published lateral and altitude distances from the noise monitor terminal (NMT) when the aircraft noise level measurement was the highest (LAmax).
In calculating aircraft noise levels by the AEDT, we used the default flight profiles offered by the AEDT. In one case, that of Tel Aviv Ben Gurion Airport, no AEDT takeoff profile enabled the AEDT to calculate the noise levels when aircraft traces were used for flight paths in the AEDT model. We wrote a takeoff profile that matches the published Noise Abatement Departure Procedure (NADP) of Tel Aviv Ben Gurion Airport.

Comparing Measured with Modeled Aircraft Noise Levels
We compared aircraft noise level measurements with AEDT noise calculations at three locations: London Heathrow Airport (LHR, ICAO identification EGLL), San Francisco International Airport (SFO, ICAO identification KSFO), and Tel Aviv Ben Gurion Airport (TLV, ICAO identification LLBG). In the following subsections, aircraft noise levels are presented for each of these locations, alongside various flight parameters, including the "default" or the actual flight paths, in an attempt to correlate between aircraft noise levels and aircraft location, to validate or improve aircraft noise models.

Hethrow (LHR)
We compared four Airbus 320 landings on runway 27 R, whose tracks were obtained from OpenSky. Three NMT stations' measurements were taken from WebTrak, whose locations are shown in Figure 1. NMT 130 is the farther, at about 8.5 km from runway 27 threshold. The maximum measured and calculated sound levels (LAmax) at these NMTs are shown in Table 1. The calculated LAmax at each NMT for the default ILS approach to runway 27 R is presented in the first row, and then the calculated and measured LAmax at each NMT for every flight path is presented. The average and standard deviation are those of the four flights calculated and measured LAmax (not the default flight path). The calculated maximum noise level (LAmax) based on the actual flight path is almost identical for all four aircraft, at all the NMTs. It is an expected result, due to the flight operation (landing using Instrument Landing System (ILS)), which guarantees a similar flight path for all of these flights. The calculated LAmax for the default flight path at the two NMTs that are closer to the runway threshold is also similar to the calculated LAmax for the actual flight paths. However, there is a significant difference between the noise level calculated for the default flight path and the noise level calculated for the actual flight paths. It implies a different flight path, used by these aircraft, or different landing parameters used during these landings (e.g., flaps, gear, or throttle configuration).
When it comes to actual noise measurements, there is a small difference between the measured noise level of each of the flights at each NMT, and the variance is meaningful only at the farther NMT. However, the measured noise level is quite similar to the calculated noise level for the default flight path, except for NMT 56, and quite different from the calculated noise level for the actual flight paths (except for NMT 11, the closest NMT to runway 27 R threshold). The differences between the measured and calculated LAmax at these NMTs are presented in Table 2, along with the slant distances between the aircraft and the NMTs at the moment of recording LAmax of the aircraft at these NMTs. As can be seen, the difference between the measured and the calculated maximum noise levels (LAmax) is significant in the two farther NMTs, and the variance in the difference increases with the slant distance between the aircraft and the NMT when LAmax of the aircraft was recorded.

San Francisco (SFO)
An SFO noise report published the measured aircraft noise levels at NMT 404 located in Palo Alto, along with the lateral distances of the aircraft from the NMT and its altitude at the moment of recording the maximum noise levels [19]. The data presented in this report were used to validate the AEDT noise model. Figure 2 presents the NMT location and the default flight path of an aircraft in the ILS approach to runway 28 R or 28 L, using Standard Arrival Route (STAR) SERFR direct.  Figure 3, and it is evident that there is no such correlation. Moreover, the AEDT calculated LAmax for Airbus 320 arriving in the "default" flight path (according to the STAR/ILS charts and procedures) is 56.7 dB(A), which is below the lower range of the measured LAmax and 7.5 dB(A) lower than the average measured LAmax. Similarly, the AEDT calculated SEL is 70.9 dB(A), 4.3 dB(A) lower than the average measured SEL.

Ben-Gurion Airport (TLV)
We compared several takeoffs and landings performed from runway 26 and on runway 21 respectively. Two sets of NMTs measurements were used for comparisons; these NMT locations are shown in Figure 4.

Landings
All 2019 landings on runway 21 were measures at two NMTs (13 and 14) by the Israeli Airport Authority (IAA). We used all 2019 noise level recordings of aircraft landings using the ILS 21 approach, which guarantees a precise flight path for all aircraft that should result in similar noise levels for each type of aircraft.
The differences between the average measured SEL and the calculated one, based on the ILS flight path, are shown in Table 3, for both NMTs and seven aircraft types. As can be seen, the differences are minor, below the standard deviation, except for Boeing 789 and Airbus 321 measured at NMT 13, the farther NMT from the runway threshold. When it comes to the maximum noise level (LAmax), the differences are bigger and meaningful, as shown in Table 4. Two comparisons were made. The first was based on 2017 measurements, published by the IAA, and AEDT version 2d noise calculations, for various aircraft types. The second was based on specific Airbus 320 aircraft takeoff noise measurements in 2019, their precise flight path, and AEDT version 3c calculations.

2017 comparison-aggregated aircraft calculations and measurements
Measured SEL was compared for 10 aircraft types, composing more than 75% of the traffic in the airport and representing some of the noisiest aircraft taking off from the airport, with the calculated AEDT SEL. The measurements are averaged over the entire 2017 year, and the calculations were performed according to the published departure procedure from runway 26 flight path, and the default takeoff profile. The differences, in SEL dB(A), are depicted in Table 5, and as can be seen, the measured noise levels are higher than the calculated noise levels for almost all aircraft, at all NMTs.
It should be noted that these differences represent an average of the calculations for each takeoff weight (represented by the stage length in the AEDT model) for each of the aircraft. The standard deviation of the measurements varies, on average, between 1.3 dB(A) for closer NMT to the aircraft flight path to 2.5 dB(A) on the farther NMT. It seems that the differences are substantial, at least for some of the aircraft (e.g., Boeing 777-200 and 747-400), beyond two standard deviations.

2019 comparison-specific A320 aircraft calculations and measurements
Measured SEL was compared for a dozen Airbus 320 aircraft taking off from runway 26, with their calculated AEDT SEL. The IAA published the measurements, and we used the specific flight paths of these aircraft for the AEDT input, using ADB-S data we collected. As mentioned above, an NADP profile had to be created and used, to enable the AEDT model to calculate the noise levels, as all the default profiles caused the model to fail. The differences, in SEL dB(A), are depicted in Table  6, and as can be seen, the measured noise levels are mostly greater than the calculated noise levels, especially in the farther NMTs. The average difference and standard deviation increase with the distance of the NMTs from the aircraft flight path. The SEL difference between the measured and the calculated SEL is presented in Figure 5 for several NMTs. There is no precision, nor accuracy between the measured and the calculated aircraft noise levels. In NMT3 the calculated noise levels are mostly above the measured noise levels, whereas in NMT9 and NMT5 it is the opposite situation, i.e., the measured noise levels are higher than the calculated ones, with a larger difference between the measured and the calculated noise levels than in NMT3.
Furthermore, there is also no correlation between measured and calculated LAmax of aircraft with similar takeoff weights, as can be seen in Figure 5d for NMT9. The takeoff weight in AEDT is represented by the stage length, which is calculated based on the flight distance, reflecting the fuel required for that distance, and affecting the aircraft weight.

Discussion
Heathrow's results show that on ILS landings the calculated aircraft noise levels based on the actual flight path are quite similar, with increasing variance as the slant distance between the aircraft flight path and the NMTs increases. This is possibly the result of the greater variance in the aircraft location, relative to the ILS flight path. Nevertheless, the actual, measured aircraft noise levels at the NMTs are 3.6 dB(A) higher than the calculated noise levels at two of the NMTs, where the slant distance between the aircraft flight path and the NMTs are greater than 1000 m. It seems that at NMT 11 and NMT 56, which are at about the same distance from threshold 27 R and differ mainly in the lateral distance from the ILS flight path, the measured aircraft noise level is about the same, whereas the calculations estimated a 3 dB(A) difference. This might suggest a wind or a miscalculation of the lateral attenuation by the AEDT model. It is also interesting to note, however, that the measured noise level in the farther NMT is similar to the calculated noise level of the Airbus 320 landing on the "default" flight path.
Contrary to Heathrow, in ILS landings at San Francisco, measured 26 km away from runways 28 thresholds, the dispersion of the aircraft "around" the ILS flight path creates more than 250 m difference in the slant distances (about 15% of the average slant distance). It also creates a 10 dB(A) interval of the recorded noise levels, and there is almost no correlation between these noise levels and the slant distances, which suggests that the recorded aircraft noise levels at the NMT might be affected by other factors. Similar to Heathrow, the measured aircraft noise levels were significantly higher than the AEDT calculated noise level for the "default" ILS flight path.
Similar results in terms of the differences between the measured aircraft noise levels and the AEDT calculated ones for landings are shown in Tel Aviv airport. Measured noise levels are higher than those calculated by the AEDT model. It should be noted that the slant distance in Tel Aviv airport in the farther NMT (NMT13) is about 500 m, much shorter than the slant distances of San Francisco. The difference is much smaller and close to none for the NMT that is near the runway 21 threshold, similar to Heathrow's results.
As for takeoffs, the correlation between aircraft noise level and its location is more complicated, at least due to the different takeoff weights, which are functions of flight distances (i.e., different fuel quantities), that require different power settings at takeoff and climb. Tel Aviv measurements of 2017 show again that the measured noise levels are considerably higher than the AEDT calculated ones for the "default" flight path, even when considering takeoff weights. The variance in the measured noise levels increases with the slant distance between the aircraft flight path and the NMTs. When the AEDT noise model calculates noise levels of specific flight paths, the measured noise levels (in 2019) are again greater than the calculated noise levels, especially in the farther NMTs, and no correlation exists between the two sets, even when we consider takeoff weight.

Conclusions and Summary
This study shows that the measured aircraft noise levels span over a range of about 10 dB(A), even if aircraft move in the same flight path and perform the same operation (takeoff or landing). The interval and the variance of the aircraft noise levels increase with the slant distance between the aircraft flight path and the measurement location (the MNT) and are usually greater for takeoffs than for landings. These findings are in line with other results in the research literature [20].
This study also shows that the AEDT model underestimates noise levels, sometimes considerably, by 4 to 7 dB(A), even when using an accurate flight path for its input. It further shows that there is no correlation between the calculated noise levels of specific aircraft to their actual, measured noise levels, even if the calculation is based on their precise flight path and speed.
The results of this study suggest that aircraft noise model validation should be separated into four cases; takeoffs and landings, and for each operation, a different approach should be used for close and far NMTs. Validation should involve correction of at least the NPD tables, as well as takeoff profiles, and a recursive process must continue until there is a match in terms of accuracy in its wider sense-trueness, and precision.