Evaluation of H-ARAIM Reference Algorithm Performance Using Flight Data ^†

Abstract

Currently, relevant efforts are being dedicated to the implementation of Advanced Receiver Autonomous Integrity Monitoring (ARAIM) in future aviation receiver standards. These contributions focus on the specific aspects of algorithm processing and performance using simulated or real static user grid data. However, significant differences in the quality of measurements made by ground receivers compared to an avionics receiver may arise due to operational constraints such as space weather (troposphere and/or ionosphere), multipath, signal outages, and cycle slips. The objective of our work is to evaluate the Horizontal-ARAIM (H-ARAIM) reference algorithm sensitivity in an operational scenario using GPS and GALILEO dual-frequency flight data. Navigation performances are analyzed for typical arrival and approach maneuvers with respect to positioning accuracy and integrity for Required Navigation Performance (RNP) specifications, along with the evaluation of algorithm computational load when subjected to the dynamics of the aircraft.

1. Introduction

ARAIM is the proposed evolution of Receiver Autonomous Integrity Monitoring (RAIM) to dual-frequency multi-constellation with diverse signals, and performance characteristics that will be communicated to user receivers in the Integrity Support Data (ISD) [1,2,3,4]. The ARAIM algorithm addresses more complex scenarios than RAIM at the cost of a higher computational load. As opposed to classical weighted least square RAIM [5], ARAIM complies with more stringent integrity requirements. The most remarkable improvement is its ability to boundposition errors under nominal, single and multiple fault conditions.

ARAIM ensures integrity by comparing the position solution estimate made with all satellites in view (all-in view) to estimates from subsets that have removed some of the satellites [1,6]. Subsets are formed by removing all sufficiently likely satellite combinations that could contain faults [1]. Integrity is ensured provided one of the evaluated subsets has no faulty satellites. ARAIM availability and continuity greatly improves by having more satellites in view and tracking more than two constellations [7]. However, many of the new core constellations do not yet have an established long-term track record of performance. Therefore, having many satellites with a relatively high likelihood of faults can lead to a very large number of satellites subsets to evaluate, increasing the algorithm computational load. To mitigate rare event faults including satellite failures, the Fault Detection (FD) algorithm can be implemented [1]. ARAIM FD exploits redundant ranging signals to achieve self-contained fault detection at the user receiver. However, while using redundant signals and multiple constellations improves integrity monitoring performance, the likelihood of satellite faults is also higher. The larger number of used satellites also causes more loss of continuity due to faults being detected, thereby increasing the continuity risk [7]. In response, the Fault Detection and Exclusion (FDE) algorithm can be implemented to reduce the continuity risk [1].

To account for the risks of loss of integrity and loss of continuity while improving availability, a reference ARAIM user algorithm has been developed [1] by the Technical Sub-Group (TSG) of EUROCAE WG-62 and RTCA SC-159 WG-2 (WG-C-ARAIM TSG). The main objectives of the group focus on detection and exclusion thresholds that limit the risk of false alerts and failed exclusions, as well as on prediction and analysis of ARAIM availability and continuity performance. The reference algorithm is a key part of the ARAIM standardization effort [2,3]. Although the algorithm is not mandatory for receiver manufacturers, it serves as a reference to assess ARAIM navigation performance.

Presently, the EUROCONTROL PEGASUS team actively assists the receiver standardization group to implement ARAIM in future aviation standards [2]. To this end, the ARAIM TSG established the tool verification sub-group to support the validation of the requirements, and to enhance the clarity and consistency of the standards with the objective to achieve a common understanding of ARAIM among aviation equipment manufacturers, regulators, and users. The contributions focus on the specific aspects of algorithm processing and performance using simulated or real static grid data. In validation and verification studies [2], baseline GALILEO and GPS constellations of 24 satellites are respectively considered [8,9]. To be able to represent different user receivers, common frequency bands are used, assuming available ranging signals on both L1/E1 and L5/E5a bands for all the satellites. Furthermore, minimum performance levels in terms of clock and ephemerides error are assumed, and a user segment fixed masking angle is used (5 degrees). Therefore, the use of real flight data offers the opportunity to analyze robustness to signal outages due to aircraft maneuvers and other operational constraints, as well as serving to demonstrate the state of readiness of the algorithm to meet its intended purpose, such as demonstrating high integrity navigation performance for aviation operations [10,11].

This work aims to provide an overall picture of the minimum operational performance capability of the ARAIM reference algorithm [1] using current GPS and GALILEO constellation performance in flight. Dual-frequency flight data are used to analyze the navigation performances for typical arrival and approach maneuvers with respect to positioning accuracy and integrity for RNP operations, examining the algorithm computational load and its sensitivity to aircraft maneuvers.

2. Flight Trials and Data Collection

In the framework of the single European sky ATM research program (SESAR) [12,13], the demonstration of runway-enhanced approaches made with satellite navigation (DREAMS) project focused on enhanced arrival procedures solutions supported by advanced GNSS navigation applications such as the Satellite-Based Augmentation System (SBAS), and Ground-Based Augmentation System (GBAS). The project aimed to demonstrate the feasibility of a single runway with dual-threshold operations or dual-glideslope (one being steeper) operations in an operational environment, while fostering the deployment of GNSS navigation applications in Europe. Demonstration flight trials took place in Twente airport, The Netherlands. The test aircraft (Cessna Citation II) was equipped with Netherlands Aerospace Centre’s flight inspection system, and the EUROCONTROL test bench (Figure 1) equipped with a latest generation aeronautical multimode receiver (MMR), along with another rover DFMC GNSS receiver.

The EUROCONTROL role in the DREAMS project was twofold, leading the work program on advanced operations demonstration, and contributing to GNSS navigation performance analysis. Several test flights were performed for two weeks, for a total of over 100 approaches and up to 18 h of flight data were collected. While the project had a predominantly operational objective using legacy SBAS and GBAS guidance, DFMC data (GPS+GALILEO L1/E1+L5/E5a) were collected at the same time. These data enable additional specific evaluations for future DFMC navigation applications such as Horizontal ARAIM (H-ARAIM or Service Type A ARAIM, for which ICAO Annex 10 standards are to be published officially soon).

3. H-ARAIM Operational Performance Analysis

The expected level of performance for future ARAIM users has been comprehensively discussed in various studies in the past [2,7,14]. These studies rely on a set of assumptions on the GNSS signals and the behavior of the airborne receiver, as follows:

• When a satellite is above an elevation angle of 5 degrees, it will be tracked. It is known, however, that aircraft banking can cause the loss of the lock of low elevation satellites;
• The multipath and ionospheric uncertainty are bounded by the elevation-dependent models specified in [15];
• The cycle slips are assumed rare enough, so that the error bound on the multipath is almost always the one provided by the bound in the carrier-smoothed code;
• Space weather conditions are considered quiet at any latitude of simulated grids excluding possible signal outages due to ionospheric disturbances;
• Baseline GPS and GALILEO constellations of 24 satellites are considered, assuming available ranging signals on both L1/E1 and L5/E5a bands for all the satellites [14].

In these cases, ARAIM performance is analyzed without consideration of the positioning performance, as well as without the limitations caused by operational constraints such as signal loss and cycle slips, due to multipath, ionosphere disturbances, and aircraft dynamics.

The objective of this work is to investigate H-ARAIM performance capability in an operational scenario using real flight data, and default ISD based on core satellite constellations (GPS and GALILEO) commitments as presently proposed in [16]. Horizontal Navigation System Error (NSE) and Horizontal Protection Level (HPL) are evaluated for both FD and FDE scenarios focusing on RNP1 and RNP0.3 navigation specifications defined in [17]. Results are obtained using the EUROCONTROL PEGASUS toolset upgraded to the latest DFMC Minimum Operational Performance Standards (MOPS) [15] and ARAIM reference algorithm specifications described in [1]. PEGASUS ARAIM module verification and validation results are described in [2]. The navigation performance for both RNP1 and RNP0.3 specifications are evaluated using error models, along with navigation requirements and design parameters inputs values as detailed in [2,15]. The NSE was evaluated using the PEGASUS Dynamics module by comparing the estimated GPS+GALILEO Iono-Free (IFree) position solution [15] with the true flight path reference position obtained by processing carrier-phase rover receiver measurements using a web-service [18].

3.1. Multipath and Ionosphere Errors Experienced During Flight Tests

In this section, we focus on the error model parameters that define the portion of the user range error that cannot be corrected. The magnitudes of both estimated multipath and ionospheric errors are compared to the corresponding standardized models [15]. The multipath and ionosphere uncertainty models that only apply after carrier smoothing with a 100 s time constant specify an elevation-dependent distribution [15]. These models are adequate only if the corresponding distribution is an upper bound of the actual distribution [19].

To be able to estimate multipath and noise error, we performed carrier levelling on continuous arcs of more than 600 s, that is, intervals with no data gaps and no cycle slips. For each arc, the carrier phase measurements are subtracted from the single frequency code measurements to obtain an estimate of the multipath. To account for the divergence of code and carrier phase measurements introduced by a change in the ionospheric delay over time, carrier phase measurements from a second frequency are used to estimate the divergence. For this process, as detailed in [19], the linear combinations of GALILEO E1+E5a and GPS L1+L5 were used for the respective signals. Over the continuous arcs of data, the bias due to the carrier phase ambiguities (resulting from subtracting the ambiguous carrier phase measurements from the unambiguous code measurements) was removed, evaluating the median of the differences to be able to exclude residual biases when very short datasets including multipaths are considered [19,20]. As the model binds the residual errors in the pseudo-ranges after code-carrier smoothing (100 s), the same IFree code-carrier smoothing [15] is applied to the obtained data. Data from all satellites are then sorted into bins of satellite elevation computed with respect to the local horizon [15]. The standard deviation of the data was then determined from all independent samples, where the rate of independent samples is twice the smoothing time constant, i.e., every 200 s [19]. The results are shown in Figure 2.

In Figure 2a, the estimated ${\sigma }_{MP\&noise}$ values for both GPS and GALILEO IFree are effectively bound by the required models [15] where the estimated multipath and noise for GALILEO appear lower than the GPS estimate from 20 degrees elevation onward, except for elevations superior to 85 degrees.

Use of multiple signals of distinct frequencies transmitted from the same satellite allows direct observation and removal of the first order (up to 99.9%) ionospheric effect, which depends on the inverse square of the frequency. The activity of the ionosphere varies significantly with latitude due to the Earth’s magnetic field and the angle at which solar radiation interacts with the atmosphere [21]. The flight tests took place in the Netherlands at mid-latitude where the ionosphere presents minor activity compared to the equatorial region. However, sporadic ionospheric irregularities and scintillation events can still occur, especially during periods of increased solar activity or geomagnetic storms. Ionospheric scintillation can lead to rapid variations in signal strength, causing signal fading, leading to loss of lock and degraded navigation performance.

Using a methodology akin to that used for multipath estimation, airborne GPS and GALILEO carrier-phase dual-frequency measurements are combined to estimate the slant ionosphere delay on continuous arcs (minimum duration of 600 s). The carrier phase estimate of slant ionospheric delay presents low noise but contains integer ambiguities of both frequencies, which are removed by subtracting the mean of differential ionospheric delays of continuous arcs. Finally, data from all satellites are then sorted into bins of satellite elevation computed with respect to the local horizon [15]. The results of estimated ionosphere uncertainty are presented in Figure 2b, showing that the estimated ${\sigma }_{UIRE\&noise}$ is also effectively bound by the required model [15]. The analysis results show the absence of multipath and ionosphere anomalies during the flight tests [13]; thus, nominal operational navigation performance behavior is presented in the next section.

3.2. Accuracy and Integrity Analysis of the Flight Data

In this section, we present the H-ARAIM accuracy and integrity performances. It is important to highlight the fact that the limited size of analyzed data samples cannot prove the integrity of the reference H-ARAIM algorithm implementation with respect to error bounds at a ${10}^{-7}$ integrity risk [1,2,16]. Another aspect that should not be neglected is that the H-ARAIM algorithm input parameter set used in this analysis depends on the present core constellation commitments (also known as default Integrity Support Data, ISD_default) [16], as well as on the latest assumptions developed in the ARAIM reference algorithm standardization group [1,2].

Figure 3 presents the results obtained for RNP1 and RNP0.3 specifications [22]. The number after RNP specifies the 95% bound of the Total System Error (TSE) expressed in nautical miles (NM) which is the combination of Flight Technical Error (FTE) and Navigation System Error (NSE) [17,22]. The NSE includes the Position Estimation Error (PEE), Path Definition Error (PDE) and display error [22]. For the scope of the analysis, the PDE and display error are assumed negligible. As the FTE is not available (arrival and approach maneuvers were manually flown), the analysis concentrates on the portion of the Position Estimation Error (PEE) that includes the signal in space error and the airborne receiver error [22]. Analysis of the NSE distributions for FD and FDE indicates that for most of epochs, the NSE hovered around 1 m, and the corresponding 95 percentiles are around 3 m. Those values are largely below the specifications for the horizontal PEE of 1611.2 m (0.87 NM) for RNP1, and 518.5 m (0.28 NM) for RNP0.3 [22]. However, upon observing the distributions of corresponding HPL, it becomes evident that only for the FD scenario were the RNP1 and RNP0.3 specifications met. Another notable observation from the distributions is the FDE mode’s capability to mitigate faults, resulting in an improved NSE (less than 1 m) for most of the time, with faulty satellites being excluded from position solution computation. Furthermore, the FDE HPL consistently remains below the PEE allocation for 95% of the time [22], specifically in the case of RNP1. Finally, for the totality of the processed epochs, the HPL corresponding to FD and FDE always bound to the corresponding NSE, and the NSE was always below the horizontal PEE allocation requirements for both RNP1 and RNP0.3 specifications [22].

3.3. Analysis of H-ARAIM Sensitivity to Aircraft Dynamics

ARAIM utilizes a multiple hypothesis approach to evaluate the potential impact of single or multiple satellite faults when computing the Protection Level (PL). Each combination of faults (or fault mode) is assessed based on its prior probability of occurrence (ISD). If a specific fault mode is likely to occur, a subset of measurements excluding potential fault candidates is established, and a subset position is estimated for comparison with the all-in-view solution. By merging all hypothetical position solutions into a union, the probability of the resulting subset containing at least one fault-free position solution can be determined [1]. The remaining fault modes not considered in the subset form the set of nonmonitored modes, chosen to ensure that the sum of their probabilities does not exceed a predefined fraction of the total integrity budget [1,2]. However, the ISD does not explicitly specify which fault modes need to be monitored. This determination is made by the algorithm based on the probabilities of events that can be treated as independent (ISD content).

Since the algorithm does not decide on fault modes for satellites involved in position computation, each position solution with integrity involves reduced subset geometries, making ARAIM more susceptible to available geometry compared to classic RAIM [6]. Nonetheless, ARAIM navigation performance significantly improves with more satellites in view. However, having many satellites in view with a relatively high likelihood of faults can result in a considerable number of subsets to evaluate. In [23], the number of monitored subsets is identified as the best parameter to monitor the ARAIM algorithm computational load. Furthermore, for a given geometry, the corresponding PL must be computed to ensure that the Probability of Hazardously Misleading Information (PHMI) remains below the required integrity allocation for horizontal error [1]. This is achieved by solving the PL equation using the half-interval search method [2], iteratively refining the solution until reaching the specified accuracy or maximum number of iterations [1,5].

These algorithm characteristics motivate the following analysis of the effects that the aircraft dynamics may have on ARAIM operational performance. The analysis focuses on HPL magnitude and algorithm computational load sensitivity to factors such as the aircraft roll angle, the number of used satellites for position solution, plus the number of monitored subsets, and the number of iterations necessary to solve the PL equation.

Figure 4 and Figure 5 illustrate the behaviors of the HPL magnitude versus the chosen analysis factors correspondent to FD and FDE scenarios. Data have been categorized for different aircraft roll angle intervals depicted in different colors. The number of satellites used for position solutions is the number of satellites satisfying the satellite use criteria specified in [15]. The number of monitored subsets is determined as outlined in [1,2]. Regarding the number of iterations, those refer to the total number of iterations needed to solve the PL equation for both horizontal components North–South and East–West. To solve the PL equation, a convergence threshold parameter (tolerance level) of 5 cm is used [2]. Moreover, a threshold of maximum 10 iterations is considered for each PL component [2].

Figure 4 and Figure 5 show that as the HPL magnitude increases, both the number of satellites used for position solution (geometry size) and the number of monitored subsets decrease. Furthermore, higher levels of HPL occur when the algorithm requires an increased number of iterations. Analyzing the color distributions reveals that aircraft dynamics appear to moderately impact the HPL magnitude. It is noteworthy that good geometry sizes are observed for aircraft roll angles above 9.5 degrees, although instances of high HPL values can be observed even when the aircraft roll angle is below 9.5 degrees. These cases can be attributed to the fact that estimated multipath errors on both frequencies are effectively by the error models, while weak geometry sizes may occur at any roll angle due to the limited availability of dual-frequency GPS satellites. For the FDE scenario, as outlined in Figure 5, we can observe high values of HPL also in correspondence of strong geometries compared to the FD scenario, as shown in Figure 4.

Figure 4 and Figure 5 also show that in both scenarios, the number of monitored subsets falls within a similar range, which may lead us to assume that FD and FDE scenarios would likely have similar computational loads. The computational load is approximately linear in the number of monitored subsets but very dependent on the details of the implementation [23]. Those elements are the ISD parameters describing the likelihood that a satellite or constellation is faulty, along with the algorithm navigation and design parameters inputs detailed in [2]. However, to be able to compare the two scenarios, in the case of FDE (Figure 5), the reported number of monitored subsets represents the worst-case monitored subsets for a given single exclusion (corresponding to the maximum protection level computed among all the subset exclusions) [1]. Indeed, the computational load of the FDE scenario could be roughly estimated as the product of the monitored subsets exclusions (including the all-in-view set) and the maximum number of iterations to compute the protection levels relative to the exclusions. Another detail that can be drawn from the obtained results is that the relatively high number of monitored subsets for high aircraft roll angles (19 degrees or more) also depends on the ratio between the number of used satellites with high and low likelihood of faults [23].

4. Conclusions

Flight data were used to characterize the minimum operational performance capability of the ARAIM reference algorithm. GPS and GALILEO dual-frequency flight data were used to analyze the performances for typical arrival and approach maneuvers with respect to positioning accuracy and integrity, as well as examining the algorithm computational load and its sensitivity to aircraft dynamics.

During the flight tests, no multipath or ionosphere anomalies were observed. Thus, nominal operational navigation performance analysis has been presented. The results show that in nominal conditions and with current constellation deployment status, using present core constellation commitments for default ISD, the navigation specifications for RNP1 and RNP0.3 operations could be met during the flight tests only for the FD scenario. However, the results in terms of navigation accuracy also show that the estimated horizontal navigation system error was below 1 m for most of the evaluated epochs for both FD and FDE scenarios.

Our analysis shows that the navigation performance and computational load of the H-ARAIM reference algorithm depend on the availability of dual-frequency GPS satellites and the current core constellation commitments to default ARAIM ISD.

However, the H-ARAIM capability of supporting more stringent navigation specifications for safety critical applications is expected to benefit from core constellation evolutions in the close future.