A Comparative Experimental Performance Assessment of RTK+OSNMA-Based Positioning for Road Vehicle Applications ^†

Abstract

To realize the societal need for greener, safer, and smarter mobility, ambitious technical challenges need to be addressed. With this aim, the H2020-EUSPA project ESRIUM investigates various aspects of highly accurate, reliable, and assured EGNSS localization information for road vehicles with a particular focus on automated vehicles. To analyze the achievable accuracy, reliability, and availability of multi-frequency and multi-GNSS mass-market receivers, we have conducted test drives under different GNSS reception conditions. In the tests, special focus was placed on using the Galileo Open Service Navigation Message Authentication (OSNMA) service, offering an additional feature for assured PVT (position, velocity, and time) information with respect to spoofing. We analyzed the performance of three Septentrio Mosaic-X5 receivers operated with different OSNMA settings. It could be shown that strict use of OSNMA provides very good positioning accuracy as long as sufficient suitable satellites are available. However, the overall performance suffers from a reduced satellite number and is therefore limited. The performance of a receiver using authenticated Galileo with GPS signals (final status of Galileo OSNMA) is very good for a mass-market receiver: 92.55% of the solutions had a 2D position error below 20 cm during 8.5 h of driving through different environments.

1. Introduction

Signals from the Global Navigation Satellite Systems (GNSS) are used in many modern devices and various applications, such as navigation, surveying and mapping, satellite communication, emergency response, location-based services, and precise time services, among many others. The robustness and resilience of the GNSS signals are very important for those applications, especially in safety-critical applications. There are particular concerns regarding the spoofing of signals, which means that a GNSS receiver can be deceived by fake GNSS signals in an attempt to modify the Position, Velocity, and Time (PVT) solution of the target receiver.

One method for preventing spoofing is the authentication of signals, which enables ensuring that a message is identical to the one transmitted at its origin and that it was generated by a trusted source [1]. The Galileo Open Service Navigation Message Authentication (OSNMA) is the first service providing this authenticity check in the civilian sector. It has currently been in the Public Observation (PO) test phase for slightly longer than a year. It uses a navigation message authentication scheme based on the Timed Efficient Stream Loss-tolerant Authentication (TESLA) protocol [2] and enables authentication at the receiver end in a manner that virtually eliminates the need for a chain of trust with dependence on third-party services. This offers the opportunity for Galileo to become the reference for many diverse new applications requiring authenticated positions.

Galileo OSNMA signals may also play a vital role in the field of road vehicle navigation. In this respect, the ESRIUM project, funded in the context of Horizon 2020, aims at creating road wear maps with accurate information about the position and shape of road damage, and to send prompt and real-time notifications to drivers and autonomous vehicles with instructions to avoid the damaged areas and route recommendations (e.g., change of lanes and driving offsets with respect to the lane center) to even out the road wear [3]. ESRIUM relies in this regard on Galileo services for increasing the positioning accuracy of vehicles receiving these notifications, and for authenticating the position estimates using Galileo OSNMA. The overall goal is to increase the safety and robustness of the localization solution and to reduce road maintenance costs as well as improve road safety and comfort at the same time [4].

Within the scope of ESRIUM, the Finnish Geospatial Research Institute (FGI) developed an open-source implementation of Galileo’s OSNMA, called FGI-OSNMA [5], which distinguishes itself, e.g., by modularity and real-time usability. Other implementations also exist already, for example Osnmalib [6] and galileo-osnma [7]. The Septentrio company included support for OSNMA in some of their products, such as the multi-constellation GNSS receiver module Mosaic-X5 [8]. Theoretical work regarding OSNMA is provided by several authors, e.g., [2,9,10,11,12,13]. Practical performance assessments are reported by, e.g., [14,15,16,17,18,19,20,21,22,23,24,25]. From an experimental performance assessment of Galileo OSNMA within the scope of ESRIUM, it was observed in a four day-long static observation that the receiver could produce authenticated signals 99.74% of the time under open sky conditions [26]. The authentication status could not be attained in cases with a low number of satellites transmitting OSNMA data to the receiver. Therefore, it was concluded that the authentication performance will significantly decrease in environments with degraded satellite visibility, such as urban environments. Hence, with regard to OSNMA, it is beneficial to also use data from satellites with low elevation, as this increases the chance to cross-authenticate visible satellites.

In the current paper, we analyze the experimental performance of RTK+OSNMA-based positioning for road vehicle applications by comparing the PVT solutions of three different Mosaic-X5 receivers that were operated using three different OSNMA configurations (for details see Section 2). We conducted about 8.5 h of test drives under different driving environments, including open sky, motorways, rural roads, hilly and forested environments, and suburban and urban regions. We analyze the overall performance aggregated over all the environments regarding the proportion of Real-Time Kinematics (RTK) solutions and the horizontal position error. A detailed description of the experimental settings is provided in Section 2 and the results are shown in Section 3.

2. Experimental Settings

A sensor car shown in Figure 1 provided by Virtual Vehicle Research GmbH (Graz, Austria) was equipped by the study consortium members with three Septentrio Mosaic-X5 GNSS receivers in different configurations and an iMAR iNAT-FSLG-01 inertial measurement system (IMU) with gyro compassing capability (high-end fiber-optic gyroscope) and an integrated GNSS receiver (Novatel OEM729-2.01). All GNSS receivers were connected to the same antenna using an active one-by-four antenna splitter. The used antenna model was a NavXperience 3G+C multi-band GNSS antenna. The architecture of this setup is shown in Figure 2. The test vehicle was equipped with the DataSpeed drive-by-wire kit (https://www.dataspeedinc.com/, accessed on 15 May 2023), which enables access to most CAN data as well as the control actuators for ADAS/AD system-development purposes. The vehicle was also equipped with additional sensors, which could be modified depending on the measurement or the use-case requirements.

The GNSS receivers were connected to the so-called car PC (industrial PC installed in the vehicle) for power supply and data monitoring. The car PC was powered over an inverter belonging to the vehicle’s DataSpeed drive-by-wire kit. The connection between the PC and the X5 receivers was achieved via USB and ethernet (cf. Figure 2). A control laptop was also connected via ethernet.

All three Mosaic-X5 receivers received the GNSS signals via the same antenna and operated in the RTK mode using the incoming RTCM messages via 4G mobile internet. To guarantee that all receivers are receiving identical RTCM data with the same latency, Septentrio Data Link software was used to act as an RTCM splitter, acting as an NTRIP client to receive correction data in the form of virtual reference station data and forward the data to all receivers via a TCP stream. GNSS receivers operating in theh RTK mode can either output RTK fixed solutions, where the carrier phase ambiguities could be solved (estimated as an integer value), which provides the best positioning accuracy, or RTK float solutions, where the carrier phase ambiguities could be estimated as float values only, resulting typically in less accurate position information. When the receiver could not compute RTK positions, it fell back to so-called Single Point Positioning (SPP), where only data from the satellites were used in the PVT computation. The resulting positions were saved together with relevant information such as the type of solution attained, the quality of the solution, the satellites that were used for the position estimation, and so forth. The data were stored using the SBF protocol with a sampling rate of 10 Hz.

The Mosaic-X5 receivers are one of the first mass-market receivers natively supporting Galileo OSNMA. For the performance analysis, one receiver was configured to use all GPS L1/L2 + Galileo E1/E5a satellite signals (called OSNMA off) for the PVT solution. The second receiver was configured in the OSNMA loose configuration. According to Septentrio, this means using all GPS signals and Galileo signals from satellites with authenticated navigation messages or unknown status. The third receiver was configured to use no GPS signals and only authenticated Galileo signals (called OSNMA strict). The reason for not using either Glonass or Beidou is that in the final stage, the Galileo OSNMA service will support Galileo and GPS, and hence, the OSNMA loose configuration should give an estimate of the final performance of Galileo OSNMA. The Galileo navigation data were recorded including I/NAV messages containing the OSNMA-related data bits.

The measurement data from the FOG-based EGNSS/IMU iMAR iNAT FSLG-01 system were post-processed to serve as a highly accurate reference solution for data analysis. Therefore, the IMU data rate was recorded at 500 Hz and the GNSS data at 1 Hz.

The sensor car with the installed equipment described above was then used in several test drives. In total, during four test drives, we recorded about 8.5 h of measurement data using the receiver settings as described above. For the study at hand, we analyzed the data in post-processing to investigate the accuracy of the PVT solution under the different OSNMA settings. The results are shown in Section 3.

3. Results and Discussion

In the analysis, the performance of the three solutions in terms of the 2D position error and fix type was compared, also in relation to the number of satellites. The current limitations of receivers using signals from only OSNMA-authenticated satellites in terms of availability and accuracy are discussed in relation to potential use cases involving highly automated driving.

Figure 3 shows the proportion of position types encompassing all driving conditions, ranging from open sky in motorways and rural roads, through hilly and forested environments, to suburban and urban areas. It can be seen that the percentage of fixed solutions (RTK fix) was 89.92% when all GPS L1/L2 and Galileo E1/E5a satellite signals were used (OSNMA off). The percentage was slightly lower in the OSNMA loose configuration (89.59%).

Regarding the receiver with OSNMA strict settings (use of authenticated Galileo signals only, no GPS signals), the percentage of (now authenticated) RTK-fixed solutions was dramatically reduced to only 55.77%. This is obvious since the number of available satellites is significantly reduced, resulting in a significant reduction of the percentage of authenticated fixes (RTK fix under the OSNMA strict settings). Moreover, and as analyzed in [26], the use of cross-authentication in OSNMA can lead to having visible but not authenticated Galileo satellites, which therefore will not take part in the PVT computation. Although to a lesser extent, this can also contribute to the overall satellite visibility reduction when OSNMA is in strict mode.

In 16.17% of the strict cases even no PVT solution at all (no-fix) could be achieved, which was most probably due to the reduced number of visible satellites. The probability of an SPP solution was 8.65% for the OSNMA strict case. For the other OSNMA configurations, the probability for an SPP solution was only about half as small (3.72% for OSNMA off, 3.53 for OSNMA loose). These results clearly underline that it is very important to expand the network of authenticated satellites so that the OSNMA service can be used reliably.

When considering Figure 4, one can derive that the percentage of RTK fixes increases with the number of satellites. Simultaneously, the percentage of RTK float solutions decreases. The percentage of authenticated fixes (OSNMA strict) is best when 9 to 10 authenticated satellites are available. There were no cases with more then 10 authenticated satellites. More than 10 satellites could only be used by applying the OSNMA off and loose configurations. When further increasing the number of satellites, the percentage of RTK fix solutions improved slightly and the percentage of SPP solutions decreased for both configurations. This can even be seen when increasing the number of satellites from 13–14 to 15–18. The increase in this case accounts for 1.52% in the OSNMA off configuration, and for 2.24% in the loose configuration. Most RTK-fix solutions could be achieved in the OSNMA loose configuration, with a probability of 97.06%. This value displays the performance that can presumably be achieved with the final Galileo OSNMA service. This is a very promising result in view of autonomous driving vehicles.

In general, the performance of the receivers with the OSNMA off and loose configurations was very similar, regardless of the number of visible satellites.

The localization accuracy was investigated by looking at the distribution of the two-dimensional (horizontal) localization error for the different OSNMA configurations. The left side of Figure 5 shows the Cumulative Distribution Function (CDF) for an open sky case. An autonomous vehicle was driving there, and all three receivers could achieve RTK-fix for the whole time. As can be seen, the 2D localization accuracy was always better than 5 cm. Totals 72.62% of the OSNMA strict solutions, 86.75% of the OSNMA loose solutions, and 86.86% of the OSNMA off solutions had an accuracy of better than 2 cm. These results show that under open sky conditions a vehicle can safely be guided within a traffic-lane, and this is also in agreement with [27,28,29].

The right hand side of Figure 5 shows CDFs for data aggregated over different driving environments. This is a result from drives where we passed motorways, hilly and forested rural roads, a mountain, and suburban and urban areas. The 2D errors are obviously significantly larger due to the more challenging environments. Hence, an RTK-fixed solution was not always possible, but all solution types occurred (no-fix, SPP, RTK float and fix). The worst performance was achieved in the OSNMA strict mode, which is mainly due to the limited satellite visibility and availability of authenticated Galileo satellites. Only 73.2% of the data had an error of less than 20 cm. For the OSNMA off case, 92.47% of the PVT solutions were better than 20 cm, and the OSNMA loose case again performed slightly better with 92.55% of the solutions being better than 20 cm. This is a very good performance for a mass-market receiver.

Looking more into detail, one can observe that the localization accuracy typically increases with the number of satellites used during the PVT calculation. The general behavior of the 2D errors is shown in Figure 6 and Figure 7. When the satellite number was between 4 to 6 (top left), the solutions for all OSNMA configurations performed moderately well. The OSNMA strict solution performed slightly better in the relevant accuracy range (below 50 cm). This most probably resulted from the fact that the satellite visibility is better in situations where 4–6 Galileo satellites are visible instead of when 4–6 GPS+Galileo satellites are visible (comparing solutions with a different number of visible satellites does not mean comparing the performance at the same locations!). In addition, Galileo signals are typically better than GPS ones, especially in multipath conditions. For the OSNMA strict configuration, 65.37% of the 2D errors were smaller than 20 cm, compared to 61.16% for OSNMA loose and 58.19% for OSNMA off. The authors would expect that the advantage of the OSNMA strict solution over the others should be maintained, but this was not the case for the group of 7 to 8 visible satellites. The reason for it is unclear. In summary, the performance significantly improved when having 7–8 satellites. For the use of 9 to 10 satellites, the OSNMA strict performance was extraordinarily good (98.00% of the errors were below 20 cm). This is obvious because having 9 or 10 satellites of one constellation means that the receiver was in a very open area compared to having 9 or 10 satellites of two constellations.

Further increasing the satellite number to more than 10 satellites only slightly improved the positioning accuracy. In principle, one can see nearly no difference between the OSNMA loose and off configurations. This indicates that nearly all Galileo satellites could be authenticated (or the status was unknown). The combined GPS+Galileo solutions with 15 or more satellites have a comparable performance to that of the OSNMA strict solution with 9 or 10 satellites, which is obvious since most probably, the receiver was in a very open region.

The performance of the OSNMA loose configuration gives an indication of what most probably can be achieved with the Galileo OSNMA service when it is fully operational. This is a very promising result in view of highly automated vehicles.

Due to some issues with the wireless connection in other tests, we were also able to investigate the position accuracy during periods without correction data. For that matter, the receiver was configured to use base station data with ages up to 50 s. In Figure 8, on the left we see the 2D position error against the mean correction’s age showing only RTK-fix solutions, while on the right side, one can see the 2D position errors for RTK-float solutions. In both figures, within the first seconds, one can see the errors that are typically encountered in urban and suburban situations. Afterwards, one can see the influence of older correction data on the RTK solution (mostly for highway and rural road conditions). As long as an integer fix is possible, one can be quite sure that the 2D position error stays below 20 cm up to 50 s, which was not expected. For float solutions, it can stay as good as 50 cm if the initial float solution is stable, and otherwise the errors can be as large as 22 m (maximum observed 2D error of float solution).

4. Conclusions

Summarizing, within the H2020-EUSPA project ESRIUM, test drives were performed to investigate the achievable localization accuracy in different Galileo OSNMA configurations for road vehicles with a particular focus on automated vehicles. Therefore, three mass-market GNSS receivers of the type Septentrio Mosaic-x5 were installed in a test vehicle and configured differently with respect to Galileo OSNMA. The results show that the performance of the receiver is quite good for a mass-market product. Currently, OSNMA can only authenticate Galileo satellites. Using only authenticated Galileo satellites (OSNMA strict mode), 73.2% of the solutions had a 2D position error below 20 cm during 8.5 h. For the OSNMA loose configuration (using in addition GPS satellites), by contrast, 92.55% of the solutions had a 2D localization error below 20 cm. The probability of achieving an RTK-fix solution in the OSNMA loose configuration was 97.06%. The OSNMA loose configuration is probably the performance that can be achieved in common environments with the Galileo OSNMA service when it is fully operational. In total, the results coincide with the theory that the number of satellites plays an important role regarding the accuracy of the PVT solutions that can be achieved. A medium number of satellites, around 10, using GPS and Galileo, showed already quite good performance in the results, while more are obviously preferable. The investigations showed that even longer RTK correction data outages did not necessarily lead to critical situations in terms of 2D positioning accuracy, since as long as the RTK fix can be kept, errors below 20 cm can be achieved. Therefore, it makes sense to increase the standard setting of the maximum RTK age in the receiver.