This section is divided into three sub-sections: test description, preliminary data analysis, and velocity domain results. In the first, the performed test is detailed in terms of equipment and scenario. In the second, the collected measurements are preliminarily analyzed by studying the measurement availability, the signal strength, and the satellite geometry. In the last, the estimated velocities are analyzed.
3.2. Preliminary Data Analysis: Measurement Availability, Signal Strength, and Geometry
Velocity estimation is strongly affected by the number of available measurements, by the geometry, and by the measurements’ quality. Before the results are presented in the velocity domain, a preliminary analysis of the data is performed. In detail, the following aspects are investigated:
As evidenced by
Figure 1, the location where the test took place has no significant obstacles, and the multipath level is low, so it is a typical open-sky scenario; consequently, a large number of measurements is expected.
The availability of Doppler and CP observables is shown in
Figure 2 for each device, GNSS, and satellite ID. In this analysis, all the tracked measurements are considered, without considering the mask angle and
limit. Specifically, panels in the first row refer to GPS satellites, panels in the second row to Glonass, panels in the third row to Galileo, and panels in the fourth row to BeiDou. Moreover, panels in the first column refer to the Novatel device, panels in the second column to uBlox, and panels in the third column to Xiaomi. Only one panel is present in the first column, i.e., panel (a), because the Novatel device is able to acquire only GPS signals. In each panel, a blue dot indicates the availability of both Doppler and CP measurements for a certain epoch, while a red dot indicates the availability of only Doppler measurements and the unavailability of CP measurements in the considered epoch. It never happens that CP is available and Doppler is not.
From an analysis of the panels (a), (b), and (c), it is evident that the tracked GPS satellites are roughly the same, except for G11, which is tracked by uBlox but not by Novatel and Xiaomi. There are no red dots for Novatel, while they are present for uBlox and even more for Xiaomi. This highlights that, in the GPS case, the CP observable is continuous for the high-grade device, while discontinuities appear for the high-sensitivity device and, above all, for the smartphone chip. This issue reverberates on the availability of a TDCP solution, which is based on time-differenced CP measurements. The comparison with the geodetic receiver in terms of the other GNSSs is not possible.
An analysis of the panels (d) and (e), regarding Glonass measurements from uBlox and Xiaomi, respectively, indicates that the tracked satellites are significantly different: measurements from R02, R10, R11, and R19 satellites are present for uBlox and not for Xiaomi. Glonass Doppler measurements for Xiaomi are often insufficient for standalone velocity estimation, and CP is often missing even when Doppler is available, causing a long unavailability of TDCP Glonass-only velocity.
A very similar consideration can be made for Galileo measurements, by analyzing panels (f) and (g): for Doppler shift, the uBlox device was able to track up to 11 satellites, while the maximum number of Galileo satellites for the smartphone was 4; for the TDCP case, only a few discontinuities were observed for uBlox, while for the smartphone case, large discontinuities were observed, and even in case of Doppler availability, CP is often missing.
Regarding BeiDou, the available Doppler and CP measurements are similar for uBlox and Xiaomi; only small differences can be noted. Specifically, satellite C30 is tracked by uBlox and not by Xiaomi, and satellites C02 and C05 are tracked by Xiaomi and not by uBlox. C02 and C05 are both geostationary (GEO) satellites, highlighting a uBlox shortage in tracking such satellites, while C30 is a medium Earth orbit (MEO) satellite and is probably not tracked by Xiaomi due to the inherent selection criteria of the device. Xiaomi is not able to store CP for C02, C05, and C08, while CP is discontinuous for C19 and C20. C02, C05, and C08 are all characterized by low values, about 25 dB-Hz, inhibiting the storage of reliable CP measurements; C08 is an inclined geosynchronous orbit (IGSO) satellite. CP discontinuities of C19 and C20 are related to frequent drops in their C/N0 values below 25 dB-Hz. The number of epochs where Doppler is present and CP is missing is significantly smaller in uBlox.
An analysis of the measurements stored by Xiaomi, panels (c), (e), (g), and (i), indicates that a measurement selection criterion appears to be implemented in the device, privileging GPS and BeiDou measurements over Glonass and Galileo ones. This behavior is probably due to the limited number of channels available for tracking.
An additional analysis of measurement availability is carried out by comparing the availability percentages of Doppler and CP measurements of two GPS satellites, G02 and G06, for the three considered devices. In
Figure 3a, the measurement availability for G02 is shown, while
Figure 3b shows that for G06. These particular satellites have been selected because CP from G02 is continuous for all three devices, while for G06, some discontinuities in CP are evident for uBlox and Xiaomi, as shown in panels (a), (b), and (c) of
Figure 2. The graphs in
Figure 3 confirm what emerges from
Figure 2, relative to satellites G02 and G06. Specifically, the availability of Doppler and CP measurements of G02 is 100% for all the considered devices, with only a few gaps for CP with Xiaomi, which has 99.9% availability. Regarding the G06 satellite, Doppler and CP availabilities are 99.9% for Novatel; they are 99.8% and 97.8%, respectively, for uBlox; and they are 92.2% and 83.1%, respectively, for Xiaomi.
The behavior of
is shown in
Figure 4, which is structured like the previously described
Figure 2. In the panels, the
values of the tracked satellites are displayed with respect to time; in black, the mean
values for each GNSS and device are shown too. Considering the GPS case, panels (a), (b), and (c), the Novatel device shows larger
values than uBlox and Xiaomi, with mean
values equal to 44.7 dB-Hz, 41.3 dB-Hz, and 37.7 dB-Hz, respectively. This could be related to the different gain antennas characterizing the three devices and to the way
is computed by each device. The
behavior for Novatel seems more regular with values never below 25 dB-Hz, while the uBlox receiver and the smartphone were able to track very weak signals with
values as low as 15 dB-Hz. For uBlox and Xiaomi, the
behavior is more fluctuating, with lower values for Xiaomi.
Considering the Glonass case, in panels (d) and (e), more satellites are tracked by uBlox than by Xiaomi; moreover, values for Xiaomi are lower, e.g., satellite R09 has values between 40 and 50 dB-Hz for uBlox (purple dots in panel (d)) and between 30 and 40 for Xiaomi (yellow dots in panel (e)). The mean values are 39.8 dB-Hz for uBlox and 31.5 dB-Hz for Xiaomi. Such behavior is, as previously mentioned, related to differences in antenna and receiver architecture between the considered devices. Similar behaviors are also evident for Galileo in panels (f) and (g); in this case, the mean value is 40.2 dB-Hz for uBlox and 30.4 dB-Hz for Xiaomi. Even for BeiDou, it is evident that for Xiaomi, panel (h), lower values are present in comparison to uBlox, panel (i); the mean values are 36.6 dB-Hz for uBlox and 30.2 dB-Hz for Xiaomi.
In
Table 1, the mean, maximum, and minimum
values for each GNSS and for each device are summarized. It is evident that for each GNSS, the higher the device grade, the higher the mean
value. GPS measurements have, on average, larger
values in comparison to the measurements from the other systems. Glonass and Galileo measurements, fixed the device, have similar mean
values, while the lowest values are found for BeiDou. The low mean values of
for BeiDou are due to the presence of GEO and IGSO satellites in its constellation.
Similar conclusions can be obtained considering the maximum and minimum values: uBlox (and Novatel, even if for the GPS-only case), reported a higher maximum value than Xiaomi; Xiaomi, on the other hand, reported a higher minimum value than uBlox, except for the BeiDou case. Although the devices were able to track very weak signals with close to 10 dB-Hz, the measurements related to signals with such low power are not used in the estimation process because a threshold for the value was set to 25 dB-Hz.
Table 2 reports the mean
values for BeiDou measurements, classified by orbit type; it demonstrates that BeiDou measurements from MEO satellites have
values similar to those for GPS, while significantly lower values characterize GEO and IGSO cases.
An additional analysis of
is carried out by comparing its trends for the G02 and G06 GPS satellites, for the three considered devices. In
Figure 5, the
behavior for G02 is shown in panel (a), and that for G06 is shown in panel (b). As shown in
Figure 5a, the
values are about 48 dB-Hz with a very stable behavior for Novatel; slightly lower values with more pronounced fluctuations are evident for uBlox and even more for Xiaomi, remaining above 30 dB-Hz in any case. From
Figure 5b, it is evident that the
values are lower than those for the G02 case: for Novatel,
values are higher than those for uBlox and even higher than those for Xiaomi. It can be noticed that for uBlox and Xiaomi, the epochs in which Doppler measurements are available and CP measurements are not, marked by red dots in
Figure 2, correspond to values of
below 25 dB-Hz.
Usually, measurements from a satellite elevation below a certain threshold and with a
value below a certain limit are not used for solution computation. In this work, a mask angle of 15 degrees and a
limit of 25 dB-Hz are adopted. So, the numbers of actually used Doppler and CP measurements are reduced in comparison to the ones shown in
Figure 2 and
Figure 4.
In
Figure 6, the numbers of used Doppler shift and TDCP measurements are shown for each GNSS and for each considered device, following the scheme described in the previous figures. TDCP measurements are obtained by differencing two consecutive CPs, so it is sufficient that at least one of the two CPs is missing for inhibiting TDCP measurement; for this reason, it is expected that the number of TDCP measurements is less than the number of Doppler measurements. In the same figure, the corresponding 3D DOP is shown too. The numbers of used Doppler and TDCP measurements are drawn in cyan and blue, respectively; the corresponding 3D DOP values are drawn in magenta and red, respectively. The number of measurements and the corresponding 3D DOP are strictly related; indeed, DOP increases when the number of measurements decreases and vice versa.
An analysis of the GPS-only case reveals that for the Novatel device (panel (a)), the numbers of Doppler and TDCP measurements are the same (except for the first epoch, when the TDCP measurements cannot be obtained). For uBlox, panel (b), some punctual reductions in measurements, can be noted but the numbers of Doppler and TDCP measurements are always larger than four, allowing the estimation of velocity. For Xiaomi, panel (c), a larger difference between TDCP and Doppler shifts can be appreciated, with the former going down to four. Regarding geometry, for the Novatel device, the 3D DOP behavior, in panel (a), is regular, with values below 2.5 and with two small jumps in correspondence with changes in measurement availability; the red and magenta lines are coincident. For uBlox, only a few additional jumps are visible in panel (b), related to the sudden variations in measurements. The frequent variations in GPS measurements in Xiaomi lead to irregularities in the red and magenta lines; when four TDCP measurements are available, the corresponding DOP value exceeds 15 (panel (c)).
For Glonass, panels (d) and (e), the number of measurements used varies between four and seven for uBlox, and few differences between the Doppler and TDCP cases can be noted. For the Xiaomi case, the number of used Doppler shift measurements is between one and five; the situation is even worse for TDCP measurements, which exhibit significant drops in comparison to the Doppler shift. An analysis of the corresponding geometry reveals that for the uBlox device, the 3D DOP values are about 2.5 for the first part of the session, while they jump to values greater than 15 in the second part, characterized by fewer available measurements (panel (d)). For Xiaomi, the number of Glonass measurements is not sufficient in the second part of the session, making the velocity estimation and DOP computation impossible (panel (e)).
For Galileo, panels (f) and (g), the behavior of the available measurements seems very regular for uBlox, with between six and eight measurements available for Doppler and between five and eight measurements available for TDCP. For Xiaomi, the number of Doppler measurements is often below four, making the estimation of velocity impossible, while TDCP measurements are always insufficient for the velocity estimation. In an analysis of the corresponding geometry, uBlox shows a good DOP behavior, comparable to the GPS-only case (panel (f)). Xiaomi has sufficient Doppler measurements for estimating velocity only in the last part of the session, while the TDCP measurements are insufficient for the entire session, as shown in panel (g).
For BeiDou, the numbers of Doppler and TDCP measurements regularly span between five and six for uBlox (panel (h)), and the number of measurements is most of the time equal to five, with drops to three for Doppler and to two for TDCP, for Xiaomi (panel (i)). For uBlox, the geometry with five measurements is poor, leading to DOP values greater than 15, while with six measurements, DOP values become below 5 (panel (h)). For Xiaomi, when five measurements are available, 3D DOP values are between 2 and 3, and during the sudden measurement drops, DOP values become large, exceeding 15, as shown in panel (i).
The availability of Doppler and TDCP measurements during the session is summarized in
Table 3, where the minimum, maximum, and average numbers of measurements are shown for each considered GNSS and device. It is evident that Novatel and uBlox are able to track approximately the same GPS satellites while Xiaomi shows a lower minimum value, just sufficient for velocity estimation. The Xiaomi device demonstrates a deficiency in available Glonass and Galileo measurements in comparison to uBlox, which leads to frequent solution unavailability. Regarding BeiDou, uBlox and Xiaomi devices have comparable measurement availability, which is reduced for Xiaomi mainly because of IGSO C08 satellite unavailability.
3.3. Velocity Domain Results
In this section, the main outcomes of this study are reported; in particular, the estimated velocities are analyzed for each considered device and for single GNSS cases. In
Figure 7 and
Figure 8, the behaviors of the horizontal and vertical velocity errors are shown; in all panels, the solution obtained using Doppler shift measurements is represented by the red line, while the blue line represents the solution obtained with the TDCP technique. In this case, the figures are composed as described in the previous section.
For the performance in the velocity domain using GPS measurements, the panels (a) (b), and (c) of
Figure 7 and
Figure 8 are considered. From the figures, it is evident that Doppler-based and TDCP-based velocities are accurately estimated when GPS measurements from Novatel are used, while when using GPS measurements provided by uBlox, the TDCP velocity error is mostly about one order of magnitude less than Doppler one, as emphasized by the mean errors. Nevertheless, frequent spikes in the TDCP solution can be noted and are caused by anomalous measurements, supposedly due to cycle slips. For GPS measurements collected by Xiaomi, panel (c), sudden climbs in horizontal and vertical errors are evident in both the Doppler and TDCP cases (more pronounced in the latter), due to poor geometry; indeed, such error increases correspond to the DOP increase shown in
Figure 6.
For Glonass measurements with uBlox, the first part of the session is characterized by frequent spikes in the TDCP solution, panel (d), while in the second part, a degradation in both the Doppler and TDCP solutions is present, due to a reduction in the available measurements and to a consequent geometry worsening; maximum errors are higher, being about a few m/s. When using Glonass measurements from the Xiaomi device, panel (e), the number of available Doppler measurements is mainly four in the first part of the session and below four in the second part, making the solution computation impossible. In the first part, sudden jumps in error values indicate the presence of blunders among the measurements. TDCP measurement availability is worse with longer solution unavailability; blunders are also present, leading to a maximum error exceeding 30 m/s.
The solutions obtained using Galileo measurements from the uBlox receiver, panel (f), are very similar to the GPS case with the same device, resulting in fewer and smaller anomalous error peaks. For Galileo with Xiaomi, panel (g), the TDCP solution is impossible to estimate due to an insufficient number of measurements during the session, while the Doppler solution is possible only in the second part of the session, with degraded performance (rms errors of about 1 m/s) due to the probable presence of anomalous measurements.
Considering BeiDou measurements from the uBlox device, panel (h), both Doppler and TDCP are degraded in the initial part of the session because of poor geometry, while in the remaining part, velocity errors are prevalently below 20 cm/s for the Doppler case and 3 cm/s for TDCP case. A large spike is evident in the TDCP solution, related to anomalous measurements. Considering BeiDou measurements from Xiaomi, panel (i), the poor geometry issue in the initial part of the session is not present, but several spikes are present in the TDCP solution. The Doppler velocity estimation seems to be better than the uBlox case, with more bounded errors.
The mean, root mean square (rms), and maximum horizontal and vertical velocity errors are computed and summarized in
Table 4 and
Table 5, respectively.
Comparing the single GNSS configurations in terms of the considered figures of merit, it is evident that the best performance is obtained by the high-grade device. For uBlox, GPS-only performance and Galileo-only performance are similar, with the Doppler solution prevailing for GPS and the TDCP solution prevailing for Galileo. The GPS-only solution with Xiaomi demonstrates good performance, better than the corresponding uBlox configuration for almost all the figures of merit. The Glonass-only performance is deficient for both the uBlox and Xiaomi devices.
From the tables considering the GPS Novatel case, it can be noted that the rms horizontal and vertical errors are 2 cm/s and 3.6 cm/s, respectively, for Doppler and 2 mm/s and 4 mm/s for TDCP; the maximum errors are 7.2 cm/s and 11.7 cm/s for Doppler and 17 mm/s and 55 mm/s for TDCP. For the uBlox case, the rms errors are at the cm/s level for both the Doppler and TDCP cases. For the TDCP case, larger maximum horizontal and vertical errors of about 1 m/s and 1.8 m/s, respectively, have been observed. Finally, for the smartphone case, Doppler-based performance is good, considering that the horizontal and vertical rms errors are 1.3 cm/s and 4.5 cm/s, comparable with the Novatel case.
In
Table 6, the solution availability values for the different configurations are reported. Solution availability is defined as the percentage of available solutions when the number of available measurements is at least four. Herein, a measurement is considered available if it comes from a satellite at an elevation angle above 15 degrees and if it has a C/N0 above 25 dB-Hz. Moreover, in the epochs where the geometry is too poor, specifically with 3D-DOP larger than 15, the solutions are considered not available. TDCP solution availability cannot be 100%, because two consecutive CP measurements should be differenced to obtain a TDCP measurement, and so it is not possible to have TDCP measurements in the first epoch.
From
Table 6, it is evident that for the GPS case, the solution availability is very close to 100% for both the Doppler shift and the TDCP case. For the BeiDou case, it can be noted that few solutions are not available for Xiaomi. The situation for Glonass and Galileo is strongly dependent on the type of receiver. Using uBlox in both cases, the solution availability is very high, about 95% for Glonass and about 100% for Galileo; only marginal differences can be appreciated between the Doppler shift and the TDCP case. The situation is completely different when the smartphone solutions are considered: for the Glonass case, the solution availability using Doppler shift measurements is around 50%; this value is reduced in the TDCP case to about 43%. For Galileo, the Doppler shift solution availability is about 42%, and it was not possible to compute the TDCP solution; hence, the solution availability in this case is 0%. Hence, the configurations characterized by the worst solution availability are Glonass-only and Galileo-only with Xiaomi with a percentage of about 50 or lower, and with the TDCP solution never being possible for Galileo-only.
From the errors reported in
Table 4 and
Table 5 and
Figure 7 and
Figure 8, the presence of anomalous measurements leading to large velocity errors is evident, above all for the TDCP solution and non-high-grade devices. In this context, the application of RAIM-FDE can be effective in improving the results. In
Figure 9 and
Figure 10 and
Table 7 and
Table 8, the same error analysis carried out previously is reported, but with the application of the Subset algorithm. RAIM-FDE evaluates the consistency of the measurements, tries to reject blunders, and indicates reliable solutions. In this error analysis, only reliable solutions are considered.
RAIM-FDE has no impact on both Doppler shift and TDCP velocities using the Novatel device, as is evident from the identical figures of merit reported in
Table 7 and
Table 8, because no blunders were detected. The benefits coming from RAIM-FDE can be clearly appreciated when using the measurements of low-cost devices.
For the GPS case with uBlox, the benefits of RAIM-FDE are evident in the TDCP solution where several spikes have been eliminated, improving all the figures of merit; no benefits are evident for Doppler velocity. A similar behavior is demonstrated for the Galileo-only case with uBlox. No significant effect is visible in the GPS case using measurements retrieved from the Xiaomi device, because the error peaks occur when DOP values are high and measurement redundancy is low, making the application of RAIM-FDE ineffective.
For the Glonass-only case with uBlox, several anomalous measurements have been identified and rejected, as demonstrated by the improvements of all the considered figures of merit, but some large errors remain in the solution owing to low redundancy. Both the Doppler and TDCP solutions are discontinuous because RAIM was not able to identify blunders in spite of the presence of measurement inconsistencies. Reliable solutions are almost completely absent for the Glonass and Galileo cases when the Xiaomi device is used; this is mainly due to the limited number of available measurements, as at least five measurements are needed to apply RAIM-FDE.
Finally, considering BeiDou measurements, RAIM has no impact with uBlox but is effective with Xiaomi, as demonstrated by the reduction in the maximum errors.
In
Table 9, the reliable availability, defined as the solution percentage indicated by RAIM as reliable, is reported for each considered device, GNSS, and processing technique. Reliable availability is always lower than or equal to solution availability. The most critical configurations, according to reliable availability, are Glonass with both uBlox and Xiaomi, and Galileo with Xiaomi. For Glonass with uBlox, about 35% of the solutions are not reliable; for Glonass with Xiaomi, about 98% of the solutions are not reliable; and for Galileo with Xiaomi, there are no reliable solutions. For Galileo, these low values are due to severe geometric conditions and a limited number of measurements, while for the uBlox Glonass case, several outliers have been identified among the measurements.