Error Investigation on Wi-Fi RTT in Commercial Consumer Devices

: Researchers have explored multiple Wi-Fi features to estimate user locations in indoor environments in the past decade, such as Received Signal Strength Indication (RSSI), Channel State Information (CSI), Time of Arrival (TOA), and Angle of Arrive (AoA). Fine Time Measurement (FTM) is a protocol standardized by IEEE 802.11-2016, which can estimate the distance between the initiator and the station using Wi-Fi Round-Trip Time (RTT). Promoted by Google, such a protocol has been explored in many mobile localization algorithms, which can provide meter-level positioning accuracy between Wi-Fi RTT-enabled smartphones and access points (APs). However, previous studies have shown that the Wi-Fi RTT measurements are sensitive to environmental changes, which leads to signiﬁcant errors in the localization algorithms. Such an error usually varies according to different environments and settings. Therefore, this paper investigates the error in Wi-Fi RTT distance measurements by setting multiple experiments with different hardware, motion status, and signal path loss conditions. The experiment results show that four categories of errors are found in RTT distance measurements, including hardware-dependent bias, blocker-dependent bias, ﬂuctuations, and outliers. Comparison and analysis are carried out to illustrate the impact of the different errors on Wi-Fi RTT distance.


Introduction
The Global Positioning System (GPS) has served for years for providing reliable Location-Based-Services (LBS) in outdoor scenarios.However, the performance of GPS signal degrades in indoor scenarios caused by blockers, such as walls and roofs.In recent years, Wi-Fi has been widely explored to provide LBS in complex indoor environments due to easy implementation and no requirements for extra infrastructure.Researchers have explored multiple Wi-Fi characteristics to estimate user locations, including Received Signal Strength Indication (RSSI) [1], Channel State Information (CSI) [2], Time of Arrival (TOA) [3], and Angle of Arrive (AoA) [4].
In recent years, a new protocol of Fine Time Measurement (FTM) has been standardized by IEEE 802.11-2016 to provide meter-level positioning accuracy [5] using the new technology of Round-Trip Time (RTT).Such a protocol is able to calculate the distance between the RTT-enabled smartphone and the Access Point (AP) by measuring the round-trip time of flight of the signal.Compared to other techniques (such as TOF), Wi-Fi RTT does not require clock synchronization between APs and smartphones.Wi-Fi RTT shows great potential to provide indoor positioning service for mobile users.Some researchers have shown efforts to calibrate the signal or develop reliable positioning algorithms in recent years [6][7][8][9][10][11][12][13][14][15][16].
In 2018, Google introduced the Wi-Fi API to the Android system for smartphones running Android 9 (Pie) or later [17].Promoted by Google, new smartphones from many manufacturers with up-to-date Android systems support Wi-Fi RTT.Such smartphones can send Wi-Fi ranging requests to nearby APs to measure their distance according to the Wi-Fi signal traveling time in a very short latency.In addition, Google has also introduced some apps for developers and researchers to test and explore the RTT signal for distance estimation and positioning use.Wi-Fi RTT has great potential to become a prevailing positioning method in the near future.
Although Google claims that Wi-Fi RTT provides highly accurate distance estimations (meter-level accuracy), the error in Wi-Fi RTT distance measurements varies according to different environments and settings.The study from Horn [18] shows that the Wi-Fi RTT distance can be affected by hardware and position.It also suggests that position-dependent error caused by user motion usually leads to a significant error in the ranging results.Moreover, the non-line-of-sight/line-of-sight (NLOS/LOS) condition is also the main factor that leads to inaccurate Wi-Fi RTT distance measurements.Yu et al. [19] conducted some experiments to block the RTT signal using different materials of blockers to evaluate the Wi-Fi RTT distance measurements in NLOS conditions.However, the experiments and analysis are not enough to cover the possible situations in complex indoor environments, particularly when considering the complicated settings of different user devices, motion status, and signal pass losses.
Therefore, this study explores the errors of Wi-Fi RTT distance measurements for mobile localization with extensive experiments.The contributions are as follows:

•
This paper fully investigates, analyzes, and summarises four categories of errors in Wi-Fi RTT distance measurements, including hardware-dependent bias, blockerdependent bias, fluctuations, and outliers.

•
Two cases of keeping the smartphone static or varying its position during continuous measurement of the Wi-Fi RTT distance are stipulated to analyze the errors against user movements.• Different materials are used to block the signal path to evaluate the errors against complex NLOS conditions.
The rest of the paper is organized as follows.In Section 2, we introduce the background knowledge of using RTT signals to measure the distance between RTT-enabled smartphones and APs.Capable hardware and software proposed in recent years for such research topics are summarised.In Section 3, various experiments are set to investigate the errors in Wi-Fi RTT distance measurements.The experimental results are illustrated and analyzed in Section 4. Conclusions and future works are made in Section 5.

Fine Time Measurement Protocol
FTM protocol is able to estimate the distance between the smartphone and the AP by calculating the round-trip time of flight of the signal.As shown in Figure 1, the smartphone sends one Initial FTM Request (iFTMR) to the AP and receives an acknowledgment (ACK) at the beginning.Then, the AP gives a burst containing at most 31 FTMs (excluding the iFTMR) with the timestamps recording when the signal is sent and received.Based on this information, the round-trip time of flight of the signal can be calculated through the equation: Hence the distance between the smartphone and the AP can be calculated by: where c denotes the speed of light (c = 3 * 10 8 m/s).

Ranging with Wi-Fi RTT in Android
The protocol of FTM has been implemented in Wi-Fi RTT API and introduced since Android 9 (API level 28).Such an API can estimate the distance from the RTT-enabled smartphones to RTT-enabled Wi-Fi access points [17] and other neighboring RTT-enabled smartphones [21].As officially claimed by Google, more than 40 flagship models of smartphones from different manufacturers are RTT-enabled, such as Google, Xiaomi, LG, Samsung, and Sharp [17].Particularly, all the phones from the Google Pixel series (from Google Pixel 1 to Google Pixel 6 Pro) support such functions.Moreover, there are also multiple retail, warehousing, and distribution center devices that support Wi-Fi RTT, such as Zebra PS20 and Skorpio X5 [17].In contrast, the only three consumer APs that officially claim to support RTT are the APs from the Google Wi-Fi series, including Google Wi-Fi, Google Nest Wi-Fi Router, and Google Nest Wi-Fi Point [17].Although Google also officially claims that Compulab WILD AP developed by Compulab [22] is RTT-capable, it is not recommended for new projects and is not available as advertised on its website [22].There are also some other enterprise options from Aruba that are RTT-enabled.Nevertheless, only three models of Google APs have proven standard compatibility officially and are available to consumers at the moment, which narrows the options.
Besides the hardware, some apps have been introduced by Google and other developers for exploring the Wi-Fi RTT API with Android smartphones.The first app developed by Google, named WiFiRttScan, can discover nearby RTT-enabled APs and estimate the distance using Wi-Fi RTT API [23].It can provide various information of the discovered AP, such as BSSID, SSID, RSSI, range, mean range, and range standard deviation.It also supports data logging by storing the measurements/estimations in an Excel file.The code of the original (old) version of this open-source project can be found in GitHub [24].Israel and Enrica [25] developed a new app based on WiFiRttScan by adding some new functions, such as conducting sampling campaigns and showing RTT capabilities of all nearby APs.However, one major limitation of these two apps is they cannot handle the simultaneous reading of multiple nearby RTT-capable APs.Compulab released one app named WILD Minimal [26] which can read from multiple APs at the same time but with limited information, including BSSID, range, range standard deviation, and RSSI.In June 2021, Google released a new app named WiFiRttLocator to demonstrate indoor positioning with Wi-Fi RTT [27].Rather than only providing distance estimations, such an app can even provide location information by calculating from multiple (at least three) Google APs.However, the complex configuration of the app should be completed before starting the Wi-Fi scanning.This includes locating the floor plan on the map, obtaining precise ground truth (latitude and longitude) of the APs, and so on.A new Android app named FTMRTT developed by Horn was released [28] in 2022.The location of the APs can be provided in any convenient Cartesian coordinate system (in meters) in the app, which is much easier to configure than WiFiRttLocator.Moreover, this app supports more functions, such as showing the measurement error and working with uncooperative APs in one-sided RTT mode [29].

Experiment Devices and Software
Many new Android-powered smartphones support the Wi-Fi RTT feature.This study chose three smartphones from different manufacturers as the initiator.This includes Google Pixel 2 (from Google), Xiaomi Redmi K20 (from Xiaomi), and LG V50S (from LG).All of them were embedded with an Android 9+ system to support the Wi-Fi RTT feature.More details can be found in Table 1.However, as discussed in Section 2.2, Google Wi-Fi is the only standard with proven compatibility available to consumers at the moment.Therefore, we chose the latest Google Nest Wi-Fi in this study.The AP operated in simultaneous dual-band (2.4 GHz/5 GHz, only 5 GHz supports Wi-Fi RTT).
To collect Wi-Fi RTT measurement samples, we installed the Android app named WiFiRTTScan (developed by Google) on each smartphone to send ranging requests and receive ranging results.
All the data processing, analysis, and algorithms were operated using Python 3.8 and Matlab on a desktop with an Intel i7-9700 CPU (3.00 GHz) and 32 GB memory.

Experimental Sites
Our study mainly investigated the measurement error caused by blockers (with different materials), phones, communication distance, and motion.All the experiments were in open outdoor space to evaluate the single source of interference or some of their combinations on the accuracy of Wi-Fi-RTT distance measurements.In general, we had three different experimental sites:

•
Range and position-fixed environment (RFPF): considering a sampling period of t s, the smartphone was fixed at a certain position at a distance d m to the AP.• Range-fixed and position-varied environment (RFPV): considering a sampling period of t s, the smartphone was held by a pedestrian (volunteer) while the distance between the pedestrian and the AP remained at d m to the AP.

•
Range and position-varied environment (RVPV): considering a sampling period of t s, the smartphone was held by a pedestrian (volunteer) while the distance between the pedestrian and the AP varied between d m to d m .
All the experimental sites were set outdoors, away from other objects (except the ground).The samples were collected in both LOS and NLOS conditions by setting different blocks in each site.The LOS between the transmitter and receiver antenna is the central direct transmission path line of an ellipsoidal-shaped region in space.Such a long ellipsoidal-shaped region is also called the Fresnel zone.To ensure the LOS condition, the primary (first) Fresnel zone at the radius of R should be ideally 60-80% clear of obstacles.The radius R can be calculated as follows: where d 1 and d 2 represent the distance from the transmitter to the blocker and the distance from the blocker to the receiver, and λ denotes the wavelength.As d 1 and d 2 are in the range of 1 m to 5 m and 1 m to 9 m, and d 1 + d 2 is less than 10 m in this study (will be illustrated later), the maximum R can be calculated to about 0.39 m when d 1 = d 2 = 5 m.So, the AP was placed 0.74 m above the ground to reduce the influence from the ground.
To obtain the NLOS condition, the ideal radius of the blocker should yield the maximum R of 0.39 m.As shown in Figure 2, the blocker was supported by a wooden frame and placed 0.4 m above the ground.The main body of the blocker was in the shape of a 1.2 m × 0.8 m rectangle with a thickness of 2 mm.With such a design, the blocker can sufficiently block the first Fresnel zone.We swapped the main body of the blocker to different materials, including glass, wood and metal for each experiment in each site.
The blockers' settings in each environment are shown in Figure 3.The detailed illustration of each environment is as follows.

Range and Position-Fixed Environment
As shown in Figure 4a, a LOS condition in the RFPF environment was first constructed.The smartphone and the AP were placed 0.74 m above the ground to reduce the reflection.We can observe from the figure that no obstacle blocked the signal transmission between the smartphone and the AP.In addition, we placed the smartphone at a specific ground truth distance dm to the AP (d = 1, 2, . . ., 10).We collected at least 3000 samples at the sampling period of 250 ms at each distance.
Based on such settings, we then constructed an NLOS condition by adding a blocker at the transmission line between the smartphone and the AP, as shown in Figure 3a.
Similarly, at least 3000 samples at the same sampling period of 250 ms were collected at each distance as shown in Figure 4b.After collecting enough samples at all distances, the material of the blocker was changed, and the same sample collection procedure was repeated.In all environments in this paper, three different blockers were used: wood, glass, and metal.
In both LOS and NLOS conditions, the sample collection procedures were conducted repeatedly for all three smartphones.

Range-Fixed and Position-Varied Environment
A volunteer was employed to hold the smartphone and collect the samples following a specific route in this environment.Similarly, a LOS condition was first set in this environment.As shown in Figure 5a, a circular route was stipulated surrounding the AP at the radius of d m so that the range between the smartphone and the AP was fixed (d = 1, 3, 5, 7, 9) while the smartphone's position changed.At each distance, samples were collected at the sampling period of 200 ms for at least 250 s.This data collection provided us with more than 1000 samples in each case and ensured that the entire circular route could be finished at least once.
Then, an NLOS condition was constructed by setting the same blockers used in RFPF.As shown in Figure 3b, different from the NLOS settings in RFPF, we added three more blockers to entirely block the signal transmission to all directions.Identical to the LOS condition, we collected samples at the sampling period of 200 ms for at least 250 s for each circular route.Once all routes had been finished, the material of the blocker was changed and the same sample collection procedure was repeated.
Similarly, we adopted the same sample collection procedures for all three different smartphones in both LOS and NLOS conditions.

Range and Position-Varied Environment
The range between the smartphone and the AP was varied in this case.Specifically, the volunteer was asked to hold the phone and walk away from the AP from 1 m to 10 m.As shown in Figure 3c, the volunteer held the smartphone pointed toward the AP and kept this pose when collecting samples.One complete collection of samples from 10 m to 1 m with one smartphone took 20 s, while the walking speed of the volunteer remained approximately constant.The sampling period was set as 200 ms.Similar to the previous environments, we swapped the smartphones and the blockers to collect samples in both NLOS and LOS conditions.The schematic of this environment is shown in Figure 6.

Evaluation Metrics
Some metrics were used to quantify the error between the measurements (RTTbased distance measurements) and the ground truth to evaluate the measurement noise.Specifically, Root Mean Squared Error (RMSE) was chosen as the main metric for all evaluations.The RMSE of n samples is calculated by: where d m,i denotes the ith measurement, and d g,i stands for the ith ground truth distance.

Verification of the Effectiveness of the Blockers
As illustrated in the experimental settings in Section 3.2, our calculation showed that the designed blockers should block the direct transmission path and attenuate the signal.To further verify the effectiveness of the blockers, we conducted a preliminary experiment by setting the smartphone and the AP at both sides and 5 m away from the blocker.We compared the RSSI values from the smartphone in LOS (no blocker) and NLOS (blocker with different materials) conditions.The mean RSSI is listed in Table 2.Note that the signal strength decreased in NLOS conditions with all blockers and smartphones.The results verify that the blocker's design and settings can effectively create an NLOS condition to attenuate the signal.

Range and Position-Fixed Environment
The samples collected in LOS in RFPF are shown in Figure 7.The Wi-Fi RTT distance usually has a negative bias to the ground truth distance.The mean of each set of Wi-Fi RTT distance at different ground truth distances was then calculated.It can be seen from Figure 8a-c that the slopes of the Wi-Fi RTT distance measured by all three smartphones are almost one.This means that the Wi-Fi RTT distance changes linearly with the ground truth distance.Hence, a positive offset was applied to each curve to compensate for such a negative bias.The offsets for Google Pixel 2, Xiaomi Redmi K20, and LG V50S were 0.45 m, 1.85 m, and 0.24 m, respectively.The RMSE between the Wi-Fi RTT distance and the ground truth distance at each distance was calculated.As it can be seen from Figure 8d-f  Next, we investigated the NLOS effects on Wi-Fi RTT distance.As illustrated in the experimental settings, we set different materials of blockers, including wood, glass, and metal.We can observe from Figure 9a-c that the Wi-Fi RTT distance still varies linearly in NLOS but has a different bias with different blockers.One fact that leads to the bias could be the diffraction effects at the edges of the finite blocker.Since the blocker was set to block the primary Fresnel zone, the signal can diffract around the blocker, which causes losses.The reflections from the blockers and the ground should also be considered.
Moreover, we can observe from Figure 9d,e that the RMSE remained stable in LOS condition, while the RMSE increased as the ground truth went further in the NLOS condition.Moreover, for all smartphones, the metal blocker impeded the transmission of the RTT signal the most among the three materials.The glass blocker showed moderate effects on Wi-Fi RTT distance, while the Wi-Fi RTT distance blocked by wood showed the lowest RMSE.This could be caused by the skin effects.The Wi-Fi RTT signals propagate as electromagnetic waves in the air at approximately the speed of light .The skin effects mean when Wi-Fi signals carried by electromagnetic waves pass through some blockers or media, some will stay on the surface of the blockers, and the rest can pass through and keep propagating to the destination.Materials with better conductivity would attenuate more Wi-Fi signals.

Range-Fixed and Position-Varied Environment
Figure 10 presents the raw Wi-Fi RTT distance measured in LOS condition in RFPV.It can be seen from the figure that there were severe fluctuations with many outliers in the measurements when the smartphone was moving and the ground truth was fixed.We can also infer that the noise here may not be Gaussian; hence, some compensation algorithms assuming Gaussian error may not apply (such as the Kalman filter).In addition, the outlier

Range and Position-Varied Environment
The volunteer was asked to hold the smartphone and walk away from the AP in this experiment so that both the position and the distance between the smartphone and the AP were varied.In this condition, the ground truth distance was not available as it varied according to the time even though we controlled the walking speed of the volunteer as uniformly (stable) as possible.Hence, Wi-Fi RTT distance was compared to the reference distance.Similar to previous experiments, the offsets were applied to the Wi-Fi RTT distance.The RMSE between the Wi-Fi RTT distance and the reference distance were calculated and listed in Table 4.It can be seen that the RMSE is similar when the signal is in LOS condition or blocked by wooden and glass blockers.However, a huge error occurs when the blocker is metal.As visualized in Figure 12, compared to the reference distance, the Wi-Fi RTT distance shows some acceptable variations when the smartphone is moving away from the AP in LOS condition.There are also some expected outliers shown in NLOS condition with wooden and glass blockers.Nevertheless, it can be seen that there is a significant number of outliers in the NLOS condition with metal blockers.As the standard deviations are listed in Table 5, the outliers show a severe impact on the stability of the Wi-Fi RTT distance measured by all three smartphones when the metal blocker blocks the signal.Unlike the fluctuations caused by the motion of smartphones, such variations are mainly caused by the outliers, which are far from the ground truth.Although such outliers are usually the minority in the measured Wi-Fi RTT distance in one journey of each smartphone, their impact on average error is huge.It can be concluded that when the smartphone is in this environment, the dominant error becomes the outliers rather than the fluctuations.It can also be inferred that the NLOS effects could be exacerbated by the motion effects, which lead to high Wi-Fi RTT distance dispersion.In addition, according to the skin effects in the NLOS condition, the higher the conductivity of the blocker, the more outliers would appear in Wi-Fi RTT distance.Moreover, it can be seen from the results that LG V50S provides the lowest error in most of the cases.We infer that the phone embedded with a better CPU performance could provide more stable and accurate measurements in such a critical environment.

Summary
To sum up, the errors caused by the hardware, motion effects, and NLOS effects have been investigated.In different conditions, the errors show distinct effects on the Wi-Fi RTT distance from several aspects, including fixed biases, fluctuations, outliers, and their mixture.As summarized in Table 6, the bias always exists in all conditions and varies according to different hardware/manufacturers.Additional bias occurs when the smartphone is in NLOS condition.Particularly, the bias would be larger when the blocker has better conductivity.Moreover, the Wi-Fi RTT distance shows higher fluctuations when the smartphone is moving rather than in static mode.The motion effects usually lead to moderate fluctuations regardless of the ground truth distance in LOS conditions.However, the motion effects exacerbate the NLOS impact on the Wi-Fi RTT distance, resulting in severe outliers.All the errors except the hardware-dependent bias increase when the blocker has better conductivity in all circumstances.In addition, we infer that the phone with a better CPU could provide better RTT performance against the fluctuations and outliers when the phone is moving in an NLOS environment.

Hardware-Dependent Bias † Blocker-Dependent Bias Fluctuations Outliers
Range-fixed, position-fixed, LOS --Range-fixed, position-varied, LOS -Range-varied, position-varied, LOS -Range-fixed, position-fixed, NLOS * Range-fixed, dynamic, NLOS * Range-varied, position-varied, NLOS * † This bias varies depending on different manufacturers (hardware).* In the NLOS case, the bias, fluctuations, and outliers may be related to the skin effect (conductivity) of the blockers.In this study, we infer that higher conductivity usually results in a higher error in Wi-Fi RTT distance.The number of denotes the error level (the more the higher).

Conclusions and Future Works
This study has fully explored, analyzed, and categorized the errors in Wi-Fi RTT distance with commercial consumer RTT-enabled smartphones and APs for mobile localization algorithms.The paper first summarized and compared the supporting hardware and software for development and research in recent years.Then, multiple experiments were conducted to investigate the errors in Wi-Fi RTT distance measurements with smartphones.Various experimental settings were made to evaluate the impact of hardware, motion status, and NLOS/LOS conditions on the errors in Wi-Fi RTT distance measurements.Through the comprehensive analysis of the experimental results in different sites, this paper summarized four errors in Wi-Fi RTT distance.The errors include hardware-dependent bias, blockerdependent bias, fluctuations, and outliers.The hardware-dependent bias varies according to different hardware/manufacturers of the smartphones; the blocker-dependent bias is caused by NLOS effects; high fluctuations usually exist when the smartphone is moving, while the mixture of NLOS and motion effects results in severe outliers.In addition, experiments with different materials of blockers showed that all nonhardware-dependent errors increase when the blocker has better conductivity.
Our future work will mitigate such errors to provide better distance estimation and positioning accuracy in complex indoor environments.

Figure 2 .Figure 3 .
Figure 2. Specification of the frame and the blocker (metal in this figure).
, the RMSE of the compensated Wi-Fi RTT distance is very close to 0 m regardless of the change of the ground truth.The offsets were applied to all the measurements in the following experiments to better visualize and analyze the errors caused by the motion and NLOS effects.

Table 1 .
Specifications of the selected smartphones.

Table 2 .
Preliminary experimental results of the average RSSI (dBm).

Table 4 .
Comparison of the RMSE in RVPV environment.

Table 5 .
Comparison of the standard deviation of the Wi-Fi RTT distance in RVPV environment.
The bold number denotes better performance.

Table 6 .
Summary of different errors and their impacts on Wi-Fi RTT distance.