Accuracy and Reliability of Local Positioning Systems for Measuring Sport Movement Patterns in Stadium-Scale: A Systematic Review

The use of valid, accurate and reliable systems is decisive for ensuring the data collection and correct interpretation of the values. Several studies have reviewed these aspects on the measurement of movement patterns by high-definition cameras (VID) and Global Positioning Systems (GPS) but not by Local Positioning Systems (LPS). Thus, the aim of the review was to summarize the evidence about the validity and reliability of LPS technology to measure movement patterns at human level in outdoor and indoor stadium-scale. The authors systematically searched three electronic databases (PubMed, Web of Science and SPORTDiscus) to extract studies published before 21 October 2019. A Boolean search phrase was created to include sport (population; 8 keywords), search terms relevant to intervention technology (intervention technology; 6 keywords) and measure outcomes of the technology (outcomes; 7 keywords). From the 62 articles found, 16 were included in the qualitative synthesis. This systematic review revealed that the tested LPS systems proved to be valid and accurate in determining the position and estimating distances and speeds, although they were not valid or their accuracy decreased when measuring instantaneous speed, peak accelerations or decelerations or monitoring particular conditions (e.g., changes of direction, turns). Considering the variability levels, the included studies showed that LPS provide a reliable way to measure distance variables and athletes’ average speed.


Introduction
Electronic Performance and Tracking Systems (EPTS) are divided into Local Positioning Systems (LPS), multiple high-definition cameras (VID) and Global Positioning Systems (GPS) [1]. Moreover, LPS and GPS based sensors are included in a Wireless Body Sensor Network [2][3][4], which is a group of wearable sensor nodes and which can include other types of sensors (e.g., microelectromechanical electronic search was computed from three databases (PubMed, Web of Science and SPORTDiscus) to identify articles published before 22 October 2019. The authors were not blinded to journal names or manuscript authors. We created a Boolean phrase to include population (team sport, soccer, football, futsal, basketball, rugby, handball, hockey), terms relevant to the intervention technology (UWB, ultra-wide band, LPS, local positioning system*, LPM, local position measurement*) and measured outcomes (agreement, accurate, accuracy, precision, reproducibility, reliability, validity). Groups of keywords (population, technology and outcomes) were connected with OR within each group and using AND to combine the three groups.

Selection of Studies
One of the authors (MRG) downloaded the main data from the articles (title, authors, date and database) to an Excel spreadsheet (Microsoft Excel, Microsoft, Redmond, WA, USA) and removed the duplicate records. Then, the referred authors (MRG, JPO, ALA) screened search results independently against inclusion/exclusion criteria. The authors were not blinded to the title or authors of the publications. Any disagreements on the final inclusion-exclusion status were resolved through discussion in both the screening and excluding phases, and the final decision was through agreement among the authors.
Abstract and conference papers from annual meetings were not included because of rigor in outcome measures. If we had any questions about the application of the inclusion-exclusion criteria, we requested further information from the authors. The additional information provided by the authors was considered during the screening process. Lack of additional information led to the article being excluded. Documents from all languages were included unless the translation could not be made.

Identification and Selection of Studies
A total of 62 documents were initially retrieved from SPORTDiscus (n = 11), PubMed (n = 16) and Web of Science (n = 35), of which 25 were duplicates. A total of 37 articles were screened. Next, the full texts and abstracts of the remaining articles were evaluated and 21 were removed because they did not report validity and reliability of LPS in team sports. Finally, 16 studies were included in the qualitative synthesis ( Figure 1).

Assessment of Methodological Quality
The quality of the included studies was individually assessed based on the information provided in the method section using the Rico-González et al., [21] checklist for the use of LPS technologies. Among the articles included in this systematic review (n = 16), 5 provided 29% of the required criteria, 3 provided 33%, 2 provided 38%, 2 provided 43%, 2 provided 48% and another 2 provided 52%.

Assessment of Methodological Quality
The quality of the included studies was individually assessed based on the information provided in the method section using the Rico-González et al., [21] checklist for the use of LPS technologies. Among the articles included in this systematic review (n = 16), 5 provided 29% of the required criteria, 3 provided 33%, 2 provided 38%, 2 provided 43%, 2 provided 48% and another 2 provided 52%. (Appendix A).
All articles assessed the precision of LPS technology on the measurement of kinematic variables such as time-motion at different intensities during linear and nonlinear locomotion. The precision was assessed comparing the LPS with a criterion measurement device (considered as a gold standard in each article). Among them, tape measurement (n = 4), timing gates (n = 5), trundle wheel (n = 1), VICON optic-system (n = 4), Laser measurement (LAVEG) (n = 1) and Geographic Information System (GIS) (n = 1) were used as comparison methods. Moreover, one article analyzed the accuracy of UWB to measure collective tactical behavior variables (i.e., surface area), comparing its validity using GIS [24] (Appendix B).

Discussion
The use of valid, accurate and reliable systems is decisive for ensuring the data collection and correct interpretation of the values. Accuracy and validity can be defined as how close a measurement is to the exact or true value that is intended to be measured; thus, are really important factors to be considered in using location-based systems. Reliability can be understood as the capacity of an instrument or a measure to be repeatable or reproducible on repeated occasions [26]. With such an idea in mind, the purpose of the present systematic review was to summarize the evidence about the validity and reliability of LPS technology to measure movement patterns in outdoor and indoor environments. In the 16 included articles, the major topics that prevail were about validity, reliability and accuracy levels of the LPS technology. One article also tested interchangeability. This section will be organized according to the articles that tested the validity, reliability and accuracy of the LPS system to simplify and organize the discussion.

Accuracy and Validity of LPS Systems
One of the first LPM systems (45-Hz; 19 antennae) to be tested [33] revealed that, in static conditions, the average positional error was 1 cm, while in dynamic conditions the LPM underestimated distances for almost all courses varying from 0 (sprinting straight) to 29 cm (combined course while walking).
Using a similar frequency (45-Hz of the Inmotio) and smaller number of antennae (N = 12), a mean absolute error of all position estimations was found of 0.234 m [22]. The LPS of Catapult ClearSky T6 was tested by two studies [38,39] in indoor conditions, despite using different methodological approaches considering that in the study by Serpiello et al. [39] there were 18 antennae and a 10-Hz sampling frequency while in the study by Luteberget et al. [38] there were 16 antennae and a 20-Hz sampling frequency. In both studies [38,39], the LPS was compared with retroreflective-marker-based systems (VICON and Qualisys). In the study by Serpiello et al. [39], the comparisons in linear locomotor activities revealed mean differences between ClearSky and Vicon with bias between 0.2 and 2.3%. The mean differences between systems in the total distance, mean and peak speed and mean and peak accelerations ranged from 0.2 to 12%; however, for the case of mean and peak decelerations differences reached 84% [39]. In the other study testing ClearSky vs. Qualisys conducted by Luteberget et al. [38], mean differences were found for all position estimations of 0.21 m (in optimal conditions) and 1.79 m (in suboptimal conditions). For comparisons of distances, the mean differences were 0.31 m (for optimal conditions) and 11.42 m (for the suboptimal conditions) while instantaneous speed had mean differences between 34.8 and 39.2% in optimal conditions and 74.4 and 90.8% in suboptimal conditions [38]. Summarizing the evidence of both studies relative to ClearSky, validity is acceptable for measuring position, distance, speed and acceleration, although instantaneous speed and decelerations are not accurate enough due to large differences obtained in comparison to gold-standard methods.
In addition, testing a LPS (Kineson One, version 1.0) using 12 antennae and a 20 Hz sampling rate, the study conducted by Hoppe et al. [31] revealed typical error of estimation (TEE) for criterion variables within a circuit between 0.1 and 1.9. In the same study, the LPS system was also compared with a 10and 18-Hz GPS system [31]. Overall, better validity values of LPS were found for determining distances covered and sprint mechanical properties, although the LPS system presented more outliers due to measurement errors compared to the 10-Hz GPS [31]. The NBN23 LPS system (Nothing But Net model) was also tested for its validity [30]. The system consisted of 12 antennae and frequencies between 9 and 50-Hz. The mean absolute error for distance variables varied between 0.10 m (in walking) and 0.18 m (in running), suggesting good values of validity [30]. For the case of time variables, mean absolute error varied between 0.2 s (at walking) and 0.14s (at walking). Time presented moderate to very high correlations [30]. The NBN23 LPS system revealed validity for monitoring distance and running time, although intensity affects the accuracy of the system [30]. Validation of an LPM system using glass fiber technology was also conducted using 19 antennae and 45 Hz [33]. Comparing average course speed to the average actual course speed, correlations were found (r = 0.71 to 0.97). Despite that, a systematic error of LPM was found in lower speed compared to actual speed [33]. Differences between LPM and actual speed were between −1.3 and 3.9% [33]. Finally, the TEE revealed a clear increasing tendency following the increase in the speed, thus TEE at low speed was more stable and less variable than in sprinting conditions [33]. Considering the values, it was possible to determine the validity of the system for measuring distance and speed [33]. A LPS using a Wireless Ad hoc System for Positioning (WASP) using 12 antennae and a sampling rate of 10-Hz was tested for its validity [32]. The results for mean error (%) varied between 1.26 (walking distance in a linear course) and 3.87% (sprinting distance in a nonlinear course). Results in indoor and outdoor conditions were consistent and revealed validity [32]. One of the included studies proposed to analyze the interchangeability of a multicamera, semiautomatic system, LPM (Inmotio) and GPS units [29]. Comparing the distance run at different speeds, the Inmotio tended to largely and moderately underestimate the distances run at 7.2 and 14.4 km/h −1 , respectively, while the multicamera system and the GPS tended to overestimate the distance run at all intensities [29]. In the study, the authors [29] proposed calibration equations for interchangeability of the systems, revealing that most of the calibration equations calculated were associated with small-to-moderate typical errors of the estimate.
An ultra-wide band (UWB) from RealTrack systems was tested in indoor conditions for its accuracy revealing mean absolute error of all position estimations of 5.2 cm (0.97%) for the x-position and 5.8 cm (0.94%) for the y-position [17]. Additionally, the estimation of errors was between 2.1 and 8.3 cm on the x-axis and 3.5 and 8.2 cm on the y-axis [17]. The results of the study [17], suggested acceptable accuracy levels of the UWB for monitoring the position of players. The same UWB (RealTrack systems) tested in outdoor conditions [24] revealed a mean absolute error of 41.23 and 47.6 cm for x-axis and y-axis, respectively. The findings confirmed the high accuracy and high transmission path of the UWB, mainly considering comparisons with GPS systems [24]. A third included study [23], testing the UWB from RealTrack systems showed a bias (%) of 0.55 to 5.85% for determining distance covered, and, moreover, a bias between −0.56 and 0.67 for determining mean velocity [23]. An additional comparison with GPS also revealed the better accuracy of UWB [23]. In brief, the studies [17,23] testing the accuracy of UWB of RealTrack systems showed a good accuracy of the system to determine players' positions, distances covered and mean velocities. An alternative brand of UWB (Ubisens Series 7000 compact tag) was also tested for its accuracy [36], also showing sufficient accuracy to test positions of players independently of the length of the recorded runs.
Summarizing the evidence about validity of LPS, all the tested systems (e.g., Catapult ClearSky T6; Kineson One; NBN23; WASP; LPM using glass fiber technology; Inmotio) revealed mean error below 5% measuring distances and average speeds, although not in measuring instantaneous speed and decelerations. It was clear that all studies confirmed good and acceptable accuracy of LPS systems to estimate the position and the distance and velocities achieved by players, although a decrease in accuracy occurs in some conditions (e.g., turns, changes of direction, sport-specific actions) and intensities (e.g., peak accelerations or decelerations).

Reliability
Commonly, reliability can be tested determining the within-subject variation, changes in the mean and retest correlation [26]. Reliability of the measures are critical for LPS systems, mainly to ensure the consistency and allow comparisons over time and in a repeated way (ref). The most common tests to be applied in reliability analysis are the coefficient of variation or typical error of measurement (TEM) and, in some cases, the intraclass correlation test (ICC) [27]. Following the suggestions of [27], reliability can be interpreted as good for variability lower than 5%, moderate between 5 and 10% and poor for 10% or above.
From the included studies of this systematic review, four of them [17,24,30,31] proposed to test the reliability levels of LPS systems. A 20-Hz LPS system (Kinexon One) using 12 antennae was tested by Hoppe et al. [31], revealing typical errors between 0.1 (criterion variable of 10m jogging with jump) and 1.7 (criterion variable of 129.6 m entire circuit). The LPS revealed good reliability for the entire distance covered, walking over 10 m with change of direction (COD), sprinting with CODs, sprinting over 30-m, sprinting over 5-20 m and theoretical maximal force and horizontal power [31]. However, in comparison to the GPS tested (10-Hz and 18-Hz), the LPS revealed greater noise at distances covered during standing, mainly caused by a shift in the zero-velocity line and increase in the velocity due to performed turning maneuvers [31]. Despite that, comparisons of reliability between the GPS and LPS was mainly favorable to LPS [31]. An UWB from RealTrack Systems was tested for its intra-and interunit reliability [17]. The intraunit reliability of UWB in mean velocity varied between 0.895 and 0.999 of ICC (95% of confidence interval) and the low and upper (for interunit variability) ranged between −0.09 and 0.42%. In the case of distance covered, the typical error of UWB varied between 0.94 and 4.87% and the lower and upper bias was between −2.65 and 2.06%. Thus, it was concluded that the UWB was reliable for distance covered and mean velocity [17]. Another study testing interunit reliability of UWB of the RealTrack system presented ICC values of 0.65 and 0.88 for x-and y-axis, respectively [17]. The NBN23 LPM system (Nothing But Net) was tested for its reliability. The coefficient of variations for walking, running and sprinting was 1.1-3.0%, 0.9-4.1% and 0.6-4.3%, respectively [30]. Comparisons between the LPM system and the taped measurement were also conducted, showing that the differences between the trials only varied for the 0-15 m at walking speed and interparticipant differences were found at 0-10 m in walking. Thus, results of the study showed that the NBN23 was reliable for monitoring distance and running time.
Summarizing the evidence regarding the reliability of LPS systems, it is possible to conclude that the three systems (Kinexon One, RealTrack Systems and NBN23) had coefficient of variations below 5% thus revealing reliability for measuring distances covered at different speeds and also for quantifying velocities achieved during the tasks.

Conclusions and Future Issues
This systematic review revealed that the tested LPS systems showed they were valid and accurate in determining the position and estimating distances and speeds, although not being valid or decreasing their accuracy when measuring instantaneous speed, peak accelerations or decelerations or monitoring particular conditions (e.g., changes of direction, turns). Considering the variability levels, the included studies showed that LPS provides a reliable way to measure distance variables and athletes' speeds. Further LPS developments could improve these systems for instantaneous speed, peak accelerations, decelerations, changes of direction or turns. Moreover, more standards for validation and reliability should be identified, aiming to define similar conditions that may allow sports scientists to easily identify the confidence thresholds for the systems. Funding: For the case of the F.M.C., this work is funded by FCT/MCTES through national funds and when applicable co-funded EU funds under the project UIDB/ EEA/50008/2020.

Conflicts of Interest:
The authors declare no conflict of interest. Table A1. Quality assessment of the studies using Rico-González et al. [21] checklist for radio-frequency technologies.

Appendix A
Frencken, Lemmink and Delleman [33]        UWB-20Hz has been recommended as accurate technology for estimating position of players on the pitch, while GPS-10Hz has substantial limitations Significance differences reported in tactical analysis between GPS and LPS that the error of using one system or another can mean a difference of more than 8%. Test-retest reliability and interunit reliability were good for the two systems assessed.

Catapult
Serpiello et al. -The mean differences for distance, mean/peak speed, and mean/peak accelerations in the linear drills were in the range of 0.2-12%, with typical errors between 1.2 and 9.3%. Mean and peak deceleration had larger differences and errors between systems.
LPS had acceptable validity to assess movements. Mean difference = 21 ± 13 cm in the optimal setup, and 179 ± 761 cm in the suboptimal setup.
Distance Average difference = < 2% for all tasks in the optimal condition, while it was < 30% in the suboptimal condition. Instantaneous speed Differences = ≥ 35% in the optimal and ≥74% suboptimal condition The differences between the LPS and reference system in instantaneous speed were speed dependent, showing increased differences with increasing speed.
The accuracy of LPS output was highly sensitive to relative positioning between field of play and walls/corners and anchor nodes. The LPS is not valid in calculating instantaneous speed from raw data.