Performance Evaluation of a UWB Positioning System Applied to Static and Mobile Use Cases in Industrial Scenarios

Abstract

Indoor positioning systems are essential in the industrial domain for optimized production and safe operation of mobile elements, such as mobile robots, especially in the presence of static machinery and human operators. In this paper, we assess the performance of a commercial UWB radio-based positioning system deployed in a realistic industrial scenario, considering both static and mobile use cases. Our goal is to characterize the accuracy of this system in the context of industrial use cases and applications. For the static case, an extensive analysis was presented based on measurements performed at 72 measurement positions at 3 different heights (above, at similar a level to, and below the average clutter level) in different industrial clutter conditions (open and cluttered spaces). The extensive analysis in the mobile case considered several runs of a route covered by an autonomous mobile robot equipped with multiple tags in different positions. The results indicate that a similar degree of accuracy with a median 2D positioning error smaller than 20 cm is possible in both static and mobile conditions with an optimized anchor deployment. The paper provides a complete statistical characterization of the system’s accuracy and addresses the multiple deployment trade-offs and system dynamics observed for the different configurations.

1. Introduction

Positioning systems have gained widespread adoption in recent decades owing to the mass market diffusion of civilian applications benefiting from the localization of objects in space. Global Navigation Satellite Systems (GNSS) such as Galileo or the Global Positioning System (GPS) offer worldwide coverage and are used for geolocation and navigation, providing baseline accuracy levels in the order of meters [1], which can be further enhanced up to the centimeter level by using terrestrial station support [2]. However, the paramount penetration losses of GNSS signals render these systems unsuitable for application in indoor scenarios, where indoor positioning systems (IPS) can be used instead [3]. With an IPS it is possible to mount beacon/tag devices on people, vehicles, and other assets in order to track their position inside buildings, stores, airports, warehouses, etc. However, instead of using satellites, IPS employs anchors (cameras, sensors, or transceivers) that are placed on walls or ceilings, and coordinated from a centralized controller unit.

IPS can be important in the industrial domain as they can help in terms of safety and production flexibility and optimization [4]. By knowing the real-time positions of different factory shopfloor elements such as human workers, production modules, pallets, forklifts or autonomous mobile robots (AMRs), it would be possible to adapt and proactively coordinate their actions, enhancing current operational schemes. For example, it would be possible to vary AMR speeds depending on how far workers and forklifts or other mobile robots are from the AMR. In other words, enabling AMRs to drive at maximum speed when located far from workers and vehicles, and slowing down when in close proximity. Another possibility is using the positions of workers, robots, and vehicles in the path planning of mobile elements in order to optimize logistic routes without interrupting human labor, leading to increased industrial productivity.

There are multiple technologies that can be used to implement IPS. These technologies are mainly radio frequency (RF)-, acoustic-, or optical signal-based. In RF-based systems, Wi-Fi, radio frequency identification (RFID), Bluetooth low energy (BLE), ZigBee, and ultra-wideband (UWB) are the most commonly adopted technologies. In terms of accuracy, Wi-Fi is in the order of 5–15 m [5], BLE achieves 2–5 m [6,7,8], and RFID 1–5 m [7,8,9]. The most accurate RF-based technologies for IPS are Zigbee and UWB, which offer sub-meter and decimeter range accuracy, respectively, [7,10]. For acoustic-based IPS, ultrasonic technology can achieve accuracy in the sub-meter range [7,8,11]. The same applies for optical-signal-based IPS, such as those built with infrared (IR) or visible light communication (VLC) technologies [4]. However, compared to RF- and acoustic-based IPS, optical-based IPS are subject to an important constraint: line-of-sight (LOS) conditions from cameras to sensors or objects are needed, which considerably limit its application.

Different system characteristics are typically considered when selecting the most appropriate IPS for a specific use case in a given scenario, i.e., coverage, scalability, agility of the installation, maintenance, and cost [12]. Leaving aside optical-signal-based technologies, such as IR, that can achieve millimeter-range accuracy under very specific deployment conditions with reduced coverage and LOS; UWB solutions offer extended coverage, even in non-line-of-sight (NLOS) conditions, as compared to ultrasonic-acoustic-based solutions and other RF-based technologies offering sub-meter accuracy such as ZigBee. Further, the nature of UWB technology makes it robust to interference from external systems, while allowing for reliable operation even in the presence of multiple nodes sharing the same UWB spectrum, ensuring good system scalability properties [13]. In terms of installation and maintenance, all IPS are very similar as they require the deployment of a cabled back-end control network between anchors and controller (typically Ethernet-based), and access to power for the anchors, e.g., via dedicated power plug access or via Power-over-Ethernet (PoE). Beacon/tag devices are normally battery-operated, except in the case of the optical-camera-based IPS, where tags are passive elements with specific patterns allowing for image recognition. The duration of the tag batteries depends on the specific positioning communication technology and mode of operation implemented. However, in industrial deployments, the targeted battery life is typically in the order of years for all IPS [14]. Considering hardware-specific prices for anchors, tags, and controllers, as well as the different operational subscription fees and other operational costs, the overall price for a UWB enterprise deployment ranges in the order of USD 10 K–20 K (for a reference scenario of 5000 m²) [15]. The total costs might be slightly higher than those for other IPS, but come with the associated benefits of increased coverage and accuracy.

Based on this, it is clear that UWB-based IPSs are a suitable and attractive solution for deployment in industrial use case scenarios. Within this context, this paper aims specifically at evaluating and verifying the accuracy of these commercial positioning systems in real-world operation conditions for different industrial use cases and deployment configurations. The main challenges faced in the paper are related to the establishment of a proper methodology for collecting and analyzing a large dataset for the performance analysis of localization accuracy in static and mobile cases. These aspects are thoroughly presented in the remainder of the paper.

1.1. Literature Review

This section provides an indepth analysis of studies that evaluate the performance of UWB systems in industrial or industrial-like environments. We categorize the related work into three main categories, those which evaluate UWB-positioning accuracy in: realistic scenarios, simplified scenarios, or simulated environments. In

Table 1. Summary of UWB-positioning accuracy levels reported in the literature considering realistic, simplified, and simulated industrial environments. (P50 = 50%-ile (median), P90 = 90%-ile, etc.).

	UWB System	Anchors	Room [m2]	Tag Height [cm]	2D|3D	GT Points	SPP	Accuracy [cm]
[16]	Woku	8	132	Static N/A Mobile $\sim 150$	2D	9	72 k	<20
[17]	Ubisense	4	110	100	2D	25	892	∼20 (P50)
	Sewio	4	110	100	2D	25	104	∼55 (P50)
[18]	Ubisense	4	150	∼138	3D	8	-	7–25 (open) 12–130 (cluttered)
[19]	Ubisense	6	336	70	3D	72	320	61 (P50)
	BeSpoon	6	336	70	3D	72	40	58 (P50)
	DecaWave	6	336	80	3D	72	56	39 (P50)
[20]	Pozyx	4	112	0–300	3D	9	20	51–86
[21]	Commercial	4	2760	-	2D	36	-	100 (P90)
[22]	Decawave	4–6	20	70	2D	1	∼75	1 (static) ∼4–11 (mobile)
[23]	Non-commercial	6	95	100	2D	32	1000	3.4
[24]	TimeDomain	4	16	∼100	2D	6	∼1250	2–38
[25]	Decawave	4	60	∼100	2D	5	1000	∼ 5–22
[26]	Decawave	4	70	-	2D	2	900	10–22
[27]	Decawave	4	60	∼100	2D, 3D	5	1000	∼ 5–40 (2D) 88 (3D)
[28]	Simulation	8	1050	150	3D	584	-	7–13
[29]	Simulation	8	900	150–400	3D	500	-	7.8

, the analyzed studies are summarized along with relevant details such as: the type of UWB system, the number of anchors in the system, the area covered by the system, the heights of the tag devices, the type of positioning output (2D/3D), the number of ground truth (GT) points and number of samples per point (SPP) considered in the evaluation and, finally, the achieved accuracy.

For realistic scenarios, studies from the literature evaluated UWB systems in real-industrial or pseudo-industrial environments [16,17,18,19,20,21], featuring production lines, robots, metal tables, etc. These studies reported accuracies ranging from 20 cm to more than 100 cm for a variety of both 2D and 3D system configurations. Ref. [16] examined both static and mobile positioning performances in an industrial plant spanning an area of 132 m² using eight anchors. A similar 2D positioning accuracy of approximately 20 cm was observed in both static and mobile cases, when human workers were standing or walking around the plant with a tag hanging around their neck. Ref. [17] evaluated two different UWB systems in an industrial plant covering an area of 110 m² using four anchors. In this case, the system configured with a higher sampling rate, which collected a higher number of samples per point, achieved a 2D accuracy of 20 cm - similar to that reported from the previous study, while the other system presented an increased positioning error close to 55 cm, on average. Ref. [18] evaluated a UWB system in a factory environment spanning an area of 150 m² with four anchors and with a focus on 3D accuracy for different clutter scenarios, finding an accuracy in the range 7–25 cm in obstacle-free areas, reduced to 12–130 cm in obstacle-filled sections. In general, the average UWB-positioning accuracy achieved by the different systems in real-world scenarios falls within the 20–90 cm range as further reported in [19] for a 336 m² warehouse with six anchors. The variation in UWB 3D position accuracy for different deployment configuration and tag heights was evaluated in [20], finding an improvement of approximately 35 cm when tag antennas were located at a height of 3 m, over the 86 cm average accuracy measured when they were located on the ground. Finally, Ref. [21] evaluated a UWB system in a large 2760 m² auction center, using only four anchors. In such poor area-to-anchor ratio situation, the 2D positioning accuracy was quite poor as compared to those reported in the other studies, with large errors of more than 1 m found in approximately 10% of the cases.

For simplified scenarios, studies from the literature were performed in small controlled scenarios under optimal radio conditions. This is the case in [22,23,24,25,26,27], which evaluated UWB systems in relatively small, obstacle-free environments with full LOS conditions to all anchors. These studies represent the best case scenarios for UWB positioning in an industrial environment, and achieved accuracies ranging from 1 cm to 40 cm. We do not believe this kind of environment is directly comparable to a real-world operational industrial environment; however, we still consider them because they provide insight into the achievable accuracy of a UWB system under optimal conditions, which could be fulfilled under very specific controlled settings. In [22], the UWB-positioning system was evaluated in a 20 m² room with 4–6 anchors using a single ground-truth point at a height of 70 cm. In static conditions, an average 2D accuracy of 1 cm was achieved. In mobile conditions, moving a tag around using a wooden cart, the 2D accuracies were roughly 11 cm, 9 cm, and 4 cm using 4, 5, and 6 anchors, respectively. Ref. [23] reported a mean static 2D accuracy of 3.4 cm in a 95 m² empty room using six anchors. Ref. [24] evaluated a small 16 m² environment using four anchors, achieving 2D accuracies in the 2–38 cm range, dropping to 1–3 cm when a person was positioned right next to a tag. Similar 2D accuracies ranging from 5–22 cm were reported in [25,26] for empty rooms measuring 60–70 m² with four anchors. The larger positioning errors in simplified controlled environments were reported in [27], which evaluated the difference in 2D and 3D positioning accuracy in a 60 m² room with four anchors. The study found an approximately 50 cm larger deviation in 3D positioning as compared to 2D positioning, with the accuracy ranging from 5–40 cm.

There are also studies that have reported evaluations of UWB systems in simulated industrial environments [28,29]. These studies, which relied on impractical scenarios with ideal environmental and geometrical conditions, focused more on theoretical evaluations of the system-level capabilities and trends, concluding that sub-decimeter accuracies are possible using UWB technologies in ideal conditions.

1.2. Shortcomings in the Related Work and Paper Contributions

From the literature survey it can be concluded that the expected accuracy for UWB-positioning systems is in the sub-decimeter range for idealized scenarios (which assume either simulations or simplified non-challenging RF deployments). However, in realistic operational scenarios, the reported accuracy of UWB positioning falls into the slightly degraded range of a few decimeters. When carefully considered and compared, it is possible to identify a number of shortcomings in the reported literature. Leaving aside simulations and simplified scenarios, and focusing on the evaluations conducted in realistic environments, it is clear that most of the evaluations focus mostly on static applications and not so much on mobile ones, which are also of interest in the industrial domain. Furthermore, most of the evaluations focus on a single tag height over a few GT points, reporting a single average accuracy and leaving aside statistical variations of the accuracy due to the environment.

These shortcomings render data from the current literature insufficient to form a decision on, for example, whether a mobile robot could be accurately controlled based on UWB positioning, or whether a factory employee needs to be equipped with a UWB tag placed on the shoes, belt, or hardhat. Further, it is not possible to conclude whether the accuracy of a tag on a worker would be maintained when in close or distant proximity to surrounding machinery. Thus, this study aims at filling the identified gaps with the following contributions:

• Extensive empirical performance evaluation of a commercial UWB-positioning system in both static and mobile conditions in a realistic industrial setting.
• Extensive evaluation with focus on static use cases: examining the effects on system accuracy of anchor deployment, tag height, nearby environment, and sample size of the positioning data sets.
• Extensive evaluation with focus on mobile use cases: examining the effects on system accuracy of anchor deployment, and tag position and orientation.
• Extensive statistical analysis addressing the impact of the system’s dynamics on the accuracy.

In summary, this paper bears clear novelty with respect to the presented literature, as it thoroughly addresses the aspects of mobility and parameters such as anchor deployment and tag heights, etc., which were only marginally addressed or disregarded earlier.

The rest of the paper is structured as follows: Section 2 describes the industrial setup and provides details on the static and mobile measurement campaigns. Section 3 presents and analyzes the UWB-positioning accuracy results from the two campaigns, elaborating on the accuracy dynamics of the system for different configurations. Section 4 provides a discussion of the results, putting them in perspective of the literature and expanding on the implications of the reported accuracy and dynamics for operational industrial systems. Finally, Section 5 concludes our study.

2. Materials and Methods

The empirical performance evaluation described in this study was performed at the “AAU 5G Smart Production Lab” at Aalborg University [30]. This laboratory is a small industrial factory hall where automated production areas equipped with real operational industrial machinery such as production line modules, robotic arms, mobile robots, or production cells, are present together with other areas containing metal shelves dedicated to storage, or with tables and workbenches dedicated to manual work. As displayed in Figure 1, such an environment corresponds to a real industrial environment, and even the structural composition of the laboratory resembles that of operational factories with high ceilings, reinforced concrete walls, network infrastructure, ventilation pipes, and power access deployed on walls and ceilings. The size of the laboratory is 14 m × 40 m (560 m²) with an average height of 6 m. The IPS system deployed in the laboratory is a commercial “Enterprise” UWB system by Pozyx [15] consisting of eight anchors that are deployed around the laboratory at a height ranging between 3.4 m and 5.3 m. In total, six anchors are installed on the walls, and two are placed on the ceiling. A detailed overview of the laboratory floorplan with clutter footprint and the location of the anchors is given in Figure 2 and Figure 3. All eight anchors are connected via Ethernet to a centralized local server/gateway in the laboratory for PoE access, time synchronization, and scheduled UWB system data collection. At the device side, battery-powered developer tags were used. A picture of the laboratory including the system and measurement setup is shown in Figure 1.

The UWB system is set to operate in time-difference-of-arrival (TDOA) mode, which means that the local server/gateway calculates the position of the deployed UWB tags by comparing the time of the UWB message readings received from a given tag at the different anchors. The system is configured to provide 2D positioning only (3D positioning would be highly unreliable as anchors are deployed in a single height tier at approximately the same height). In terms of UWB RF signal configuration, the following settings were used: UWB channel 2 (3774–4243.2 MHz), 500 MHz bandwidth (timing resolution of 1.6 ns), 110 kbps data rate, preamble length of 1024 bits, 64 MHz pulse repetition frequency, and 20 dBm transmission power. Further, optimized settings based on movement models and freedom of movement were enabled for the static and mobile use case evaluations. There are two movement models to choose from: predictable and unpredictable. The unpredictable model is used when tags are assumed to have velocity distributed around 0. Thus, this setting is used for our static use case evaluation. The predictable model is used when tags are assumed to have acceleration distributed around 0, which makes it applicable to our mobile use case evaluation. Regarding the freedom of movement, the setting depends on the expected variance of the speed or acceleration of a tag. In our static use case evaluation, there is no variance in speed or acceleration, which is why a weak freedom of movement is used. On the other hand, in our mobile use case evaluation, there is variance in speed and acceleration and, thus, a medium freedom of movement is used.

2.1. Initial Testing for Fine Tuning of the UWB System Configuration

The considered UWB system can be configured for operation in two different modes: high/low speed. The high-speed mode is recommended for short-range positioning with the least power consumption, while the low-speed mode is recommended for long ranges and accuracy. Another aspect regarding the system configuration is the update rate (e.g., the speed at which the tags transmit their UWB signals) which can be set to a maximum update rate of 100 Hz. Prior to the extensive measurement campaign, a sensitivity analysis was performed by sweeping all the possible configuration value combinations at selected calibrated measurement reference positions in order to find the optimal configuration of the UWB system. From this test, the low-speed mode was selected as it offered superior performance (higher accuracy and lower variability of the measurement in static conditions) than the high-speed mode. In terms of tag update rate, the choice was made based on a trade-off between low measurement variability in static conditions and the impact of the sampling time on mobile conditions (which should be as low as possible to ensure safety operation of mobile use cases when UWB is used as positioning method). The trade-off analysis led to an optimal choice of 50 Hz as the update rate, which offered good performance in static positions and will guarantee that a positioning sample is obtained every 2 cm when a tag is moving at a constant speed of 1 m/s.

2.2. Measurement Setup for the Static Use Case Evaluation

For the static evaluation, 72 measurement positions were selected. Out of the 72 points, 47 were classified as open-space, while the remaining 25 were characterized as cluttered-space. The distribution of these measurement points is shown in Figure 2 over a floor plan of the laboratory. Open-space points (green dots) are defined as obstacle-free spaces within the industrial laboratory. These points are far from objects, such as production lines, shelves, and machines. Open-space points are typically placed in aisles or empty storage areas. Cluttered-space points (red dots) are defined as locations with a high density of obstacles in the near vicinity surrounding area of the point within a 1 m radius. These points are close to obstacles such as robot cages, production lines, shelves, or workbenches.

In order to establish the reference coordinate system and ground truth (GT) coordinate positions, all measurement points were measured with a Leica TS16 total station [31], which allows for the accurate measurement of points in space with millimeter precision, by measuring horizontal angles, vertical angles and distances. As LOS is needed between the total station and the points under measurement, the station was placed at different locations in the laboratory to cover all measurement positions at floor level and the anchor positions close to ceiling level. All points were measured several times from different locations, and the GT coordinates of the measurement points were computed based on a least-squares procedure that includes all the measured angles and distances from all the positions. The standard deviation of the estimated GT coordinates of the anchors was less than 5 mm, and the standard deviation of the estimated GT coordinates of the measurement points at floor level was less than 2 mm.

For the static use case, each measurement position (GT point) is evaluated at three different heights using the tripod displayed in Figure 1. The three selected heights of interest are: low (34 cm), medium (98 cm), and high (198 cm). The low height is chosen to resemble the height of an AMR, operating close to the ground level. The medium height corresponds approximately to the height of a production line, representing an average device height within the industrial clutter. Finally, the high height corresponds to a height above average clutter, and can be mapped approximately to the height of a person wearing a safety hat. In order to ensure accurate 2D positioning during the static measurement campaign, the tripod was placed over the reference GT points using a laser reference for precise positioning. For each measurement point, positioning at all three measurement heights was evaluated simultaneously, based on 90 s of measurement per position. As the tags were configured with a sampling rate of 50 Hz, the static use case measurement resulted in 72 GT points × 3 heights = 216 static positions for evaluation, and a total of 216 × 90 s × 50 Hz = 972 k raw measurement samples. Effectively, this number was slightly lower (approximately 815 k–830 k samples) due to unsuccessful positioning of sample computation in some cases, especially in heavily cluttered positions.

2.3. Measurement Setup for the Mobile Use Case Evaluation

For the mobility evaluation, an AMR (in this case, an MiR200 [32], typically used in industry for internal factory transportation and logistics) was set to drive around the laboratory with five tags attached to it at different positions. Four of the tags were placed on the four sides of the robotic platform (front, back, left side, right side) at a height of 34 cm, and one tag was placed on top of the robot at a height of 44 cm, see Figure 1 for further visual reference. The AMR drove at an average speed of 0.5 m/s, over the route/path displayed in Figure 3. The dynamic use case measurement route was driven twice, from one end of the laboratory to the other, and then the other way around, covering a total distance of approximately 200 m. At the configured tag update rate of 50 Hz, 100 positioning samples per meter were obtained on average, resulting in a total of approximately 100 k raw measurement samples available for evaluation of the accuracy in the mobile use case.

For the mobile evaluation, it was not possible to use the same method for GT reference point characterization. Therefore, in this case, the GT was based on the internal positioning data from the AMR. The chosen AMR implements a simultaneous localization and mapping (SLAM) algorithm [33] based on a calibrated reference system and the input from its on-board light detection and ranging (LiDAR) system, cameras, and proximity sensors. In general, the SLAM-based reference GT measurements are accurate within a 0–5 cm range. Since this GT positioning error is in the order of centimeters, the positioning results from this mobile campaign will have a slightly larger uncertainty than the results from the static use case campaign.

2.4. Anchor Deployment Configurations

Different anchor deployment configurations are evaluated in this study in order to examine the impact of deployment planning and optimization strategies on the performance of the UWB-positioning system.

Table 2. Summary of the five anchor deployment configurations used in this study.

Deployment	Anchor Groups	Evaluated in
Setup A (Reference)	{1–8}	static and mobile
Setup B	{1–4}{1–8}	static and mobile
Setup C	{1–3}{1–8}	static
Setup D	{1–4}{2,3,4,6}{1–8}	static and mobile
Setup E	{1–3}{1–8}{4–8}	mobile

summarizes the different anchor deployment configurations considered. In Setup A, all eight anchors are used everywhere in the laboratory. The other four deployments are obtained via selecting a subset of the anchors by splitting them into two independent different operational area subsystems, such as in Setup B and Setup C, or in three area subsystems, such as in Setup D and Setup E. This means, for example, that in Setup B, one part of the laboratory only would use the subsystem with anchors 1–4, and the other part would use the subsystem with anchors 1–8. Similarly, in Setup E, the first third of the laboratory would use the subsystem with anchors 1–3, the next third would use the subsystem with anchors 1–8, and the last third would use the subsystem comprising anchors 4–8. These specific configurations were chosen based on preliminary tests on the reference UWB deployment in terms of RF including LOS conditions and power levels from the different anchors at different positions within the environment. In the next section, we refer to such setups as optimized configurations.

2.5. Accuracy Evaluation Metrics and Data Processing

In order to evaluate the UWB positioning performance, different accuracy metrics are used. For the static evaluation, Equation (1) is used to calculate the non-aggregated 2D Euclidean distance between all the different tag position samples and GT position:

$${\epsilon }_{static,i}=\left|\right|(p-q_i)\left|\right|=\sqrt{{(p_1-q_{1,i})}^2+{(p_2-q_{2,i})}^2}$$

(1)

where:

• ${\epsilon }_{static,i}$ is the Euclidean distance between point p and $q_i$
• p is the ground truth position
• $q_i$ is the ith tag position sample

The result of applying this metric is a vector containing all the estimated positioning errors from all the individual samples acquired at a given position q, which allows for statistical characterization of the UWB system performance and gaining some insight on the potential dispersion of the positioning samples around a GT point in the different evaluated conditions. A different signal processing is possible, which would first consist in combining (averaging) all the tag position samples obtained at a given location q, and then computing a single error sample with respect to the GT point. This is achieved by applying the aggregated accuracy metric indicated in Equation (2):

$${\overline{\epsilon }}_{static}=\left|\right|(p-\overline{q})\left|\right|=\sqrt{{(p_1-{\overline{q}}_1)}^2+{(p_2-{\overline{q}}_2)}^2}$$

(2)

where:

• ${\overline{\epsilon }}_{static}$ is the Euclidean distance between point p and $\overline{q}$
• p is the ground truth position
• $\overline{q}$ is the average tag position estimated from a total number of N tag position samples, calculated as described in Equation (3):

  $$\overline{q}=\frac{1}{N}\sum _{i=1}^N{q}_i$$

  (3)

A visual representation of the two different 2D accuracy metrics is given in Figure 4A,B for Equations (1) and (2), respectively.

For the mobile use case, the metric used is the 2D Euclidean distance between tag position and GT point, with the addition of an offset value to compensate for the horizontal non-collocation of the UWB tag mounted on the AMR and the SLAM-based GT measurement system. This metric is given in Equation (4):

$${\epsilon }_{mobile}=\left|\right|(p-q)\left|\right|-offset=\sqrt{{(p_1-q_1)}^2+{(p_2-q_2)}^2}-offset$$

(4)

where:

• ${\epsilon }_{mobile}$ is the Euclidean distance between points p and q
• p is the AMR SLAM-based ground truth reference position
• q is the tag position
• offset is the fixed distance between the AMRS SLAM-based ground truth point and the UWB tag position

Further, in this mobile case, in order to be able to estimate the positioning accuracy by comparing the readings from the two systems, compensation for the different sampling rates of the UWB and AMR SLAM-based systems is required. As the sampling rate of the AMR SLAM-based positioning was 1 Hz, GT points were linearly interpolated to match the 50 Hz sampling from the UWB measurements. A visual representation of this process is shown in Figure 5.

3. Results

3.1. Static Use Case Evaluation

First, the measurement results were analyzed with regard to the effect of anchor deployment on the overall positioning accuracy. Figure 6 displays the cumulative distribution function (CDF) of the non-aggregated accuracy achieved with the original reference deployment (Setup A in Table 2), as well as for the optimized deployment (Setup B, C, and D in Table 2). In this case, the selection of anchors was calculated on a per ground truth point basis. At each ground truth point, an algorithm computes and analyzes which of the anchor group optimized setups (i.e., Setup B, C, and D) detailed in Table 2 gives the most stable and accurate positioning reading for that point. The original deployment presented a median accuracy of 21 cm, while the optimized one exhibited 17 cm. Thus, a small deployment gain of 4 cm was observed, on average. However, there is a large difference between the original and optimized anchor deployment in the upper 20% of the distributions. While the original anchor deployment exhibits positioning errors in the order of many meters, with the optimized anchor deployment, accuracy levels well-contained below 40 cm are achieved for 90% of the data. For further reference,

Table 3. Summary of non-aggregated accuracy statistics for the industrial static use case.

	Median	90%-ile	99%-ile	Samples	min/med/99%-ile
	[cm]	[cm]	[cm]		SD [cm]
Anchor deployments
Original	21	148	735	833 k	4/8/45
Optimized	17	40	86	814 k	4/7/24
—
Environments
Open	16	33	85	529 k	4/6/12
Cluttered	21	48	90	285 k	4/7/40
—
Tag heights
High	14	28	43	276 k	4/6/11
Medium	19	41	94	267 k	4/6/16
Low	20	50	93	270 k	5/8/40

summarizes the main statistical values for the presented empirical distributions.

The effects of the environment and the tag surroundings on the achieved accuracy were also analyzed. As illustrated in Figure 7, tags placed in open-space environments with a median accuracy of 16 cm are, on average, 5 cm more accurate than tags placed in cluttered-space environments. This is due to better radio propagation conditions and increased LOS conditions between anchors an tags in open-space positions as compared to cluttered-space positions. It is worth mentioning that, despite tags in open-space positions clearly demonstrating better accuracy performance, some large errors are observed for both open-space and cluttered-space environments in the upper 3% of the distributions. As in the previous case, Table 3 summarizes the main values of the analyzed empirical distributions.

Further insights on the effect of the industrial environment concerning the performance of the UWB positioning system were obtained by observing the dispersion of the positioning data readings at a given position. Examples of measured static positioning data from two positions with very different clutter conditions are shown in Figure 8. The figure on the left corresponds to an open-space position where the tag was placed at high height and thus, above average clutter height, in good radio propagation conditions. At this position, the measurements exhibit high stability with high accuracy and small variance or dispersion, which led to a small UWB ranging error of approximately 5 cm. On the other hand, the figure on the right shows data from a cluttered-space position with the tag placed at low height. Here, it is observed how the blockage and presence of nearby metallic elements deteriorate the radio propagation conditions, resulting in higher instability of the measurement data leading to higher dispersion and inaccuracy (with an observed UWB ranging error larger than 60 cm). Numerically, these observed effects in open-space and cluttered-space positions can be quantified by analyzing the standard deviation (SD) of the obtained accuracy distributions. Key SD values are given for the different scenarios in Table 3. While in the open-space scenario, a SD of 12 cm was observed at the 99%-ile, this reached up to 40 cm in the cluttered-space scenario.

The performance of the positioning system for static tag positions at different heights was also studied. Figure 9 shows the CDFs of the empirical non-aggregated accuracy obtained for the different cases. Tags placed at high positions present a median accuracy of 14 cm and are, on average, 5–6 cm more accurate than tags placed at medium and low positions. Tags placed at medium height are slightly (~1 cm) more accurate than tags placed at low height, on average, but follow a very similar distribution leading to similar error values of approximately 95 cm at the 99%-ile. At this level, a reduced error of approximately half (~43 cm) was experienced at the high height. Once again, this can be explained from the radio propagation perspective, as LOS and overall radio propagation conditions between UWB anchors and tags are worsened at lower heights as the tags become embedded in the industrial clutter. The same effect is noticeable in the dispersion of the data, as observed from the SD statistics summary in Table 3. While, on average, all tag deployment heights exhibit a similar SD median value between 6–8 cm, there is a trend of increasing SD with tag height at the higher distribution percentiles. In particular, at the 99%-ile level, low, medium, and high tag heights exhibit SD values of 40, 16, and 11 cm, respectively.

Given the observed performance trends with respect to impact of the environment and tag height, a stability analysis was performed by noting the correlation between the achieved median accuracy and SD for the different configurations. Figure 10 displays two scatter plots of median accuracy on one axis and SD on the other axis. It was found that, in general, the UWB system demonstrates stable behavior with consistent dispersion of measurement data at a given position independently of the achieved accuracy, surrounding environment or tag height. The outliers indicate unstable performance of the UWB system at certain cluttered-space positions with low or medium tag heights, where large errors (inaccuracy larger than 40 cm) and high SD were jointly observed.

Finally, an analysis on the potential aggregation of measured data to improve the accuracy of the UWB system for use in static conditions was performed. Measurements at each given static position were taken with a sampling rate of 50 Hz over a 90 s period, and relevant aggregation intervals were fixed to slots for a duration of 0.1, 1, 10, and 90 s, which lead to aggregation sizes of 5, 50, 500, 4500 positioning samples, respectively. The CDFs of the achieved aggregated accuracy for the different aggregation sizes are shown in Figure 11 and

Table 4. Summary of aggregated accuracy statistics for the industrial static use case.

Aggregation	Median	90%-ile	99%-ile	Samples	Median SD
	[cm]	[cm]	[cm]		[cm]
Non-aggregated	17	40	86	814 k	7
0.1 s (5 samples)	17	40	86	170 k	7
1 s (50 samples)	17	40	86	17 k	6
10 s (500 samples)	15	35	86	1587	3
90 s (4500 samples)	14	38	78	216	-

summarizes the main statistical reference values. There were no differences between non-aggregating data or aggregating in 0.1 and 1 s slots. However, a clear gain was observed when aggregating measurement data for 10 or 90 s. In this case, the accuracy of the system was improved by an average of 2–3 cm. This improvement was also observed in the upper 1% of the distributions, where the positioning error was also improved by 2–8 cm.

3.2. Mobile Use Case Evaluation

Similar to the static use case evaluation, the effect of the different anchor deployments was evaluated for the mobile use case. In this case, the optimized anchor deployment was calculated slightly differently from the static case. Since the ground truth points are mobile, the laboratory was split up in a grid with 1 m × 1 m tiles in order to be able to apply the same anchor selection scheme as in the static optimized anchor deployment. The results comparing the accuracy obtained for the original reference deployment and the optimized deployment are displayed in Figure 12. On average, with the original deployment, an accuracy of 23 cm was observed. This accuracy was improved by 4 cm with the optimized deployment. A deployment gain was also achieved in the upper tails with the optimized deployment with respect to the original deployment, where the positioning error was reduced by 33–37% (from a reference of up to approximately 1.5 m) at the 90-99%-ile. For further reference,

Table 5. Summary of accuracy statistics for the industrial mobile use case.

	Median	90%-ile	99%-ile	Samples
	[cm]	[cm]	[cm]
Anchor deployments
Original	23	67	153	91 k
Optimized	19	52	102	101 k
—
Tag placements
Top	28	55	90	24 k
Left	14	48	91	20 k
Right	15	55	112	19 k
Back	14	39	82	20 k
Front	14	45	91	19 k

summarizes the main statistical values of the presented empirical distributions.

The impact of AMR tag placement on the accuracy of the system in mobility conditions was also analyzed. Figure 13 shows the positioning measurement data for the best (back) and worst (top) performing tag placement configurations, together with the AMR SLAM-based GT points. The CDFs of the achieved accuracy performance for the mobile use case with the different tag placement configurations are displayed in Figure 14. On average, tags placed on the sides of the AMR (front, back, left, and right) showed a similar median accuracy in the 14–15 cm range. However, tags placed on top of the AMR exhibited twice the error (28 cm). It is believed that the main cause for this large error is the change in radiation properties of the tag. While the tags placed on the sides of the AMR experience filtering and back-reinforcing of their radiation properties, those on top of the AMR have their radiation pattern distorted by the top platform of the robot, resulting in reduced overall performance of the system. In the upper tails of the accuracy distributions, values of approximately 90–110 cm were observed in all cases, except for the back tag, which exhibited an error of approximately 80 cm at this level. Thus, overall, tags placed on the back of the AMR demonstrated the best performance. Exact values for the different distributions are given in Table 5. Note that the SD and dispersion of the measurement data are not discussed for this mobile use case due to the inherent uncertainty of the SLAM-based GT point measurement, which is already subject to a baseline accuracy of 0–5 cm and would, thus, render an analysis of the SD very unreliable.

3.3. Comparison of Static Use Case and Mobile Use Case Accuracies

To make a fair comparison of the accuracy achieved by the UWB system for the industrial static and mobile use cases, measurement data sets obtained in similar conditions were identified. Since the height of the tags in the AMR mobility evaluation are comparable to the tags in the low height static evaluation, those sets of data were processed for comparison. In order to produce a straightforward comparison, all data from the mobile use case (front, back, right, left, and top tags) were combined to obtain an overall representative data set. As illustrated in Figure 15, which shows the CDFs of the accuracy for both use cases, the median accuracy for both the static and mobile use cases are quite similar (20 cm for the static use case with low tag height and 19 cm for the overall mobile use case). This similarity holds in the upper percentiles, where at the 90%-ile, the static use case presents a 50 cm error, while the overall mobile use case exhibits an inaccuracy of 52 cm. In the lower percentiles, i.e., 10%-ile, mobile use cases appear to have a better accuracy.

Moreover, it should be noted that the individual median accuracy of the four tags placed on the sides of the AMR in the mobile use case (summarized in Table 5) is 5–6 cm lower than that from the static use case with low tag height (summarized in Table 3). This improvement observed in the mobile use case is likely due to the configured filtering performed by the Pozyx algorithm, enabled by the predictable movement setting along with the medium freedom of movement as explained previously in Section 2.

4. Discussion

The results presented in Section 3 are now compared to those from the literature studies discussed in Section 1.1 and reported in Table 1.

Generally, our results are party aligned with some of the studies reported in the literature that evaluated their UWB systems in realistic indoor industrial environments. In terms of overall static accuracy, our study is well aligned with [16], who reported an accuracy better than 20 cm in both static and mobility conditions. The results obtained in our study for the static use case are also comparable to those in addressed in [17]. They reported median and 90%-ile accuracies of 20 and 40 cm, respectively, with UWB tags located at a height of 1 m. Similarly, in our case, with the medium height tag placed at 0.98 m, the median accuracy demonstrated was 19 cm and the 90%-ile accuracy was 41 cm. Ref. [18] explored different clutter environments and reported a UWB accuracy of 7–25 cm and 12–130 cm with tags located at a height of 1.38 m in open-space and cluttered-space conditions, respectively. Although the trends and average accuracy values described for both scenarios are aligned with those observed in our study, the dispersion of the data in their study differed slightly from ours. More specifically, they observed lower dispersion than us in open-space conditions. This is due to the limited number of their samples (only 8 GT positions) and also their environment being approximately four times smaller that our AAU 5G Smart Production Lab. The rest of the reviewed literature studies with a focus on realistic industrial environments reported much less accurate results. The reason behind the difference in accuracy in [19] and [20] as compared to the results in our evaluation is mainly due to differences in the size of the environment and the fact that they reported 3D accuracy, which might be worse than 2D accuracy; especially with the low anchor deployment density utilized based on 4–6 anchors only. In order to achieve accurate 3D positioning in an operational environment, anchors need to be deployed in a cubic 3D fashion [15], ideally replicating at close-to-floor level positions the deployment of anchors at ceiling level, thus with only 4–6 anchors it is impractical that a 3D-accurate deployment can be achieved. Ref. [21] reported an accuracy of 100 cm at the 90%-ile, while in our case that value is approximately 40 cm. This large deviation in the tails of the distribution can be explained by the low accuracy of their implementation based on the limited GT points explored (only 36) and the fact that their study noted a deployment considering only four anchors to cover a total area of 2760 m${}^2$ resulting in an expected anchor coverage area of 690 m${}^2$/anchor. As a reference, our AAU 5G Smart Production Lab deployment considers an anchor coverage area of 70 m${}^2$/anchor. In view of the above, it should be clear that our study validates some of the literature results, complements others and, in general, extends the statistical characterization of UWB system performance in realistic industrial settings for a larger number of deployment configurations and use cases.

Naturally, if we compare our results with those from the studies that evaluated positioning in simplified clutter-free environments [22,23,25,27] or in ideal simulated scenarios [28,29], the accuracy reported in our case is generally lower than that reported in those studies (i.e., sub-decimeter). These studies should still be taken into consideration as they indicate that it might be possible to enhance the performance of the UWB system deployment by optimized anchor placement or (mainly) by anchor densification until reaching accuracy values in the lower-bound range. However, achieving an anchor deployment that guarantees optimal clutter-free LOS conditions from at least three anchors to all positions in industrial environments requires complex planning and may require a large number of anchors to be set. This may result in excessive increases in cost for a not-so-important accuracy gain.

In general, the absolute accuracy results are specific to the analyzed environment, although the overall magnitude is similar in other scenarios (tenths of cm), as observed from the literature review. The UWB system behavior and performance trends discussed for the different deployments (heights, clutter, optimization of anchor position) will still hold in other indoor industrial environments, independently of the specific achievable accuracy values. This highlights the importance and novelty of this study, as it provides useful insights for practical deployment of a UWB localization system in industrial environments.

4.1. Implications of the Results

The UWB system performance results reported in our study can be put in the context of current indoor positioning needs in operational industrial systems. According to our evaluation, it is realistic to mount UWB tag devices on employees, pallets, forklifts, or AMRs in order to track their real-time position within the scenario and optimize safety and production operational schemes. As guidelines for general deployment, it is suggested to optimally deploy tags in the highest possible positions, e.g., on human operator helmets or roofs (or similar elevated positions) of forklifts or other large vehicles. These devices will experience an accuracy of 14 cm throughout the entire scenario. In those elements where it is not possible to deploy the tag device atop an elevated position, UWB accuracy will be reduced, on average, to 19 cm for AMRs or other middle-height devices, or to 20 cm for elements deployed close to ground such as pallets in certain operational conditions. With the observed achieved accuracy, the granularity of the UWB-based real-time tracking can be achieved in cycles at short durations of 0.5 s without compromising performance as compared to higher sampling configurations. Such update rates would be sufficient to coordinate the movements of humans and mobile robots in an industrial scenario with AMRs driving at an average speed of approximately 0.5 m/s. In this respect, the current UWB system performance guarantees that its readings can be trusted as coarse/broad accuracy for general navigation, but not for the fine positioning needed in order to replace the intelligent LiDAR/SLAM mechanisms inside the AMRs. In those cases where the elements are expected to be in static positions (e.g., pallets), with averaging of multiple UWB temporal readings it is possible to achieve a gain of 17.6% in terms of average accuracy, at the cost of increased battery use.

4.2. Future Research Directions

The presented research constitutes, to the best of the authors’ knowledge, the most complete and extensive study of UWB system performance in industrial settings reported to date, however, further aspects could be considered for future analysis. It would be possible to further examine the performance of the UWB system under different radio parametrizations and quantify their impact on the accuracy. In particular, for the mobile use case, adjusting the update rate of the UWB radio transmissions may potentially lead to optimized trade-offs between sampling rate and accuracy performance, allowing for the support of reliable positioning for general navigation of faster AMR with average speeds higher than 0.5 m/s.

This study showed that anchor placement has a great impact on the accuracy of the UWB system, thus, another important aspect that would require some attention is the development of deployment optimization schemes to automate the calculation of the optimized anchor positions during installation for a given clutter, or simply calculating the optimal subset of anchors for impromptu use after some quick calibration procedures.

Another interesting topic would be to compare the performance of the indoor UWB system with other systems based on different technologies. In this line, our work presented in [34] compared a subset of the UWB measurements reported in this paper with measurements obtained with a system based on ultrasound in the same industrial scenario. As a reference, comparable accuracies to those reported in this study with eight UWB anchors were achieved with an optimal ultra-sonic system based on 14 beacons. Future work will extend the comparison to other important radio technologies such as Wi-Fi or BLE.

In addition, the integration of different technologies for the sake of further improving the positioning accuracy is to be pursued. In this respect, one can leverage the approach presented in [35], where UWB positioning is fused with information obtained from LiDAR sensors, to a large-scale industrial setup. Furthermore, the possibility of using UWB tags for relative positioning in infrastructure-less deployments is to be investigated [36].

5. Conclusions

In this study, we have presented an extensive evaluation of the performance of a commercial UWB-based IPS in a realistic 560 m${}^2$ industrial setting, with a focus on both static and mobile applications. The measurement results revealed a similar performance with an overall median 2D accuracy of 17 cm in both static and mobile conditions, which is sufficient to track workers, robots, or vehicles in a factory.

The effect of the nearby industrial clutter environment on the accuracy of the system was also explored, finding that UWB tags located in open-space will experience a median accuracy of 16 cm, 5 cm better than UWB tags located close to machinery in cluttered-space positions. In general, for the static use case, the best median accuracy (14 cm) was obtained with UWB tags located in high positions at 1.98 m above average clutter level and the worst (20 cm) with UWB tags located close to the ground at 0.34 m. For the mobile use case, the best median accuracy (14 cm) was obtained placing the UWB at the back side of an AMR. The study also highlighted the importance of optimizing the radio deployment, finding the combinations of anchors which led to the most accurate results at a given position. It was shown that the optimized deployment provides a gain of up to 4 cm at the median accuracy level but, most importantly, improves the tails of the accuracy distributions by more than 100 cm at levels above the 90%-ile. In terms of radio configuration and post-processing, it was found that averaging multiple UWB values at a given position will only provide a small degree of improvement (up to 2–3 cm at the median level) when aggregating more than 500–4500 samples with associated integration periods of 10–90 s.