Static and Dynamic Evaluation of an UWB Localization System for Industrial Applications

Round 1

Reviewer 1 Report

Authors have improved the overall quality of the work with respect to the former submission, but some minor details should be corrected:

- In my opinion, authors must mention in the abstract that a contribution of their work is "...to define if UWB localization sensors can be used for gesture recognition in 3D space", as it is said in the Introduction.

- Subsection 3.4.3. Dynamic Measurement Evaluation and Precision of Mapping: "The third test is to realize a mapping of the inner area (Figure 5a) and outer area (Figure 5a)" ==> please correct according to the captions of those figures.

Author Response

In my opinion, authors must mention in the abstract that a contribution of their work is "...to define if UWB localization sensors can be used for gesture recognition in 3D space", as it is said in the Introduction. This is the ultimate aim of the whole study but the article presented here is a first step that does not fully answer that question. Therefore, it should be deleted from the introduction. We removed "for gesture recognition" so the sentence becomes: to see if UWB can be used to obtain an accurate 3D trajectory. Subsection 3.4.3. Dynamic Measurement Evaluation and Precision of Mapping: "The third test is to realize a mapping of the inner area (Figure 5a) and outer area (Figure 5a)" ==> please correct according to the captions of those figures. Fixed

Reviewer 2 Report

The authors resume several localization technologies and methods, while focusing on UWB systems. They show an experimental setup with four to six anchors and one tag. The ground truth position of the UWB devices is measured with a motion capture system. Both measured ground truth positions and the estimated UWB tag positions are expressed in a common reference frame, which transformation is calculated using a Singular Value Decomposition (SVD).

While adequately described up to this point, there seem to be some flaws in the Tests and Evaluation section (3.4).

-Missing explanation of how the estimate of the tag UWB position (Theta_i) is calculated.

-On the Z Anchor Change Test (3.4.4):  Having coplanar anchors can result in a very high (bad) VDOP. Furthermore, depending on how the position of the tag is calculated, there can be two valid solutions: One below the plane formed by the 4 anchors and one above them. Changing the position of the z-axis of the anchors, should reduce the VDOP value and thus the standard deviation error in the z-axis. This is especially true for unbiased distance measurements (mean error near zero). It is easy to calculate the DOP values in order to verify this. Placing one anchor high and one low should provide you a good VDOP value.

-It is not clear to me why the results between static and dynamic measurements should differ, and if the anchor setup is the same, why the bad VDOP does not affect the solution. Is it possible that an error in time synchronization between the UWB data and the ground truth data is increasing the actual UWB localization error?

-The performance with more anchors should in theory be better (again you can calculate DOP values). There is something probable wrong there, for example some NLOS situation or some offset errors in the distance measurement(s).

There are some english grammar errors in the text that should be adressed.

Author Response

The authors resume several localization technologies and methods, while focusing on UWB systems. They show an experimental setup with four to six anchors and one tag. The ground truth position of the UWB devices is measured with a motion capture system. Both measured ground truth positions and the estimated UWB tag positions are expressed in a common reference frame, which transformation is calculated using a Singular Value Decomposition (SVD). While adequately described up to this point, there seem to be some flaws in the Tests and Evaluation section (3.4). -Missing explanation of how the estimate of the tag UWB position (Theta_i) is calculated. The objective of the article is to evaluate an existing system, and the location data is provided directly by this system. The algorithm is implemented by Decawave and uses Time of fight calculation. We added the clarification: We use the two-way ranging (TWR) standard algorithm embedded by Decawave. -On the Z Anchor Change Test (3.4.4): Having coplanar anchors can result in a very high (bad) VDOP. Furthermore, depending on how the position of the tag is calculated, there can be two valid solutions: One below the plane formed by the 4 anchors and one above them. Changing the position of the z-axis of the anchors, should reduce the VDOP value and thus the standard deviation error in the z-axis. This is especially true for unbiased distance measurements (mean error near zero). It is easy to calculate the DOP values in order to verify this. Placing one anchor high and one low should provide you a good VDOP value. In theory the results should be better, but the manufacturer's recommendations are to place the anchors in a coplanar configuration. This hypothesis should therefore be used in their algorithms and this explains the results obtained experimentally. We will take into account the remarks on DOP in future works where we will develop our own algorithms. -It is not clear to me why the results between static and dynamic measurements should differ, and if the anchor setup is the same, why the bad VDOP does not affect the solution. Is it possible that an error in time synchronization between the UWB data and the ground truth data is increasing the actual UWB localization error? In the static case we have calculated a mean error while in the dynamic case we calculate an instantaneous error for each point. This is the main reason for the increase of the error in the dynamic case. The performance with more anchors should in theory be better (again you can calculate DOP values). There is something probable wrong there, for example some NLOS situation or some offset errors in the distance measurement(s). We agree that it should be better in theory. However, when we increased the number of anchors, we placed them in a non-coplanar configuration to try to increase the Z-accuracy. Not following the configuration recommended by the manufacturer may explain this slight increase in error. Moreover, the variation is small compared to the announced accuracy of the system and is therefore not significant. There are some english grammar errors in the text that should be adressed. We have tried to correct as many mistakes as possible.

Reviewer 3 Report

How are the estimated values from the UWB system obtained? Which algorithm is used on which acquired data to achieve this? Please elaborate.

For comparison, the measurements should also be done in conditions with less multipath, such as an outdoors environment or an anechoic chamber.

In which way has the cart been moved? Was it self-propelled with a remote control, or was it pushed by a person? Did this influence the measurements?

Please clarify whether NLOS scenarios been investigated.

More care should be invested in the text, language-wise. This English is often sloppy and multiple typos exist.

There is a lot of repetition in the content between paragraphs 1 and 2. This can and should be done more concise.

The document needs improvements. Multiple additional questions and suggestions have been added in the attached file.

Author Response

How are the estimated values from the UWB system obtained? Which algorithm is used on which acquired data to achieve this? Please elaborate. The objective of the article is to evaluate an existing system, and the location data is provided directly by this system. The algorithm is implemented by Decawave and uses Time of flight calculation. We added the clarification: We use the two-way ranging (TWR) standard algorithm embedded by Decawave. For comparison, the measurements should also be done in conditions with less multipath, such as an outdoors environment or an anechoic chamber. This is a good point, but the aim of our work is to evaluate the system in real conditions, not in perfect conditions. This is why we have carried out these tests in conditions representative of an industrial environment. In which way has the cart been moved? Was it self-propelled with a remote control, or was it pushed by a person? Did this influence the measurements? The cart was being pushed by a person. We first assessed the impact of the person on the accuracy. We found that there was no in uence because the person was far enough away from the UWB tag. We added the clarification: To limit interference, we placed the tag on a wooden cart pushed by a person far enough away from the tag. Please clarify whether NLOS scenarios been investigated. For this study, NLOS tests were not performed because it is more difficult to obtain a ground truth since the VICON system also requires LOS conditions. We are working on an evaluation under these conditions on an industrial environment (production line) with NLOS conditions, but this will be the subject of another study. There is a lot of repetition in the content between paragraphs 1 and 2. This can and should be done more concise. The document needs improvements. Multiple additional questions and suggestions have been added in the attached file. The have amended the article accordingly.

Round 2

Reviewer 1 Report

The authors have clarified all my questions about their work

Author Response

The authors have clarified all my questions about their work. Thank you for reviewing.

Reviewer 2 Report

Two-Way Ranging (TWR) is usually used to determine the distance between the UWB transmitters. Maybe you could elaborate just a little on how the position theta_i is estimated.

Why is a coplanar configuration recomended? Is this recomendation for 2D positioning? If available please provide this reference.

Author Response

Two-Way Ranging (TWR) is usually used to determine the distance between the UWB transmitters. Maybe you could elaborate just a little on how the position theta_i is estimated. We agree Two-Way Ranging (TWR) is used for determine distance between UWB transmitters. Decawave location engine uses maximum likelihood estimation to determine theta_i explained in the Datasheet (https://www.decawave.com/sites/default/files/dwm1001_system_overview.pdf). Why is a coplanar configuration recomended? Is this recomendation for 2D positioning? If available please provide this reference. A coplanar configuration is recommended for 2D positioning (cf Datasheet of Decawave). Future work will focus on Geometry Dilution Of Precision know as GDOP to find best configuration for 2D or 3D positioning.

Reviewer 3 Report

Significant improvement over previous version.

It would be useful to mention the position of the anchors in or around the figures showing the measurement results.

Although some tables have been adapted, not all significant numbers issues have been resolved. Please address this.

Please also ensure the entire document uses 'GHz' instead of 'Ghz', as also mentioned in my first review.

Comments for author File: Comments.pdf

Author Response

Significant improvement over previous version. It would be useful to mention the position of the anchors in or around the figures showing the measurement results. We add the precision Anchors a place respectively at (-2;5.03),(3.10;5.03),(-2;1),(3.10;1) in the article. Although some tables have been adapted, not all significant numbers issues have been resolved. Please address this. Thank you, This is corrected Please also ensure the entire document uses ’GHz’ instead of ’Ghz’, as also mentioned in my first review. Thank you, This is corrected

Round 3

Reviewer 2 Report

Some of the propositions have been implemented. Overall the paper has been improved. I have no further comments to make.

Author Response

Some of the propositions have been implemented. Overall the paper has been improved. I have no further comments to make. Thank for reviewing

Reviewer 3 Report

I noticed that the article is not clear about which Decawave product is used in the setup. Do you use the TREK1000 kit? Do you use you own pcb with a custom made antenna? Please be sure to clarifiy in the beginning of paragraph 3.1-experimental setup.

Furthermore, I see that, in spite of my remarks in the two previous reviews, there is still 'GHz' and 'Ghz' in the document. Please do correct this to the former form.

Author Response

I noticed that the article is not clear about which Decawave product is used in the setup. Do you use the TREK1000 kit? Do you use you own pcb with a custom made antenna? Please be sure to clarifiy in the beginning of paragraph 3.1-experimental setup. We used the MDEK1001 kit, we didn't use custom antenna. Furthermore, I see that, in spite of my remarks in the two previous reviews, there is still 'GHz' and 'Ghz' in the document. Please do correct this to the former form. This is corrected