Improving Plane Fitting Accuracy with Rigorous Error Models of Structured Light-Based RGB-D Sensors
Round 1
Reviewer 1 Report
This paper presents a new strategy for plane fitting based on RANSAC focused on low cost structured light 3d cameras. The authors perform error modeling of the camera used for validation and apply a RANSAC based plane fitting method that uses radial distances as input for the cost function instead of perpendicular distances.
The paper is well written and presents the problems and solutions in a concise way. The contribution of the paper is backed by the results found but, given the lack of multiple experiments, I believe the experiment does not provide sufficiently strong evidence to back the contribution in a reliable way. There are also many spelling and formatting mistakes as well, but I am not considering these in this review as they can be easily fixed.
The authors repeat the acquisitions 20 times but for only one scenario and one camera. I believe there should be more experiments to validate the claim of increased accuracy. Either changing the situation by using several camera positions, using a different environment, repeating the test with one other camera (kinect v2, realsense, xtion, etc) or by performing additional tests on a non-controlled environment so that the applicability could be compared against the other methods. One of these options, in my opinion, should be enough to further back the claim of increased accuracy against the other established methods.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The presented method is promising and should be presented in a clear, accurate and comprehensive way. Currently I cannot agree that this paper is ready for publication, it may not even be ready to be reviewed.
I would recommend to improve the paper with the following aspects in mind:
Reconsider the names of your variables. If the disparity value has d_i, then a error value should not have a 'd' too (e.g. not d_e^s), for errors you could use \epsilon instead or something else.More importantly, if d_i is disparity, why is D_i a direction? Introduce vectors for points that have 3 coordinates. Aim at showing your equations in a concise, clear and short form. Write about the content of a sections before starting with your derivations, in particular make clear what your goals are, e.g. for Section 2: "In this section we will explain the fundamentals of SL sensors, how disparites are converted to distances and how the errors of such sensors can be modeled."
In section 2 you, for example, never state why all these derivations are required! Please make sure to 'guide' he reader through your paper. Define all your variables and define them only ONCE! There is no comparison to any other established methods. The authors argue that RANSAC is superior to for example Least Squares- and Hough Transform methods. This needs to be proven by experimental results! There are public implementations available, some on Github, some in the PCL and other sources.
Please find some related methods in the papers below. Please provide a complete list of related work Please justify why your errors in angle and distance are larger than the errors reported in [4]. In [4] angle errors of less than a degree and distance errors of less than 2 cms are achieved even with the LS-base baseline methods! Make sure that you consistently add spaces before commas and cites. Make sure that your variable descriptions are clearly separated from your text paragraphs
[1] J. Poppinga, N. Vaskevicius, A. Birk, and K. Pathak. “Fast plane detection and polygonalization in noisy 3D range images”. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3378–3383, IEEE, Nice, France, 2008.
[2] D. Holz and S. Behnke. “Approximate triangulation and region growing for efficient segmentation and smoothing of range images”. Robotics and Autonomous Systems, Vol. 62, No. 9, pp. 1282–1293, 2014.
[3] C. Feng, Y. Taguchi, and V. R. Kamat. “Fast plane extraction in organized point clouds using agglomerative hierarchical clustering”. In: IEEE International Conference on Robotics and Automation (ICRA 2014), pp. 6218–6225, IEEE, Hong Kong, China, 2014.
[4] P. Fuersattel, S. Placht, A. Maier, and C. Riess. “Geometric Primitive Refinement for Structured Light Cameras”. Machine Vision and Applications, Vol. 29, No. 2, pp. 313–327, 2018.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 3 Report
This paper presented an approach to improve plane fitting accuracy in SL based sensor by adopting a new cost function in RANSAC based plane fitting method. First, the author derived a rigorous error model for the SL-based RGB-D sensor based on its working principle and error propagation law. Then, they illustrated their improvement in RANSAC based plane fitting method by replacing the perpendicular distance with radial distance. Finally, they addressed their conclusion by experiments and comparison with other methods. Overall, the work is fine and publishable.
This manuscript modified the cost function in RANSAC from perpendicular offset to the weighted radial offset significantly improved the fitting accuracy, and It was significant in the domain of low-cost SL sensor data capturing.
However, there are some issues in this paper.
Through the introduction, the author not only needs to present the main work in this paper but also needs to present the originality of this work or improvement to any previous research. However, it’s not clear to see it in this introduction.
There are some errors in the manuscript like formula (5), (14) and so on.
There are some typesetting questions like formula (17), (18) and so on.
In Figure 8, the author should provide a top view of result comparison should be more convincing.
In line 254, "Figure 9 shows the range errors before and after the plane fitting. Compared with Figure 6 the author presented approach is robust for plane fitting, can it be extended to curve plane or irregular object fitting?
As the main contribution, can explain what's the advantage of weighted radial offset? (b)" may cite error figure.
There are some grammar errors. Please check carefully.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 4 Report
The concept behind this work, that sensor readings with higher amounts of noise need to be weighted less than those with lower amounts of noise, is not new. However, the application to depth sensing in this context may possibly be. That being said, the way the error was calculated and applied needs to be significantly improved.
Firstly, the authors state in multiple locations that fitting point cloud data to multiple types of regular shapes, in particular planes, spheres, and cylinders, is important. However, only one example of one object from one position, a single wall from a single vantage point, is ever presented. Data viewing the wall from multiple different poses is required as too is examples of additional objects. One with additional examples can the method be seen to be an improvement rather than a one-off test.
Could the authors try modelling the radial weighted error and applying the weightings to other objects to test the applicability of their findings to more than the single context presented?
Could the authors please differentiate the systematic from the random in their error estimations? The example data shown in Figure 8 appears to have the majority of the error as systematic. As such it should be possible to determine any bias and calibrate it out.
How exactly were the data points weighted based on estimated error? The standard Kalman Filter approach is to weight a sample based on the inverse of the variance of the population. Equation (15) indicates that variance could have been used (even though line 189 incorrectly states it is standard deviation) but equation (16) indicates standard deviation was used. Why was standard deviation rather than variance chosen?
The end of the abstract should have a sentence (or two) on applications or implementations of the work. Where could this work be applied or what does it actually mean for the future use of such sensors if this work was incorporated into their use.
On line 50 the authors state that RGB-D cameras are a recent development. This is not actually true. They have been used in commercial gaming systems for over 9 years and were developed long before then.
In the paragraph commencing on line 156 the authors make a claim of fact without evidence or scale. The claim is that previous assumptions made are valid for less noisy data but not for noisy data. Could the authors please expand on this point by providing evidence of this? Showing how much error results under these assumptions for different levels of noise at the input would make it possible to know when the assumptions are valid and how much the error could be reduced using the proposed method. This is vital as it will show the ideal operating conditions for the proposed method and the expected scale of the improvement under different conditions.
Can the authors please quantify the difference in the example results shown in Figure 5? By how much is the result improved?
Minor Grammar Corrections:
Line 20: “Most existing…”
Line 38: “…are widely used for 3D model generation [1, 2], HD…”
Line 39: “Instead of directly using millions…”
Line 43-44: “…are mainly generated by terrestrial laser scanner (TLS) or RGB camera.”
Line 45: “…surveying time of TLS restricts the wide…”
Line 50: “Recently developed RGB-D sensors are another choice for point cloud collection…”
Line 51: “RGB-D sensors can be divided into…”
Line 68: “…low and with a large proportion of outliers, or an occlusion…”
Line 69: “…based on feature space transformations…”
Line 71: “…PCA is sensitive to outliers as well, similar to LS, while…”
Line 72: “…difficult to apply to low…”
Line 74: “…can provide robust…”
Line 80: “…practical on high-quality Lidar data.”
Line 81: “…shortages to handling datasets collected…”
Line 82: “…the measurements are both significantly larger and distance related.”
Line 97: “Figure 1. Working Principle of Structured Light Depth Sensor”
Line 105: “For most SL based depth sensors…”
Line 163: “…because the datasets are much noisier…”
Line 208: “…a width of 6.4m was used…”
Line 209: “…control points were surveyed…”
Line 210: “…IR camera was calibrated…”
Line 250: “…is the one closer to…”
Line 255: “…the control points, the range errors…”
Line 256: “…the larger the error.”
Line 257: “…errors are controlled to an accuracy…”
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
This paper presents a new strategy for plane fitting based on RANSAC focused on low cost structured light 3d cameras. The authors achieve validation of the proposed method by comparing their algorithm to other well-established methods.
The new experiments added by the authors provide sufficient proof to validate the algorithm as more accurate than the existing methods. The experiments were well designed and fit the paper by backing the claims made by the authors. I have no additional remarks to make about this paper.
Author Response
we would like to express our sincere appreciation to the comments and suggestions from you.
Reviewer 2 Report
The authors have improved their manuscript and successfully addressed the previously mentioned issues.
Author Response
we would like to express our sincere appreciation to the comments and suggestions from you.
Reviewer 4 Report
While it would be a stronger paper if the authors had investigated additional object types, rather than just a flat wall, the inclusion of multiple distances has improved it. All other questions and concerns appear to have been adequately addressed.
Some care should be taken to edit the new content for correct English. For example line 30: "...our method is robustness and..." should be "...our method is robust and...". Also, all discussions of results should be in the past tense. For example line 32: "...errors are about one degree..." should be "...errors were about one degree...".
Lines 147 and 149 refer to standard deviations, however equations 10 and 11 are both variance.
Line 252 and Figure 8 refer to far range and edge area as well as centre area. Could the authors please define precisely what these are?
Author Response
Please see the attachment.
Author Response File: Author Response.docx
