A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots

Round 1

Reviewer 1 Report

 

actuators-2507692

 

In this article, the authors propose an adaptive backstepping method based on Double Q-learning for tracking and controlling the trajectory of mobile robot. They design the incremental model-free algorithm of Double-Q learning, which can quickly learn to rectify the trajectory tracking controller gain online. And for the controller gain rectification problem in non-uniform state space exploration, they propose an incremental active learning exploration algorithm, which incorporates memory playback as well as experience playback mechanisms to achieve online fast learning and controller gain rectification for agent. The authors work is timely new and interesting but to still I have some comments and suggestions that must be considered.

 

1.     The English grammar should be improved and there are also some typos. The authors should carefully revise the paper to avoid such errors.

2.     The key contribution of this work should be added in bullet form at the last second paragraph of the introduction section.

3.     Furthermore, the paper organization should be added at the last paragraph of the introduction section.

4.     Further increase the resolution of all figures.

5.     Fig. or Figure? Keep consistency throughout the paper. Use similar form format throughout.

6.     Table 1 is not cited inside the text.

7.     For experimental evaluation and validation, the authors should add a separate table for comparison with other state-of-the-art approaches. If there is no proper comparison, how can the novel reader understand that this model is novel or effective? Therefore, I highly advise comparing your model to the most recent models, particularly those from the last 3-5 years.

8.     To know more about mobile robot, the authors can refer to “A Localization based on Unscented Kalman Filter and Particle Filter Localization Algorithms,” IEEE ACCESS.”

9.     Some numerical equations are missing proper explanation and citations, please revise.

 

10.  The readability of the article needs to be improved further. 

Extensive editing of English language required

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

This manuscript cannot be published because:

1) The state of the art is weak: there are many ways of controlling the trajectory tracking of mobile robots that have not been cited, e.g.,

J.A. Chocoteco, R. Morales, V. Feliu and H. Sira-Ramírez, “Robust Output Feedback Control for the Trajectory Tracking of Robotic Wheelchairs”, Robotica, vol. 33 nº1, pp. 41-59, January 2015.

and many other papers about control schemes not based on AI.

2) Moreover, some papers are cited that do not have direct relation with the addressed problem, e.g., in lines 141-143: Subudhi [28] ....... multi-link flexible manipulators under different load conditions.??

3) Consequently, motivation for using the proposed approach is unclear. The advantage over other simpler techniques should had been clearly stated. For example, what is exactly being optimized?, does it really make a difference in the performance compared to other controllers?.

4) In the simulation and experimental section, a comparison of performances of this control method with others should had been presented.

5) The nomenclature is careless, e.g.;

- v and \omega of (1) are not defined. Instead v_c and \omega_c have been defined.

- In (2): what is \theta?, and \theta_a?, and Te?.

- In line 225, separations among  X_e, Y_e, \theta_e is incorrect.

-  What is q_c in Figure 2?..........

6) Some important issues are not mentioned, e.g.:

- Which sensor are you using in your control?.

- The exact description of the shape of the disturbances, what they represent and where they are applied are missing.

7) Simulated and experimental results do not seem to be impressive. For the described trajectories, and the required velocities (which are not high), the performance shown does not seem to be better than what can be obtained with other controllers. 

8) The impact of the different elements that constitute the control system must be illustrated, e.g., what is the impact of the adaptive gains in the overall performance?.

 

English needs an extensive review, e.g.:

- line 45: the task required requirements proposed by human beings.

- line 93:  that compensate for errors.

- line 213: the robot particle are and, respectively, which can simplify the motion control of the mobile robot.

- Many verbs use incorrectly the 3rd person singular.......

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

-The paper presents another method of commanding autonomous vehicles. In the specialized literature, there are many works dealing with this issue.

-In my opinion, the work does not demonstrate the advantages stated in the abstract: better robustness, generalization, real-time and stronger anti-disturbance capability. The authors must demonstrate these qualitative aspects by referring to quantitative comparisons with the results of known solutions.

-The introduction presents some solutions taken from the bibliography. It is recommended to focus on works that have solutions close to the one in the work.

-The mobile robot kinematics model is well known. Authors must specify bibliographic sources where this model can be found.

-The work must be corrected from the point of view of editing errors.

-A discussion regarding an obstacle course is required.

-Since the work is proposed to be published in an actuator journal, a presentation of the command and execution equipment is necessary.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

NA

 

Minor editing of English language required

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Some of my questions have been addressed but other important ones not. Then I cannot recommend this paper:

- The state of the art already has to be improved. It is not enough citing some papers on non-AI control techniques of mobile robots. You must clearly state which problems are not solved using these techniques that you foresee to solve with your approach. Moreover, you must clearly state why you expect to solve these problems with your approach.

- Nomenclature still has serious flaws, e.g., in (2), X_A, Y_A, \theta_A have not been defined. What is \theta in this expression?. Moreover, is expression (2) correct?.

- What are "random human disturbances"?. The precise characterization of the disturbances and the explanation of what they exactly mean are still missing. The explanation promised by the authors in line 635 does not exist.

- What is "Backstepping-Fractional-Older PID controller"?, is it "order"?. In line 689, it is stated that this technique was developed in [51]. [51] has nothing to do with fractional order PID controllers. 

- And the main drawback is still the motivation: ¿why to use this technique which requires a lot of parameters heuristically chosen (see lines 691-694) and a training procedure instead of a simpler control system designed based on a well-known model?.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

-

Author Response

Thank you for your valuable feedback on my article, which has improved its readability.