Decentralized Multi-Agent Control of a Manipulator in Continuous Task Learning
, Dario Piga 1 and Loris Roveda 1,*Round 1
Reviewer 1 Report
The authors propose in this paper an MA control for a robot manipulator using continuous representation. The paper is interesting, sound, and easy to read. However, I have a few comments which could help to improve the manuscript:
1. Some Figures (especially 5, 14, 15) have a very low-quality resolution. In general, authors should prefer vectorial images (such as pdf, svg, or eps). These formats avoid being pixeled when the image is resized.
2. As this work discusses reward shaping, a relevant paper published in the same journal using deep RL in a continuous domain for a manipulation task [1] should be added and discussed.
[1] https://www.mdpi.com/2076-3417/10/16/5574
3. On some occasions, the experimental results show that the accumulated reward plummet to a local minimum. In order to avoid such behavior, authors should run their experiments multiple times (at least ten), and use averaged reward for each approach, otherwise experiments lack statistical significance. Additionally and in this regard, the standard deviation should be shown on the plots.
4. The manuscript would benefit by adding a final table summarizing the results. For instance, all approaches may be compared by using the reward under the curve, or simply the total accumulated reward, i.e. adding the reward obtained each episode. This way, a more fair comparison between the methods is presented.
5. as stated in the introduction, it is very important to consider new alternatives to simplify the complex learning problem. Therefore, I think this paper should at least discuss (either in the introduction or conclusions as possible future work) on other alternatives for speeding up learning in human-robot interaction. Some examples: contextual affordances [2], explainable robotic systems [3], interactive feedback [4], and action selection methods [5].
[2] https://www2.informatik.uni-hamburg.de/wtm/ps/Cruz_ESANN_2016.pdf
[3] https://link.springer.com/article/10.1007/s00521-021-06425-5
[4] https://ieeexplore.ieee.org/abstract/document/8329809
[5] https://ieeexplore.ieee.org/abstract/document/8625243
6. Minor: don't --> do not
Author Response
Dear Reviewer,
please find attached the reply to your comments.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
1. This well-written paper presents an interesting decentralized multi-agent control of a manipulator in learning continuous task.
2. It will be nice to see more experimental results conducted on an industrial manipulator platform.
Author Response
Dear Reviewer,
please find the reply attached.
Kind regards,
the authors
Author Response File: Author Response.pdf
Reviewer 3 Report
The presented article has a very good scientific quality.
The paper has a real objective, methodology, experiment and results.
Literature review is sufficient and up-to-date. I have comments on the used literature, where, in addition to IEEE conference papers, I also expect a comparison of results published in quality journals.
The abstract is written as it should and looks good.
Only that you write that
“the proposed framework is capable of accelerating the learning process at the beginning”. What kind of framework is it?
The defined goal of the work is clearly specified, but you also write it in line 35 and at the same time line 82. You should unify this.
Overall, I can evaluate that the defined intent is clear and the evaluation of the relevant results.
Author Response
The authors revised the paper based on the Reviewer's suggestions. Please find the reply to the comments attached.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The authors addressed all my comments satisfactorily and, therefore, I would suggest accepting the paper for publication.
