Stereoscopic Projection Policy Optimization Method Based on Deep Reinforcement Learning
Round 1
Reviewer 1 Report
Dear Authors,
I read your article carefully. It deals with an important and difficult topic of optimization of operations in the theater of military operations. The work deals with numerous aspects, some of them were only briefly mentioned, without giving any reason. Especially:
- title: DRL - very important word, given as an abbreviation
- abstract: A3C algorithm used without explanation. There is no mention of the neural network, which is an essential part of the work
- Introduction - citations [2-4] are too general
- Introduction - the aim of the work was indicated - multi-dimensional projection - where was the aim of the work achieved in the work?
- Introduction - A3C algorithm - no reference to the literature
- Introduction - the work structure has not been presented
- Section 1 - Difficult to read and understand, what does DQN stand for?
- Section 1.1 - What exactly does "p" stand for?
- Section 1.2 - no reference to literature - what about SARAS, Epsilon-Greedy, Q-learning, TD learning?
- Section 1.4 - notations in equations not explained, no references to literature,
- Section 1.5 - no definition of "state spaces" and "action spaces", what do LSTM, CNN, DDPG, A2C, PPO mean?
- Section 1.5 - equation (4) left without explanation
- Section 1.5 - no introduction and discussion of the A3C method
- Section 1.5 - last paragraph - very good part of the work, but left without reference to the literature.
- Section 2 - Is the example from literature or your own?
- Section 2.4 - How to understand "projection strategy"?
- Section 3 - What does a CMO mean?
- Conclusion - does the presented method work only to the considered specific scenario or it can be extended to the other cases ?
What computing power were used to perform the necessary simulations?
References - cursory literature sources
- Figure - figure markings are in two languages
For this reason, I believe that the work can be published after considering the indicated issues.
The work requires a major revision.
Author Response
Reviewer 1
1.- title: DRL - very important word, given as an abbreviation
- reply: Replace abbreviations with full names
2.- abstract: A3C algorithm used without explanation. There is no mention of the neural network, which is an essential part of the work
- reply: The A3C algorithm is explained and the neural network is added
3.- Introduction - citations [2-4] are too general
- reply: Replaced some references
4.- Introduction - the aim of the work was indicated - multi-dimensional projection - where was the aim of the work achieved in the work?
- reply: Unified terminology，raplaced “ multi-dimensional projection” with “ stereoscopic projection”
5.- Introduction - A3C algorithm - no reference to the literature
- reply: Added the references
6.- Introduction - the work structure has not been presented
- reply: Added the description of work structure
7.- Section 1 - Difficult to read and understand, what does DQN stand for?
- reply: Replace abbreviations DQN with full names
8.- Section 1.1 - What exactly does "p" stand for?
- reply: Removed the representation of p
9.- Section 1.2 - no reference to literature - what about SARAS, Epsilon-Greedy, Q-learning, TD learning?
- reply: The explanation of abbreviations and reference to literature are added
10.- Section 1.4 - notations in equations not explained, no references to literature,
- reply: Explained the symbols in the equation and added references
11.- Section 1.5 - no definition of "state spaces" and "action spaces", what do LSTM, CNN, DDPG, A2C, PPO mean?
- reply: The explanation of abbreviations and reference to literature are added
12.- Section 1.5 - equation (4) left without explanation
- reply: Explained the symbols in the equation
13.- Section 1.5 - no introduction and discussion of the A3C method
- reply: Added the references,and the A3C is discussed in 1.4
14- Section 1.5 - last paragraph - very good part of the work, but left without reference to the literature.
- reply: Added the references
15.- Section 2 - Is the example from literature or your own?
- reply: The example is from our own experiments
16.- Section 2.4 - How to understand "projection strategy"?
- reply: replaced "projection strategy" with” stereoscopic projection action strategy”. It means the action strategy of stereoscopic projection
17.- Section 3 - What does a CMO mean?
- reply:It is a simulation platform,command.It is corrected.
18.- Conclusion - does the presented method work only to the considered specific scenario or it can be extended to the other cases ?
- reply: The presented method work can be extended to the other cases.
19.What computing power were used to perform the necessary simulations?
- reply:We used 20 virtual machines to perform the necessary simulations.
20.References - cursory literature sources
- reply: It is corrected.
21- Figure - figure markings are in two languages
- reply: It is corrected.
Reviewer 2 Report
In the opinion of the reviewer, the edition of the paper is very careless and makes it difficult to understand. The presented diagrams and drawings are illegible and contain untranslated text. The text is not visible even when zoomed in. Reference titles are written in lower case, without punctuation marks, as well as subsection titles (e.g. 2.5).
The authors present the A3C algorithm as their achievement: “In order to improve the learning efficiency, the A3C (Asynchronous Advantage Actor-Critic) algorithm is proposed”. This algorithm is well known with the code available: https://paperswithcode.com/paper/asynchronous-methods-for-deep-reinforcement#code. The A3C algorithm was chosen, not proposed. In this case, it is appropriate to make a comparison with other calculations.
The training sets and the methods themselves are not sufficiently described. Nothing can be read from Fig. 3. The description of Fig.4 and the diagram itself do not explain anything else. According to the reviewer, this interesting work requires more careful editing, additions and a major revision.
Author Response
Reviewer 2
1.The presented diagrams and drawings are illegible and contain untranslated text.
- reply: It is corrected.
- The text is not visible even when zoomed in.
- reply: fig.1 is just a schematic.
3.Reference titles are written in lower case, without punctuation marks, as well as subsection titles (e.g. 2.5).
- reply: It is corrected.
4.The authors present the A3C algorithm as their achievement: “In order to improve the learning efficiency, the A3C (Asynchronous Advantage Actor-Critic) algorithm is proposed”. This algorithm is well known with the code available: https://paperswithcode.com/paper/asynchronous-methods-for-deep-reinforcement#code. The A3C algorithm was chosen, not proposed. In this case, it is appropriate to make a comparison with other calculations.
- reply: It is corrected. And a comparison with other calculations is show in 3.4.
5.The training sets and the methods themselves are not sufficiently described. Nothing can be read from Fig. 3.
- reply: The simulation experiment environment for model training is shown in Figure 3, which mainly includes three parts.
6.The description of Fig.4 and the diagram itself do not explain anything else.
- reply: the change trend of learning times and awards is shown in Figure 4
Reviewer 3 Report
A stereoscopic projection policy optimization method using deep reinforcement learning is proposed. The paper is well organized. The readability is also good. Some revisions are needed. All Figures except for Fig. 1 should be replaced to remove the non-English characters and the overlapping layers. Fig. 1 (a) is not clear enough to read the numbers.
Author Response
Reviewer 3
1. All Figures except for Fig. 1 should be replaced to remove the non-English characters and the overlapping layers.
- reply: It is corrected.
2. Fig. 1 (a) is not clear enough to read the thnumbers.
- reply: There are too much data . Fig. 1 (a) is just a schematic diagram.
Round 2
Reviewer 1 Report
The quality of work is much better. All gaps have been filled.
The work may be published
Author Response
Thank you for giving us the opportunity to revise and improve our paper, and thank all the reviewers for their careful review and valuable comments.
Now we are replying to the comments by the reviewers one by one, and all the revised parts have been marked in highlight in the revised version.
Once again, I would like to thank you for the opportunity to publish our papers.
Reviewer 2 Report
The authors did not avoid correcting errors, although they limited themselves to basic corrections, e.g. :
line 45 - Firstly,the
line 47 - structed.Secondly, By
line 360 - Figure 3. simulation
line 397 - Figure 5. cumulative
References are still not properly described.
Author Response
1.line 45 - Firstly,the
Reply: The error is corrected.
2.line 47 - structed.Secondly, By
Reply: The error is corrected.
3.line 360 - Figure 3. Simulation
Reply: The same errors in paper are corrected.
4.line 397 - Figure 5. Cumulative
Reply: The same errors in paper are corrected.
5.References are still not properly described.
Reply: The errors are corrected.
