Review Reports

Round 1

Reviewer 1 Report

This article is very interesting. The scientific approach is very good and the topic of the article fits the journal scopes. The Figures and tables are very clear and the conclusions are supported by data.

I suggest improving the section "references" by inserting more scientific papers about deep learning applications in transportation engineering. For example, I suggest the following papers  published in MDPI journals:

 

- Guerrieri, M,.; Parla, G. Deep learning and yolov3 systems for automatic traffic data measurement by moving car observer technique, Infrastructures, Volume 6, Issue 9, 2021 Article number 134

- Huang, R., Guan, W., Zhai, G., He, J., Chu, X. Deep Graph Reinforcement Learning Based Intelligent Traffic Routing Control for Software-Defined Wireless Sensor Networks (2022) Applied Sciences (Switzerland), 12 (4), art. no. 1951, . 

- Chen, G., Cheng, Y. A lightweight model for multi-traffic object detection based on deep learning under complex traffic conditions (2022) Journal of Applied Science and Engineering (Taiwan), 25 (3), pp. 527-535. 

- Ahmad, A.B., Tsuji, T. Traffic monitoring system based on deep learning and seismometer data (2021) Applied Sciences (Switzerland), 11 (10), art. no. 4590, .

Author Response

Dear Reviewer:

We are very grateful to Reviewer for reviewing the paper (Manuscript ID: applsci-1880143) so carefully. We have tried our best to improve the manuscript and revised some of the problems you pointed out in the paper.

 Responds to the reviewers' comments:Reviewer:l  I suggest improving the section "references" by inserting more scientific papers about deep learning applications in transportation engineering. Response: We appreciate it very much for this good suggestion, and we have done it according to your ideas. We have inserted several papers provided by you in the section "references"，such as [22][26].

 

Your sincerely,
First author: Yong Huang
E-mail: huangyonghy5250@163.com
Corresponding author: Yong Huang
E-mail: huangyonghy5250@163.com

Reviewer 2 Report

Urban Road Following Control Based on Deep Reinforcement Learning

The title is a bit confusing: the paper is about vehicle-following or car-following not road following; there is no urban road in this paper as this paper is using simulation. 

1. Grammar issues.

Line 41-42, Check the grammar for ‘The stronger the exploration ability, the more comprehensive the mastery of the environment’. 

Check the grammar for Line 132-135.

Line 138, ‘the action space and action space’ duplication errors, should be state space and …

Not capitalised for the first word in Figure 1 caption.

Line 130, missing ‘a’ before ‘Markov decision process’.

Line 188, ‘optimization.’ Not a sentence.

Line 200, ‘But’ is redundant. 

Line 164, ‘we proposes’ should be ‘we propose’

Line 258, parentheses not necessary.

Typo errors: ‘longitudina’ in Table 1.

2. Format issues.

Spacing problems: A space is missing between a comma and the next sentence, between a period and the next sentence, between words and left parentheses, and even in author names (XinXu). For example, Line 233-237. Line 63 the TORCS(The Open Racing Car Simulator). And a space that should be deleted before a period (Line 237). Line 169. 

Citations problems: strange format and wrong citations. Please check citation format related requirements of this journal, and go through all this paper to change where appropriate. For example, 

- Ma and Qu [16] (Line 88) is wrong (reference No. 16 is by Sharma et al. rather than Ma and Qu).

- et al. is not correctly used: E.g., Line 105, Yihuan Zhang[19] should be Zhang et al [19]; Line 74, Gazis, Herman, and Rothery et al.[10] should be Gazis et al. [10], 

- First names are rarely seen in citations. I would use (Line 99) Li et al. [18] instead of W Li[18]; Line 90, Zhu and Zhang [17] instead of Z Wen-Xing, Z Li-Dong[17]; Line 112, Masmoudi et al. [21] instead of Mehdi Masmoudi[21] et al. Same for M Zhu[22], H Gao[24] and others.

- First names were used as last names in citations. For example, citation No. 17: Wen-Xing, Z.; Li-Dong, Z. A new car-following model for autonomous vehicles flow with mean expected velocity field. Physica A: Statistical Mechanics and its Applications 2018, 492, 2154–2165. Wen-Xing and Li-Dong are first names.

3.Questions. 

How the values of r1=1, r2=0.2, r3=0.05 are determined? How the method is sensitive to these values.

Line 202-203, ‘The following will introduce the division method of trajectories in the start-stop phase:’, but directly follows are bullet points for both start-stop phase and steady phase.

Line 204-209, the definitions of start-stop phase and steady phase are unclear and have grammar issues. Line 212, the speed (of the rear vehicle?) remains the same? Considering these definitions, there seems to be a lot of gaps in the trajectories which do not belong to either start-stop phase or steady phase: For example, consider the following scenario: Starting at the steady phase when the front car and the rear car are maintaining a following space of (speedX_r x Tmth + 2), then the front car decelerated, so the rear vehicle will slow down as well, but distance is still within [speedX_r x Tmth, speedX_r x Tmth + 2], but at this timestep, the state does not belongs to steady phase (speed not remaining the same) or start-stop phase (stop phase requires inter-vehicle space to be smaller than speedX_r x Tmth).

The meaning of x-axis in Figure 4 and 6 is unclear. The meaning of the two dash lines in Figure 6 are also unclear. 

The number of training timesteps is confusing: Line 236 said Training timesteps are 10000, but Figure 4 only showed 1600 steps (What are the other steps) and said ‘1000000 Timesteps So Far’ - What are the 1000000 timesteps? Figure 6 showed 2000 (timesteps) but labelled 10000 timesteps.

Line 160, the paper said ‘the maximum speed of the preceding vehicle is set as speedX_p = 25 km/h’, but in Figure 6, the preceding vehicle exceeded 25 many times and even reached over 70 in the beginning, where 14 metres is not safe at all for these speeds.

There is no specific value for PPO in Figure 5, but from my observations from Figure 4 and 5, normal PPO is outperforming the proposed Subsection-PPO, so what is the advantage of the proposed Subsection-PPO over normal PPO?

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Most of my questions have been addressed.

Line 328, first names and last names were still in wrong order.

Line 208~212, steady phase and start-stop phase have been redefined in the revised version: steady phase removed the speed constraint; start-stop phase removed dist constraint but added new symbols 'flag' and 'max_speedX_p', the meaning of which has not been explained, making the definition of start-stop phase and steady phase still confusing. Make sure the definition of start-stop and steady phase are exclusive of each other and their union includes all possible situlations. 

Make sure the numbers in Figure 5 are still clearly readable if printed in A4 papers.

The two subplots in Figure 6 are two close - their y axises overlaped.

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf