Perspectives on Safety for Autonomous Vehicles
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis manuscript reviews and prospects the safety issues of autonomous systems. The authors conduct a review using autonomous vehicles as an example, and claim to propose future research directions from three aspects— that is, guardian-based safety architectures, functional decompositions and abstraction mechanisms. The research presented in this manuscript is comprehensive, with clear prospects, and plays a certain role in promoting the development of the industry. Below are my suggestions for this manuscript:
1. In the title, the authors state that the object under consideration is autonomous "systems". Correspondingly, in the first sentence of the abstract, the authors use autonomous "vehicles". I believe these two cannot be completely equated, and it is recommended to make adjustments.
2. Although this paper focuses on safety, given that the object under study is Cyber-Physical Systems, the network layer is a crucial component of the CPS. Security issues arising from the network can lead to safety concerns, so I recommend mentioning this issue at least in the Introduction section.
3. For a review article, an excessively long length is not reader-friendly. Therefore, I suggest adding the main framework of this paper after the first section. This will help readers quickly grasp the authors' thinking and locate the parts they are interested in for focused reading.
4. Regarding Table 1 on Page 8, the capitalization in the table header needs to be revised, such as "iso" and "sotif". There is a missing period at the end of the paragraph above this table. Additionally, the main text seems to lack a brief description of the content of Table 1.
5. Section 6 is the core content of this manuscript. In my view, there seems to be a lack of discussion on the relationship between specific AI technologies and the research content here, which makes this part relatively broad and not specific enough when read as a whole.
6. The title of Figure 6 does not seem to align with its content; please check this.
7. The first paragraph in the Conclusions section appears out of place and unclear. In my view, this part should summarize the main findings of this paper rather than present some descriptive knowledge.
Author Response
First, thank you for taking the time to review the manuscript, it was helpful. The paper has three objectives:
- Using first principles, outline in high level terms the crux of how governance/safety/V&V was historically done, how it has evolved, and the current approaches. While not new, this sort of information does not seem to be readily available, and we thought it was important to state.
- Define the challenges introduced by AI and the reasons that the current methods do not work.
- Introduce the tooling from another space (semiconductors) which has very different approach to the problem.
The three communities (Safety, IT V&V, Semi V&V) are quite siloed, and we feel strongly that the leverage of cross-domain techniques has merit. This thought is not well represented in the literature and forms the motivation for our work. We also believe that this research vector has a great deal of potential, so we want to expose it to the broader community.
With that, let me address your more specific points...
Below are my suggestions for this manuscript:
1. In the title, the authors state that the object under consideration is autonomous "systems". Correspondingly, in the first sentence of the abstract, the authors use autonomous "vehicles". I believe these two cannot be completely equated, and it is recommended to make adjustments.
>> fair enough .... we do believe that the fundamental issues cross domains (ground, marine, airborne, space), but it is fair to limit ... I have changed the title.
2. Although this paper focuses on safety, given that the object under study is Cyber-Physical Systems, the network layer is a crucial component of the CPS. Security issues arising from the network can lead to safety concerns, so I recommend mentioning this issue at least in the Introduction section.
>> again a very fair point ... in fact, security is part of assurance... given the already broad nature of the paper, we decided not to build on this point. I do agree with you that a broader assurance argument is proper.
3. For a review article, an excessively long length is not reader-friendly. Therefore, I suggest adding the main framework of this paper after the first section. This will help readers quickly grasp the authors' thinking and locate the parts they are interested in for focused reading.
>> agree ... restructured to put into appendix content which is more explanatory in nature.
4. Regarding Table 1 on Page 8, the capitalization in the table header needs to be revised, such as "iso" and "sotif". There is a missing period at the end of the paragraph above this table. Additionally, the main text seems to lack a brief description of the content of Table 1.
>> I think fixed
5. Section 6 is the core content of this manuscript. In my view, there seems to be a lack of discussion on the relationship between specific AI technologies and the research content here, which makes this part relatively broad and not specific enough when read as a whole.
>> updated introduction to better connect the focus of the paper
6. The title of Figure 6 does not seem to align with its content; please check this.
>> fixed
7. The first paragraph in the Conclusions section appears out of place and unclear. In my view, this part should summarize the main findings of this paper rather than present some descriptive knowledge.
>> LOL... you are very diplomatic ... again agree ... took it out
Reviewer 2 Report
Comments and Suggestions for AuthorsPlease find my comments in the attached PDF
Comments for author File: 
Comments.pdf
Author Response
First, thank you for taking the time to review the manuscript, it was helpful. The paper has three objectives:
- Using first principles, outline in high level terms the crux of how governance/safety/V&V was historically done, how it has evolved, and the current approaches. While not new, this sort of information does not seem to be readily available, and we thought it was important to state.
- Define the challenges introduced by AI and the reasons that the current methods do not work.
- Introduce the tooling from another space (semiconductors) which has very different approach to the problem.
The three communities (Safety, IT V&V, Semi V&V) are quite siloed, and we feel strongly that the leverage of cross-domain techniques has merit. This thought is not well represented in the literature and forms the motivation for our work. We also believe that this research vector has a great deal of potential, so we want to expose it to the broader community.
From your review, it seems that you are not very familiar with the semiconductor V&V process... which is quite different from the other processes discussed. This is likely reflective of some percentage of the reading audience. Based on your feedback, I have refactored the paper on three points:
1) Build an introduction with outline (as suggested) and more content on Why we are interested in semiconductor V&V.
2) addressed directly your point about "aren't semiconductors already part of current designs validated by ISO 26262" As you see, the level of rigor is quite a bit different.
3) combined various sections for a clear flow based on the high-level objectives.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe manuscript seems to talk about something different from the assessment by the title. The title says safety perspectives but safety is marginally considered, it is not the strict focus but a consequence of all the argumentation.
The first three chapters cover several aspects but the goal remains still unclear as well as some practical example and application of the desired intent of the manuscript.
The last chapters are very well described and detailed going to some conclusions. However, the safety intended in a road and user perspective is marginally discussed. It is all about the V&V and how to maximize systems to improve the AV capabilities.
I suggest authors to rearrange part of the manuscript keeping in mind the goals of research. Maybe, uttering at the beginning the goals and how they want to pursue them would help readers in getting the scope of research and reading adequately the manuscript. Otherwise, it could be ambigous and distant from the declarations made by the title.
Author Response
First, thank you for taking the time to review the manuscript, it was helpful. The paper has three objectives:
- Using first principles, outline in high level terms the crux of how governance/safety/V&V was historically done, how it has evolved, and the current approaches. While not new, this sort of information does not seem to be readily available, and we thought it was important to state as a baseline.
- Define the challenges introduced by AI and the reasons that the current methods do not work.
- Introduce the tooling from another space (semiconductors) which has very different approach to the problem.
The three communities (Safety, IT V&V, Semi V&V) are quite siloed, and we feel strongly that the leverage of cross-domain techniques has merit. This thought is not well represented in the literature and forms the motivation for our work. We also believe that this research vector has a great deal of potential, so we want to expose it to the broader community.
In terms of your feedback, it is fair enough comment .... it is difficult to structure cross-community scope. In the revision, I have taken three actions:
1) refactored out into Appendix sections content which is good for completeness but not directly on point
2) added an outline to the paper to better guide reader
3) brought into the introduction more directly the fundamental reasons that Semiconductor methods are interesting in this context.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors' revisions have addressed my concerns, and I therefore recommend acceptance of this manuscript, provided that the similarity rate meets the requirements.
Reviewer 2 Report
Comments and Suggestions for AuthorsI appreciate the authors for their efforts in addressing my previous comments. I do not have additional comments. I am ok to accept the paper in current form.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe authors massively revised the manuscript and provide a solid structure to the work. It seems extremely modified pointing towards a well-defined goal.
The manuscript investigates deeply the perspectives that V&V has on safety, discussing point by point all the possible criticalities and challenges.
