The Use of the K-Sim Polaris Simulator in the Process of Automatic Assessment of Navigator Competence in the Aspect of Anticollision Activities

Round 1

Reviewer 1 Report

Require proofread of the paper

 

1. There are grammatical errors like: "are a big challenge'

2. The standard of English is not so good, that's why recommended proofreading the text

3. MAIB data mentioned is very old 1994-2003, recommend to include recent data.

4. What do you mean by 12 or 6 Mm in line 326

5.  In line 456, please mention the value of co-relation

6.  In this paper the assessment is done for the Colreg Rules 6,7, 15,16 & 17, what about the assessment for other Rules

7. Recommend improving the conclusion and the limitations of the SEA assessment

 

Author Response

Thank you for your submitted review of the article and your comments and suggestions. The following corrections have been made as a result:

• The text of the paper has been corrected to remove grammatical and linguistic errors.
• I agree that the data in the MAIB report was based on older data (1994-2003), but such a cross-sectional analysis allows to better highlight the human factor as the primary cause of accidents (lack of proper observation) compared to annual reports. In contrast, an analysis of more recent data from 135 accidents (2010-19) is presented in the article [2], in which the human factor is also identified as the main cause of collisions.
• This was a translation error, should be NM of course. The initial observation range (at the beginning of the exercise) should allow the acquisition of objects detected by radar. Therefore, choosing a range other than 6/12 Mm resulted in penalty points (poorly conducted radar observation).
• The term correlation was inaccurate, taken directly from the cited article, a more appropriate term was used now: large degree of agreement.
• Due to the assessment parameters available in the SEA programme, it was decided to use an open water scenario in restricted visibility conditions during the exercise. This allowed a concentration on radar as the main device used to obtain information about objects and its interpretation (tracking). Consequently, the assessment is related to the use of the following regulations: 5, 6, 7, 8 and 19. Regulations 9, 10 and 11 to 18 were not applicable in this case as they concern restricted areas (9,10) or good visibility conditions (Chapter II of COLREG). For other navigational conditions, a separate test scenario should be prepared with different values and dependencies of the assessment criteria.
• The conclusions section has been extended to include additional suggestions for the development of the SEA system.

Reviewer 2 Report

Work describes the use of full bridge simulator in the assessment process. As noted in the article, STCW Convention does not specify competency assessment methods to be used during the examination. This leaves a considerable scope for interpretation of these methods and the possibility of using different evaluation methods.

With the increasing use of ICT technologies in the maritime domain, it's of utmost importance to use simulators in the training and assessment of future officers. The analysis presented in this paper is mainly needed to define appropriate assessment methods that can be included in the IMO or STCW extension. I've no further comments on this paper.

Author Response

Thank you for your submitted review of the article and your comments and suggestions.

Reviewer 3 Report

In the manuscript it is analyzed the automatic assessment of ships officers’ competencies using a certain full mission bridge simulators, specifically, in the application of COLREG rules (anti-collision). Considering that, the application of simulation is a reality today in the field of education/training, the fact that instructors are confident that they are assessing appropriately to future officers, it contributes to improving the safety in the shipping sector. For that reason, thank you for the interesting manuscript on a very important and interesting topic in the maritime studies.

The presentation of the problem is good and the methods and the results are in general consistent. However, some points should be reviewed before consideration for publication:

-          Although they are well known in the shipping sector, the first time that acronyms are introduced, its meaning should be included in parentheses [e.g. IMO (line 65), STCW (line 68), SEA (line 144), DWT and LOA (Table 1)].

-          It is recommended that the Introduction section is finalized with a brief paragraph which serves as a presentation of the problem dealt with in the following sections.

-          Denomination must be reviewed because in some parts it is very difficult to follow the arguments. Authors name ALPHA and BRAVO to ships (lines 374 and 378) but also they use the same name to routes (line 417). Furthermore, I do not know what you are referring with “bridge name” (Table 2).

-          Although in the presentation of the problem it is mentioned that a group of students (future deck cadets) were used, in the sample breakdown (Table 2), it is not observed any student. Please, an explanation is needed.  

-          Furthermore, in the sample used, only 13 participants were considered. From a statistical point of view may not be enough (usually, it is studied a minimum of 25 elements in the sample). Therefore, authors should be justify how such a small sample allows them to obtain valid and reasonable conclusions.

-          Please, a revision is required in the section 6, where authors define “the main conclusions are” (line 487), because the conclusions there mentioned are so generic that they can be read in any simulator user’s manual. Authors must finalize the manuscript with the main conclusions referring to the present research.

-          Authors must revise the Instructions for Authors of JMSE template and correct accordingly the last sections (lines 505-506 and lines 551-548).

Author Response

Thank you for your submitted review of the article and your comments and suggestions.

Regarding specific comments:

• Obviously, the acronyms used should have been explained when they are used first time, this has been corrected where it was missing.
• As suggested, a paragraph at the end of the Introduction section has been added.
• The designations/names of the respective simulator bridges have been added in Figure 1 which should make the text easier to understand and analyze. In this sense the ALPHA route means the route taken by the ALPHA vessel.
• The mentioned group of students participated as evaluated person but not assessors or experts. This is the reason for the absence of their opinions in Table 2. The data from these evaluations formed the basis for the evaluation of the SEA application work and therefore more experienced navigators were used.
• I agree that a larger number of participants in the study would have been more appropriate. During the verification of the evaluation criteria, a larger number of experts participated, but in a less formalized form. A different group of experts were used at the stage of evaluation of the SEA application work than at the start of the SEA application testing. Hence, a smaller number of evaluations are included.