A Comprehensive Radar-Based Berthing-Aid Dataset (R-BAD) and Onboard System for Safe Vessel Docking^†

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper presents a multi-radar ship berthing aid platform on a ferry in addition to a curated dataset named R-BAD with synchronized radar point clouds and video, and an evaluation pipeline with DBSCAN clustering which works with a Kalman tracking in conjunction with supervised classifiers. The comments below can be used to improve the paper:

 

1. Besides short range FMCW radars, authors can also mention chirps and chirping such as the Dual-mode time domain multiplexed chirp spread spectrum in [REF01] and the dual-mode chirp spread spectrum modulation. Kindly include the works.
2. Please clarify how it was managed to prevent mutual radar interference when three devices in the 77-81 GHz band operate at the same time with similar chirp parameters while another device runs at 60-64 GHz? Clarify if you used time division or antenna isolation or any other sort of interference management.
3. Is the R-BAD radar operating in monostatic or bistatic mode ? How do you cope with self-interference inherent to monostatic architectures ?
4. Kindly add more explanations on noise reduction and data clutter removal.
5. Why is Doppler velocity not used in clustering or in the tracker measurement even though Doppler is logged per detection and later shown to be among the top features in classification importance plots?
6. How to measure the end to end detection delay from first visibility to a Tracking ID, and what is the effect of this delay on berth approach timing at typical approach speeds?
7. Please explain more the kalman filtering and indicate the data association rule in Kalman tracking.
8. Please provide more information on how the five docking stages are used beyond descriptive statistics.
9. In simulations, please provide a timing budget in addition to distribution of total delay from first return to class decision in the arrival category that you target for docking support.
10. Also, is it possible to provide results for smaller buffers ?
11. In addition, in simulations, the MCSS geofence starts capture when the vessel crosses a boundary computed with the Haversine method. Did the chosen radii ever start logging after the dock already entered the radars field of view, which would bias the simulated online chain by losing the early part of Arrival?
12. There are some typos like in the conclusions section where you have “classifier types (Random Forest, XGBoost, PointNet, and Graph Neural Networks) Notably,”, you are missing a fullstop that should be “classifier types (Random Forest, XGBoost, PointNet, and Graph Neural Networks). Notably,”

 

 

References

[REF01] "Dual-mode time domain multiplexed chirp spread spectrum." IEEE Transactions on Vehicular Technology 72.12 (2023): 16086-16097.

Comments on the Quality of English Language

The English could be improved to more clearly express the research.

Author Response

Answers to Reviewer #1

Dear Reviewer,

thank you very much for your constructive comments. We sincerely appreciate the time and effort you dedicated to reviewing our work. Below, we provide our detailed responses to your comments, together with notes on the changes implemented in the revised manuscript. We also proceeded to minor changes in various typos and duplicate references.

Comment #1: “Besides short range FMCW radars, authors can also mention chirps and chirping such as the Dual-mode time domain multiplexed chirp spread spectrum in [REF01] and the dual-mode chirp spread spectrum modulation. Kindly include the works.”

Answer: Thank you very much for bringing [REF01] to our attention. We fully acknowledge the contribution of [REF01] in the field of chirping methods, particularly its innovative multiplexed chirped spread spectrum scheme (DM-TDM-CSS) for low-power wide-area networks (LPWANs).

However, the scope of our paper is different. Our contribution focuses on providing a comprehensive dataset of radar measurements for maritime collision avoidance, without introducing advances in radar technology or in specific chirping techniques. In our work, we employ the selected radar sensors (TI AWR1443, AWR1642, AWR1843, and IWR6843, see Table 1, line 155 of the revised paper) as-is, without delving into the technological aspects of their internal operation. For this reason, we have not included references on radar hardware technologies (e.g., IR-UWB, FMCW) or chirping methods in general. Instead, our citations of radar systems (e.g., ELVA, Refs. [7–8]) and prior works [5–6] aim solely to contextualize radar applications in maritime collision avoidance.

While [REF01] presents an important advancement in communication-oriented chirping methods that could potentially be extended to radar, to the best of our knowledge it has not yet been applied in maritime collision avoidance systems, which is the core focus of our study. Nevertheless, we remain open to further elaborating on this aspect and would be willing to consider citing [REF01] in a future revision should it be directly connected to radar applications in the context of maritime collision avoidance.

 

Comment #2: “Please clarify how it was managed to prevent mutual radar interference when three devices in the 77-81 GHz band operate at the same time with similar chirp parameters while another device runs at 60-64 GHz? Clarify if you used time division or antenna isolation or any other sort of interference management.”

Answer: Thank you very much for this valuable comment. We have observed that the specific radar sensors provided by TI do not interfere with each other, at least for the targets that were detected and recorded during the development of our proposed dataset. On top of that, we have used the same sensors in multiple experiments in the premises of our University and we never observed any interference whatsoever, even for targets of different sizes that are much closer to the radar sensors. Please note that, as mentioned in our answer to Comment #1 above, we did not develop these sensors and we are using them as-is.

On the other hand, there is direct coupling (=self-interference) from a radar sensor’s transmitter (Tx) path to the same sensor’s receiver (Rx) path, and this causes erroneous detections for targets at close proximity to the radar (roughly, 0-1 m away from the radar Tx-Rx antennas). We specifically refer to removing direct coupling artefacts in our previous work (Ref. [10] of the original paper). In this respect, and to clarify this point for the interested reader, we added the following lines to the revised paper:

“Coupling artefacts—false detections appearing near the radar transmit-receive antennas due to transmitter–receiver direct coupling—were eliminated by removing all detected points that are closer than 3 m away from the radar transmitter and receiver antennas.” (lines 298-301 of the revised paper)

“Please note that no interference between different co-located radar sensors was observed during our experiments.” (lines 303-304 of the revised paper).

 

Comment #3: “Is the R-BAD radar operating in monostatic or bistatic mode? How do you cope with self-interference inherent to monostatic architectures?”

Answer: Thank you for this comment that elaborates on the discussion of the previous Comment #2. Indeed, all TI’s radar sensors that are used in our research are operating in monostatic mode. As discussed in our answer to Comment #2, to remove self-interference we remove all detected points that are in close proximity to the radar sensor. In this respect, we added the lines 298-301 and 303-304 of the revised paper (please see our answer to Comment #2 above).

 

Comment #4: “Kindly add more explanations on noise reduction and data clutter removal.”

Answer: Thank you for this valuable comment. As explained in Section 3.7, an initial coordinate-based filtering is applied to the entire dataset to remove irrelevant detections (e.g. due to direct coupling effect). In addition, a second filtering stage is implemented during the annotation process, where DBSCAN is employed to discard unclustered detections and retain only coherent, physically meaningful clusters. This second filtering is applied exclusively to the annotated data used for training and testing the machine learning models, thereby reducing noise and clutter while ensuring the quality of the supervised dataset. Following your suggestion, an additional explanation of this procedure has now been added in Section 5.4, lines 603-607 of the revised paper, to better explain this aspect.

 

Comment #5: “Why is Doppler velocity not used in clustering or in the tracker measurement even though Doppler is logged per detection and later shown to be among the top features in classification importance plots?”

Answer: Thank you for this insightful comment. Indeed, we also had a similar train of thought: how to use the Doppler shift of each radar output point for clustering with DBSCAN. It turns out that one needs to carefully select a scaling factor to account for the different units of Euclidean coordinates x, y, z and Doppler shift or radial velocity (or any other velocity metric for that matter, e.g. v_x, v_y, v_z) – i.e. m vs. m/s. After some experimentation, it was observed that to assign a global scaling factor between coordinate and velocity figures is not straightforward and an appropriate scaling factor value changes with each measurement to maintain superior performance. It was, thus, decided to proceed with DBSCAN using spatial coordinates only (x-y-z), since it provided fairly good results (e.g. please see Table 5 of our original paper) – and the same is also true for Kalman-based tracking. In this respect, even though Doppler metrics are proven to be among the top features for classification, we decided not to proceed further on that direction for clustering and tracking. Also, please note that we consider our main contribution in this paper to be the collected and curated dataset per se, while the performance of the proposed system serves mainly for showcasing the applicability and eligibility of the proposed dataset for developing berthing-aid scenarios and techniques.

 

Comment #6: “How to measure the end-to-end detection delay from first visibility to a Tracking ID, and what is the effect of this delay on berth approach timing at typical approach speeds?”

Answer: Thank you for this valuable comment. While the proposed R-BAD dataset constitutes the primary contribution of this work, you are right in that the applicability of the dataset is affected by the timely provision of detection and classification output. Therefore, according to your suggestion, we hereby provide the total delay time from first target detection to classification output. This delay was measured on a standard PC equipped with an Intel i7-4790K processor and an NVIDIA GTX 750 GPU. The preprocessing step, from raw radar signals to filtered clusters, requires approximately 1.27 ms. The subsequent classification delay varies with the model employed: Random Forest ≈ 2.61 ms, XGBoost ≈ 2.68 ms, PointNet ≈ 4 ms, and GNN ≈ 14 ms. Considering that the anticipated ship velocity during berthing does not exceed 1 m/s, these delays are regarded as fit for real-time berthing operations. Overall, the results indicate that all tested methods are fast enough for practical deployment, with only the GNN showing comparatively higher but still acceptable latency. Even though the aforementioned time delay measures are retrieved from a standard PC, not much deviation is expected upon implementation on board powerful embedded computing units (contemporary Raspberry or NVIDIA Nano or similar equipment). All this discussion is also reflected in the revised paper, please see Section 6.8.2, lines 832–846.

 

Comment #7: “Please explain more the Kalman filtering and indicate the data association rule in Kalman tracking.”

Answer: Thank you very much for bringing this into our attention. Following your suggestion, Section 5.2 (lines 500-545 of the revised paper) has undergone significant revision to clarify the data association process. New cluster detections are now described as being matched to existing Kalman tracks using a nearest-neighbor method with a 5 m gating threshold. Unmatched tracks are maintained through prediction and removed after one MRS, while unmatched detections initialize new tracks.

Comment #8: “Please provide more information on how the five docking stages are used beyond descriptive statistics.”

Answer: Thank you for this valuable comment. Please note that the criterion for partitioning the collected data into five docking stages (Arrival, Departure, Port-Idle, Cruising – Ramp Opening, and Cruising – Ramp Closing) is two-fold: on the one hand, each stage corresponds to a different docking phase in terms of operational characteristics, safety concerns, available time for reaction by the ship crew etc. On the other hand, the radar environment also changes significantly: cruising with the ship ramp open vs. close corresponds to a major metal object in the close vicinity of the radar sensors and, subsequently, a large cluster of points that remains present for the entire corresponding stage. To better demonstrate this criterion, we added the following in the revised paper: “This partitioning criterion is two-fold: on the one hand, each stage represents a different operational situation with different requirements for crew alertness, safety concerns and available reaction time in case of emergency. On the other hand, different docking stages correspond to different radar environments.” (lines 282-285 of the revised paper).

Furthermore, to further clarify the usage of the proposed dataset categories, we changed and added the following lines 335-345 in the revised paper: “Among them, the most critical for analysis is the Arrival category, since it represents the most critical docking stage in terms of operational alertness, safety concerns, reaction time available to the crew due to the ship being in close quarters to the dock, etc. This category will be the primary focus in the next section and will be used for the classification of the dock. The remaining categories are provided in the aforementioned open-source R-BAD dataset with the aim of inspiring researchers to use them for their own projects, clustering, tracking and classification techniques development, etc. Having said that, the Arrival category has an average measurement duration of 4 minutes and 30,480 annotated rows and corresponds to 123 recordings from 13 different ports (see Table 2); these ports are mainly located in the South-East Aegean Sea and the wider Attica regions, as illustrated in Fig. 5.”

 

Comment #9: “In simulations, please provide a timing budget in addition to distribution of total delay from first return to class decision in the arrival category that you target for docking support.”

Answer: Please see our answer to your Comment #6 above.

 

Comment #10: “Also, is it possible to provide results for smaller buffers?”

Answer: Thank you for this insightful comment. It is true that we tried many different sizes for the buffer that we use and we ended up with a size of 20 after extensive trials. We added a paragraph that contains results for a smaller buffer size (MRS of 10 frames, corresponding to 0.5 s) in Section 5.3 (lines 569–579); there results are also tabulated in Table 5 of the revised paper. These results serve to showcase the inferior performance of this new buffer size as compared to the 20-frame MRS, which was ultimately selected.

 

Comment #11: “In addition, in simulations, the MCSS geofence starts capture when the vessel crosses a boundary computed with the Haversine method. Did the chosen radii ever start logging after the dock already entered the radars field of view, which would bias the simulated online chain by losing the early part of Arrival?”

Answer: The geofences are manually crafted for each port and correspond to a safe distance from the port to allow ample time for the device boot-up and start of radar measurements. Subsequently, we did not observe any incident where the radar logging started after the dock already entered the radars’ range. To clarify that for the interest reader, we added the following lines 195-198 of the revised paper: “It should be noted that the geofence boundaries are manually designated at a safe distance from the port to allow ample time for device boot-up and start of radar sensor logging be-fore the dock enters the radars’ operational range.”

 

Comment #12: “There are some typos like in the conclusions section where you have “classifier types (Random Forest, XGBoost, PointNet, and Graph Neural Networks) Notably,”, you are missing a fullstop that should be “classifier types (Random Forest, XGBoost, PointNet, and Graph Neural Networks). Notably,”

Answer: Thank you very much for pointing this out. We have now corrected it in the revised paper.

 

References:

[REF01] "Dual-mode time domain multiplexed chirp spread spectrum." IEEE Transactions on Vehicular Technology 72.12 (2023): 16086-16097.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

This is a new and interesting paper focused on the vessel navigation in a harbor. It has merit but I have some concerns that should be addressed.

Why are all radars placed close by? Deploying radars in different positions could improve the detection by adding more diversity.

Did you address the problem of having radars working in a similar frequency range, e.g., frequency interferences?

Please, improve your literature review with recent articles in the field, e.g., E. Cardillo and L. Ferro, "Multi-Frequency Analysis of Microwave and Millimeter-Wave Radars for Ship Collision Avoidance," 2022 Microwave Mediterranean Symposium (MMS), Pizzo Calabro, Italy, 2022, pp. 1-4, doi: 10.1109/MMS55062.2022.9825520.

Is the maximum detectable radial speed of 1 m/s enough for each case of this application?

Are the angular capabilities of your radars exploited? Consider also that they are placed practically in the same position, whereas different positions can also enhance the angular information.

How does your work compare with scientific literature?

Author Response

Answers to Reviewer #2

Dear Reviewer,

thank you very much for your constructive comments. We sincerely appreciate the time and effort you dedicated to reviewing our work. Below, we provide our detailed responses to your comments, together with notes on the changes implemented in the revised manuscript. We also proceeded to minor changes in various typos and duplicate references.

 

Comment #1: “Why are all radars placed close by? Deploying radars in different positions could improve the detection by adding more diversity.”

Answer: Thank you for bringing this subject to discussion. We had similar concerns during the development and deployment of our platform. Unfortunately, this layout was the only one that was allowed by the company that owns the passenger ship. For safety reasons, they allowed us to put the radar at a high level from the sea and away from passenger areas. Similarly, for reasons that had to do with supplied power and the day-to-day work of the ship crew, they allowed us one single location for radar locations. Please note that the company generously supplied the ship as a carrier platform for our device without financial interest whatsoever; we are deeply grateful to them for this and also to the ship crew for their help in installing the platform.

To better clarify this concern for the interested reader, we added the following lines 867-872 to the revised paper (“Conclusions” section): “Another point of interest is that, due to safety concerns, the radar was deployed at a position on the ship that is sub-optimal. The high rise from the sea level and the high angle of detection are competing with radar range and field-of-view exploitation. Despite this fact, the performance of all clustering, tracking and classification methods is very satisfactory, thus demonstrating a promising pathway for future implementations - either retrofits or system installments at newly built ships.”

 

Comment #2: “Did you address the problem of having radars working in a similar frequency range, e.g., frequency interferences?”

Answer: Thank you very much for this insightful comment. During our experiments, we never observed any indications of interference between different radar sensors working in the same frequency range, at least for the targets that were detected and recorded during the development of our proposed dataset. On top of that, we have used the same sensors in multiple experiments in the premises of our University and we never observed any interference whatsoever, even for targets of different sizes that are much closer to the radar sensors. Furthermore, please consider that we employ the radar sensors (TI AWR1443, AWR1642, AWR1843, and IWR6843, see Table 1, line 161 of the revised paper) as-is, without delving into the technological aspects of their internal operation.

On the other hand, we observed self-interference in terms of direct coupling between Tx-Rx antennas. This causes erroneous detections for targets at close proximity to the radar (roughly, 0-1 m away from the radar Tx-Rx antennas), therefore we eliminate nearby detected points. In this respect, and to clarify this point for the interested reader, we added the following lines to the revised paper:

“Coupling artefacts—false detections appearing near the radar transmit-receive antennas due to transmitter–receiver direct coupling—were eliminated by removing all detected points that are closer than 3 m away from the radar transmitter and receiver antennas.” (lines 298-301 of the revised paper)

 

Comment #3: “Please, improve your literature review with recent articles in the field, e.g., E. Cardillo and L. Ferro, "Multi-Frequency Analysis of Microwave and Millimeter-Wave Radars for Ship Collision Avoidance," 2022 Microwave Mediterranean Symposium (MMS), Pizzo Calabro, Italy, 2022, pp. 1-4, doi: 10.1109/MMS55062.2022.9825520.”

Answer: Thank you very much for bringing this paper to our attention. We have added the following lines 69-71 in the revised paper: “Indeed, there is recent evidence in the literature that microwave and millimeter wave radar sensors have the capacity to capture obstacles and objects within the harbor area [7].”

We also added Ref. [7] in the References section and renumbered accordingly the remaining references in both the main body of the paper and the References section.

 

Comment #4: “Is the maximum detectable radial speed of 1 m/s enough for each case of this application?”

Answer: Thank you for highlighting this important aspect of the radar parameter settings. The maximum speed of this type of Ro-Ro/Passenger ferry ship during docking is very low, typically around 0–2 knots, which corresponds to approximately 0–1 m/s. Careful examination of our measurements shows that once an obstacle enters the radar’s Field of View (FoV), it is almost never lost. This observation confirms that setting the maximum speed to 1 m/s is an appropriate choice.

 

Comment #5: “Are the angular capabilities of your radars exploited? Consider also that they are placed practically in the same position, whereas different positions can also enhance the angular information.”

Answer: Thank you for this comment. Yes, totally, the angular capabilities of our radars are exploited. We have used the maximum number of onboard available transmitters and receivers, as indicated in Table 1 of the original paper (please see entry “Number of Antennas” in Table 1). Subsequently, the AWR1443, AWR1843 and IWR6843 radar sensors provide output with complete range and angular information (complete x-y-z coordinates of the detected points), whereas AWR1642 provides output with angular information in the azimuth plane only. To further clarify this aspect we added the following 156-160 lines in the revised paper: “Please also note that, according to the radar sensor specifications, the number of antennas used (entry “Number of Antennas”, Table 1) allows for angular output information at both the azimuth and elevation planes for the AWR1443, AWR1843 and IWR6843 sensors, and for the azimuth plane for the AWR1642 sensor.”

 

Comment #6: “How does your work compare with scientific literature?”

Answer: Please note that we have included a literature survey in our original paper. This literature survey is now improved with the newly added reference [7] to the work of Cardillo and Ferro. Given that neither this paper includes measurements or a dataset but rather simulation results only, as well as that it is a fact that there is no evidence on the performance of clustering, tracking or classification of radar output in docking scenarios in the literature, we still consider that there is a gap (please see lines “Despite this gap, and in fact motivated precisely by it” in our original paper) with regards to radar employment for autonomous navigation in close quarters and especially during docking. Thereupon, we invested substantial effort to not only provide a dataset as-is but also showcase its applicability and eligibility by jointly offering a complete roadmap for clustering, tracking and classification techniques and relevant performance results. Furthermore, our system underwent extended testing under real-world operational conditions. To the best of our knowledge, this is the first time that comprehensive results are presented as of the applicability of radar sensors installed onboard a ship (rather than included in the port infrastructure) serving for autonomous navigation during docking.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The reviewer has no more comments.

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for addressing my concerns, I recommend publishing in present form.