Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

1. Chapter 2 reviews the current state of research on LLM, RAG, and agents, but lacks an analysis of the shortcomings and limitations of existing work. It is recommended that Chapter 2 include the research needs and innovation points of this paper.
2. The third paragraph of Section 2.1 contains “[13?]”, and the fourth paragraph of Section 2.2 contains “[?]”, indicating errors in the citation format. It is recommended that the reference numbering throughout the entire document be uniformly checked and corrected.
3. The paper repeatedly emphasizes the need for real-time and near-real-time data processing throughout the text. However, the ONTUS and Ferry Trips datasets used in Section 3.2 are static historical data, which contradicts the preceding discussion. It is recommended to supplement the experimental design section with relevant content such as streaming data processing, real-time response evaluation, or system latency assessment.
4. The term “passenger car equivalents (PCE)” in Section 3.2 and “PCE (Passenger Car Equivalent)” in Chapter 5 exhibit redundant phrasing and inconsistent formatting. It is recommended to standardize the terminology by ensuring consistent usage upon first and subsequent appearances throughout the document to maintain uniformity.
5. The comparison models in Section 3.3 only include classic models such as Claude, Llama, and Qwen, which may not fully demonstrate the performance advantages of the proposed model. It is recommended to supplement with other classic models to ensure the comprehensiveness of the comparative experiments.
6. Chapter 4's experimental section only presents overall model performance results, lacking ablation studies to validate the independent contributions of each module. We recommend adding ablation experiments, such as removing modules like RAG, SQL Agent, and Python Agent individually, to assess the impact of a single module on the model's overall performance.
7. Figure 1 includes an “embedding model fine-tuning” module, but the main text does not describe this module. We recommend modifying the relevant content in the figure or supplementing the main text with a description of its function and implementation method.
8. The main text lacks an explanation and data analysis for Table 1. It is recommended to add corresponding descriptions and analysis where the table first appears to help readers understand the experimental results.
9. Some test results in Figure 3 are shown as 0.000, but the paper does not explain the reason for this. It is recommended to supplement Section 4.1 with an explanation of this situation and discuss its impact on the experimental conclusions.

Author Response

Thank you very much for taking the time to review this manuscript. We genuinely appreciate your detailed feedback, which has helped us identify inconsistencies in our terminology, refine our experimental descriptions, and strengthen the contextual background of our research. Please find the detailed responses below and the corresponding revisions highlighted in the re-submitted files.

Point-by-point response to Comments and Suggestions 

Comments 1: Chapter 2 reviews the current state of research on LLM, RAG, and agents, but lacks an analysis of the shortcomings and limitations of existing work. It is recommended that Chapter 2 include the research needs and innovation points of this paper.

Response 1: Thank you for this valuable suggestion. We agree that a more critical analysis of the current landscape was necessary to highlight our contribution. Therefore, we have revised Chapter 2 to critically analyze existing shortcomings rather than strictly describing prior work. New paragraphs now explicitly address the limitations of standard RAG for structured data (Section 2.1) and the lack of verification in current agentic frameworks (Section 2.2). We concluded the chapter with a dedicated summary of our specific innovation points (Section 2.3) to clearly situate our contributions within these gaps.

Comments 2: The third paragraph of Section 2.1 contains “[13?]”, and the fourth paragraph of Section 2.2 contains “[?]”, indicating errors in the citation format. It is recommended that the reference numbering throughout the entire document be uniformly checked and corrected.

Response 2:  We have thoroughly checked the citation numbering throughout the entire manuscript and corrected the formatting errors in Sections 2.1 and 2.2, ensuring consistency across all references.

Comments 3: The paper repeatedly emphasizes the need for real-time and near-real-time data processing throughout the text. However, the ONTUS and Ferry Trips datasets used in Section 3.2 are static historical data, which contradicts the preceding discussion. It is recommended to supplement the experimental design section with relevant content such as streaming data processing, real-time response evaluation, or system latency assessment.

Response 3: We agree with the reviewer that the initial emphasis on real-time processing was misaligned with the static nature of the datasets used for validation. While the proposed architecture is capable of handling real-time streams, the current experimental validation relies on historical data. To address this, we have significantly tuned down the emphasis on real-time data processing throughout the paper to accurately reflect the experiments performed. We have also added a discussion regarding this limitation and noted that live streaming data integration is a subject for future work.

Comments 4: The term “passenger car equivalents (PCE)” in Section 3.2 and “PCE (Passenger Car Equivalent)” in Chapter 5 exhibit redundant phrasing and inconsistent formatting. It is recommended to standardize the terminology by ensuring consistent usage upon first and subsequent appearances throughout the document to maintain uniformity.

Response 4: We agree. We have standardized the terminology throughout the manuscript. The term is now defined upon its first appearance and used consistently thereafter to ensure uniformity.

Comments 5: The comparison models in Section 3.3 only include classic models such as Claude, Llama, and Qwen, which may not fully demonstrate the performance advantages of the proposed model. It is recommended to supplement with other classic models to ensure the comprehensiveness of the comparative experiments.

Response 5: We acknowledge the reviewer’s suggestion to expand the model selection. The current selection of models (Claude, Llama, and Qwen) was established based on the framework of the Bachelor’s thesis cited within this paper [Citation X], where these specific models were identified as the most promising candidates for this specific maritime domain task. As our study builds directly upon those findings to establish a baseline, we have opted to maintain this selection for consistency. We have added a comment in the text to clarify this selection criteria.

Comments 6: Chapter 4's experimental section only presents overall model performance results, lacking ablation studies to validate the independent contributions of each module. We recommend adding ablation experiments, such as removing modules like RAG, SQL Agent, and Python Agent individually, to assess the impact of a single module on the model's overall performance.

Response 6: Thank you for this suggestion. While we understand the value of ablation studies in many contexts, our system is designed as an interdependent agentic workflow where modules are functional requirements rather than additive features. For example, removing the SQL Agent would render the system incapable of retrieving structured database information entirely, rather than just degrading performance. Therefore, we believe an ablation study is out of scope for this specific paper, which focuses on the efficacy of the complete, integrated agentic pipeline.

Comments 7: Figure 1 includes an “embedding model fine-tuning” module, but the main text does not describe this module. We recommend modifying the relevant content in the figure or supplementing the main text with a description of its function and implementation method.

Response 7: Thank you for noting this discrepancy. We have clarified the text to align with the figure. We added the following description to the manuscript: "In the current implementation, we utilize a pre-trained embedding model without fine-tuning; however, the modular architecture supports replacing this with a domain-specific fine-tuned embedding model should improved retrieval performance for maritime-specific terminology and concepts be required in future deployments."

Comments 8: The main text lacks an explanation and data analysis for Table 1. It is recommended to add corresponding descriptions and analysis where the table first appears to help readers understand the experimental results.

Response 8: We agree that the table requires context. We have added a reference to Table 1 in Section 4.1, accompanied by a textual description and analysis of the data presented to assist the reader in interpreting the results.

Comments 9: Some test results in Figure 3 are shown as 0.000, but the paper does not explain the reason for this. It is recommended to supplement Section 4.1 with an explanation of this situation and discuss its impact on the experimental conclusions.

Response 9: Thank you for pointing this out. We have added a dedicated paragraph to Section 4.1 specifically addressing the instances of 0.000 scores. We explain the causes for these results (e.g., failure in retrieval or code generation) and discuss how these specific failures impact the overall assessment of the model's reliability.

Reviewer 2 Report

Comments and Suggestions for Authors

1. Title and Abstract

 

• Recommendation: Clarify the specific "structured data" types in the title or abstract.

  • Rationale: The term "structured data" is broad. The abstract mentions "complex maritime sensor data" and "time-series analysis". Specifying "Time-Series and Relational Sensor Data" in the title or opening would improve discoverability for specialists.

       

• Recommendation: Explicitly state the "gap" in current Agentic RAG research regarding industrial reliability.

  •  

    Rationale: The abstract mentions the need for "secure, role-constrained access", but does not explicitly state that current Agentic RAG frameworks lack robust validation protocols for numerical accuracy in high-stakes industries.

     

 

2. Introduction

 

• Recommendation: Refine Research Question 1 (RQ1).

  •  

    Rationale: Currently, RQ1 asks "How can an Agentic AI system be designed...". This is an engineering implementation question, not a research question. It should be reframed to focus on the challenges or architectural trade-offs necessary to enable such a system (e.g., "What architectural adaptations are required to bridge natural language indeterminacy with deterministic maritime sensor data?").

     

• Recommendation: Expand on the "Out-of-the-loop familiarity" concept.

  •  

    Rationale: The introduction cites DNV regarding seafarers becoming reliant on automation. The authors should explicitly connect how a Chatbot interface mitigates this, rather than exacerbating it. Does natural language query capability actually increase operator engagement, or does it further abstract the raw data?

     

 

3. Background

 

• Recommendation: Differentiate between "Text-based RAG" and "Numeric/Tabular RAG."

  •  

    Rationale: Section 2.1 discusses RAG generally. The authors must distinguish the specific difficulties of applying RAG to numerical time-series data (tokenization issues, hallucination of digits) versus standard text retrieval. This provides better context for why the "Agentic" approach (using tools) is necessary over standard semantic search.

     

 

4. Methodology (Major Revision Required)

 

• Recommendation: CRITICAL: Move the "Ground Truth Generation" methodology from the Results section to the Methods section.

  •  

    Rationale: Currently, the explanation of how ground truth was established (using Z-scores and Pearson correlation) is buried in Section 4.3 (Results), lines 686–691. This is a methodological protocol, not a result. A reader cannot evaluate the validity of the results in Section 4.1 without first understanding how the ground truth was derived.

     

• Recommendation: Detail the "Orchestrator" logic.

  •  

    Rationale: Figure 1 shows an "Orchestrating Agent" , but the text does not explain the decision logic used to route a query to the Python Agent vs. the DuckDB Agent. Is this purely prompt-based, or is there a router classification layer? This detail is vital for reproducibility.

       

• Recommendation: Include "Latency" as a core metric in Section 3.4.

  •  

    Rationale: The study focuses on "real-time sensor data". Agentic workflows involving multiple LLM calls and Python execution are inherently slow. The methodology must describe how inference time was measured, as this is a critical constraint for "Maritime AIoT" applications.

     

 

5. Results

 

• Recommendation: Improve Figure 3 (Heatmap) readability.

  •  

    Rationale: The labels in Figure 3 are illegible. The "Test Number" axis and the specific color gradients need to be clearer. The figure caption should explicitly state what "Calculated Factual Correctness" represents without forcing the reader to refer back to the text.

     

• Recommendation: Condense Section 4.3 "An End-to-End Test Case Scenario."

  •  

    Rationale: This section reads more like a discussion or a tutorial than empirical results. It should be shortened significantly to focus only on the output metrics of that specific test case, with the narrative description moved to a supplementary appendix or the Discussion section to illustrate the "Action Grounding" problem.

     

• Recommendation: Quantify the "PCE" error in Section 4.1.

  •  

    Rationale: The text mentions open-source models struggled with "statistical analysis". The authors should provide specific examples of how the answer deviated (e.g., magnitude of error) rather than just a binary correctness score.

     

 

6. Discussion

 

• Recommendation: Address the "Black Box" nature of Proprietary Models.

  •  

    Rationale: The results show Claude 3.7 is superior. However, for a maritime safety context, the inability to inspect the weights or hosting infrastructure of a proprietary model is a major risk. The discussion on "self-hosted solutions" needs to be more critical about the trade-off between accuracy (Claude) and data sovereignty (Qwen/Llama).

       

• Recommendation: Discuss Latency vs. Real-Time requirements.

  •  

    Rationale: The paper claims to address "real-time sensor data". If the Agentic RAG takes 30+ seconds to generate a Python script and execute it, does this meet the definition of "real-time" for ferry operations? This limitation must be addressed.

     

 

7. Conclusion

 

• Recommendation: Remove the phrase "exceptional 99.9% overall factual correctness."

  •  

    Rationale: In Section 4.1, the text states Claude 3.7 achieved "0.896" (89.6%). The Conclusion claims "99.9%". This discrepancy must be corrected. If 99.9% refers only to "Simple" queries, the conclusion must explicitly state "99.9% on simple queries" to avoid misleading the reader.

Author Response

Response to Reviewer 2 Comments

1. Summary

Thank you for your detailed and constructive feedback. Your comments regarding the structural organization of our methodology, the definition of our research scope, and the critical analysis of proprietary models have significantly improved the quality and clarity of this manuscript. We have addressed all your points, including moving the ground truth methodology to the appropriate section, refining our research questions, and adding necessary nuances regarding latency and data sovereignty.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: Clarify the specific "structured data" types in the title or abstract. The term "structured data" is broad. The abstract mentions "complex maritime sensor data" and "time-series analysis". Specifying "Time-Series and Relational Sensor Data" in the title or opening would improve discoverability for specialists.

Response 1: We agree that specificity improves discoverability. We have amended the abstract to explicitly specify "time-series and relational sensor data" instead of the generic "structured data" to accurately reflect the scope of the study.

Comment 2: Explicitly state the "gap" in current Agentic RAG research regarding industrial reliability. The abstract mentions the need for "secure, role-constrained access", but does not explicitly state that current Agentic RAG frameworks lack robust validation protocols for numerical accuracy in high-stakes industries.

Response 2: Thank you for this suggestion. We have revised the Abstract, Introduction, and the Agentic RAG subsection (Section 2.1) to explicitly articulate this gap. The text now clearly states that current frameworks lack robust validation protocols for numerical accuracy in high-stakes industrial environments, positioning our work as a direct response to this deficiency.

Comment 3: Refine Research Question 1 (RQ1). Currently, RQ1 asks "How can an Agentic AI system be designed...". This is an engineering implementation question, not a research question. It should be reframed to focus on the challenges or architectural trade-offs.

Response 3: We agree that the original formulation was too implementation-focused. We have reformulated RQ1 to focus on the architectural challenges: "What architectural adaptations are required to bridge non-deterministic natural language queries with deterministic sensor data in an Agentic RAG system?" We have updated all references to RQ1 throughout the text to reflect this change.

Comment 4: Expand on the "Out-of-the-loop familiarity" concept. Rationale: The introduction cites DNV regarding seafarers becoming reliant on automation. The authors should explicitly connect how a Chatbot interface mitigates this, rather than exacerbating it.

Response 4: We have added a new paragraph to the Discussion section addressing this crucial point. We explicitly detail how the system’s "Transparent Execution" and "Active Inquiry" mechanisms require the operator to engage with the underlying logic rather than passively accepting outputs. We have cited established human-factors research (Onnasch et al., 2014; Endsley, 2017) to support the argument that this design preserves operator competency.

Comment 5: Differentiate between "Text-based RAG" and "Numeric/Tabular RAG." Section 2.1 discusses RAG generally. The authors must distinguish the specific difficulties of applying RAG to numerical time-series data versus standard text retrieval.

Response 5: Thank you for this insightful suggestion. We have revised Section 2.1 to explicitly differentiate between "Text-based RAG" and "Numeric/Tabular RAG." The updated text details specific challenges inherent to numerical data—such as tokenization inefficiencies and digit hallucination—to clearly explain why standard semantic search is insufficient and why an Agentic approach with executable tools is necessary.

Comment 6: CRITICAL: Move the "Ground Truth Generation" methodology from the Results section to the Methods section. Rationale: Currently, the explanation is buried in Section 4.3. This is a methodological protocol, not a result.

Response 6: We fully agree with this correction regarding the manuscript structure. We have moved the systematic process for establishing ground truths (using Z-scores and Pearson correlation) to the Methods section (new Section 3.5). The Results section has been modified to retain only the illustrative example of Test Case 23, with a reference back to Section 3.5 for the methodology.

Comment 7: Detail the "Orchestrator" logic. Rationale: Figure 1 shows an "Orchestrating Agent", but the text does not explain the decision logic used to route a query to the Python Agent vs. the DuckDB Agent.

Response 7: We have updated Section 3.1 (System Architecture) to explicitly detail the orchestrator's routing mechanism. We clarified the decision logic used to determine when to route a query to the Python Agent versus the DuckDB Agent to ensure the reproducibility of our architecture.

Comment 8: Include "Latency" as a core metric in Section 3.4. Rationale: The methodology must describe how inference time was measured, as this is a critical constraint for "Maritime AIoT" applications.

Response 8: While we acknowledge latency is critical for deployment, the primary focus of this specific study was validating architectural accuracy and reliability. However, to address your concern, we have added a discussion on latency in the text. We clarified that while current inference times (30–60s) are experimental, they do not represent inherent architectural limitations, and we discussed strategies for optimization.

Comment 9: Improve Figure 3 (Heatmap) readability. Rationale: The labels in Figure 3 are illegible. The "Test Number" axis and the specific color gradients need to be clearer.

Response 9: We have increased the resolution and size of Figure 3 in the LaTeX build to ensure legibility. We also updated the caption to explicitly state what "Calculated Factual Correctness" represents to improve immediate comprehension.

Comment 10: Condense Section 4.3 "An End-to-End Test Case Scenario." Rationale: This section reads more like a discussion or a tutorial than empirical results.

Response 10: We have significantly condensed Section 4.3. We removed the tutorial-like enumerated steps and shortened the narrative description to focus strictly on the output metrics of the test case, moving the ground truth methodology to the Methods section as requested in Comment 6.

Comment 11: Quantify the "PCE" error in Section 4.1. Rationale: The text mentions open-source models struggled with "statistical analysis". The authors should provide specific examples of how the answer deviated.

Response 11: We have updated Section 4.1 to include concrete quantitative examples. We now explicitly mention the magnitude of error in specific test cases (e.g., CO2 emission calculations in Test Case #20) to substantiate the claims regarding the performance struggles of open-source models.

Comment 12: Address the "Black Box" nature of Proprietary Models. Rationale: The discussion on "self-hosted solutions" needs to be more critical about the trade-off between accuracy (Claude) and data sovereignty (Qwen/Llama).

Response 12: We agree that this is a major risk factor. We have added a new paragraph to the Discussion section that critically examines the trade-off between the superior accuracy of proprietary models (like Claude) and the necessity for data sovereignty and infrastructure inspection offered by open-source models (like Qwen/Llama) in safety-critical maritime contexts.

Comment 13: Discuss Latency vs. Real-Time requirements. Rationale: If the Agentic RAG takes 30+ seconds to generate a script, does this meet the definition of "real-time"?

Response 13: We have refined the discussion to clearly distinguish between "real-time data ingestion" (which the system handles) and "real-time control latency" (which requires faster inference). We clarified that the reported latency reflects the current experimental infrastructure rather than an architectural bottleneck, and that future work will focus on optimization for operational speeds.

Comment 14: Remove the phrase "exceptional 99.9% overall factual correctness." Rationale: In Section 4.1, the text states Claude 3.7 achieved "0.896" (89.6%). The Conclusion claims "99.9%". This discrepancy must be corrected.

Response 14: We apologize for this misleading phrasing. We have corrected the conclusion to accurately state "89.6% overall factual correctness (99.9% on simple queries)" to ensure the claim perfectly aligns with the data presented in Section 4.1.

Reviewer 3 Report

Comments and Suggestions for Authors

There are some recommendations that I hope will assist the author in improving their paper.

1) 1. Introduction

- In Lines 45, 68, 135, etc, use “ARAG” instead of “agentic RAG”. You already stated an abbreviation in Line 44. Use it.

- You should include a paragraph after Line 79 in which you can summarize the content of each section. For instance, Section 2 presents…….. Section 3 describes ……

2) 2.1. Large Language Models (LLMs) and Their Role in Supporting Data Analysis for IoT

- Please check the source in Line 104. It is “[13? ].”

- Move Figure 1 after Line 118.

3) 2.2. Agentic Retrieval-Augmented Generation (Agentic RAG)

- The title of this subsection should be only Agentic RAG. You already explained this abbreviation in Line 44.

- Please check the source in Line 151. It is: “[? ].”

4) 2.3. AI-driven Data Analysis in Maritime Applications

- Please check the source in Line 163. It is: “[? ].”

5) 3.2. Data

- In Lines 249-254, please provide the time period of data.

6) 3.4. Evaluation Metrics

- For each metric, please provide the computation and the range interval.

7) 3.5. Test Suite Design

- Mention the significance and unit of measurement for the X-axis.

- Please increase the resolution and the size of Figure 2.

- Use the template for the Figures with multiple subfigures. Use (a), (b), etc. in the caption of the figure (https://www.mdpi.com/files/word-templates/sensors-template.dot).

- Mention the meaning of the RT, FY, TP, HR, and DNR abbreviations as a note to Figure 2.

8) 4.1. Calculated Factual Correctness Metric

- Move Figure 3 after Line 347.

- Increase the size and resolution of Figure 3. The labels are hardly legible.

- Please cite Table 1 in the text.

9) 6. Conclusion

- Add limitations and future work of the review.

- Mention to whom the research is useful.

10) References

- Please provide the link or DOI for the datasets mentioned in sources [28] and [29]

- Where DOI is not available, please provide the link, for instance, for sources [2], [4], [5], etc.

Author Response

Thank you for your thorough review and precise editorial suggestions. Your feedback regarding the consistency of abbreviations, the visualization of our results, and the completeness of our references has been invaluable in polishing the manuscript. We have carefully addressed every point raised, particularly focusing on improving the resolution and formatting of Figures 2 and 3, ensuring all metrics are mathematically defined, and correcting citation errors.

2. Point-by-point response to Comments and Suggestions for Authors

Comment 1: 1. Introduction

• In Lines 45, 68, 135, etc, use “ARAG” instead of “agentic RAG”. You already stated an abbreviation in Line 44. Use it.

• You should include a paragraph after Line 79 in which you can summarize the content of each section. For instance, Section 2 presents…….. Section 3 describes ……

Response 1: Thank you for noting these inconsistencies.

1. We have standardized the text to use the abbreviation "ARAG" consistently throughout the manuscript after its first definition.

2. We have added a "roadmap" paragraph at the end of the Introduction (Section 1) that clearly summarizes the structure and content of the subsequent sections to guide the reader.

Comment 2: 2.1. Large Language Models (LLMs) and Their Role in Supporting Data Analysis for IoT

• Please check the source in Line 104. It is “[13? ].”

• Move Figure 1 after Line 118.

Response 2:

1. We have corrected the broken citation "[13?]" in Line 104.

2. We have adjusted the LaTeX float placement to ensure Figure 1 appears after Line 118 as requested.

Comment 3: 2.2. Agentic Retrieval-Augmented Generation (Agentic RAG)

• The title of this subsection should be only Agentic RAG. You already explained this abbreviation in Line 44.

• Please check the source in Line 151. It is: “[? ].”

Response 3:

1. We have renamed the subsection title to "Agentic RAG" to avoid redundancy.

2. We have corrected the citation formatting error "[?]" in Line 151.

Comment 4: 2.3. AI-driven Data Analysis in Maritime Applications

• Please check the source in Line 163. It is: “[? ].”

Response 4: We have identified and corrected the missing reference associated with the marker "[?]" in Line 163.

Comment 5: 3.2. Data

• In Lines 249-254, please provide the time period of data.

Response 5: We have updated Section 3.2 to explicitly state the specific time periods covered by both the ONTUS and Ferry Trips datasets to provide better context for the experimental results.

Comment 6: 3.4. Evaluation Metrics

• For each metric, please provide the computation and the range interval.

Response 6: We have revised Section 3.4. We added the specific mathematical formulas used for computation and explicitly defined the valid range intervals for each evaluation metric to ensure reproducibility and clarity.

Comment 7: 3.5. Test Suite Design

• Mention the significance and unit of measurement for the X-axis.

• Please increase the resolution and the size of Figure 2.

• Use the template for the Figures with multiple subfigures. Use (a), (b), etc. in the caption of the figure.

• Mention the meaning of the RT, FY, TP, HR, and DNR abbreviations as a note to Figure 2.

Response 7: We have improved Figure 2 based on your recommendations:

1. We adopted the MDPI template style for subfigures, labeling them (a), (b), etc

2. We increased the figure display size in the layout. The size and resolution should be visible better in the full html version as we are submitting full size png/pdfs. 

3. We adopted the MDPI template style for subfigures, labeling them (a), (b), etc.

4. We added a legend/note to the figure caption explicitly defining the abbreviations RT, FY, TP, HR, and DNR and the x-axis description

Comment 8: 4.1. Calculated Factual Correctness Metric

• Move Figure 3 after Line 347.

• Increase the size and resolution of Figure 3. The labels are hardly legible.

• Please cite Table 1 in the text.

Response 8:

1. We have moved Figure 3 to the appropriate location after Line 347.

2. We have increased the size of Figure 3 in the LaTeX build to improve legibility. We confirm that the source image is high-resolution, allowing readers to zoom in on the PDF for fine details.

3. We have ensured Table 1 is now explicitly cited and analyzed in the main text.

Comment 9: 6. Conclusion

• Add limitations and future work of the review.

• Mention to whom the research is useful.

Response 9: We have expanded the Conclusion section. It now includes a dedicated discussion on the limitations of the current study (specifically regarding real-time latency), outlines directions for future work, and explicitly identifies the stakeholders (e.g., maritime operators, system architects) who will benefit from this research.

Comment 10: References

• Please provide the link or DOI for the datasets mentioned in sources [28] and [29].

• Where DOI is not available, please provide the link, for instance, for sources [2], [4], [5], etc.

Response 10: We have carefully curated the bibliography. We addressed a BibTeX parsing error that was suppressing URLs. We have now provided direct links for datasets [28] and [29] (where DOIs are unavailable) and ensured that valid URLs are present for all other sources ([2], [4], [5], etc.) lacking a DOI.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

1. Section 3.1 mentions that the system input consists of “heterogeneous maritime AIoT data,” but Figure 1 fails to clearly distinguish between the input paths for real-time data streams and historical batch data. Recommendation: Supplement Section 3.1 and Figure 1 with additional explanations regarding the import paths for real-time streaming data versus historical batch data, along with subsequent data fusion processes.
2. Section 3.1 states that the system achieves verifiable analysis by generating auditable code. However, the paper only evaluates the correctness of final answers without verifying the security of the generated code. Recommendation: Add assessments of intermediate code, such as security checks for infinite loops in Python code.
3. Section 3.2 introduces the datasets used in this paper but does not explicitly define the division methods for training, validation, and test sets. It is recommended to supplement Section 3.2 with a detailed explanation of the specific strategies for dataset partitioning.
4. The “Calculated Factual Correctness” metric proposed in Section 3.4 is a custom evaluation metric. Its core formula (1) applies overly stringent treatment to values near zero, disregarding the actual error tolerance in maritime scenarios. Furthermore, it lacks comparative validation against traditional evaluation metrics. We recommend adding discussions on edge cases and comparing it with commonly used metrics to demonstrate its validity.
5. This paper focuses on system functionality implementation and accuracy evaluation but does not address safety and robustness considerations essential for deploying such systems in real maritime industrial environments. It is recommended to add discussions in Section 5 regarding potential safety risks, failure modes, and deployment strategies required for future practical production.

Author Response

1. Section 3.1 mentions that the system input consists of “heterogeneous maritime AIoT data,” but Figure 1 fails to clearly distinguish between the input paths for real-time data streams and historical batch data. Recommendation: Supplement Section 3.1 and Figure 1 with additional explanations regarding the import paths for real-time streaming data versus historical batch data, along with subsequent data fusion processes.

We thank the reviewer for this observation. We have updated Section 3.1 to clarify the distinction between historical batch data and real-time streaming data paths and the limitations of this paper not fully testing the real-time streaming data at the moment.

2. Section 3.1 states that the system achieves verifiable analysis by generating auditable code. However, the paper only evaluates the correctness of final answers without verifying the security of the generated code. Recommendation: Add assessments of intermediate code, such as security checks for infinite loops in Python code.

We thank the reviewer for raising this important security consideration. The system includes several safety mechanisms for code execution:

• Tool call limiting: The agent is configured with tool_call_limit=20, preventing excessive tool invocations that could indicate problematic code generation patterns.
• Request-level timeouts: The system enforces a 120-second timeout at the Gunicorn server level, which automatically terminates long-running requests, including those that might result from infinite loops.
• Framework-level safety: The agno framework's PythonTools executes code within a controlled environment with exception handling, providing isolation from the host system.

While we acknowledge that explicit static code analysis for infinite loop detection (e.g., AST-based pattern matching) would provide additional security guarantees, the current multi-layered approach—combining tool call limits, execution timeouts, and framework-level isolation—provides practical protection against infinite loops and runaway processes in our evaluation context. Future work could incorporate static code analysis tools for enhanced security validation, particularly for production deployments with stricter security requirements (we noted that in conclusions/future work as well)

3. Section 3.2 introduces the datasets used in this paper but does not explicitly define the division methods for training, validation, and test sets. It is recommended to supplement Section 3.2 with a detailed explanation of the specific strategies for dataset partitioning.

We appreciate the reviewer's attention to methodological rigor. However, this paper does not involve machine learning model training, so train/validation/test splits are not applicable.

• This is an evaluation study of an Agentic RAG system: the datasets (PONTUS-HUB and Ferry Trips Dataset) serve as the knowledge base that the system queries, not as training data. No model training occurs; the LLMs are pre-trained and used as-is.
• The evaluation methodology is described in Section 3.3 (Test Suite Design): we use a fixed test suite of 24 natural language queries with verified ground-truth answers. These queries are not split into train/validation/test sets; instead, all 24 queries serve as the evaluation set to assess the system's performance across different query complexities and data interaction types.
• The data partitioning strategy is straightforward: the full datasets are loaded into the database and made available for querying. The test suite queries this complete dataset to evaluate the system's analytical capabilities. This approach aligns with standard RAG evaluation methodologies, where the focus is on retrieval and generation performance rather than model training.

4. The “Calculated Factual Correctness” metric proposed in Section 3.4 is a custom evaluation metric. Its core formula (1) applies overly stringent treatment to values near zero, disregarding the actual error tolerance in maritime scenarios. Furthermore, it lacks comparative validation against traditional evaluation metrics. We recommend adding discussions on edge cases and comparing it with commonly used metrics to demonstrate its validity.

We appreciate the reviewer's feedback on the Calculated Factual Correctness metric. The metric uses a relative error formula: 1 - (|A - B| / max(|A|, |B|)), which is standard for numerical comparisons and treats errors proportionally regardless of magnitude.

• Regarding values near zero: the metric is intentionally strict when the ground truth is zero (returning 0.0 if the response is non-zero), as a non-zero answer when zero is expected indicates a fundamental error. For maritime operational data (fuel consumption, emissions, trip counts), answers near zero are rare and typically represent significant errors, so strict treatment is appropriate. In the previous revision we have added a paragraph about values near zero.
• The metric is already compared against traditional NLP metrics (ROUGE-L F1, Semantic Similarity, Faithfulness) in Section 3.5 and Table \ref{tab:qualitative_metrics}, which demonstrates their limitations for numerical evaluation. The Results section (Section 4.1) shows that Calculated Factual Correctness provides meaningful discrimination (Claude: 0.896, Qwen: 0.667, Llama: 0.35), while traditional metrics show limited utility (ROUGE-L F1: 0.08-0.15, Faithfulness: 0.14-0.21).

5. This paper focuses on system functionality implementation and accuracy evaluation but does not address safety and robustness considerations essential for deploying such systems in real maritime industrial environments. It is recommended to add discussions in Section 5 regarding potential safety risks, failure modes, and deployment strategies required for future practical production.

We thank the reviewer for highlighting this important gap. We have added a new paragraph to Section 5 (Discussion) regarding further considerations for production deployment that addresses: safety risks, failure modes and deployment strategies.