Optimizing Water Use in Maize Irrigation with Reinforcement Learning

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

By applying reinforcement learning, author optimized the water usage in maize irrigation. Authors used reinforcement learning in AquaCrop-OSPy simulations and developed adaptive irrigation policies for maize. Obtained results are interesting in which authors achieved a water use efficiency around 40% improvement over optimized soil moisture threshold methods. There are some minor comments as follows.

It is better when authors present briefly about Proximal Policy Optimization (PPO) and how it was applied in their approach.

Authors conducted 50 trials, each involving training the PPO agent for 100,000 timesteps 358 with a unique hyperparameter set sampled from defined ranges (Table 1). Please explain more about these trials.

Author wrote that “By focusing on final performance, we ensured that the agent not only learned effectively but also generalized well by the end of training.”. What is the final performance?

I think that the obtained results were reported clearly. However, I wonder about the sub-section 4.4. Bridging Simulation to Practical Implementation. From the practical point of view, this sub-section is very important and should be discussed further.

Comments on the Quality of English Language

The English could be improved to more clearly express the research.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The paper presents an innovative application of reinforcement learning (RL) to optimize irrigation scheduling, addressing a critical challenge in sustainable agriculture. The authors propose a novel reward mechanism that effectively balances water conservation and yield maximization. The study builds upon the previously introduced AquaCrop-OSPy framework to evaluate the proposed approach. The primary contribution of the paper lies in the design and implementation of the reward mechanism. However, several aspects could be clarified or expanded to strengthen the paper:

1. Reward Mechanism Analysis. The paper would benefit from a more detailed analysis of how the reward mechanism operates within the AquaCrop-OSPy framework. Specifically, a mathematical interpretation of how the incremental penalties and terminal rewards are integrated into the simulation environment would enhance clarity and reproducibility. For example, how are the penalties and rewards calculated at each timestep, and how do they influence the agent's learning process?

2. Integrated Methodology. While the paper describes the individual components, a more comprehensive explanation of how these components are integrated into a unified framework would be valuable.

3. Action Space Design. The authors state, "Through this iterative feedback process, the agent refines its strategy, learning when and how much to irrigate to maximize yield while conserving water." However, the action space in the framework is limited to two discrete options: 0 mm (no irrigation) and 25 mm (apply 25 mm of water). This raises two questions: 1) Why is the action space restricted to these two options? Is this simplification based on practical irrigation constraints, or is it a limitation of the framework? 2) How was the value of 25 mm determined? Is this value optimal for the specific crop and region studied, or was it chosen arbitrarily? A justification for this choice would strengthen the paper.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

In this latest version, the paper effectively presents its valuable contributions to the field of agricultural water management. The integration of RL with robust simulation tools paves the way for future studies, encouraging the exploration of varied crops and irrigation systems while pushing the boundaries of precision agriculture.