Development of a Reinforcement Learning-Based Intelligent Irrigation Decision-Making Model

Abstract

Originating from the practical demands of digital irrigation district construction, this study aims to provide support for precise irrigation management. This study developed a reinforcement learning-based intelligent irrigation decision-making model for districts employing traditional surface flood irrigation methods. Grounded in the theoretical framework of water cycle processes within the Soil–Crop–Atmosphere Continuum (SPAC) system and incorporating district-specific irrigation management experience, the model achieves intelligent and precise irrigation decision-making through agent–environment interactive learning. Simulation results show that in the selected typical area of the irrigation district, during the 10-year validation period from 2014 to 2023, the model triggered a total of 22 irrigation events with an average annual irrigation volume of 251 mm. Among these, the model triggered irrigation 18 times during the winter wheat growing season and 4 times during the corn growing season. The intelligent irrigation decision-making model effectively captures the coupling relationship between crop water requirements during critical periods and the temporal distribution of precipitation, and achieves preset objectives through adaptive decisions such as peak-shifting preemptive irrigation in spring, limited irrigation under low-temperature conditions, no irrigation during non-irrigation periods, delayed irrigation during the rainy season, and timely irrigation during crop planting periods. These outcomes validate the model’s scientific rigor and operational adaptability, providing both a scientific water management tool for irrigation districts and a new technical pathway for the intelligent development of irrigation decision-making systems.

1. Introduction

Irrigation is a critical measure to ensure timely and appropriate water supply for normal crop growth, mitigate regional droughts, and maintain high and stable agricultural yields. In 2023, China’s irrigated cropland area reached 47.78 million hectares [1], with a total agricultural water consumption of 367.24 billion cubic meters, accounting for 62.2% of the nation’s total water usage [2]. Selecting scientifically optimal irrigation timing and determining suitable water volumes are prerequisites for precision irrigation, playing a vital role in improving water use efficiency and enhancing agricultural productivity [3,4]. Currently, traditional flood irrigation remains the primary method for most farmers. Due to the influence of actual soil moisture content, farmers have limited means to regulate irrigation water volumes, therefore making precise determination of irrigation timing particularly critical. Even in regions with advanced irrigation infrastructure, irrigation scheduling still heavily relies on the experience of farmers or district managers. While this empirical knowledge offers practical guidance, its unquantifiable nature, subjectivity, and poor transferability struggle to adapt to dynamic climate change, resulting in significant limitations when addressing new challenges [5,6]. Therefore, there is an urgent need to introduce modern management models; to collect meteorological data, soil monitoring data, and crop growth data, as well as the management experience of farmers in a timely manner; and to use information fusion technology to achieve intelligent irrigation management decisions, providing scientific support for the sustainable development of agricultural production [7,8].

Irrigation decisions typically need to be made several days prior to actual implementation and depend on multiple critical factors, including soil moisture, meteorological conditions, crop water requirements at specific growth stages, and the capacity of the irrigation system to deliver water [9]. However, limited by technical expertise, farmers and irrigation district managers often struggle to comprehensively integrate these variables in practice. Most farmers still primarily rely on visual inspections of soil conditions and crop growth status to determine irrigation needs when formulating or executing irrigation plans. In addition, the security of water supply of the irrigation system and the historical precipitation in the same period are also important bases for farmers to judge whether to irrigate or not. However, this experience-based approach exhibits significant limitations, frequently leading to delayed irrigation or excessive water application, resulting in crop water stress or resource waste. The construction of intelligent irrigation decision-making systems provides scientific grounding and technical support for achieving precision irrigation through data-driven optimization.

The water cycle process within the Soil–Plant–Atmosphere Continuum (SPAC) provides a theoretical foundation for intelligent irrigation decision-making systems. Soil moisture sensors, weather stations, and crop growth monitoring devices deliver real-time, high-precision multi-source data to the system. A dedicated data analytics platform rapidly processes this heterogeneous data and generates corresponding irrigation decisions [10]. Intelligent irrigation decision-making systems continue to develop with the progress of technology, gradually moving towards automation, information technology, and intelligent irrigation modes. Early intelligent irrigation systems were mainly based on setting irrigation thresholds for simple irrigation management, real-time monitoring of soil moisture, temperature data, and meteorological data through sensors, and calculations to form irrigation decisions and regulate the amount of irrigation water [11,12]. With the introduction of Internet of Things (IoT) technology, various sensors have achieved network interconnection, enabling real-time data acquisition and transmission while supporting remote monitoring and management. This advancement has significantly enhanced the intelligent capabilities of irrigation systems [13,14,15]. Entering the stage of big data-driven decision-making, the intelligent irrigation system makes accurate irrigation decisions through multidimensional data analysis and continuously optimizes the irrigation strategy with the help of machine learning algorithms to achieve adaptive adjustment to adapt to the growth needs of different crops and climate change [16,17]. With the advancement of artificial intelligence and adaptive control technologies, modern intelligent irrigation systems now utilize deep learning models to analyze historical data and predict meteorological/soil variations, delivering precise irrigation decision services through intelligent algorithms that optimize resource utilization, driving irrigation decision-making to a new stage [18,19,20]. With the continuous development of intelligent irrigation decision-making systems, in order to further improve the accuracy and adaptivity of irrigation decision-making, researchers have begun to explore the introduction of reinforcement learning techniques into irrigation decision-making systems.

Reinforcement learning is a branch of machine learning which enables an agent to learn optimal strategies through continuous interactions with its environment. By taking actions and receiving feedback signals (rewards or penalties), the agent iteratively adjusts its policy to maximize long-term cumulative rewards [21]. Reinforcement learning has been widely applied in gaming [22], robotics [23,24,25], autonomous driving [26,27], and natural language processing (NLP) [28]. Recent studies have framed agricultural irrigation decision-making as a sequential control problem and integrated reinforcement learning into intelligent irrigation models. For instance, Chen proposed a distributed reinforcement learning-based irrigation method that holistically considers weather, soil, and crop conditions to optimize long-term irrigation benefits [29]. Chen developed a rice irrigation decision model using deep reinforcement learning, incorporating field water balance and crop water production functions. This model accounts for both the long-term impacts of irrigation on crop growth and improved rainfall utilization [30]. Alibabaei designed a deep Q-network intelligent irrigation system, leveraging LSTM models to predict soil moisture and crop yields for decision guidance [31]. However, the aforementioned studies have not incorporated the irrigation management experience of irrigation districts into the reinforcement learning framework, making it difficult to align with actual decision-making logic. To address this, this study intends to integrate irrigation management experience with reinforcement learning to construct an intelligent decision-making model with stronger practical adaptability, assisting managers in formulating irrigation strategies.

Currently, constrained by the inadequacy of digital management tools, irrigation management in irrigation districts remains predominantly reliant on empirical practices, resulting in a lack of scientific underpinnings for decisions regarding irrigation timing. As the development of digital irrigation districts accelerates, the creation of an intelligent decision-making model capable of precisely identifying critical irrigation windows has emerged as a core technical priority demanding urgent breakthroughs. In this paper, an intelligent irrigation decision-making model based on reinforcement learning is constructed, which is based on the water cycle process of the SPAC system; it constructs the environmental space of the intelligent agent and innovatively introduces the irrigation management experience, so as to provide scientific irrigation management decision-making support for irrigation districts adopting surface irrigation methods. The introduction of reinforcement learning has the following two significant advantages: on the one hand, it can interact in real time with the crop growth environment; on the other hand, it can effectively integrate farmers’ management experience into irrigation decision-making, realizing the organic combination of data-driven and empirical knowledge, and providing a new solution for the intelligence and refinement of irrigation management decision-making.

2. Materials and Methods

2.1. Study Area

The Xiaokaihe Irrigation District is situated on the left bank of the lower Yellow River in Binzhou City, Shandong Province (longitude: 117°49′29.46″ E, latitude: 37°16′59.03″ N), one of the large irrigation districts in China. With an effective irrigated area of 44,000 hectares, it primarily relies on Yellow River water to meet diversified water requirements across six administrative areas in Binzhou—the urban district, Huimin County, Yangxin County, Zhanhua District, Wudi County, and Beihai New District—serving agricultural, industrial, domestic, and ecological needs. The district predominantly cultivates winter wheat and corn in a crop rotation system, employing traditional surface flood irrigation methods. Winter wheat is typically planted from mid-October to early June of the following year, while corn grows from early June to late September. The region has an average annual precipitation of 580 mm, with a highly uneven distribution—428 mm (75% of the total) falling between June and September, and significantly less rainfall occurring during the dry season from October to May. The mean annual temperature ranges from 11.7 °C to 12.9 °C. The irrigation experimental station, located in the central part of the district under Zhanhua District’s jurisdiction, supports research and management. The geographical location of the study area is shown in Figure 1.

2.2. Crop Evapotranspiration and Soil Water Balance Calculation

The daily water requirement of the crop is calculated by the crop coefficient method:

$${ET}_0={\displaystyle \frac{0.408∆\left(R_n-G\right)+\gamma \left[\frac{900}{\left(T+273\right)}\right]{\mu }_2\left(e_s-e_a\right)}{∆+\gamma \left(1+0.34{\mu }_2\right)}}$$

(1)

$${ET}_c=K_c{K}_s{ET}_0$$

(2)

where $R_n$ denotes the net radiation at the surface (W/m²); $G$ denotes the soil heat flux density (W/m²); $T$ denotes the mean daily air temperature (℃); ${\mu }_2$ denotes the wind speed at 2 m height (m/s); $e_s$ denotes the saturation vapor pressure (kPa); $e_a$ denotes the actual vapor pressure (kPa); $∆$ denotes the slope of the saturation vapor pressure–temperature curve (kPa/°C); $\gamma$ denotes the psychrometric constant (kPa/°C); ${ET}_c$ denotes crop evapotranspiration (mm/day); $K_c$ denotes the single crop coefficient; $K_s$ denotes the water stress coefficient, for soil water limiting conditions $K_s<1$, or otherwise, $K_s=1$; ${ET}_0$ denotes reference evapotranspiration (mm/day).

Irrigation and crop water consumption processes adhere to the principle of soil water balance. The designed moisturized soil layer for irrigation is typically defined as the primary root water uptake zone, and water balance calculations are conducted within this predetermined soil layer:

$$W_t-W_0=W_T+P_0+K+M-{ET}_c$$

(3)

where $W_t$ denotes soil water storage in the planned wetting depth at the end of the period (mm); $W_0$ denotes the initial soil water storage in the planned wetting depth (mm); $W_T$ denotes the incremental water storage due to expansion of the wetting depth (mm); $P_0$ denotes the effective precipitation retained within the wetting depth (mm); $K$ denotes groundwater utilization during period t, calculated as $K=kt$, where k denotes the average daily groundwater utilization rate (mm/day); M denotes the irrigation depth (mm).

Calculation of soil moisture content:

$${\theta }_c={\displaystyle \frac{W_t}{1000H}}$$

(4)

where ${\theta }_c$ denotes the soil moisture content, and $H$ denotes the planned wetting depth (m).

Calculation of irrigation depth:

$$M=\left({\theta }_{Fc}-{\theta }_c\right)\ast H\ast 1000$$

(5)

where ${\theta }_{Fc}$ denotes the field capacity.

2.3. Development Model

The core mechanism of reinforcement learning operates through an iterative trial-and-error process between an agent and its environment, progressively optimizing policies to maximize cumulative rewards. The theoretical foundation is built upon the Markov Decision Process (MDP), with reinforcement learning systems comprising five essential elements: agent, environment, state, action, and reward. During initial training, the agent explores the environment by executing random actions, which induce state transitions and generate the corresponding reward/penalty feedback. Through repeated interactions, the agent incrementally accumulates experience and refines its policy, dynamically balancing exploration (testing new actions) and exploitation (leveraging known high-reward actions). As training advances, the agent transitions from predominantly exploratory behavior to exploiting learned knowledge, increasingly favoring actions associated with higher expected returns. Ultimately, through continuous policy optimization, the agent converges toward near-optimal decision-making. The integration of reinforcement learning into intelligent irrigation decision-making models enables agents to select optimal irrigation timing through continuous interaction with the SPAC environment, achieving precision irrigation. When constructing such models, it is essential to systematically design the environment space, state space, action space, and reward mechanism to ensure scientific and practical applicability. By integrating multi-source data including soil moisture data, meteorological data, crop phenological data, and agricultural management expertise, a multidimensional state space is constructed. Irrigation strategies are then developed through reinforcement learning algorithms. The model framework diagram is shown in Figure 2.

Environment space: The environment space of reinforcement learning can be expressed as follows:

$$E=\left\{S,A,P,R\right\}$$

(6)

where $E$ denotes the environment space, $S$ denotes the state space; A denotes the action space; $P$ denotes the state transition probability; $R$ denotes the reward function.

State space: As the core of reinforcement learning, this refers to the complete set of all possible states of the environment in which the intelligent body is located at a specific time node. The state space is designed to fully reflect the dynamic changes and key features of the agricultural environment, while also ensuring that its dimensions remain manageable to prevent excessive computation. The state space of this study can be represented as follows:

$$S=(D,{\theta }_c,T_{min},P_t)$$

(7)

where $D$ denotes the date; $T_{min}$ denotes the minimum temperature; $P_t$ denotes the precipitation sequence for the next several days.

Action space refers to the complete set of possible actions an agent can take in a given state. In the intelligent irrigation decision-making model, the action space consists of two choices: irrigation or no irrigation. Given that most farmers in the study area rely on traditional flood irrigation methods, the actual effective irrigation volume applied to the field is determined by the soil moisture content within the planned wetting layer.

$$A=\left(\mathrm{1,0}\right)$$

(8)

where $A=1$ denotes irrigation action; $A=0$ denotes no irrigation action.

The transfer probability P represents the probability that the environment will transfer from the current state to another state after executing an irrigation decision during the decision cycle.

The reward function serves as the core mechanism in reinforcement learning, where an agent receives positive rewards or negative penalties based on its chosen actions and resulting environmental state transitions, thereby guiding the optimization of its behavioral policies. The efficacy of reinforcement learning critically depends on the design of the reward function, as a scientifically structured incentive mechanism significantly enhances the agent’s learning efficiency and decision-making capabilities.

In developing the intelligent irrigation decision-making model, the reward function evaluates the quality of irrigation decisions (to irrigation or not) through immediate feedback. To ensure optimal decision-making across diverse environmental conditions, the reward function’s architecture must be meticulously designed and closely aligned with irrigation objectives. Grounded in the operational realities and management experience of the study area, the reward function in our model is formulated to achieve the following goals. Its mathematical expression is defined as follows:

$$R=R_0+{\sum }_{i=1}^5{\lambda }_i{R}_i$$

(9)

where $R$ denotes the total reward; $R_0$ denotes the basic reward; $R_i$ denotes the reward for the i-th objective; ${\lambda }_i$ denotes the weighting coefficient for the i-th objective; $i$ denotes the index of objectives.

The basic reward function should align with conventional irrigation practices by evaluating whether the soil moisture content meets traditional irrigation criteria—whether the soil moisture content has reached the crop-specific lower threshold of the optimal moisture range—while no precipitation events capable of providing irrigation-equivalent effects are forecasted within the defined decision horizon.

Based on reliability analysis of rainfall forecasts and regional precipitation characteristics, this study defines a precipitation event with irrigation-substitution effects as a situation where the 7-day forecast predicts that at least one day has a daily precipitation amount exceeding the precipitation threshold. Under such conditions, the model should prohibit irrigation triggering.

$$P_e=\left\{\begin{array}{c}1 \mathrm{i}\mathrm{f} \exists p_j\ge p_{th} \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{n}\mathrm{y} j\in \left[\mathrm{1,7}\right]\\ 0 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(10)

where $P_e=1$ denotes that a precipitation event with irrigation-substitution effects exists; $P_e=0$ denotes no precipitation event with irrigation-substitution effects exists; $p_{th}$ denotes the precipitation threshold; $j$ denotes the day $(j=\mathrm{1,2},\dots ,7)$.

$$R_0=\left\{\begin{array}{c}1 \mathrm{i}\mathrm{f} A=1 \wedge {\theta }_c\le {\theta }_{lower} \wedge P_e=0\\ 1 \mathrm{i}\mathrm{f} A=0 \wedge {\theta }_c>{\theta }_{lower}\\ 1 \mathrm{i}\mathrm{f} A=0 \wedge {\theta }_c\le {\theta }_{lower} \wedge P_e=1\\ -1 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(11)

where ${\theta }_{lower}$ denotes the crop-specific lower limit of plant-available soil water.

Objective reward: This incorporates irrigation management experience into the reinforcement learning model through reward shaping, thereby guiding the model to make scientifically sound irrigation decisions.

Objective reward 1: Peak-shifting preemptive irrigation in spring.

Spring irrigation should follow the principle of preventive irrigation. Given the study area’s scant spring precipitation and limited water diversion capacity of the irrigation district, while the water requirement of wheat increases significantly during this stage, in order to cope with high-frequency spring droughts, the irrigation area usually adopts a staggered and early irrigation strategy. This approach can effectively guarantee the supply of water to crops in the region and avoid drought caused by untimely irrigation. Objective reward 1 is expressed as follows:

$$R_1=\left\{\begin{array}{c}5 \mathrm{i}\mathrm{f} A=1 \wedge {\theta }_c\le {\theta }_{th} \wedge p_e=0 \wedge D\in D_{Pip}\\ 0 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(12)

where $R_1$ denotes rewards obtained for Objective 1, ${\theta }_{th}$ denotes the moisture content threshold for triggering irrigation in spring, and $D_{Pip}$ denotes the period during which irrigation can occur in advance.

Objective reward 2: Limiting irrigation under low-temperature conditions.

When temperatures drop below the optimal range for crop growth, irrigation may lower soil temperature, inhibit root activity, and increase the risk of disease occurrence. Additionally, low temperatures can cause ice formation in irrigation channels, posing safety hazards to water delivery. Since crop water requirements are significantly reduced under cold conditions, plants can tolerate moderate water stress. Therefore, irrigation should be suspended during low-temperature periods to ensure normal crop growth and safe water conveyance. Objective reward 2 is expressed as follows:

$$R_2=\left\{\begin{array}{c}-5 \mathrm{i}\mathrm{f} A=1 \wedge T_{min}\le T_{th}\\ 0 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(13)

where $R_2$ denotes rewards obtained for Objective 2, and $T_{th}$ denotes the low-temperature threshold.

Objective reward 3: No irrigation during non-irrigation periods.

During the maturity stages of wheat and corn, irrigation should not be implemented even if short-term drought conditions occur, to avoid excessive soil moisture that could hinder mechanized harvesting operations. This strategy aims to balance crop water requirements with the operational efficiency of mechanized harvesting. Objective reward 3 is expressed as follows:

$$R_3=\left\{\begin{array}{c}-5 \mathrm{i}\mathrm{f} A=1 \wedge D\in D_{Nip}\\ 0 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(14)

where $R_3$ denotes rewards obtained for Objective 3; $D_{Nip}$ denotes non-irrigation periods.

Objective reward 4: Delayed irrigation during the rainy season.

During the overlapping period between the corn growing season and the rainy season in the study area, crops can tolerate moderate water stress rather than receiving immediate irrigation. By appropriately delaying irrigation, natural rainfall resources can be fully utilized. This approach not only enhances rainfall use efficiency but also optimizes water resource allocation and reduces unnecessary irrigation costs. Objective reward 4 is expressed as follows:

$$R_4=\left\{\begin{array}{c}-5 \mathrm{i}\mathrm{f} A=1 \wedge {\theta }_c{\ge \theta }_{rth} \wedge D\in D_{Rs}\\ 0 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(15)

where $R_4$ denotes rewards obtained for Objective 4; ${\theta }_{rth}$ denotes the moisture content threshold for triggering irrigation during the rainy season; $D_{Rs}$ denotes the rainy season, in which crops are able to withstand drought stress.

Objective reward 5: Timely irrigation during the crop planting period.

Under crop rotation systems, particular attention must be paid to soil moisture conditions during the initial crop establishment period. When soil water content falls below optimal levels and no precipitation events capable of replacing irrigation are forecasted in the near term, irrigation should be promptly implemented. This practice ensures proper seedling emergence and early growth establishment, thereby laying a solid foundation for water supply throughout the entire growth cycle. Objective reward 5 is expressed as follows:

$$R_5=\left\{\begin{array}{c}5 \mathrm{i}\mathrm{f} A=1 \wedge {\theta }_c\le {\theta }_{min} \wedge P_e=0 \wedge D\in D_{Cps}\\ 0 \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.$$

(16)

where $R_5$ denotes the rewards obtained for Objective 5; $D_{Cps}$ denotes the crop planting period.

2.4. Model Solving

This study employs the Q-learning algorithm, a value function-based approach, to derive solutions. The optimal policy is selected by learning the state-action value function (Q-function). As a model-free reinforcement learning method, Q-learning requires no prior knowledge of environmental dynamics and achieves learning solely through interactions with the environment.

According to the fundamental Q-learning algorithm, the Q-function is updated using the following formula, a method known as temporal difference learning:

$$Q\left(s_t,a_t\right)\leftarrow Q\left(s_t,a_t\right)+\alpha \left[r_{t+1}+\gamma maxQ\left(s_{t+1},a_{t+1}\right)-Q\left(s_t,a_t\right)\right]$$

(17)

where $a_t$ denotes the action taken at time step t; $s_t$ denotes the current state at time step t; $s_{t+1}$ denotes the next state after executing action $a_t$; $a_{t+1}$ denotes any possible action in state $s_{t+1}$; $\alpha$ denotes the learning rate, which was set to 0.001 in this study; $\gamma$ denotes the discount factor; $r_{t+1}$ denotes the immediate reward received after executing action $a_t$. In addition to directly extracting policies from the Q-table, the Q-learning algorithm can also balance exploration and exploitation during training through a strategy called ε-greedy. Under the ε-greedy strategy, the algorithm selects the action with the highest Q-value in most cases, but with probability ε, it randomly chooses other actions to ensure better exploratory behavior.

This study models crop growth as a continuous Markov process, where daily environmental conditions are treated as discrete states. The model is trained to determine optimal decisions (to irrigate or not) for each daily state. This framework enables effective irrigation strategy development for specific environments by significantly reducing the state space and accelerating policy convergence, and the model achieves stable action outputs after 300 training iterations. The model solution procedure is shown in Figure 3.

2.5. Data Collection

This study selected the Xiaokaihe Irrigation Experimental Station as a typical area to carry out this research.

The meteorological data were obtained from the China Meteorological Data Network (http://data.cma.cn. accessed on 21 October 2024), including key climatic variables such as daily minimum temperature, daily maximum temperature, daily mean temperature, average wind speed, sunshine duration, mean relative humidity, and daily precipitation.

By collecting historical records and field-measured data, the growth periods, crop coefficients, and optimal soil moisture ranges for each growth stage of the major crops in the study area were determined, providing reliable data support for the research. Wheat and corn are the core crops grown in the irrigation district of the study area, with their total planting area accounting for more than 80% of the district’s effective irrigated area. The growth stage parameters for winter wheat are presented in

Table 1. The growth stage parameters for winter wheat.

Winter Wheat Growth Stages	Sowing to Tillering	Tillering to Overwintering	Overwintering to Green-up	Green-up to Jointing	Jointing to Grain Filling	Grain Filling to Wax Maturity	Wax Maturity to Harvest
Starting and ending dates	11 October to 10 November	11 November to 10 December	11 December to 10 March	11 March to 10 April	11 April to 10 May	11 May to 31 May	1 June to 15 June
Number of days	31	30	90	31	30	31	15
Maximum volumetric moisture content	36.26%	36.26%	36.26%	36.26%	36.26%	36.26%	36.26%
Minimum volumetric moisture content	19.95%	23.57%	23.57%	23.57%	23.57%	21.61%	21.61%

, while those for corn are listed in

Table 2. The growth stage parameters for corn.

Corn Growth Stages	Sowing to Germination	Germination to Jointing	Jointing to Booting	Booting to Grain Filling	Grain Filling to Milk Stage
Starting and ending dates	11 June to 30 June	1 July to 20 July	21 July to 10 August	11 August to 31 August	1 September to 30 September
Number of days	20	20	21	20	27
Maximum volumetric moisture content	36.26%	36.26%	36.26%	36.26%	36.26%
Minimum volumetric moisture content	19.95%	21.76%	21.76%	23.57%	20.22%

.

3. Results

3.1. Analysis of Simulation Results

An intelligent irrigation decision-making model was developed in this study based on selected typical areas within the irrigation district and meteorological monitoring data from Zhanhua District. The study employed simulation methods to systematically reconstruct the dynamic variations in soil water content and potential occurrences of irrigation events over a decade. Validation of the model’s reliability was achieved by assessing its performance under different environmental conditions.

Over the past decade (2014–2023), the average annual crop water requirement in the study area was 881 mm, with winter wheat and corn accounting for 422 mm and 459 mm during their respective growing seasons. The mean annual precipitation was 642 mm, and there was a significant mismatch between the precipitation distribution and crop water requirement pattern. The average annual precipitation during the growing period of wheat (from early October to early June of the following year) is only 154 mm, much lower than the crop water requirement, while the average annual precipitation during the growing period of corn (from mid-June to late September) is 488 mm, close to the crop water requirement, but due to the uneven spatial and temporal distribution of precipitation in some years and the limited capacity of the soil to hold water and other factors, there is still a risk of staged water deficit.

As shown in

Table 3. Model irrigation simulation results.

Year	Irrigation Time	Irrigation Water Volume (mm)	Irrigation Time	Irrigation Water Volume (mm)	Irrigation Time	Irrigation Water Volume (mm)	Irrigation Time	Irrigation Water Volume (mm)
2014	24 March	79	28 April	107	26 July	150	11 October	121
2015	22 April	97
2016	20 March	84	26 April	109
2017	26 February	128	30 April	109
2018	3 March	121	8 October	130
2019	11 April	101	11 June	176	7 October	173
2020	13 April	74	10 June	136
2021	2 March	80	5 May	101
2022	30 April	110
2023	1 March	84	8 May	115	6 September	129

, the simulation results indicate that the model triggered 22 irrigation events over the decade, with an annual average irrigation requirement of 251 mm, exhibiting significant interannual and seasonal variations. The year 2014 emerged as the peak irrigation requirement year, with 456 mm of irrigation water applied through four irrigation events, driven by drastically reduced precipitation measuring 316 mm—the lowest annual total recorded in the decade. Conversely, only one irrigation event was necessary in 2015 and 2022. Seasonal analysis revealed spring as the most irrigation-intensive season (15 irrigation events), followed by autumn (4 irrigation events) and summer (3 irrigation events). From a crop growth cycle perspective, distinct irrigation requirement patterns emerged between the winter wheat growing season (October–May) and corn growing season (June–September). The model triggered 18 irrigation events (81.8% of total events) during wheat cultivation, with an annual average required irrigation amount of 182 mm. In contrast, corn cultivation (early June to late September), despite overlapping with the rainy season, still required four supplemental irrigation events over the decade, averaging 127 mm per time.

Based on the analysis of simulation deduction results, winter wheat in the study area typically requires two irrigation events during its growing season, which is consistent with the current irrigation practices in the region. The initial irrigation covers two critical growth stages, the early planting stage and the green-up to jointing stage. In the early stage of wheat planting, three irrigations are carried out. This not only meets the water requirements for the normal germination and seedling growth of wheat but also ensures adequate water supply for the subsequent green-up and jointing stages, as wheat has relatively low water consumption in autumn and winter. Six irrigations are conducted during the wheat’s green-up to jointing stages. During this period, the water requirement of wheat increases significantly, while the rainfall is often insufficient. Thus, the primary purpose of this irrigation is to meet the water needs for the rapid growth of wheat and prevent drought stress from affecting the crop. The second irrigation occurs during the heading and grain-filling period. At this stage, wheat consumes a large amount of water, and soil moisture is consumed quickly. Irrigation at this stage helps to ensure the smooth progress of grain formation and grouting. Simulation results show that irrigation is triggered nine times during this period.

During the corn growth period, four irrigations are mainly related to rainfall during the rainy season. Two of these irrigations occur during the early stages of corn planting, primarily due to the high soil moisture consumption during the late stages of wheat growth and the lack of timely and effective rainfall in June. Therefore, irrigation is necessary to ensure proper seedling emergence and growth of corn. The irrigations that took place during the mid-late stages of corn growth in 2014 and 2023 were due to insufficient rainfall during the rainy season, which caused the soil moisture to be inadequate to meet the subsequent growth needs of corn, requiring irrigation to ensure crop yield. Studies have shown that rainfall during the rainy season, particularly in early to mid-June, significantly impacts whether irrigation is needed during the corn growth period.

Irrigation, as a supplementary measure in agricultural management, is used to compensate for insufficient rainfall. As shown in

Table 4. Multi-year average of monthly precipitation and crop water requirements.

Crop	Volume of Water	Multi-Year Monthly Average
Winter wheat growth stages		October	November	December	January	February	March	April	May
	Precipitation (mm)	28	31	5	4	8	9	31	38
	crop water requirements (mm)	38	17	6	7	12	54	130	158
Corn growth stages		June	July	August	September
	Precipitation (mm)	79	190	185	34
	crop water requirements (mm)	83	113	145	118

, the temporal and spatial distribution of precipitation during the wheat growing period in the study area shows a significant imbalance with the crop water requirement pattern. From March to May, during key growth stages such as jointing, heading, and grain filling, the water requirement of wheat increases significantly, with a daily average water requirement of 3–6 mm. However, during this period, precipitation only accounts for 36.5% of the water requirement, and the Water Stress Deficit Index (WSDI) reaches 0.63; at least one irrigation event will occur during this period. Without irrigation intervention, soil moisture will decrease rapidly, leading to severe drought stress and yield loss. In contrast, during the corn growing period (June to September), although it falls within the rainy season, precipitation can meet 106.8% of the crop’s water requirement. However, the precipitation is concentrated and unevenly distributed over time, especially during the first half of June and the latter part of September, where short-term drought conditions are likely to occur. The first half of June coincides with the seedling and early growth stages of corn, which are sensitive to soil moisture, and low soil moisture content will significantly affect the germination rate and seedling growth. In the latter part of September, corn enters the grain filling and maturation stages, which are still at a peak water requirement. A reduction in precipitation could lead to soil moisture deficits, affecting grain filling and final yield. Therefore, despite the overall higher precipitation during the corn growing season, irrigation measures are still required to compensate for the water shortages during critical growth periods. Irrigation simulation results indicate that the model can accurately capture the coupling relationship between crop water requirement critical periods and the temporal and spatial distribution of precipitation. The simulated irrigation events align with the current situation in the study area, demonstrating high scientific accuracy and reliability.

3.2. Analyses of Model Objectives

Based on the irrigation management experience in the study area, the following five specific objectives were established during model development:

Objective 1: Peak-shifting preemptive irrigation in spring.

Objective 2: Limiting irrigation under low-temperature conditions.

Objective 3: No irrigation during non-irrigation periods.

Objective 4: Delayed irrigation during the rainy season.

Objective 5: Timely irrigation during the crop planting period.

An analysis of the intelligent irrigation decision model developed in this study on irrigation decisions over the past 10 years, with the goal achievement status, is shown in

Table 5. Irrigation decision goal statistics.

Year	Objective 1	Objective 2	Objective 3	Objective 4	Objective 5
2014	➕			➕	➕
2015	➕			➕
2016	➕
2017		➕		➕
2018		➕		➕	➕
2019			➕
2020	➕				➕
2021		➕		➕
2022				➕
2023	➕			➕

➕: Represent the model achieving the irrigation Objective.

.

Based on the results of the simulation, the intelligent irrigation decision-making model consistently met predefined objectives across different years over the past decade, reflecting the model’s scientific and adaptive nature in irrigation decision-making. As shown in Figure 4, the model implemented early spring irrigation in 2014, 2015, 2016, 2020, and 2023, alleviating water stress during the jointing and heading stages of wheat through preventive irrigation, which aligns with Objective 1 (peak-shifting preemptive irrigation in spring). In early 2017, 2018, and 2021, even though soil moisture was low, the model did not execute irrigation due to low temperatures, thus preventing soil temperature drop and the inhibition of root activity caused by irrigation, demonstrating the model’s sensitivity to low-temperature conditions, in line with Objective 2 (limiting irrigation under low-temperature conditions). In 2018 and 2019, during a short drought period at the wheat maturity stage, the model refrained from irrigation, avoiding soil over-wetting and ensuring smooth subsequent mechanical harvesting, which aligns with Goal 3 (no irrigation during non-irrigation periods). The model made 22 irrigation decisions, none of which were made during or around periods of effective precipitation. Furthermore, in 2015 and 2017, the model allowed for short-term drought during the corn growing season and did not implement irrigation, effectively improving rainfall utilization, in line with Goal 4 (delayed irrigation during the rainy season). The model also implemented one irrigation at the early wheat planting stages in 2014, 2018, and 2019, as well as the early corn planting stages in 2019 and 2020, replenishing soil moisture to ensure normal seedling emergence and early growth, laying a foundation for the full growing season’s water supply, in line with Goal 5 (timely irrigation during the crop planting period). In conclusion, the irrigation decision model, by comprehensively considering factors such as precipitation, soil moisture, crop water requirement patterns, and temperature, has achieved precise decision-making, meeting the preset goals in different scenarios, and provides scientific support for the efficient use of regional agricultural water resources and stable, high crop yields.

4. Discussion

This study introduces reinforcement learning technology into surface irrigation decision-making. When formulating irrigation decisions, the model achieves the integration of data-driven and experience-guided approaches through dynamic interaction with the crop growth environment and in-depth integration of irrigation practice experience. Specifically, the introduction of reinforcement learning technology integrates the long-accumulated unique irrigation experience of the irrigation district into the design of its reward function, avoiding the decision-making deviation of pure data-driven models. By implementing a weighted summation mechanism, this approach resolves decision deadlocks arising from multi experiential conflicts, ensures robustness in model outputs, and provides an extensible interface for the subsequent supplementation of empirical management rules.

During the research process, it was found that there are significant differences in the irrigation water requirements of crops at different growth stages. For example, the average irrigation water amounts for wheat during the sowing, green-up and jointing, and heading and grain-filling periods are 141 mm, 96 mm, and 102 mm, respectively, and the irrigation water amount during the sowing period is 46.9% and 38.2% higher than that during the green-up and jointing period and the heading and grain-filling period, respectively; meanwhile, for corn, the average irrigation amounts during the sowing period and rainy season supplemental irrigation are 155 mm and 139 mm, respectively, and the irrigation water amount during the sowing period is 11.5% higher than that during the rainy season supplemental irrigation. For irrigation events occurring in the early stage of crop planting, the irrigation water requirement is relatively high. This regularity imposes refined requirements on the operation and management of irrigation systems. In actual irrigation water management processes, neglecting the irrigation needs of crops at different growth stages and adopting a unified irrigation quota can easily lead to waste or insufficient supply of water resources. Therefore, implementing stage-specific water allocation strategies based on the water demand characteristics of crops at different growth stages can achieve precise coupling between irrigation decisions and crop physiological needs, improving the level of irrigation management.

The constructed intelligent decision-making model can accurately capture feasible irrigation events under different environmental conditions. However, it has limitations in quantifying the yield risks that may be caused by active irrigation suspension measures, and the research mainly focuses on the precise control of irrigation timing, failing to achieve irrigation water conservation through the collaborative optimization of irrigation timing. To address these limitations, subsequent research will focus on two key aspects for improvement. First, introduce a yield prediction model to establish a quantitative relationship between proactive irrigation suspension measures and crop yield, providing managers with specific numerical references for assessing yield losses. Second, construct a multi-stage irrigation timing optimization framework, where dynamic adjustment of irrigation timing across various stages enables the achievement of irrigation water conservation.

5. Conclusions

This study introduces reinforcement learning into irrigation decision-making and develops an intelligent irrigation decision model based on this approach. Grounded in the hydrological processes of the SPAC system, the model constructs the agent’s environmental space and innovatively integrates irrigation management experience, thereby supporting scientific water management in surface flood irrigation systems. Long-term simulation results from typical experimental zones in the irrigation district demonstrate that the model can generate irrigation decision schemes meeting agronomic requirements under multiple scenarios. Specifically, a decade of simulation results revealed that the model triggered 22 irrigation decisions in total, with 18 during wheat growing seasons and 4 during maize growing seasons. The decision timing aligns well with crop water consumption patterns, validating the model’s precise responsiveness to regional crop water requirements. Furthermore, these results suggest that when formulating the annual water allocation plan for the study area, priority should be given to ensuring the irrigation water needs of winter wheat in the spring, while reasonably adjusting the irrigation water for the summer and autumn seasons to achieve efficient water resource utilization and ensure crop growth.

The intelligent irrigation decision-making model developed in this study successfully integrates traditional irrigation decision-making with the unique management experience of the irrigation district. It can accurately capture the coupling relationship between the key periods of crop water requirement and the temporal distribution of precipitation, and meanwhile, the model provides underlying technical support for the construction of digital irrigation district platforms.