# Modeling Soil Water Content and Reference Evapotranspiration from Climate Data Using Deep Learning Method

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Data Collection

#### 2.2. Data Preprocessing

#### 2.3. Deep Learning Methods

#### 2.3.1. LSTM Architectures

#### 2.3.2. Bidirectional LSTM

#### 2.3.3. CNN-LSTM Model

#### 2.3.4. Dropout

#### 2.4. Bayesian Optimization

- Choose a surrogate function to approximate the objective function and define its prior. The choice of this function is optional; the algorithm is convergent, and the choice of the prior function can help increase the speed of convergence.
- Given a set of observations, Bayes’ rule is used to obtain the posterior distribution over the objective function. Bayes rule is given by Equation (10):$$P\left(A\right|B)=\frac{P\left(B\right|A\left)P\right(A)}{P\left(B\right)}$$
- Use an acquisition function $\alpha $ which is a function of the posterior distribution, to determine the next sample point as ${x}_{t}=\underset{x}{\mathrm{argmax}}\phantom{\rule{4pt}{0ex}}\alpha \left(x\right)$.
- Add the new sample to the observation set and go back to step 2 until convergence or budget has elapsed.

#### 2.5. Evaluating the Models

- Mean Square Error (MSE): is the average of the squared differences between prediction and actual observation. In other words, the MSE is the variance of the error [41]. It is calculated by Equation (11):$$MSE=\frac{1}{n}\sum _{i=1}^{n}{({y}_{i}-{\widehat{y}}_{i})}^{2}$$
- Root Mean Square Error (RMSE): is the square root of the MSE. In other words, the RMSE is the standard deviation of the error [41]. In the MSE metric, the errors are first squared before averaging, resulting in a high penalty for large errors. Therefore, the RMSE is useful when large errors are undesirable.
- Mean Absolute Error: is the average of the absolute differences between predictions and actual observations and is calculated by Equation (12) [41]:$$MAE=\frac{1}{n}\sum _{i=1}^{n}\left(\right)open="|"\; close="|">{y}_{i}-{\widehat{y}}_{i}$$
- Mean Bias Error (MBE): captures the average bias in the prediction. A positive bias in a variable means that the data from the datasets are underestimated, while a negative bias means that the model overestimates the observed data. It is calculated by Equation (13):$$MBE=\frac{1}{n}\sum _{i=1}^{n}({y}_{i}-{\widehat{y}}_{i})$$
- ${R}^{2}$: is a statistical measure that calculates the variance explained by the model over the total variance. The higher ${R}^{2}$ is, the smaller the differences between the actual observations and the predicted values. It is calculated by Equation (14) [19]:$${R}^{2}=1-\frac{{\sum}_{i=1}^{n}{({y}_{i}-{\widehat{y}}_{i})}^{2}}{{\sum}_{i=1}^{n}{({y}_{i}-\overline{y})}^{2}}$$

## 3. Results and Discussion

## 4. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A

Variables | Unit | Data Source | Locations | Temporal Resolution | Max | Min | Mean | SD |
---|---|---|---|---|---|---|---|---|

${T}_{Min}$ | °C | AppiZezere, | Loc. 1 | daily | 26.76 | −5.26 | 9.14 | 5.86 |

DRAP-Centro | Loc. 2 | daily | 21.89 | −7.43 | 7.30 | 5.39 | ||

Loc. 3 | 15-min | 27 | −4.7 | 9.76 | 5.63 | |||

${T}_{Avg}$ | °C | AppiZezere, | Loc. 1 | daily | 35.94 | 1.96 | 16.13 | 7 |

DRAP-Centro | Loc. 2 | daily | 32.2 | 0 | 14.66 | 6.65 | ||

Loc. 3 | 15-min | 34.84 | −0.12 | 15.68 | 6.90 | |||

${T}_{Max}$ | °C | AppiZezere, | Loc. 1 | daily | 44.8 | 5.56 | 23.97 | 8.68 |

DRAP-Centro | Loc. 2 | daily | 42.9 | 4.43 | 22.99 | 8.42 | ||

Loc. 3 | 15-min | 42.7 | 1.8 | 21.84 | 8.42 | |||

$H{R}_{Min}$ | % | AppiZezere, | Loc. 1 | daily | 99.1 | 0 | 41.91 | 20.56 |

DRAP-Centro | Loc. 2 | daily | 99.1 | 0 | 42.78 | 20.17 | ||

Loc. 3 | 15-min | 95 | 0 | 38.72 | 20.20 | |||

$H{R}_{Avg}$ | % | AppiZezere, | Loc. 1 | daily | 99.1 | 21.84 | 67.16 | 18.45 |

DRAP-Centro | Loc. 2 | daily | 99.1 | 0 | 70.96 | 16.34 | ||

Loc. 3 | 15-min | 95.89 | 27.75 | 60.38 | 18.78 | |||

$H{R}_{Max}$ | % | AppiZezere, | Loc. 1 | daily | 100 | 33.33 | 89.55 | 13.87 |

DRAP-Centro | Loc. 2 | daily | 99.1 | 48.37 | 93.93 | 10.08 | ||

Loc. 3 | 15-min | 97 | 24 | 81.29 | 15.05 | |||

$S{R}_{Avg}$ | Wm${}^{-2}$ | AppiZezere, | Loc. 1 | daily | 592.08 | 3.90 | 214.67 | 140.11 |

DRAP-Centro | Loc. 2 | daily | 592.08 | 3.91 | 215.57 | 140.87 | ||

Loc. 3 | 15-min | 346.66 | 6.35 | 172.02 | 89.25 | |||

$W{S}_{Avg}$ | ms${}^{-1}$ | AppiZezere, | Loc. 1 | daily | 7.28 | 0.014 | 1.32 | 0.81 |

DRAP-Centro | Loc. 2 | daily | 5.91 | 0 | 1.06 | 0.82 | ||

Loc. 3 | 15-min | 28.85 | 0.031 | 4.62 | 3.80 | |||

$W{S}_{Max}$ | ms ${}^{-1}$ | AppiZezere, | Loc. 1 | daily | 74.04 | 1.06 | 4.82 | 2.16 |

DRAP-Centro | Loc. 2 | daily | 14.65 | 0 | 4.87 | 1.92 | ||

Loc. 3 | 15-min | 86.5 | 3.5 | 24.67 | 10.61 | |||

$Prec$ | mm d${}^{-1}$ | AppiZezere, | Loc. 1 | daily | 101.59 | 0 | 2 | 6.84 |

DRAP-Centro | Loc. 2 | daily | 112.80 | 0 | 2.92 | 8.95 | ||

Loc. 3 | 15-min | 101.6 | 0 | 2.28 | 7.20 | |||

$ETo$ | mm d${}^{-1}$ | Penman-Monteith | Loc. 1 | daily | 8.5 | 0.3 | 3.260 | 2.09 |

equation | Loc. 2 | daily | 7.4 | 0.3 | 2.83 | 1.964 | ||

Loc. 3 | daily | 9.8 | 0.2 | 3.68 | 2.088 | |||

$SWC{l}_{1}$ | m${}^{3}$m${}^{-3}$ | Loc. 1 | daily at 23:00 | 0.43 | 0.097 | 0.26 | 0.11 | |

ECMWF | Loc. 2 | daily at 23:00 | 0.43 | 0.07 | 0.27 | 0.11 | ||

Loc. 3 | daily at 23:00 | 0.43 | 0.11 | 0.26 | 0.10 | |||

$SWC{l}_{2}$ | m${}^{3}$m${}^{-3}$ | Loc. 1 | daily at 23:00 | 0.43 | 0.15 | 0.28 | 0.086 | |

ECMWF | Loc. 2 | daily at 23:00 | 0.43 | 0.15 | 0.29 | 0.08 | ||

Loc. 3 | daily at 23:00 | 0.43 | 0.15 | 0.27 | 0.086 | |||

$SWC{l}_{3}$ | m${}^{3}$m${}^{-3}$ | Loc. 1 | daily at 23:00 | 0.43 | 0.18 | 0.29 | 0.06 | |

ECMWF | Loc. 2 | daily at 23:00 | 0.42 | 0.2 | 0.30 | 0.061 | ||

Loc. 3 | daily at 23:00 | 0.43 | 0.17 | 0.28 | 0.064 | |||

$SWC{l}_{4}$ | m${}^{3}$m${}^{-3}$ | Loc. 1 | daily at 23:00 | 0.41 | 0.27 | 0.32 | 0.02 | |

ECMWF | Loc. 2 | daily at 23:00 | 0.41 | 0.29 | 0.33 | 0.03 | ||

Loc. 3 | daily at 23:00 | 0.4 | 0.24 | 0.30 | 0.037 |

_{i}: Volumetric Soil Water Level i (i ϵ {1,2,3,4}).

Variables | ${\mathbf{T}}_{\mathbf{Min}}$ | ${\mathbf{T}}_{\mathbf{Avg}}$ | ${\mathbf{T}}_{\mathbf{Max}}$ | ${\mathbf{HR}}_{\mathbf{Min}}$ | ${\mathbf{HR}}_{\mathbf{Avg}}$ | ${\mathbf{HR}}_{\mathbf{Max}}$ | ${\mathbf{SR}}_{\mathbf{Avg}}$ | ${\mathbf{WS}}_{\mathbf{Avg}}$ | ${\mathbf{WS}}_{\mathbf{Max}}$ | $\mathbf{Prec}$ | $\mathbf{ETo}$ | ${\mathbf{SWCl}}_{1}$ | ${\mathbf{SWCl}}_{2}$ | ${\mathbf{SWCl}}_{3}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Loc. 1 | ||||||||||||||

${T}_{Min}$ | 1 | 0.93 | 0.82 | −0.44 | −0.56 | −0.56 | 0.24 | 0.01 | −0.02 | −0.04 | 0.77 | −0.66 | −0.64 | −0.56 |

${T}_{Avg}$ | 0.93 | 1 | 0.97 | −0.66 | −0.72 | −0.58 | 0.37 | −0.11 | −0.09 | −0.18 | 0.82 | −0.82 | −0.78 | −0.63 |

${T}_{Max}$ | 0.82 | 0.97 | 1 | −0.78 | −0.77 | −0.56 | 0.40 | −0.20 | −0.15 | −0.27 | 0.89 | −0.86 | −0.81 | −0.64 |

$H{R}_{Min}$ | −0.44 | −0.66 | −0.78 | 1 | 0.91 | 0.62 | −0.45 | −0.01 | 0.03 | 0.46 | −0.75 | 0.79 | 0.73 | 0.50 |

$H{R}_{Avg}$ | −0.56 | −0.72 | −0.77 | 0.91 | 1 | 0.84 | −0.41 | −0.15 | −0.04 | -79 | −0.79 | 0.81 | 0.73 | 0.51 |

$H{R}_{Max}$ | −0.56 | −0.58 | −0.56 | 0.62 | 0.84 | 1 | −0.22 | −0.28 | −0.08 | 0.21 | −0.62 | 0.61 | 0.54 | 0.38 |

$S{R}_{Avg}$ | 0.24 | 0.37 | 0.40 | −0.45 | −0.41 | −0.22 | 1 | 0.02 | 0.10 | −0.20 | 0.027 | −0.39 | −0.34 | −0.17 |

$W{S}_{Avg}$ | 0.01 | −0.11 | −0.20 | −0.01 | −0.15 | −0.28 | 0.02 | 1 | 0.57 | 0.10 | 0.069 | 0.09 | 0.13 | 0.18 |

$W{S}_{Max}$ | −0.02 | −0.09 | −0.15 | 0.03 | −0.04 | −0.08 | 0.10 | 0.57 | 1 | 0.14 | −0.007 | 0.10 | 0.10 | 0.12 |

Prec | −0.04 | −0.18 | −0.27 | 0.46 | 0.37 | 0.21 | −0.20 | 0.10 | 0.14 | 1 | −0.24 | 0.31 | 0.27 | 0.13 |

ETo | 0.77 | 0.89 | 0.89 | −0.75 | −0.79 | −0.62 | 0.027 | 0.069 | −0.007 | −0.24 | 1 | −0.85 | −0.79 | −0.57 |

$SWC{l}_{1}$ | −0.66 | −0.82 | −0.86 | 0.79 | 0.81 | 0.61 | −0.39 | 0.09 | 0.10 | 0.31 | −0.85 | 1 | 0.95 | 0.72 |

$SWC{l}_{2}$ | −0.64 | −0.78 | −0.81 | 0.73 | 0.73 | 0.54 | −0.34 | 0.13 | 0.10 | 0.27 | −0.79 | 0.95 | 1 | 0.83 |

$SWC{l}_{3}$ | −0.56 | −0.63 | −0.64 | 0.50 | 0.51 | 0.38 | −0.17 | 0.18 | 0.12 | 0.13 | −0.57 | 0.72 | 0.83 | 1 |

Loc. 2 | ||||||||||||||

${T}_{Min}$ | 1 | 0.90 | 0.73 | −0.30 | −0.44 | −0.39 | 0.23 | 0.04 | 0.05 | 0.03 | 0.49 | −0.60 | −0.60 | −0.54 |

${T}_{Avg}$ | 0.90 | 1 | 0.94 | −0.60 | −0.66 | −0.43 | 0.37 | −0.08 | −0.04 | −0.17 | 0.60 | −0.81 | −0.77 | −0.63 |

${T}_{Max}$ | 0.73 | 0.94 | 1 | −0.76 | −0.72 | −0.40 | 0.39 | −0.20 | −0.16 | −0.30 | 0.58 | −0.86 | −0.81 | −0.63 |

$H{R}_{Min}$ | −0.30 | −0.60 | −0.76 | 1 | 0.88 | 0.49 | −0.42 | −0.04 | −0.02 | 0.50 | −0.47 | 0.75 | 0.68 | 0.45 |

$H{R}_{Avg}$ | −0.44 | −0.66 | −0.72 | 0.88 | 1 | 0.75 | −0.43 | −0.23 | −0.15 | 0.40 | −0.44 | 0.78 | 0.70 | 0.47 |

$H{R}_{Max}$ | −0.39 | −0.43 | −0.40 | 0.49 | 0.75 | 1 | −0.17 | −0.34 | −0.15 | 0.19 | −0.18 | 0.51 | 0.45 | 0.32 |

$S{R}_{Avg}$ | 0.23 | 0.37 | 0.39 | −0.42 | −0.43 | −0.17 | 1 | −0.0006 | 0.13 | −0.19 | 0.45 | −0.35 | −0.30 | −0.12 |

$W{S}_{Avg}$ | 0.04 | −0.08 | −0.20 | −0.04 | −0.23 | −0.34 | −0.001 | 1 | 0.78 | 0.10 | −0.001 | 0.06 | 0.10 | 0.13 |

$W{S}_{Max}$ | 0.05 | −0.04 | −0.16 | −0.02 | −0.15 | −0.15 | 0.13 | 0.78 | 1 | 0.19 | −0.01 | 0.09 | 0.11 | 0.13 |

Prec | 0.03 | −0.17 | −0.30 | 0.50 | 0.40 | 0.19 | −0.19 | 0.10 | 0.19 | 1 | −0.16 | 0.34 | 0.30 | 0.16 |

ETo | 0.49 | 0.60 | 0.58 | −0.47 | −0.44 | −0.18 | 0.45 | −0.0006 | −0.01 | −0.16 | 1 | −0.46 | −0.38 | −0.17 |

$SWC{l}_{1}$ | −0.60 | −0.81 | −0.86 | 0.75 | 0.78 | 0.51 | −0.35 | 0.06 | 0.09 | 0.34 | −0.46 | 1 | 0.95 | 0.74 |

$SWC{l}_{2}$ | −0.60 | −0.77 | −0.81 | 0.68 | 0.70 | 0.45 | −0.30 | 0.10 | 0.11 | 0.30 | −0.38 | 0.95 | 1 | 0.85 |

$SWC{l}_{3}$ | −0.54 | −0.63 | −0.63 | 0.45 | 0.47 | 0.32 | −0.12 | 0.13 | 0.13 | 0.16 | −0.17 | 0.74 | 0.85 | 1 |

Variables | ${\mathbf{T}}_{\mathbf{Min}}$ | ${\mathbf{T}}_{\mathbf{Avg}}$ | ${\mathbf{T}}_{\mathbf{Max}}$ | ${\mathbf{HR}}_{\mathbf{Min}}$ | ${\mathbf{HR}}_{\mathbf{Avg}}$ | ${\mathbf{HR}}_{\mathbf{Max}}$ | ${\mathbf{SR}}_{\mathbf{Avg}}$ | ${\mathbf{WS}}_{\mathbf{Avg}}$ | ${\mathbf{WS}}_{\mathbf{Max}}$ | $\mathbf{Prec}$ | $\mathbf{ETo}$ | ${\mathbf{SWCl}}_{1}$ | ${\mathbf{SWCl}}_{2}$ | ${\mathbf{SWCl}}_{3}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Loc. 3 | ||||||||||||||

${T}_{Min}$ | 1 | 0.94 | 0.86 | −0.49 | −0.55 | −0.52 | −0.07 | 0.57 | −0.09 | −0.08 | 0.73 | −0.66 | −0.64 | −0.50 |

${T}_{Avg}$ | 0.94 | 1 | 0.98 | −0.68 | −0.70 | −0.56 | −0.21 | 0.74 | −0.18 | −0.14 | 0.833 | −0.79 | −0.74 | −0.55 |

${T}_{Max}$ | 0.86 | 0.98 | 1 | −0.77 | −0.75 | −0.56 | −0.28 | 0.79 | −0.22 | −0.18 | 0.84 | −0.82 | −0.77 | −0.55 |

$H{R}_{Min}$ | −0.49 | −0.68 | −0.77 | 1 | 0.93 | 0.68 | 0.49 | −0.81 | −0.05 | −0.04 | −0.77 | 0.77 | 0.68 | 0.42 |

$H{R}_{Avg}$ | −0.55 | −0.70 | −0.75 | 0.93 | 1 | 0.87 | 0.44 | −0.76 | −0.17 | −0.12 | −0.80 | 0.77 | 0.66 | 0.40 |

$H{R}_{Max}$ | −0.52 | −0.56 | −0.56 | 0.68 | 0.87 | 1 | 0.28 | −0.51 | −0.28 | −0.17 | −0.67 | 0.59 | 70.49 | 0.31 |

$S{R}_{Avg}$ | 0.57 | 0.74 | 0.79 | −0.81 | −0.76 | −0.51 | −0.40 | 1 | 0.02 | 0.03 | 0.86 | −0.70 | −0.60 | −0.30 |

$W{S}_{Avg}$ | −0.09 | −0.18 | −0.22 | −0.05 | −0.17 | −0.28 | 0.05 | 0.02 | 1 | 0.85 | 0.121 | 0.11 | 0.13 | 0.18 |

$W{S}_{Max}$ | −0.08 | −0.14 | −0.18 | −0.04 | −0.12 | −0.17 | 0.12 | 0.03 | 0.85 | 1 | 0.118 | 0.08 | 0.09 | 0.12 |

Prec | −0.07 | −0.21 | −0.28 | 0.49 | 0.44 | 0.28 | 1 | −0.40 | 0.05 | 0.12 | −0.284 | 0.35 | 0.25 | 0.09 |

ETo | 0.73 | 0.833 | 0.84 | −0.77 | −0.80 | −0.67 | 0.86 | 0.121 | 0.118 | −0.284 | 1 | −0.79 | −0.70 | −0.40 |

$SWC{l}_{1}$ | −0.66 | −0.79 | −0.82 | 0.77 | 0.77 | 0.59 | 0.35 | −0.70 | 0.11 | 0.08 | −0.79 | 1 | 0.95 | 0.68 |

$SWC{l}_{2}$ | −0.64 | −0.74 | −0.77 | 0.68 | 0.66 | 0.49 | 0.25 | −0.60 | 0.13 | 0.09 | −0.70 | 0.95 | 1 | 0.79 |

$SWC{l}_{3}$ | −0.50 | −0.55 | −0.55 | 0.42 | 0.40 | 0.31 | 0.09 | −0.30 | 0.18 | 0.12 | −0.40 | 0.68 | 0.79 | 1 |

$SWC{l}_{4}$ | 0.03 | 0.04 | 0.03 | −0.05 | −0.05 | −0.03 | −0.02 | 0.23 | 0.10 | 0.06 | 0.154 | 0.06 | 0.15 | 0.431 |

## References

- World Urbanization Prospects: The 2018 Revision (ST/ESA/SER.A/420); Technical Report; United Nations, Department of Economic and Social Affairs, Population Division: New York, NY, USA, 2019.
- Sundmaeker, H.; Verdouw, C.N.; Wolfert, J.; Freire, L.P. Internet of Food and Farm 2020. In Digitising the Industry; Vermesan, O., Friess, P., Eds.; River Publishers: Aalborg, Denmark, 2016; pp. 129–150. [Google Scholar]
- FAO. World Agriculture 2030: Main Findings. 2002. Available online: http://www.fao.org/english/newsroom/news/2002/7833-en.html (accessed on 1 May 2020).
- Laaboudi, A.; Mouhouche, B.; Draoui, B. Neural network approach to reference evapotranspiration modeling from limited climatic data in arid regions. Int. J. Biometeorol.
**2012**, 56, 831–841. [Google Scholar] [CrossRef] [Green Version] - Achieng, K.O. Modelling of soil moisture retention curve using machine learning techniques: Artificial and deep neural networks vs support vector regression models. Comput. Geosci.
**2019**, 133, 104320. [Google Scholar] [CrossRef] - Allen, R.G.; Pereira, L.S.; Raes, D.; Smith, M. Crop Evapotranspiration—Guidelines for Computing Crop Water Requirements FAO Irrigation and Drainage Paper 56; FAO—Food and Agriculture Organization of the United Nations: Rome, Italy, 1998. [Google Scholar]
- Jimenez, A.F.; Cardenas, P.F.; Canales, A.; Jimenez, F.; Portacio, A. A survey on intelligent agents and multi-agents for irrigation scheduling. Comput. Electron. Agric.
**2020**, 176, 105474. [Google Scholar] [CrossRef] - Clulow, A.D.; Everson, C.S.; Mengistu, M.G.; Price, J.S.; Nickless, A.; Jewitt, G.P.W. Extending periodic eddy covariance latent heat fluxes through tree sap-flow measurements to estimate long-term total evaporation in a peat swamp forest. Hydrol. Earth Syst. Sci.
**2015**, 19, 2513–2534. [Google Scholar] [CrossRef] [Green Version] - Kumar, M.; Raghuwanshi, N.S.; Singh, R. Artificial neural networks approach in evapotranspiration modeling: A review. Irrig. Sci.
**2011**, 29, 11–25. [Google Scholar] [CrossRef] - Karandish, F.; Šimůnek, J. A comparison of numerical and machine-learning modeling of soil water content with limited input data. J. Hydrol.
**2016**, 543, 892–909. [Google Scholar] [CrossRef] [Green Version] - Adeyemi, O.; Grove, I.; Peets, S.; Domun, Y.; Norton, T. Dynamic Neural Network Modelling of Soil Moisture Content for Predictive Irrigation Scheduling. Sensors
**2018**, 18, 3408. [Google Scholar] [CrossRef] [Green Version] - Yamac, S.S.; Seker, C.; Negis, H. Evaluation of machine learning methods to predict soil moisture constants with different combinations of soil input data for calcareous soils in a semi arid area. Agric. Water Manag.
**2020**, 234. [Google Scholar] [CrossRef] - Fernandez-Lopez, A.; Marin-Sanchez, D.; Garcia-Mateos, G.; Ruiz-Canales, A.; Ferrandez-Villena-Garcia, M.; Molina-Martinez, J.M. A Machine Learning Method to Estimate Reference Evapotranspiration Using Soil Moisture Sensors. Appl. Sci.
**2020**, 10, 1912. [Google Scholar] [CrossRef] [Green Version] - Tseng, D.; Wang, D.; Chen, C.; Miller, L.; Song, W.; Viers, J.; Vougioukas, S.; Carpin, S.; Ojea, J.A.; Goldberg, K. Towards Automating Precision Irrigation: Deep Learning to Infer Local Soil Moisture Conditions from Synthetic Aerial Agricultural Images. In Proceedings of the 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), Munich, Germany, 20–24 August 2018; pp. 284–291. [Google Scholar]
- Song, X.; Zhang, G.; Liu, F.; Li, D.; Zhao, Y.; Yang, J. Modeling spatio-temporal distribution of soil moisture by deep learning-based cellular automata model. J. Arid. Land
**2016**, 8, 734–748. [Google Scholar] [CrossRef] [Green Version] - Adamala, S. Temperature based generalized wavelet-neural network models to estimate evapotranspiration in India. Inf. Process. Agric.
**2018**, 5, 149–155. [Google Scholar] [CrossRef] - Saggi, M.K.; Jain, S. Reference evapotranspiration estimation and modeling of the Punjab Northern India using deep learning. Comput. Electron. Agric.
**2019**, 156, 387–398. [Google Scholar] [CrossRef] - De Oliveira e Lucas, P.; Alves, M.A.; de Lima e Silva, P.C.; Guimarães, F.G. Reference evapotranspiration time series forecasting with ensemble of convolutional neural networks. Comput. Electron. Agric.
**2020**, 177, 105700. [Google Scholar] [CrossRef] - Zhang, J.; Zhu, Y.; Zhang, X.; Ye, M.; Yang, J. Developing a Long Short-Term Memory (LSTM) based model for predicting water table depth in agricultural areas. J. Hydrol.
**2018**, 561, 918–929. [Google Scholar] [CrossRef] - Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw.
**2015**, 61, 85–117. [Google Scholar] [CrossRef] [Green Version] - Drucker, H.; Burges, C.J.C.; Kaufman, L.; Smola, A.; Vapnik, V. Support Vector Regression Machines. In Proceedings of the 9th International Conference on Neural Information Processing Systems, NIPS’96, Denver, CO, USA, 2–5 December 1996; MIT Press: Cambridge, MA, USA, 1996; pp. 155–161. [Google Scholar]
- Breiman, L. Random Forests. Mach. Learn.
**2001**, 45, 5–32. [Google Scholar] [CrossRef] [Green Version] - Huber, P.J. Robust estimation of a location parameter. Ann. Math. Stat.
**1964**, 35, 73–101. [Google Scholar] [CrossRef] - Muñoz Sabater, J. ERA5-Land hourly data from 1981 to present. Copernicus Climate Change Service (C3S) Climate Data Store (CDS). 2019. Available online: https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-land?tab=overview (accessed on 15 April 2021).
- IFS Documentation CY45R1—Part IV: Physical processes. In IFS Documentation CY45R1; Number 4 in IFS Documentation; ECMWF: Reading, UK, 2018. [CrossRef]
- Balsamo, G.; Albergel, C.; Beljaars, A.; Boussetta, S.; Brun, E.; Cloke, H.; Dee, D.; Dutra, E.; Muñoz Sabater, J.; Pappenberger, F.; et al. ERA-Interim/Land: A global land surface reanalysis data set. Hydrol. Earth Syst. Sci.
**2015**, 19, 389–407. [Google Scholar] [CrossRef] [Green Version] - Richards, L.A. Capillary conduction of liquids through porous mediums. Physics
**1931**, 1, 318–333. [Google Scholar] [CrossRef] - Montgomery, D.C.; Jennings, C.L.; Kulahci, M. Introduction to Time Series Analysis and Forecasting; Wiley Series in Probability and Statistics; Wiley: Hoboken, NJ, USA, 2011. [Google Scholar]
- Benesty, J.; Chen, J.; Huang, Y.; Cohen, I. Pearson Correlation Coefficient. In Noise Reduction in Speech Processing; Springer: Berlin/Heidelberg, Germany, 2009; pp. 1–4. [Google Scholar]
- Kreyszig, E.; Kreyszig, H.; Norminton, E.J. Advanced Engineering Mathematics, 10th ed.; Wiley: Hoboken, NJ, USA, 2011. [Google Scholar]
- Patterson, J.; Gibson, A. Deep Learning: A Practitioner’s Approach; O’Reilly: Beijing, China, 2017. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput.
**1997**, 9, 1735–1780. [Google Scholar] [CrossRef] - Schuster, M.; Paliwal, K.K. Bidirectional recurrent neural networks. IEEE Trans. Signal Process.
**1997**, 45, 2673–2681. [Google Scholar] [CrossRef] [Green Version] - Sermanet, P.; Eigen, D.; Zhang, X.; Mathieu, M.; Fergus, R.; LeCun, Y. OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. arXiv
**2013**, arXiv:1312.6229. [Google Scholar] - Donahue, J.; Hendricks, L.A.; Rohrbach, M.; Venugopalan, S.; Guadarrama, S.; Saenko, K.; Darrell, T. Long-Term Recurrent Convolutional Networks for Visual Recognition and Description. IEEE Trans. Pattern Anal. Mach. Intell.
**2017**, 39, 677–691. [Google Scholar] [CrossRef] - Gal, Y.; Ghahramani, Z. A Theoretically Grounded Application of Dropout in Recurrent Neural Networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16, Barcelona, Spain, 5–10 December 2016 ; Curran Associates Inc.: Red Hook, NY, USA, 2016; pp. 1027–1035. [Google Scholar]
- Mockus, J. Bayesian approach to global optimization. In Mathematics and its Applications (Soviet Series); Theory and Applications, with a 5.25-inch IBM PC Floppy Disk; Kluwer Academic Publishers Group: Dordrecht, The Netherlands, 1989; Volume 37, p. xiv+254. [Google Scholar]
- Bergstra, J.; Bardenet, R.; Bengio, Y.; Kégl, B. Algorithms for Hyper-Parameter Optimization. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS’11, Granada, Spain, 12–14 December 2011; Curran Associates Inc.: Red Hook, NY, USA, 2011; pp. 2546–2554. [Google Scholar]
- Brochu, E.; Cora, V.M.; de Freitas, N. A Tutorial on Bayesian Optimization of Expensive Cost Functions with Application to Active User Modeling and Hierarchical Reinforcement Learning; Technical Report; University of British Columbia‚ Department of Computer Science: Vancouver, BC, Canada, 2009. [Google Scholar]
- Nogueira, F. Bayesian Optimization: Open Source Constrained Global Optimization Tool for Python. 2014. Available online: https://github.com/fmfn/BayesianOptimization (accessed on 1 August 2020).
- Willmott, C.J.; Ackleson, S.G.; Davis, R.E.; Feddema, J.J.; Klink, K.M.; Legates, D.R.; O’Donnell, J.; Rowe, C.M. Statistics for the evaluation and comparison of models. J. Geophys. Res. Ocean.
**1985**, 90, 8995–9005. [Google Scholar] [CrossRef] [Green Version] - Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. 2015. Software Available. Available online: tensorflow.org (accessed on 1 April 2020).
- Chollet, F. Keras. 2015. Available online: https://keras.io (accessed on 1 April 2020).
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization, 2014. In Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA, 7–9 May 2015. arxiv:1412.6980. [Google Scholar]
- Chang, Y.W.; Hsieh, C.J.; Chang, K.W.; Ringgaard, M.; Lin, C.J. Training and Testing Low-degree Polynomial Data Mappings via Linear SVM. J. Mach. Learn. Res.
**2010**, 11, 1471–1490. [Google Scholar] - Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction, 2nd ed.; The MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]

**Figure 4.**(

**A**). Diagram of LSTM unit and (

**B**). regular RNN unit. Left side shows LSTM cell, right side shows RNN cell.

**Figure 6.**The left-hand side is the general CNN architecture, and the right-hand side is the convolutional operation.

**Figure 8.**${R}^{2}$ and RMSE during training. The

**left side**shows the ${R}^{2}$-score and the

**right side**shows the RMSE.

**Figure 9.**True values vs. Predicted values. The x-axis indicates true values, and the y-axis indicates prediction values by models.

**Figure 10.**The RMSE and ${R}^{2}$-score of CNN-BLSTM model during training. The

**left side**shows the ${R}^{2}$-score and the

**right side**shows the RMSE.

Location | Póvoa de Atalaia (Loc. 1) | Borralheira (Loc. 2) | Fadagosa (Loc. 3) |
---|---|---|---|

Longitude | ${40}^{\xb0}{4}^{\prime}{15.43}^{\u2033}$ N | ${40}^{\xb0}{19}^{\prime}{14.14}^{\u2033}$ N | ${40}^{\xb0}{1}^{\prime}{46.55}^{\u2033}$ N |

Latitude | ${7}^{\xb0}{24}^{\prime}{26.02}^{\u2033}$ W | ${7}^{\xb0}{25}^{\prime}{23.67}^{\u2033}$ W | ${7}^{\xb0}{26}^{\prime}{36.27}^{\u2033}$ W |

Start date | 19 September 2013 | 19 September 2013 | 16 May 2020 |

End date | 16 May 2020 | 07 January 2010 | 23 March 2020 |

Temporal resolution | daily | daily | 15 min |

Dropout Size | |||
---|---|---|---|

Dropout on | 0.00000 | 0.10000 | 0.20000 |

outputs | 0.02050 | 0.01970 | 0.01960 |

inputs | - | 0.01996 | 0.02000 |

recurrent-outputs | - | 0.01950 | 0.09136 |

Model Architecture | Locations | Number of Nodes per Layer | |||
---|---|---|---|---|---|

64 | 128 | 256 | 512 | ||

LSTM-2 layers | Loc. 1 | 0.01713 | 0.01197 | 0.01135 | 0.01222 |

Loc. 2 | 0.01906 | 0.01591 | 0.01446 | 0.01466 | |

Loc. 3 | 0.02064 | 0.02059 | 0.02065 | 0.02035 | |

Mean | 0.018943 | 0.016156 | 0.015486 | 0.015743 | |

LSTM-3 layers | Loc. 1 | 0.01973 | 0.01221 | 0.01114 | 0.01128 |

Loc. 2 | 0.01923 | 0.01566 | 0.01463 | 0.01460 | |

Loc. 3 | 0.02168 | 0.02089 | 0.02106 | 0.02134 | |

Mean | 0.020213 | 0.016253 | 0.01561 | 0.01574 | |

LSTM-4 layers | Loc. 1 | 0.01911 | 0.01401 | 0.01317 | 0.01156 |

Loc. 2 | 0.02280 | 0.01623 | 0.01490 | 0.01490 | |

Loc. 3 | 0.21685 | 0.15875 | 0.12353 | 0.11088 | |

Mean | 0.086253 | 0.0629 | 0.05053 | 0.045886 | |

BLSTM-1 layer | Loc. 1 | 0.01527 | 0.01207 | 0.01104 | 0.0115 |

Loc. 2 | 0.01870 | 0.01715 | 0.01556 | 0.01522 | |

Loc. 3 | 0.02001 | 0.02023 | 0.01949 | 0.01936 | |

Mean | 0.017993 | 0.016483 | 0.015363 | 0.01536 | |

BLSTM-2 layers | Loc. 1 | 0.01574 | 0.01374 | 0.01268 | 0.01289 |

Loc. 2 | 0.01893 | 0.01689 | 0.01583 | 0.01583 | |

Loc. 3 | 0.02017 | 0.01959 | 0.01945 | 0.01982 | |

Mean | 0.01828 | 0.01674 | 0.015986 | 0.01618 |

Model | Learning Rate | Decay | Batch Size | Dropout Size |
---|---|---|---|---|

BLSTM | ${10}^{-4}$ | ${10}^{-5}$ | 124 | 0.2 |

**Table 5.**Evaluation of the LSTM model on the test set for each location. The model was trained on all datasets.

Locations | Metrics | ETo | ${\mathbf{SWCl}}_{1}$ | ${\mathbf{SWCl}}_{2}$ | ${\mathbf{SWCl}}_{3}$ | In-General |
---|---|---|---|---|---|---|

Loc. 1 | MSE | 0.07 | 0.00074 | 0.00027 | $4.94\times {10}^{-5}$ | 0.0144 |

RMSE | 0.29 | 0.021 | 0.01 | 0.004 | 0.13 | |

MAE | 0.115 | 0.02 | 0.013 | 0.0072 | 0.12 | |

${R}^{2}$ | 0.94 | 0.9 | 0.94 | 0.98 | 0.98 | |

MBE | 0.032 | −0.0013 | 0.0016 | 0.003 | 0.0072 | |

Loc. 2 | MSE | 0.093 | 0.0004 | $7.48\times {10}^{-5}$ | $1.86\times {10}^{-5}$ | $0.018$ |

RMSE | 0.3 | 0.02 | 0.0086 | 0.0043 | 0.137 | |

MAE | 0.23 | 0.012 | 0.005 | 0.002 | 0.05 | |

${R}^{2}$ | 0.91 | 0.92 | 0.98 | 0.99 | 0.96 | |

MBE | −0.024 | 0.0034 | 0.0006 | 0.0004 | −0.004 | |

Loc. 3 | MSE | 0.27 | 0.0002 | $3.8\times {10}^{-5}$ | $7.9\times {10}^{-6}$ | 0.056 |

RMSE | 0.51 | 0.016 | 0.006 | 0.002 | 0.23 | |

MAE | 0.36 | 0.01 | 0.003 | 0.0016 | 0.69 | |

${R}^{2}$ | 0.93 | 0.97 | 0.99 | 0.99 | 0.98 | |

MBE | −0.06 | −0.0007 | −0.0007 | −0.00027 | −0.013 |

**Table 6.**Evaluation of the LSTM model on the test set for each site. The model was trained on two datasets and tested on the third dataset.

Locations | Metrics | ETo | ${\mathbf{SWCl}}_{1}$ | ${\mathbf{SWCl}}_{2}$ | ${\mathbf{SWCl}}_{3}$ | In-General |
---|---|---|---|---|---|---|

Loc. 1 | MSE | 0.08 | 0.0004 | $9.4\times {10}^{-5}$ | $2.3\times {10}^{-5}$ | 0.016 |

RMSE | 0.28 | 0.021 | 0.009 | 0.004 | 0.129 | |

MAE | 0.21 | 0.013 | 0.0061 | 0.0029 | 0.12 | |

${R}^{2}$ | 0.94 | 0.9 | 0.94 | 0.98 | 0.98 | |

MBE | 0.032 | −0.0013 | 0.0016 | 0.003 | 0.0072 | |

Loc. 2 | MSE | 0.1 | 0.00047 | $7.8\times {10}^{-5}$ | $1.89\times {10}^{-5}$ | 0.02 |

RMSE | 0.33 | 0.021 | 0.0088 | 0.0043 | 0.15 | |

MAE | 0.25 | 0.014 | 0.006 | 0.0029 | 0.055 | |

${R}^{2}$ | 0.89 | 0.91 | 0.98 | 0.99 | 0.95 | |

MBE | −0.024 | 0.0034 | 0.0006 | 0.0004 | −0.004 | |

Loc. 3 | MSE | 0.35 | 0.0004 | $7.9\times {10}^{-5}$ | $1.2\times {10}^{-5}$ | 0.075 |

RMSE | 0.59 | 0.02 | 0.008 | 0.003 | 0.28 | |

MAE | 0.46 | 0.014 | 0.005 | 0.002 | 0.9 | |

${R}^{2}$ | 0.9 | 0.95 | 0.98 | 0.99 | 0.97 | |

MBE | −0.26 | 0.0012 | 0.0002 | 0.0005 | −0.004 |

Layer (Type) | Output Shape |
---|---|

Input | (Batch Size, 2, 4, 11) |

TimeDistributed(Conv1D) | (Batch Size, 2, 4, N. Filters) |

(kernel-size = 3, padding = same) | |

Dropout 1 | (Batch Size, 2, 4, N. Filters) |

TimeDistributed(Conv1D) | (Batch Size, 2, 4, N. Filters) |

(kernel-size = 2, padding = same) | |

Dropout 1 | (Batch Size, 2, 4, N. Filters) |

TimeDistributed(Averagepooling) | (Batch Size, 2, 2, N. Filters) |

(pool-size = 2) | |

TimeDistributed(Flatten) | (Batch Size, 2, $2\times \mathrm{N}.\phantom{\rule{4.pt}{0ex}}\mathrm{Filters}$) |

Models | Locations | MSE | RMSE | MAE | ${\mathit{R}}^{2}$ | MBE |
---|---|---|---|---|---|---|

Loc. 1 | 0.161 | 0.127 | 0.047 | 0.974 | −0.006 | |

CNN-LSTM | Loc. 2 | 0.019 | 0.138 | 0.051 | 0.96 | −0.003 |

(AveragePooling+$\mathrm{tanh}$) | Loc. 3 | 0.063 | 0.25 | 0.083 | 0.976 | −0.016 |

Loc. 1 | 0.016 | 0.12 | 0.05 | 0.975 | −0.014 | |

CNN-LSTM | Loc. 2 | 0.021 | 0.14 | 0.055 | 0.95 | −0.017 |

(MaxPooling, $\mathrm{Relu}$) | Loc. 3 | 0.61 | 0.24 | 0.082 | 0.977 | −0.012 |

Models | T(E/S) | N. Param. | N. E | N. P | Tr. S | val. S | Te. S | TT |
---|---|---|---|---|---|---|---|---|

CNN-LSTM | 2 | 6,842,885 | 160 | 20 | 6197 | 1550 | 365 | <1 s |

LSTM | 1 | 2,151,429 | 250 | 20 | 6197 | 1550 | 365 | <1 s |

Layer (Type) | Output Shape | Param |
---|---|---|

Input | (Batch size, 8, 11) | 0 |

(Conv1D)(kernel-size = 3) | (Batch size, 6, N. Filters) | 17,408 |

Dropout 1 | (Batch size, 6, 512) | 0 |

(Conv1D)(kernel-size = 2) | (Batch size, 5, N. Filters) | 524,800 |

Dropout 1 | (Batch size, 5, N. Filters) | 0 |

(Averagepooling)(pool-size = 2) | (Batch size, 2, N. Filters) | 0 |

Flatten | (Batch size, $2\times \mathrm{N}.\phantom{\rule{4.pt}{0ex}}\mathrm{Filters}$) | 0 |

Dense | (Batch size, N. Filters) | 524,800 |

Dense | (Batch size, 5) | 2565 |

**Table 11.**Performance of the traditional machine learning algorithms and CNN model on the dataset (MSE).

Locations | Metrics | SVR-RBF | RF | CNN |
---|---|---|---|---|

Loc. 1 | MSE | 0.2 | 0.085 | 0.033 |

RMSE | 0.45 | 0.29 | 0.18 | |

MAE | 0.31 | 0.21 | 0.7 | |

${R}^{2}$ | 0.81 | 0.92 | 0.94 | |

MBE | 0.14 | 0.002 | 0.03 | |

Loc. 2 | MSE | 0.17 | 0.11 | 0.027 |

RMSE | 0.42 | 0.34 | 0.16 | |

MAE | 0.31 | 0.26 | 0.06 | |

${R}^{2}$ | 0.83 | 0.88 | 0.94 | |

MBE | 0.14 | 0.02 | −0.03 | |

Loc. 3 | MSE | 0.33 | 0.32 | 0.2 |

RMSE | 0.58 | 0.56 | 0.45 | |

MAE | 0.39 | 0.39 | 0.16 | |

${R}^{2}$ | 0.92 | 0.92 | 0.93 | |

MBE | −0.023 | −0.02 | −0.016 |

Loss Functions | Metrics | |||
---|---|---|---|---|

MSE | RMSE | MAE | ${\mathit{R}}^{2}$ | |

MSE | 0.056 | 0.23 | 0.69 | 0.98 |

RMSE | 0.067 | 0.25 | 0.8 | 0.975 |

MAE | 0.07 | 0.07 | 0.27 | 0.97 |

${H}_{\delta =1}$ | 0.063 | 0.25 | 0.79 | 0.976 |

${H}_{\delta =0.1}$ | 0.068 | 0.26 | 0.77 | 0.974 |

${H}_{\delta =0.01}$ | 0.074 | 0.27 | 0.8 | 0.972 |

${H}_{\delta =0.001}$ | 0.062 | 0.27 | 0.07 | 0.977 |

${H}_{\delta =0.0001}$ | 0.09 | 0.3 | 0.1 | 0.965 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Alibabaei, K.; Gaspar, P.D.; Lima, T.M.
Modeling Soil Water Content and Reference Evapotranspiration from Climate Data Using Deep Learning Method. *Appl. Sci.* **2021**, *11*, 5029.
https://doi.org/10.3390/app11115029

**AMA Style**

Alibabaei K, Gaspar PD, Lima TM.
Modeling Soil Water Content and Reference Evapotranspiration from Climate Data Using Deep Learning Method. *Applied Sciences*. 2021; 11(11):5029.
https://doi.org/10.3390/app11115029

**Chicago/Turabian Style**

Alibabaei, Khadijeh, Pedro D. Gaspar, and Tânia M. Lima.
2021. "Modeling Soil Water Content and Reference Evapotranspiration from Climate Data Using Deep Learning Method" *Applied Sciences* 11, no. 11: 5029.
https://doi.org/10.3390/app11115029