# A Physics-Informed, Machine Learning Emulator of a 2D Surface Water Model: What Temporal Networks and Simulation-Based Inference Can Help Us Learn about Hydrologic Processes

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. The Tilted V benchmark Problem

_{rain}is the rainfall flux (mh

^{−1}), t is time (h), and v is the velocity given by the depth-discharge relationship:

_{x}

_{,y}are the topographic slopes in the x and y directions (-), and n is the Manning’s roughness coefficient (m

^{1/3}h

^{−1}). There are many approaches to solving for surface water flow; however, no analytical solutions exist for the Tilted V test problem. In this work, the integrated hydrologic model ParFlow [26,28,29] was used to solve these equations.

#### 2.2. Numerical Solution of the Tilted V and General Training Approach

#### 2.3. ML Model Architectures

- CNN3D, a deep 3D convolution (third dimension is time) detailed in Table A1;
- CNN2D, a deep 2D convolution detailed in Table A2;
- CNN2D_B1, a moderately deep 2D convolution detailed in Table A3;
- UNet2D_E7, a three-level U-Net detailed in Table A4;
- CNN2D_A1, a deep 2D convolution with no pooling detailed in Table A5; and
- CNN2D_B3, a very shallow 2D convolution detailed in Table A6.

#### 2.4. ML Model Training

Training | Test In Range | Test Full Range | ||||||||
---|---|---|---|---|---|---|---|---|---|---|

Variable | Units | Lower | Mid1 | Mid2 | Upper | Lower | Mid | Upper | Lower | Upper |

channel slope | (-) | 0.001 | 0.004 | 0.007 | 0.01 | 0.0015 | 0.0055 | 0.0095 | 0.01 | 0.02 |

hill slopes | (-) | 0.05 | 0.083 | 0.117 | 0.15 | 0.055 | 0.0775 | 0.1 | 0.01 | 0.2 |

Manning’s n | (m^{1/3}h^{−1}) | 8.3 × 10^{−6} | 8.8 × 10^{−6} | 9.3 × 10^{−6} | 9.7 × 10^{−6} | 8.5 × 10^{−6} | 8.8 × 10^{−6} | 9.0 × 10^{−6} | 5.0 × 10^{−6} | 2.1 × 10^{−5} |

rain rate | (mh^{−1}) | −0.005 | −0.01 | −0.015 | −0.02 | −0.007 | −0.011 | −0.015 | −0.004 | −0.02 |

rain start | (h) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

rain length | (h) | 0.1 | 0.15 | 0.2 | 0.25 | 0.1 | 0.15 | 0.25 | 0.1 | 0.3 |

^{−4}using the Adam optimizer [36], with 2500 epochs of training for each case. A Smooth L1 loss function was calculated between the ML-predicted pressure field and the field produced by the ParFlow simulation at each snapshot in time. Only two hyperparameters (learning rate and number of epochs) were chosen through experimentation on the training data. Note that other hyperparameters in the models themselves (e.g., number of output channels, size of the linear layer) were not tuned but vary between model simulations in the experiments presented here. The final loss values for all models was below 10

^{−4}, with some being below 10

^{−5}. Training times were not rigorously tracked but ranged from approximately 2 min for the 2DCNN_B3 model to about an hour for the U-Net.

^{2}statistic), and Spearman’s Rank Correlation Coefficient (commonly the Spearman’s Rho) of the ML predicted and ParFlow simulated hydrographs. These metrics were calculated for each realization in the ensemble, and the mean and variance were reported across the test ensembles.

#### 2.5. Parameter Evaluation

- Three ParFlow simulations were selected at random from the In Range ensemble and their hydrographs calculated, and these are defined as our target ParFlow “truth” simulations;
- A full suite of simulations were conducted with the three ML models varying parameters shown in Table 1 across the complete set of In Range ensemble values;
- Hydrographs were calculated from the entire suite of the ML models, and the RMSE was calculated for every realization compared to each of the ParFlow “truth” simulations;
- The five realizations with the lowest RMSE values were selected as the best performing ML realizations;
- The parameters for the best-matching ML realizations were compared to the parameters from the ParFlow “truth” simulations to evaluate whether the emulators would preserve the behavior of the ParFlow parameters.

## 3. Results and Discussion

#### 3.1. Base-Case Model Performance and In Range Test Cases

#### 3.2. Model Sensitivity to Training Data

**Figure 5.**Heatmap for three metrics (Pearson Correlation Coefficient, Spearman’s Rho, and RMSE) calculated for the hydrographs derived from all ML simulations compared to ParFlow across all 243 In Range test realizations. As indicated in the titles, the ensemble mean of each metric is shown in the top row, and the standard deviation of each metric is shown in the bottom row. Models were trained on a different number of realizations, as indicated by the x-axis in each subplot.

#### 3.3. Full Range Test Cases

**Figure 9.**Surface pressure averaged across all 32 Full Range test realizations for the six ML models compared to the ParFlow simulations, at five snapshots in time. All models were trained on 1024 realizations.

**Figure 10.**RMSE of surface pressure between each of the six ML models and ParFlow simulations calculated across all 32 Full Range test realizations, at five snapshots in time. All models were trained on 1024 realizations.

**Figure 11.**Ensemble hydrograph comparisons between each of the six ML models and ParFlow simulations across all 32 Full Range test realizations. The ensemble mean is shown for both ML (red) and ParFlow (blue) along with a shaded range that represents the entire spread across all ensemble members. All models were trained on 1024 realizations.

#### 3.4. Parameter Evaluation

## 4. Conclusions

- ML models can be trained as general emulators of hydrologic behavior. ML models that are given the same input data and are run in the same manner as a physical hydrologic model (e.g., CNN2D) are possible and still exhibit good performance across a range of metrics. They can be given the same input as a physically based hydrologic model and produce the same outputs, in this case, time-dependent maps of surface pressure. This is exciting and suggests that true ML-emulator models for more complex problems are also possible.
- This is a distinctly new approach to applications of ML in hydrology and suggests that a ML model can learn general physical behavior.
- ML models that were provided with explicit temporal information during training (CNN3D) showed the best performance across a range of metrics. This suggests that the ML models without explicit temporal dependence do not contain the ability to simulate underlying hydrologic processes the same as the physically based model. The physical processes represented by hydrologic models allow them to convert spatial patterns in pressure to temporal behavior at the outlet. For best performance, ML models should be trained on time series to emulate the dynamics of hydrologic models.
- Deeper networks (CNN3D, CNN2D) perform better than models with fewer parameters (CNN2D_B1, CNN2D_B3). Some hydrograph metrics, such as total flow, are easier for the models to capture, while others, such as peak timing, are more challenging.
- ML-emulator models perform best when tested on cases with inputs that fall within the range of the parameter sets used for training. While some of the deeper models (CNN3D) exhibited good performance for the Full Range test set, other models exhibited very poor and even unrealistic behavior.
- We demonstrate that ML models can be successfully calibrated using an ABC approach. This approach calibrated the parameters of the ML model directly on a synthetic observation. We found that this approach could consistently improve model performance. Moreover, in many cases, the resulting calibrated model parameter sets correlated well with the original ParFlow inputs, indicating a good match between the learned model behaviors and ParFlow relationships between parameters and simulated outputs.

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Detailed Tables of the ML Model Architectures

Layer | Output Shape | Number of Parameters |
---|---|---|

├─Sequential: 1-1 | [18, 10, 6, 6] | |

│ └─Conv3d: 2-1 | [18, 40, 25, 25] | 2448 |

│ └─ReLU: 2-2 | [18, 40, 25, 25] | |

│ └─Conv3d: 2-3 | [18, 40, 25, 25] | 8766 |

│ └─ReLU: 2-4 | [18, 40, 25, 25] | |

│ └─MaxPool3d: 2-5 | [18, 20, 12, 12] | |

│ └─Conv3d: 2-6 | [18, 20, 12, 12] | 8766 |

│ └─ReLU: 2-7 | [18, 20, 12, 12] | |

│ └─MaxPool3d: 2-8 | [18, 10, 6, 6] | |

├─Linear: 1-2 | [7000] | 45,367,000 |

├─Linear: 1-3 | [25,000] | 175,025,000 |

Total: | 220,411,980 |

Layer | Output Shape | Number of Parameters |
---|---|---|

├─Sequential: 1-1 | [18, 6, 6] | |

│ └─Conv2d: 2-1 | [18, 25, 25] | 828 |

│ └─ReLU: 2-2 | [18, 25, 25] | |

│ └─Conv2d: 2-3 | [18, 25, 25] | 2934 |

│ └─ReLU: 2-4 | [18, 25, 25] | |

│ └─MaxPool2d: 2-5 | [18, 12, 12] | |

│ └─Conv2d: 2-6 | [18, 12, 12] | 2934 |

│ └─ReLU: 2-7 | [18, 12, 12] | |

│ └─MaxPool2d: 2-8 | [18, 6, 6] | |

├─Linear: 1-2 | [20,000] | 12,980,000 |

├─Linear: 1-3 | [625] | 12,500,625 |

Total: | 25,487,321 |

Layer | Output Shape | Number of Parameters |
---|---|---|

├─Sequential: 1-1 | [6, 6, 6] | |

│ └─Conv2d: 2-1 | [6, 25, 25] | 276 |

│ └─ReLU: 2-2 | [6, 25, 25] | |

│ └─MaxPool2d: 2-3 | [6, 12, 12] | |

│ └─Conv2d: 2-4 | [6, 12, 12] | 330 |

│ └─ReLU: 2-5 | [6, 12, 12] | |

│ └─MaxPool2d: 2-6 | [6, 6, 6] | |

├─Linear: 1-2 | [7000] | 1,519,000 |

├─Linear: 1-3 | [625] | 4,375,625 |

Total: | 5,895,231 |

Layer | Output Shape | Number of Parameters |
---|---|---|

├─Sequential: 1-1 | [16, 12, 12] | |

│ └─Conv2d: 2-1 | [16, 25, 25] | 736 |

│ └─BatchNorm2d: 2-2 | [16, 25, 25] | 32 |

│ └─ReLU: 2-3 | [16, 25, 25] | |

│ └─Conv2d: 2-4 | [16, 25, 25] | 2320 |

│ └─BatchNorm2d: 2-5 | [16, 25, 25] | 32 |

│ └─ReLU: 2-6 | [16, 25, 25] | |

│ └─MaxPool2d: 2-7 | [16, 12, 12] | |

├─Sequential: 1-2 | [32, 6, 6] | |

│ └─Conv2d: 2-8 | [32, 12, 12] | 4640 |

│ └─BatchNorm2d: 2-9 | [32, 12, 12] | 64 |

│ └─ReLU: 2-10 | [32, 12, 12] | |

│ └─Conv2d: 2-11 | [32, 12, 12] | 9248 |

│ └─BatchNorm2d: 2-12 | [32, 12, 12] | 64 |

│ └─ReLU: 2-13 | [32, 12, 12] | |

│ └─MaxPool2d: 2-14 | [32, 6, 6] | |

├─Sequential: 1-3 | [64, 3, 3] | |

│ └─Conv2d: 2-15 | [64, 6, 6] | 18,496 |

│ └─BatchNorm2d: 2-16 | [64, 6, 6] | 128 |

│ └─ReLU: 2-17 | [64, 6, 6] | |

│ └─Conv2d: 2-18 | [64, 6, 6] | 36,928 |

│ └─BatchNorm2d: 2-19 | [64, 6, 6] | 128 |

│ └─ReLU: 2-20 | [64, 6, 6] | |

│ └─MaxPool2d: 2-21 | [64, 3, 3] | |

├─Sequential: 1-4 | [32, 6, 6] | |

│ └─Conv2d: 2-22 | [32, 3, 3] | 18,464 |

│ └─BatchNorm2d: 2-23 | [32, 3, 3] | 64 |

│ └─ReLU: 2-24 | [32, 3, 3] | |

│ └─Conv2d: 2-25 | [32, 3, 3] | 9248 |

│ └─BatchNorm2d: 2-26 | [32, 3, 3] | 64 |

│ └─ReLU: 2-27 | [32, 3, 3] | |

│ └─ConvTranspose2d: 2-28 | [32, 6, 6] | 4128 |

├─Sequential: 1-5 | [16, 12, 12] | |

│ └─Conv2d: 2-29 | [32, 6, 6] | 18,464 |

│ └─BatchNorm2d: 2-30 | [32, 6, 6] | 64 |

│ └─ReLU: 2-31 | [32, 6, 6] | |

│ └─Conv2d: 2-32 | [16, 6, 6] | 4624 |

│ └─BatchNorm2d: 2-33 | [16, 6, 6] | 32 |

│ └─ReLU: 2-34 | [16, 6, 6] | |

│ └─ConvTranspose2d: 2-35 | [16, 12, 12] | 1040 |

├─Sequential: 1-6 | [1, 25, 25] | |

│ └─Conv2d: 2-36 | [32, 12, 12] | 9248 |

│ └─BatchNorm2d: 2-37 | [32, 12, 12] | 64 |

│ └─ReLU: 2-38 | [32, 12, 12] | |

│ └─Conv2d: 2-39 | [16, 12, 12] | 4624 |

│ └─BatchNorm2d: 2-40 | [16, 12, 12] | 32 |

│ └─ReLU: 2-41 | [16, 12, 12] | |

│ └─ConvTranspose2d: 2-42 | [1, 25, 25] | 65 |

Total: | 143,041 |

Layer | Output Shape | Number of Parameters |
---|---|---|

├─Sequential: 1-1 | [16, 25, 25] | |

│ └─Conv2d: 2-1 | [16, 25, 25] | 736 |

│ └─ReLU: 2-2 | [16, 25, 25] | |

│ └─Conv2d: 2-3 | [16, 25, 25] | 2320 |

│ └─ReLU: 2-4 | [16, 25, 25] | |

│ └─Conv2d: 2-5 | [16, 25, 25] | 2320 |

│ └─ReLU: 2-6 | [16, 25, 25] | |

├─Linear: 1-2 | [7000] | 70,007,000 |

├─Linear: 1-3 | [625] | 4,375,625 |

Total: | 74,388,001 |

Layer | Output Shape | Number of Parameters |
---|---|---|

├─Sequential: 1-1 | [5, 6, 6] | |

│ └─Conv2d: 2-1 | [5, 25, 25] | 230 |

│ └─ReLU: 2-2 | [5, 25, 25] | |

│ └─MaxPool2d: 2-3 | [5, 12, 12] | |

│ └─Conv2d: 2-4 | [5, 12, 12] | 230 |

│ └─ReLU: 2-5 | [5, 12, 12] | |

│ └─MaxPool2d: 2-6 | [5, 6, 6] | |

├─Linear: 1-2 | [100] | 18,100 |

├─Linear: 1-3 | [625] | 63,125 |

Total: | 81,685 |

## References

- Paniconi, C.; Putti, M. Physically based modeling in catchment hydrology at 50: Survey and outlook. Water Resour. Res.
**2015**, 51, 7090–7129. [Google Scholar] [CrossRef] [Green Version] - Rogers, L.L.; Dowla, F.U. Optimization of groundwater remediation using artificial neural networks with parallel solute transport modeling. Water Resour. Res.
**1994**, 30, 457–481. [Google Scholar] [CrossRef] - Artificial Neural Networks in Hydrology. II: Hydrologic Applications. J. Hydrol. Eng.
**2000**, 5, 124–137. [Google Scholar] [CrossRef] - Artificial Neural Networks in Hydrology. I: Preliminary Concepts. J. Hydrol. Eng.
**2000**, 5, 115–123. [Google Scholar] [CrossRef] - Kratzert, F.; Klotz, D.; Brenner, C.; Schulz, K.; Herrnegger, M. Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks. Hydrol. Earth Syst. Sci.
**2018**, 22, 6005–6022. [Google Scholar] [CrossRef] [Green Version] - Wilkinson, M.D.; Dumontier, M.; Aalbersberg, I.J.; Appleton, G.; Axton, M.; Baak, A.; Blomberg, N.; Boiten, J.-W.; da Silva Santos, L.B.; Bourne, P.E.; et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data
**2016**, 3, 160018. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Nevo, S. The Technology Behind our Recent Improvements in Flood Forecasting. Google AI Blog
**2020**. Available online: http://ai.googleblog.com/2020/09/the-technology-behind-our-recent.html (accessed on 21 November 2021). - Moshe, Z.; Metzger, A.; Elidan, G.; Kratzert, F.; Nevo, S.; El-Yaniv, R. Hydronets: Leveraging river structure for hydrologic modeling. arXiv
**2020**, arXiv:2007.00595. [Google Scholar] - Maskey, M.; Alemohammad, H.; Murphy, K.J.; Ramachandran, R. Advancing AI for Earth science: A data systems perspective. Eos Trans. Am. Geophys. Union
**2020**, 101. Available online: https://eos.org/science-updates/advancing-ai-for-earth-science-a-data-systems-perspective (accessed on 21 November 2021). [CrossRef] - Karpatne, A.; Atluri, G.; Faghmous, J.H.; Steinbach, M.; Banerjee, A.; Ganguly, A.; Shekhar, S.; Samatova, N.; Kumar, V. Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data. IEEE Trans. Knowl. Data Eng.
**2017**, 29, 2318–2331. [Google Scholar] [CrossRef] - Jiang, S.; Zheng, Y.; Solomatine, D. Improving AI System Awareness of Geoscience Knowledge: Symbiotic Integration of Physical Approaches and Deep Learning. Geophys. Res. Lett.
**2020**, 47, e2020GL088229. [Google Scholar] [CrossRef] - Zhao, W.L.; Gentine, P.; Reichstein, M.; Zhang, Y.; Zhou, S.; Wen, Y.; Lin, C.; Li, X.; Qiu, G.Y. Physics-Constrained Machine Learning of Evapotranspiration. Geophys. Res. Lett.
**2019**, 46, 14496–14507. [Google Scholar] [CrossRef] - Bergen, K.J.; Johnson, P.A.; Hoop, M.V.d.; Beroza, G.C. Machine learning for data-driven discovery in solid Earth geoscience. Science
**2019**, 363, eaau0323. [Google Scholar] [CrossRef] - Chen, C.; He, W.; Zhou, H.; Xue, Y.; Zhu, M. A comparative study among machine learning and numerical models for simulating groundwater dynamics in the Heihe River Basin, northwestern China. Sci. Rep.
**2020**, 10, 3904. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Ma, Y.; Montzka, C.; Bayat, B.; Kollet, S. Using Long Short-Term Memory networks to connect water table depth anomalies to precipitation anomalies over Europe. Hydrol. Earth Syst. Sci. Discuss.
**2020**, 2020, 1–30. [Google Scholar] [CrossRef] - Ehret, U.; van Pruijssen, R.; Bortoli, M.; Loritz, R.; Azmi, E.; Zehe, E. Adaptive clustering: Reducing the computational costs of distributed (hydrological) modelling by exploiting time-variable similarity among model elements. Hydrol. Earth Syst. Sci.
**2020**, 24, 4389–4411. [Google Scholar] [CrossRef] - Sun, A.Y.; Scanlon, B.R.; Zhang, Z.; Walling, D.; Bhanja, S.N.; Mukherjee, A.; Zhong, Z. Combining Physically Based Modeling and Deep Learning for Fusing GRACE Satellite Data: Can We Learn From Mismatch? Water Resour. Res.
**2019**, 55, 1179–1195. [Google Scholar] [CrossRef] [Green Version] - Lu, D.; Konapala, G.; Painter, S.L.; Kao, S.-C.; Gangrade, S. Streamflow Simulation in Data-Scarce Basins Using Bayesian and Physics-Informed Machine Learning Models. J. Hydrometeorol.
**2021**, 22, 1421–1438. [Google Scholar] [CrossRef] - Tartakovsky, A.M.; Marrero, C.O.; Perdikaris, P.; Tartakovsky, G.D.; Barajas-Solano, D. Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems. Water Resour. Res.
**2020**, 56, e2019WR026731. [Google Scholar] [CrossRef] - Bandai, T.; Ghezzehei, T.A. Physics-Informed Neural Networks With Monotonicity Constraints for Richardson-Richards Equation: Estimation of Constitutive Relationships and Soil Water Flux Density From Volumetric Water Content Measurements. Water Resour. Res.
**2021**, 57, e2020WR027642. [Google Scholar] [CrossRef] - Zahura, F.T.; Goodall, J.L.; Sadler, J.M.; Shen, Y.; Morsy, M.M.; Behl, M. Training Machine Learning Surrogate Models From a High-Fidelity Physics-Based Model: Application for Real-Time Street-Scale Flood Prediction in an Urban Coastal Community. Water Resour. Res.
**2020**, 56, e2019WR027038. [Google Scholar] [CrossRef] - Tran, H.; Leonarduzzi, E.; De la Fuente, L.; Hull, R.B.; Bansal, V.; Chennault, C.; Gentine, P.; Melchior, P.; Condon, L.E.; Maxwell, R.M. Development of a Deep Learning Emulator for a Distributed Groundwater–Surface Water Model: ParFlow-ML. Water
**2021**, 13, 3393. [Google Scholar] [CrossRef] - Di Giammarco, P.; Todini, E.; Lamberti, P. A conservative finite elements approach to overland flow: The control volume finite element formulation. J. Hydrol.
**1996**, 175, 267–291. [Google Scholar] [CrossRef] - Vanderkwaak, J.; Sudicky, E. Application of a Physically-Based Numerical Model of Surface and Subsurface Water Flow and Solute Transport; IAHS-AISH Publisher: Wallingford, UK, 2000; pp. 515–523. [Google Scholar]
- Panday, S.; Huyakorn, P.S. A fully coupled physically-based spatially-distributed model for evaluating surface/subsurface flow. Adv. Water Resour.
**2004**, 27, 361–382. [Google Scholar] [CrossRef] - Kollet, S.J.; Maxwell, R.M. Integrated surface-groundwater flow modeling: A free-surface overland flow boundary condition in a parallel groundwater flow model. Adv. Water Resour.
**2006**, 29, 945–958. [Google Scholar] [CrossRef] [Green Version] - Maxwell, R.M.; Putti, M.; Meyerhoff, S.; Delfs, J.-O.; Ferguson, I.M.; Ivanov, V.; Kim, J.; Kolditz, O.; Kollet, S.J.; Kumar, M.; et al. Surface-subsurface model intercomparison: A first set of benchmark results to diagnose integrated hydrology and feedbacks. Water Resour. Res.
**2014**, 50, 1531–1549. [Google Scholar] [CrossRef] [Green Version] - Ashby, S.F.; Falgout, R.D. A parallel multigrid preconditioned conjugate gradient algorithm for groundwater flow simulations. Nucl. Sci. Eng.
**1996**, 124, 145–159. [Google Scholar] [CrossRef] - Jones, J.E.; Woodward, C.S. Newton-Krylov-multigrid solvers for large-scale, highly heterogeneous, variably saturated flow problems. Adv. Water Resour.
**2001**, 24, 763–774. [Google Scholar] [CrossRef] [Green Version] - Kuffour, B.N.O.; Engdahl, N.B.; Woodward, C.S.; Condon, L.E.; Kollet, S.; Maxwell, R.M. Simulating coupled surface–subsurface flows with ParFlow v3.5.0: Capabilities, applications, and ongoing development of an open-source, massively parallel, integrated hydrologic model. Geosci. Model. Dev.
**2020**, 13, 1373–1397. [Google Scholar] [CrossRef] [Green Version] - LeCun, Y.; Boser, B.; Denker, J.S.; Henderson, D.; Howard, R.E.; Hubbard, W.; Jackel, L.D. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Comput.
**1989**, 1, 541–551. [Google Scholar] [CrossRef] - Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM
**2017**, 60, 84–90. [Google Scholar] [CrossRef] - Keçeli, A.S.; Kaya, A.; Can, A.B. Combining 2D and 3D deep models for action recognition with depth information. Signal. Image Video Process.
**2018**, 12, 1197–1205. [Google Scholar] [CrossRef] - Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation; Springer: Cham, Switzerland, 2015; pp. 234–241. Available online: https://link.springer.com/chapter/10.1007/978-3-319-24574-4_28 (accessed on 21 November 2021).
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style; High-Performance Deep Learning Library, 2019; pp. 8024–8035. Available online: https://arxiv.org/abs/1912.01703 (accessed on 21 November 2021).
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Du, L. How Much Deep Learning does Neural Style Transfer Really Need? An Ablation Study. 2020. Available online: https://openaccess.thecvf.com/content_WACV_2020/papers/Du_How_Much_Deep_Learning_does_Neural_Style_Transfer_Really_Need_WACV_2020_paper.pdf (accessed on 21 November 2021).
- Duan, Q.; Schaake, J.; Andreassian, V.; Franks, S.; Goteti, G.; Gupta, H.V.; Gusev, Y.M.; Habets, F.; Hall, A.; Hay, L.; et al. Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results from the second and third workshops. J. Hydrol.
**2006**, 320, 3–17. [Google Scholar] [CrossRef] [Green Version] - Cranmer, K.; Brehmer, J.; Louppe, G. The frontier of simulation-based inference. Proc. Natl. Acad. Sci. USA
**2020**, 117, 30055–30062. [Google Scholar] [CrossRef] - Beaumont, M.A.; Zhang, W.; Balding, D.J. Approximate Bayesian Computation in Population Genetics. Genetics
**2002**, 162, 2025–2035. [Google Scholar] [CrossRef] - Duan, Q.; Sorooshian, S.; Gupta, V.K. Optimal use of the SCE-UA global optimization method for calibrating watershed models. J. Hydrol.
**1994**, 158, 265–284. [Google Scholar] [CrossRef] - Sorooshian, S.; Duan, Q.Y.; Gupta, V.K. Calibration of Rainfall-Runoff Models—Application of Global Optimization to the Sacramento Soil-Moisture Accounting Model. Water Resour. Res.
**1993**, 29, 1185–1194. [Google Scholar] [CrossRef] - Tsai, W.-P.; Feng, D.; Pan, M.; Beck, H.; Lawson, K.; Yang, Y.; Liu, J.; Shen, C. From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling. Nat. Commun.
**2021**, 12, 5988. [Google Scholar] [CrossRef] [PubMed] - Lanusse, F.; Melchior, P.; Moolekamp, F. Hybrid Physical-Deep Learning Model for Astronomical Inverse Problems. arXiv
**2019**, arXiv:1912.03980. [Google Scholar]

**Figure 2.**Surface pressure averaged across all 243 In Range test realizations for the six ML models compared to the ParFlow simulations, at five snapshots in time. All models were trained on 1024 realizations.

**Figure 3.**RMSE of surface pressure between each of the six ML models and ParFlow simulations calculated across all 243 In Range test realizations, at five snapshots in time. All models were trained on 1024 realizations.

**Figure 4.**Ensemble hydrograph comparisons between each of the six ML models and ParFlow simulations across all 243 In Range test realizations. The ensemble mean is shown for both ML (red) and ParFlow (blue) along with a shaded range that represents the entire spread across all ensemble members and the dashed lines that indicate the maximums within that range. All models were trained on 1024 realizations.

**Figure 6.**Scatterplot of three hydrograph metrics—peak time, peak flow, and total flow—for the ParFlow and CNN3D.1024 ML model. Note points are colored by the channel slope.

**Figure 7.**Scatterplot of three hydrograph metrics—peak time, peak flow, and total flow—for the ParFlow and CNN2D.1024 ML model. Note points are colored by the channel slope.

**Figure 8.**Scatterplot of three hydrograph metrics—peak time, peak flow, and total flow—for the ParFlow and UNet2D_E7.1024 ML model. Note points are colored by the channel slope.

**Figure 12.**Results of the simulation-based inference process for the CNN2D.1024 model and three randomly chosen ParFlow realizations as indicated. Plotted are the five ML model simulations (blue lines) with the smallest RMSE when compared to the hydrograph of the ParFlow simulation (red line) with full ensemble range (blue shading).

**Figure 13.**Results of the simulation-based inference process for the CNN2D_B3.1024 model and three randomly chosen ParFlow realizations as indicated. Plotted are the five ML model simulations (blue lines) with the smallest RMSE when compared to the hydrograph of the ParFlow simulation (red line) with full ensemble range (blue shading).

**Figure 14.**Plot of the ability of the simulation-based inference process for the CNN2D_B3.1024, CNN2D_B3.1024, and UNet2D_E7.1024 model to invert the original ParFlow parameters. Box and whisker plots for the normalized RMSE (calculated as the difference between each parameter estimate and the true parameter divided by the true parameter value) for the three realizations for each case as indicated. The horizontal line is the median, and the x-symbol represents the mean of each model parameter NRMSE distribution.

Base Case | num Realizations | |||||
---|---|---|---|---|---|---|

Model | 1024 | 512 | 256 | 128 | 64 | 32 |

CNN3D | X | X | X | X | X | X |

CNN2D | X | |||||

CNN2D_B1 | X | X | X | X | X | X |

UNet2D_E7 | X | X | X | |||

CNN2D_A1 | X | X | X | X | X | |

CNN2D_B3 | X | X | X | X | X | X |

Channel Slope | Hill Slopes | Manning’s n | Rain Rate | Rain Length | |
---|---|---|---|---|---|

ML1 | 0.002 | 0.077 | 0.031 | 0.011 | 2 |

ML2 | 0.002 | 0.055 | 0.032 | 0.011 | 2 |

ML3 | 0.002 | 0.055 | 0.031 | 0.011 | 2 |

ML4 | 0.002 | 0.055 | 0.032 | 0.011 | 2 |

ML5 | 0.002 | 0.077 | 0.032 | 0.011 | 2 |

ParFlow | 0.002 | 0.055 | 0.032 | 0.015 | 2 |

NRMSE | 0.00 | 0.57 | 0.04 | 0.60 | 0.00 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Maxwell, R.M.; Condon, L.E.; Melchior, P.
A Physics-Informed, Machine Learning Emulator of a 2D Surface Water Model: What Temporal Networks and Simulation-Based Inference Can Help Us Learn about Hydrologic Processes. *Water* **2021**, *13*, 3633.
https://doi.org/10.3390/w13243633

**AMA Style**

Maxwell RM, Condon LE, Melchior P.
A Physics-Informed, Machine Learning Emulator of a 2D Surface Water Model: What Temporal Networks and Simulation-Based Inference Can Help Us Learn about Hydrologic Processes. *Water*. 2021; 13(24):3633.
https://doi.org/10.3390/w13243633

**Chicago/Turabian Style**

Maxwell, Reed M., Laura E. Condon, and Peter Melchior.
2021. "A Physics-Informed, Machine Learning Emulator of a 2D Surface Water Model: What Temporal Networks and Simulation-Based Inference Can Help Us Learn about Hydrologic Processes" *Water* 13, no. 24: 3633.
https://doi.org/10.3390/w13243633