# Can Deep Learning Extract Useful Information about Energy Dissipation and Effective Hydraulic Conductivity from Gridded Conductivity Fields?

^{*}

## Abstract

**:**

_{eff}) of a binary K grid. A deep learning algorithm (UNET) can infer K

_{eff}with extremely high accuracy (R

^{2}> 0.99). The UNET architecture could be trained to infer the energy dissipation weighting pattern from an image of the K distribution, although it was less accurate for cases with highly localized structures that controlled flow. Furthermore, the UNET architecture learned to infer the energy dissipation weighting even if it was not trained directly on this information. However, the weights were represented within the UNET in a way that was not immediately interpretable by a human user. This reiterates the idea that even if ML/DL algorithms are trained to make some hydrologic predictions accurately, they must be designed and trained to provide each user-required output if their results are to be used to improve our understanding of hydrologic systems.

## 1. Introduction

_{eff}) is based on upscaling by a recursive calculation whereby the extent of each grid unit is doubled along each direction at each step [21,22]. This approach essentially allows for the use of arithmetic and harmonic averaging at the local scale, thereby simplifying the computation of effective conductivity. However, while the method is very fast and efficient, severe errors can occur in the final estimates at the scale of the largest blocks due to unrealistic boundary representations during the recursive upscaling process [23]. Further, as with the exponential approach, the renormalization method is only applicable to statistically isotropic, lognormal conductivity fields having no clear structure [19,24].

_{eff}of a medium characterized by a strong contrast between low and high conductivities, with the assumption that the upscaled value of conductivity is primarily a consequence of flows through connected high permeability pathways, when they exist [6,25,26]. Subsequent studies in which percolation theory was used to assess K

_{eff}[27,28,29] have generally found that percolation theory is appropriate when the proportion of the high conductivity medium is close to the percolation transition threshold [20].

_{eff}. In this regard, Knight [36] and Indelman and Dagan [37] suggested that K

_{eff}can be determined from a grid of cells by assuming that dissipated energy must be preserved during the equivalent block conductivity computation.

_{eff}. That is, the local K values in those areas within which most energy is dissipated contribute most to K

_{eff}. This approach is general, not limited to hydrologic problems. For example, Ferre et al. [38] used energy dissipation to define the sample area of time domain reflectometry probes, showing that relatively small areas of the domain contribute disproportionately to TDR-measured water content.

_{eff}. Here, we make use of recently developed approaches that facilitate comparing the activation patterns of different DL models [50] to examine how these DL tools extract and use the knowledge that is relevant to the process of upscaling (i.e., energy dissipation weighting). To our knowledge, this is a first attempt to include knowledge about the system to design a DL architecture for hydrogeologic upscaling.

_{eff}in two modes: with and without energy dissipation weighting information provided. The second objective is to understand whether UNET learns to use energy dissipation weighting as an intermediate step for inferring K

_{eff}if those weights are not provided. That is, we first compared the ability of a UNET to infer K

_{eff}from a binary K grid when trained on both the K grid and the energy dissipation weighting, and when trained only on the K grid. Then, we examined the trained weights within the UNET trained only on the K grid to examine how it processes information. Based on this examination, we drew conclusions on whether the UNET is learning the energy dissipation weighting independently.

## 2. Methodology

_{eff}, using MODFLOW, a well-known finite difference numerical groundwater flow model. MODFLOW was used to produce the steady-state head distribution over a square grid with a 1-D applied gradient. That is, the left and right boundaries each had applied constant head values (Type I, Dirichlet) and the top and bottom boundaries were no flow (Type II, Neumann). We computed the steady-state flow and then calculated K

_{eff}as the homogeneous K necessary to achieve that flow for the same boundary conditions in a 1D flow system. The local head gradient was used to define the energy dissipation weighting in each cell, which can be combined with the local K values to determine K

_{eff}.

#### 2.1. Flow through Heterogeneous Binary Grids (Dataset Generation)

^{3}/T], A is cross section area [L

^{2}], and $\frac{\mathbf{dL}}{\mathbf{dH}}$ is hydraulic gradient [-]:

_{eff}was calculated based on the flow into the left boundary and the flow out of the right boundary. The resulting K

_{eff}values calculated with both of these flow rates always agreed within 1%, and the average value was used for all analyses.

#### 2.2. Energy Dissipation Weighting Method

_{eff}can be thought of as a weighted average of the spatially distributed values of K, in which the weight at each point is equal to the normalized energy dissipation of the field at that point:

_{eff}) as the sum of the local K weighted by the energy dissipation weighting factor over the domain, as:

_{eff}with the energy dissipation approach for these conditions. First, the gradient can be computed at each cell edge and the value of K

_{eff}at the edge can be determined based on the K value in the two neighboring cells. Second, the head values can be interpolated to the edges, allowing for gradients to be computed at the nodes, matching the locations of the K grid. Both of these approaches were tested and were found to agree within 1%; accordingly, the average of these two estimates of K

_{eff}was used for each grid for further analyses. Hereafter, the energy dissipation weights are referred to as ED weights, or simply as weights.

#### 2.3. Estimating K_{eff} with and without ED Weights

_{eff}. In particular, given that knowledge of energy dissipation has been shown [36,37,38] to provide valuable information regarding the weighting required to define K

_{eff}, the problem of estimating K

_{ef}

_{f}from a grid of K values can be seen as a problem that has two stages. The first step is to estimate the energy dissipation weighting at each cell, and the second is to use the estimates of the spatially distributed ED weights to estimate K

_{eff}.

_{eff}or resolve energy dissipation weightings of a grid because of their input and output formats. In contrast, DL methods, containing multiple hidden layers, provide us with both flexible input and output formats and the power to model a complex process. Hidden layers are layers of DL architecture that are located between inputs and outputs and are responsible for nonlinear transformation of the inputs. Each layer consists of several processing units (neurons). Each neuron is connected to adjacent layers with an individual weight assigned to each interlayer link. All inputs into a neuron are multiplied by their associated weight and summed to form a single output. Finally, each of these outputs is subject to a nonlinear transformation referred to as the activation function.

_{eff}from an image of the binary conductivity field. Figure 2 illustrates our proposed model. The model is composed of two sub-models, named “Energy Dissipation” and “K

_{eff}”. In this study, we proposed a modified UNET model. The architecture of the model is inspired by our physical domain knowledge, which suggests that knowledge of the energy dissipation weightings will result in improved estimation of K

_{eff}.

_{eff}. In the first implementation, referred to as ‘informed’, the model is trained using the freeze-training technique [54,55], in which the lower branch of the model (labeled “Energy Dissipation” on Figure 2) is first trained to estimate the spatially distributed ED weights. This is achieved by providing the K grids and the associated ED weights to the intermediate layers during training. Once partially trained, the weights of the lower branch were frozen and training was then continued by feeding only the K grid into the UNET. The trained algorithm was provided with only the K grid and it would first estimate the ED weights and then concatenate those with the K grid as input to the final fully connected layer to estimate K

_{eff}.

_{eff}.

_{eff}. It should be noted that as part of preprocessing, we padded the input image to 32 × 32 to make the final output of the UNET the same as the original image.

#### 2.4. Model Evaluation

_{eff}values, and (for the informed UNETs) ED weights. The inputs and targets were divided into training, validation, and testing subsets. A random selection of 65% of the input cases were used for training and 15% were used as a validation dataset for hyperparameter tuning. The same training/validation/testing sets were used for all of the analyses reported herein. Model performance is reported using the testing data set, comprising the remaining 20% of the data. Before training, the inputs were standardized by subtracting the mean value and dividing by the standard deviation. All hyper-parameters were tuned using a grid search approach. The root mean squared error (RMSE) between the observed and model-calculated values (of K

_{eff}or ED weight) is used to assess the prediction quality of each model. The R

^{2}value was also calculated, but it was only used to further illustrate the quality of the predictions.

#### 2.5. Deep Learning Implementation

^{−4}as the optimizer. Training was stopped when performance on the validation dataset stopped improving within a patience value equal to 50.

#### 2.6. CKA and Similarity Analysis

_{eff}using gridded binary K information, we also wanted to determine whether these tools can infer the underlying pattern of energy dissipation in the process of inferring K

_{eff}. If it can be shown that the deep learning procedure naturally infers the spatial distribution of energy dissipation, then it would provide an example of how DL tools can learn underlying concepts. Further, because the distribution of energy dissipation indicates which parts of the medium have the largest impact on steady-state flow, the ability to make inferences regarding these patterns would also enable an understanding of the relationship between K

_{eff}and the structure of the K distribution. Such knowledge would also be valuable for understanding soil property distributions that may impact dispersion, colloid trapping/mobilization, and erosion/piping.

## 3. Results

_{eff}) of a binary heterogeneous medium. We examined this for multiple realizations of random fields that contain different percentages of the higher K material.

_{eff}. We first confirmed this finding for the set of binary grids examined. Then, we examined whether deep learning algorithms can predict K

_{eff}with and without information regarding the ED weights. By comparing DL algorithms trained with and without access to energy dissipation weightings’ information, we sought to understand the mechanism by which K

_{eff}is inferred by the DL.

#### 3.1. Analysis of the Effective Hydraulic Conductivity (K_{eff}) and High K Percentage

_{eff}was computed from the overall gradient applied over the domain and the steady-state flow through the domain. Figure 3 indicates how K

_{eff}varies as a function of the percent high K material present in the realization. The parallel (layers in the direction of flow) and series’ (layers perpendicular to flow) arrangements for each percent high K realization were calculated analytically and are shown by blue and orange color lines to place limits on the ranges that K

_{eff}values can take. The mean value of K

_{eff}for each high K percentage is shown as a solid red line.

_{eff}on percent high conductivity. At low percentages of high conductivity, K

_{eff}is only minimally affected by the addition of higher K material and remains approximately equal to the conductivity of the lower K material. A nonlinear transition zone is seen to occur at approximately 40 to 70% high K, and the relationship becomes approximately linear above 70%. For a given percentage of high K, the maximum variance of K

_{eff}occurs in the transition zone. These results illustrate the two related but different challenges for inferring K

_{eff}from a binary grid: predicting mean K

_{eff}as a function of the percent high K material, and predicting K

_{eff}for a specific grid given knowledge regarding the percentage of high K material present—especially in the transition zone of percent high K material.

#### 3.2. Analysis of the Energy-Dissipation Weighting Method to Explain the K_{eff}

_{eff}. This fact is confirmed by our study (Figure 4). The energy dissipation approach can be thought of as computing a weighted average of the local K values on the grid that perfectly recovers the flow-based K

_{eff}.

_{eff}values (0.53 and 0.24 cm/s, respectively). The corresponding maps of the ED weights are shown in Figure 5C,D, illustrating that the grid with the lower K

_{eff}has a much more localized pattern of ED weighting. While it might be tempting to attribute this localized weighting to the connected pattern of low K cells running vertically through Figure 5B, beyond this qualitative assessment, it is essentially impossible to visually infer the values of the ED weights from the knowledge of the spatial organization of K. Of course, both the pattern of ED weights and their values can be computed readily by solving the steady-state flow problem, but then the value of K

_{eff}can be determined directly, and knowledge of the ED weights is superfluous. Accordingly, the ED weighting approach is best seen as a method for understanding spatial organization [38] rather than a practical approach for inferring K

_{eff}from a K grid.

_{eff}and the fraction of high energy dissipation cells (defined with a threshold of 95%), but with some interesting complications to that relationship in the range of 50 to 60% high conductivity material. These results suggest that information regarding the fraction of high energy cells may be informative for inferring K

_{eff}for most percent high conductivity material fractions, but this relationship varies in the nature and quality of the correlation as a function of the fraction of high dissipation cells and percent of high conductivity material.

#### 3.3. Inferring K_{eff} with UNET with and without ED Weights

^{2}= 0.9984 when evaluated on the testing data (Figure 7A). Interestingly, the informed UNET (Figure 7B) offered only marginal improvement over the uninformed UNET with an RMSE = 0.0106 and an R

^{2}= 0.9986. It is important to note that, although the informed UNET was provided information regarding the ED weights during training, its predictions of K

_{eff}are made based solely on the K grid. In other words, the uninformed UNET independently determined weights that are as good as the ED-informed. This leads to the question: Is the uninformed UNET discovering the ED weights, or has it found some other weighting scheme that is as effective as ED weighting? Regardless of the outcome, it is a contribution that the uninformed UNET has achieved this performance without requiring that a flow model be run. However, it would be even more interesting if it could be shown that the UNET was discovering ED weighting without being given the physical insight that underpins this approach.

#### 3.4. Inferring ED Weights with an Informed UNET

^{2}(0.9549). This can also be aggregated to infer the fraction of high energy cells, showing similarly good results (RMSE = 0.04876 and R

^{2}= 0.9832). However, considering all 625 cells in all 297,000 simulations, there are cases that show significant mismatch (Figure 9A). In particular, UNET consistently under-predicts the ED weights for cells that have very high actual weight (top right quadrant of Figure 9A). This leads to an over-prediction of the fraction of high energy cells for cases with intermediate percent high K (Figure 9B). From Figure 6B, these are the conditions that give rise to the most concentrated ED weighting. Taken together, these results suggest that the UNET has difficulty in inferring the ED weights when they are concentrated in highly localized areas (e.g., 60–75% high K material in Figure 6B).

## 4. Discussion

_{eff}from a binary K grid, with or without ED weighting information being provided. Second, if an uninformed UNET can predict K

_{eff}, then does the weighting on the intermediate layers represent the DL independent learning of the ED weighting?

#### 4.1. Dependence of the ED Weighting Distribution on the K Field

_{eff}associated with binary grids showed a highly nonlinear dependence on the percentage of high K material (Figure 3). Specifically, K

_{eff}is closer to the arithmetic mean for materials with low to medium percentage of high K, while being approximately halfway between the arithmetic and harmonic means for materials with a higher percentage of high K. The variation in this trend is due to the influence of specific structural patterns in the spatial distribution of high and low K cells among grid realizations. The maximum degree of variability occurs for materials with intermediate percentages of high K values. In general, both the trend and the specific variations in K

_{eff}are very well explained by ED-weighted averaging (Figure 4).

#### 4.2. Comparison of Performance

^{2}, both informed and uninformed UNET performed extremely well.

_{eff}values were low (Figure 7). The performance was also relatively poor for intermediate percentage levels of high K material (Figure 10). That is, the UNET algorithms had the most difficulty when localized structures acted to impede flow, leading to a low K

_{eff}.

#### 4.3. Hidden Layer Representation Analysis

_{eff}without having to solve the flow equation. In contrast, direct use of ED weighting requires the flow problem to be solved for every K

_{eff}inference.

_{eff}during training and validation, but the ED weights are not determined. Given that the ED weights are thought to represent a key mechanism linking the K grid to the value of K

_{eff}, this raises the question of whether the uninformed UNET is somehow inferring information regarding the distribution of ED without being explicitly provided with such information during training.

_{eff}, represents the ED weight distribution. That is, as in the direct use of ED weights, this matrix is combined with the K grid matrix to infer K

_{eff}. Examining the corresponding layer of the uninformed UNET shows no correlation with the true ED weights. However, a more advanced analysis, based on computing the centered kernel alignment similarity (CKA) [50], provides a more complete picture of the information flows through the informed and uninformed UNETs. These results are visualized as a similarity matrix (Figure 11). The output of each layer of the informed model is compared to other layers of the uninformed model to examine the degree of similarity between them while accounting for the presence of invertible linear transformations. A similarity value of zero between two layers indicates that their representations are not invertible linear transformations of each other, while a similarity value of 1 indicates that the two layers are equivalent up to a linear transformation.

_{eff}.

_{eff}achieved by the informed and uninformed UNETs.

_{eff}. However, when not required to produce an ED map (training under uninformed conditions), the UNET does not develop a layer to translate the information to a user-readable ED map. Rather, the latent information about the ED weights propagates through the UNET, with an associated change in the final dense layer to produce high-quality inferences of K

_{eff}.

_{eff}and these predicted with the ‘swapped’ network until the substitutions reached the conv2d_12 layer. This is consistent with the high CKA representation’s similarity to this layer (Figure 11). There is a strongly nonlinear relationship for conv2d_13, which corresponds with a low CKA value at this layer. In the final layer (i.e., the output layer), we see a strong negative linear correlation between the output of the mixed structure model and that of the informed model. This pattern is consistent with the high CKA value observed in Figure 11, and suggests that an orthogonal transformation between the weights was necessary to overcome the changes applied in the deeper layers to recover the correct K

_{eff}values. This analysis suggests that both the informed and uninformed UNET are implementing similar computational processes, ostensibly extracting information corresponding to the ED distribution from the K grid, but representing it differently in n-d dimensional space. Further, the user-imposed requirement to produce a readable ED map results in a nonlinear transformation that must be compensated in later layers to produce accurate inferred K

_{eff}values.

## 5. Conclusions

_{eff}with extremely high accuracy (R

^{2}> 0.99) when provided with only the binary grid.

_{eff}, we examined whether providing such information improved the DL performance. While adding information derived from the ED distribution improved the performance of each algorithm, the improvement was marginal.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A

MODIFIED UNET MODEL |
---|

3 × 3 CONV. 16-SAME PADDING-STRIDE 1-RELU ×22 × 2 MAXPOOLING STRIDE 2 |

3 × 3 CONV. 32-SAME PADDING-STRIDE 1-RELU ×22 × 2 MAXPOOLING STRIDE 2 |

3 × 3 CONV. 64-SAME PADDING-STRIDE 1-RELU ×22 × 2 MAXPOOLING STRIDE 2 DROPOUT 0.64 |

3 × 3 CONV. 128-SAME PADDING-STRIDE1-RELU ×2 |

2 × 2 CONV2DTRANSPOSE. 64-SAME PADDING-STRIDE 2-NO ACTIVATION ×1CROPPINGCONCATENATION3 × 3 CONV. 64-SAME PADDING-STRIDE 1-RELU ×2 |

2 × 2 CONV2DTRANSPOSE. 32-SAME PADDING-STRIDE 2-NO ACTIVATION ×1CROPPINGCONCATENATION3 × 3 CONV. 32-SAME PADDING-STRIDE 1-RELU ×2 |

2 × 2 CONV2DTRANSPOSE. 16-SAME PADDING-STRIDE 2-NO ACTIVATION ×1CROPPINGCONCATENATION3 × 3 CONV. 16-SAME PADDING-STRIDE 1-RELU ×2 |

1 × 1 CONV. 1-SAME PADDING-STRIDE 1-NO ACTIVATION ×1 |

CONCATENATION3 × 3 CONV. 10-SAME PADDING-STRIDE 1-TANH ×1FLATTEN1 DENSE-LINEAR |

## References

- Ahuja, L.R.; Ma, L.; Green, T.R. Effective Soil Properties of Heterogeneous Areas for Modeling Infiltration and Redistribution. Soil Sci. Soc. Am. J.
**2010**, 74, 1469–1482. [Google Scholar] [CrossRef] - Chan, S.; Elsheikh, A.H. Parametrization and generation of geological models with generative adversarial networks. arXiv
**2017**, arXiv:1708.01810. [Google Scholar] - Aliyari, F.; Bailey, R.T.; Tasdighi, A.; Dozier, A.; Arabi, M.; Zeiler, K. Coupled SWAT-MODFLOW Model for Large-Scale Mixed Agro-Urban River Basins. Environ. Model. Softw.
**2019**, 115, 200–210. [Google Scholar] [CrossRef] - Shamsudduha, M.; Zahid, A.; Burgess, W.G. Security of Deep Groundwater against Arsenic Contamination in the Bengal Aquifer System: A Numerical Modeling Study in Southeast Bangladesh. Sustain. Water Resour. Manag.
**2019**, 5, 1073–1087. [Google Scholar] [CrossRef] [Green Version] - Adhikari, A.; Ehsani, M.R.; Song, Y.; Behrangi, A. Comparative Assessment of Snowfall Retrieval from Microwave Humidity Sounders Using Machine Learning Methods. Earth Sp. Sci.
**2020**, 7. [Google Scholar] [CrossRef] - Slater, L. Near Surface Electrical Characterization of Hydraulic Conductivity: From Petrophysical Properties to Aquifer Geometries—A. Review. Surv. Geophys.
**2007**, 28, 169–197. [Google Scholar] [CrossRef] - Hertrich, M. Imaging of Groundwater with Nuclear Magnetic Resonance. Prog. Nucl. Magn. Reson. Spectrosc.
**2008**, 53, 227–248. [Google Scholar] [CrossRef] - Dlubac, K.; Knight, R.; Song, Y.-Q.; Bachman, N.; Grau, B.; Cannia, J.; Williams, J. Use of NMR Logging to Obtain Estimates of Hydraulic Conductivity in the High Plains Aquifer, Nebraska, USA. Water Resour. Res.
**2013**, 49, 1871–1886. [Google Scholar] [CrossRef] [Green Version] - Ehsani, M.R.; Behrangi, A.; Adhikari, A.; Song, Y.; Huffman, G.J.; Adler, R.F.; Bolvin, D.T.; Nelkin, E.J. Assessment of the Advanced Very High Resolution Radiometer (AVHRR) for Snowfall Retrieval in High Latitudes Using ClouSat and Machine Learning. J. Hydrometeorol.
**2021**, 22, 1591–1608. Available online: https://journals.ametsoc.org/view/journals/hydr/22/6/JHM-D-20-0240.1.xml (accessed on 11 June 2021). - Wen, X.-H.; Gómez-Hernández, J.J. Upscaling Hydraulic Conductivities in Heterogeneous Media: An Overview. J. Hydrol.
**1996**, 183, ix–xxxii. [Google Scholar] [CrossRef] - Journel, A.G.; Deutsch, C.; Desbarats, A.J. Power Averaging for Block Effective Permeability. In Proceedings of the SPE California Regional Meeting, Oakland, CA, USA, 2–4 April 1986. Paper Number: SPE-15128-MS. [Google Scholar]
- Matheron, G. Matheron’s Theory of Regionalised Variables; Pawlowsky-Glahn, V., Serra, J., Eds.; Oxford Scholarship Online: Oxford, UK, 2019; pp. 48–70. [Google Scholar]
- Desbarats, A.J.; Srivastava, R.M. Geostatistical Characterization of Groundwater Flow Parameters in a Simulated Aquifer. Water Resour. Res.
**1992**, 27, 687–698. [Google Scholar] [CrossRef] - Zhu, J.; Mohanty, B.P. Upscaling of Soil Hydraulic Properties for Steady State Evaporation and Infiltration. Water Resour. Res.
**2002**, 38, 17-1–17-13. [Google Scholar] [CrossRef] - Masihi, M.; Gago, P.A.; King, P.R. Estimation of the Effective Permeability of Heterogeneous Porous Media by Using Percolation Concepts. Transp. Porous Media
**2016**, 114, 169–199. [Google Scholar] [CrossRef] [Green Version] - Arabzadeh, A.; Ehsani, M.R.; Guan, B.; Heflin, S.; Behrangi, A. Global Intercomparison of Atmospheric Rivers Precipitation in Remote Sensing and Reanalysis Products. J. Geophys. Res. Atmos.
**2020**, 125. [Google Scholar] [CrossRef] - Cardwell, W.; Parsons, R. Average Permeability of Heterogeneous Oil Sands. Trans. AIME
**1945**, 160, 34–42. [Google Scholar] [CrossRef] - Durlofsky, L.J. Representation of Grid Block Permeability in Coarse Scale Models of Randomly Heterogeneous Porous Media. Water Resour. Res.
**1992**, 28, 1791–1800. [Google Scholar] [CrossRef] - Wen, X.-H.; Gómez-Hernández, J.J. Selective upscaling of hydraulic conductivities. In Geostatistics Wollongong ’96; Baafi, E., Schofield, N., Eds.; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1996; Volume 2, pp. 1112–1123. [Google Scholar]
- Colecchio, I.; Boschan, A.; Otero, A.D.; Noetinger, B. On the Multiscale Characterization of Effective Hydraulic Conductivity in Random Heterogeneous Media: A Historical Survey and Some New Perspectives. Adv. Water Resour.
**2020**, 140, 4. [Google Scholar] [CrossRef] - King, P.R.; Neuwelier, I. Probability Upscaling. Comput. Geosci.
**2002**, 6, 101–114. [Google Scholar] [CrossRef] - King, P.R. The Use of Renormalization for Calculating Effective Permeability. Transp. Porous Media
**1989**, 4, 37–58. [Google Scholar] [CrossRef] - Malick, K. Boundary effects in the successive upscaling of absolute permeability. Master’s Thesis, Stanford University, Stanford, CA, USA, June 1995. [Google Scholar]
- Sánchez-Vila, X.; Girardi, J.P.; Carrera, J. A Synthesis of Approaches to Upscaling of Hydraulic Conductivities. Water Resour. Res.
**1995**, 31, 867–882. [Google Scholar] [CrossRef] - Ambegaokar, V.; Halperin, B.I.; Langer, J.S. Hopping Conductivity in Disordered Systems. Phys. Rev. B.
**1971**, 4, 2612–2620. [Google Scholar] [CrossRef] - Ehsani, M.R.; Arevalo, J.; Risanto, C.B.; Javadian, M.; Devine, C.J.; Arabzadeh, A.; Venegas-Quiñones, H.L.; Dell’Oro, A.P.; Behrangi, A. 2019–2020 Australia Fire and Its Relationship to Hydroclimatological and Vegetation Variabilities. Water
**2020**, 12, 3067. [Google Scholar] [CrossRef] - Berkowitz, B.; Balberg, I. Percolation Theory and Its Application to Groundwater Hydrology. Water Resour. Res.
**1993**, 29, 775–794. [Google Scholar] [CrossRef] - Hunt, A.G.; Sahimi, M. Flow, Transport, and Reaction in Porous Media: Percolation Scaling, Critical-Path Analysis, and Effective Medium Approximation. Rev. Geophys.
**2017**, 55, 993–1078. [Google Scholar] [CrossRef] - Hunt, A.; Ewing, R.; Ghanbarian, B. Percolation Theory for Flow in Porous Media; Springer International Publishing: Cham, Switzerland, 2014; Volume 880. [Google Scholar]
- Hassanzadegan, A.; Cacace, M.; Sippel, J.; Scheck-Wenderoth, M. The Application of Inverse Modeling in Characterizing Hydraulic Conductivity beneath the City of Berlin, Germany. Environ. Earth Sci.
**2016**, 75, 1–17. [Google Scholar] [CrossRef] - Kotlar, A.M.; Varvaris, I.; van Lier, Q.J.; de Jonge, L.W.; Møldrup, P.; Iversen, B.V. Soil Hydraulic Properties Determined by Inverse Modeling of Drip Infiltrometer Experiments Extended with Pedotransfer Functions. Vadose Zone J.
**2019**, 18, 1–11. [Google Scholar] [CrossRef] [Green Version] - Cheng, Q.; Qiang, X.; Xianglin, C.; Song, Y.; Zhongyi, W.; Yurui, S.; Xiaofei, Y.; Jones, S.B. In-Situ Estimation of Unsaturated Hydraulic Conductivity in Freezing Soil Using Improved Field Data and Inverse Numerical Modeling. Agric. For. Meteorol.
**2019**, 279, 6. [Google Scholar] [CrossRef] - De Oliveira, L.F.C.; de Souza, G.R.; Corrêa, F.V.; E Silva, J.R.M. Parameter Estimation of Soil Hydraulic Characteristics by Inverse Modeling of the Analytical Equation for Unsaturated Subsurface Water Flow. J. Hydroinform.
**2020**, 22, 1270–1282. [Google Scholar] [CrossRef] - Vrugt, J.A.; Stauffer, P.H.; Wöhling, T.; Robinson, B.A.; Vesselinov, V.V. Inverse Modeling of Subsurface Flow and Transport Properties: A Review with New Developments. Vadose Zone J.
**2008**, 7, 843–864. [Google Scholar] [CrossRef] - Lai, J.; Ren, L. Estimation of Effective Hydraulic Parameters in Heterogeneous Soils at Field Scale. Geoderma
**2016**, 264, 28–41. [Google Scholar] [CrossRef] - Knight, J.H. Sensitivity of Time Domain Reflectometry Measurements to Lateral Variations in Soil Water Content. Water Resour. Res.
**1992**, 28, 2345–2352. [Google Scholar] [CrossRef] - Indelman, P.; Dagan, G. Upscaling of Permeability of Anisotropic Heterogeneous Formations: 2. General Structure and Small Perturbation Analysis. Water Resour. Res.
**1993**, 29, 925–933. [Google Scholar] [CrossRef] - Ferré, P.A.; Knight, J.H.; Rudolph, D.L.; Kachanoski, R.G. The Sample Areas of Conventional and Alternative Time Domain Reflectometry Probes. Water Resour. Res.
**1998**, 34, 2971–2979. [Google Scholar] [CrossRef] - Afzaal, H.; Farooque, A.A.; Abbas, F.; Acharya, B.; Esau, T. Groundwater Estimation from Major Physical Hydrology Components Using Artificial Neural Networks and Deep Learning. Water
**2020**, 12, 5. [Google Scholar] [CrossRef] [Green Version] - Mo, S.; Zabaras, N.; Shi, X.; Wu, J. Integration of Adversarial Autoencoders with Residual Dense Convolutional Networks for Estimation of Non-Gaussian Hydraulic Conductivities. Water Resour. Res.
**2020**, 56, 1–24. [Google Scholar] [CrossRef] [Green Version] - Wang, N.; Zhang, D.; Chang, H.; Li, H. Deep Learning of Subsurface Flow Via Theory-Guided Neural Network. J. Hydrol.
**2020**, 584, 124700. [Google Scholar] [CrossRef] [Green Version] - Tartakovsky, A.M.; Marrero, C.O.; Perdikaris, P.; Tartakovsky, G.D.; Barajas-Solano, D. Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems. Water Resour. Res.
**2020**, 56, 1–16. [Google Scholar] [CrossRef] - Canchumuni, S.W.A.; Emerick, A.A.; Pacheco, M.A.C. Towards a Robust Parameterization for Conditioning Facies Models Using Deep Variational Autoencoders and Ensemble Smoother. Comput. Geosci.
**2018**, 128, 87–102. [Google Scholar] [CrossRef] [Green Version] - Zhou, Z.; Shi, L.; Zha, Y. Seeing Macro-Dispersivity from Hydraulic Conductivity Field with Convolutional Neural Network. Adv. Water Resour.
**2020**, 138, 103545. [Google Scholar] [CrossRef] - Wu, J.; Yin, X.; Xiao, H. Seeing Permeability from Images: Fast Prediction with Convolutional Neural Networks. Sci. Bull.
**2018**, 63, 1215–1222. [Google Scholar] [CrossRef] [Green Version] - Apley, D.W.; Zhu, J. Visualizing the effects of predictor variables in black box supervised learning models. J. R. Stat. Soc. Ser. B
**2020**, 82, 1059–1086. [Google Scholar] [CrossRef] - Chakraborty, S.; Tomsett, R.; Raghavendra, R.; Harborne, D.; Alzantot, M.; Cerutti, F.; Srivastava, M.; Preece, A.; Julier, S.; Rao, R.M.; et al. Interpretability of deep learning models: A survey of results. In Proceedings of the 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation, San Francisco, CA, USA, 4–8 August 2017. [Google Scholar] [CrossRef]
- Mosser, L.; Dubrule, O.; Martin, J.B. Reconstruction of three-dimensional porous media using generative adversarial neural networks. Phys. Rev. E
**2017**, 96, 43309. [Google Scholar] [CrossRef] [Green Version] - Srisutthiyakorn, N. Deep Learning Methods for Predicting Permeability from 2-D/3-D Binary Segmented Images. SEG Tech. Progr. Expand. Abstr.
**2016**, 35, 3042–3046. [Google Scholar] [CrossRef] - Kornblith, S.; Norouzi, M.; Lee, H.; Hinton, G. Similarity of Neural Network Representations Revisited. arXiv
**2019**, arXiv:1905.004142019, 6156–6175. [Google Scholar] - Knudby, C.; Carrera, J.; Bumgardner, J.D.; Fogg, G.E. Binary Upscaling—The Role of Connectivity and a New Formula. Adv. Water Resour.
**2006**, 29, 590–604. [Google Scholar] [CrossRef] - Pozdniakov, S.; Tsang, C.-F. A Self-Consistent Approach for Calculating the Effective Hydraulic Conductivity of a Binary, Heterogeneous Medium. Water Resour. Res.
**2004**, 40. [Google Scholar] [CrossRef] - Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional Networks for Biomedical Image Segmentation. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Cham, Switzerland, 2015; Volume 9351, pp. 234–241. [Google Scholar] [CrossRef] [Green Version]
- Zoph, B.; Yuret, D.; May, J.; Knight, K. Transfer Learning for Low-Resource Neural Machine Translation. arXIv
**2016**, arXiv:1604.02201, 1568–1575. [Google Scholar] - Brock, A.; Lim, T.; Ritchie, J.M.; Weston, N. FreezeOut: Accelerate Training by Progressively Freezing Layers. arXiv
**2017**, arXiv:1706.04983. [Google Scholar] - Gretton, A.; Bousquet, O.; Smola, A.; Scḧlkopf, B. Measuring Statistical Dependence with Hilbert-Schmidt Norms. In Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2005; Volume 3734, pp. 63–77. [Google Scholar] [CrossRef] [Green Version]
- Cristianini, N.; Kandola, J.; Elisseeff, A.; Shawe-Taylor, J. On Kernel Target Alignment BT—Innovations in Machine Learning: Theory and Applications. In On Kernel Target Alignment; Holmes, E., Jain, L.C., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 205–256. [Google Scholar]
- Cortes, C.; Mohri, M.; Rostamizadeh, A. Algorithms for Learning Kernels Based on Centered Alignment. J. Mach. Learn. Res.
**2012**, 13, 795–828. [Google Scholar] [CrossRef] - Thompson, J.; Bengio, Y.; Schoenwiesner, M. The Effect of Task and Training on Intermediate Representations in Convolutional Neural Networks Revealed with Modified RV Similarity Analysis. arXiv
**2019**, arXiv:1912.02260. [Google Scholar] [CrossRef]

**Figure 1.**Sample 25 × 25 cell grid with 50% high K (white) and 50% low K (gray) cells, constant head boundaries (blue), and no flow boundaries (diagonal hash marks). The left boundary has a constant head of 2 m and the right boundary has a constant head of 1 m, with flow occurring from left to right.

**Figure 2.**Proposed U-net architecture. The architecture is composed of two sub-models. The energy dissipation model has a UNET-shaped structure followed by a CNN model to map output of the UNET to K

_{eff}. Blue boxes correspond to a multi-channel feature map. The number of channels is denoted on top of the box. The x-y size is provided at the lower left edge of the box. White boxes represent skipped connection. The arrows are operations performed on feature maps described in the legend.

**Figure 5.**Effects of structure on K

_{eff}for the structures with the same percent high conductivity: (

**A**,

**B**) Grid samples with percent high conductivity values of 80; (

**C**,

**D**) Corresponding energy dissipation weightings.

**Figure 6.**The energy dissipation pattern for different percent of high K materials: (

**A**) Grid samples and their corresponding energy dissipation weightings as a function of percent of high K material; (

**B**) Average fraction of high energy dissipation cells as a function of the percent high K for different thresholds; (

**C**) Correlation between observed K

_{eff}and fraction of high energy dissipation cells (HDC) as a function of percent of high K material. A threshold of 95% was used to define high energy cells.

**Figure 7.**The testing performance of K

_{eff}estimation using different methods: (

**A**) K

_{eff}estimation using the energy dissipation uninformed UNET model; (

**B**) K

_{eff}estimation using informed UNET model with pre-training on energy dissipation.

**Figure 8.**Samples of energy dissipation weight distributions’ prediction for different ranges of percent of high K material: (

**A**) Observation; (

**B**) Predicted values.

**Figure 9.**Performance of informed UNET model in energy dissipation estimation: (

**A**) Energy dissipation weighting prediction for all grids; (

**B**) Fraction of high energy dissipation cells’ prediction performance as function of percent of high K material. With reference to Figure 6B, a threshold of 95% was used to define high energy cells.

**Figure 10.**Difference between inferred and actual fraction of high K cells for each grid. To compare the errors of grids at each high k percentage, the values of the left y-axis is scaled by average of actual number of high k cells at each k percentage. The fraction of high K cells for a 95% threshold is presented by a blue line.

**Figure 11.**CKA similarity matrix between: (

**A**) Informed UNET and untrained UNET (Random net); (

**B**) Informed UNET and uninformed UNET.

**Figure 12.**Correlation between true K

_{eff}and the output of UNET model built up by sequential substitution of informed model weights with uninformed UNET collectively. Each subplot labeled with the corresponding layer name in DL model.

Uninformed | Informed | |
---|---|---|

K_{eff} RMSE (Train) | 0.00626774 | 0.00964671 |

K_{eff} RMSE (Val) | 0.01129849 | 0.01077667 |

K_{eff} RMSE (Test) | 0.01129849 | 0.01064088 |

K_{eff} R (Train) | 0.99975328 | 0.99941291 |

K_{eff} R (Val) | 0.99920198 | 0.99926396 |

K_{eff} R (Test) | 0.99918991 | 0.99928351 |

Energy Dissipation Weighting RMSE (Train) | 0.02693278 | 0.00248980 |

Energy Dissipation Weighting RMSE (Val) | 0.02703661 | 0.00548620 |

Energy Dissipation Weighting RMSE (Test) | 0.03300000 | 0.00695936 |

Energy Dissipation Weighting R (Train) | −0.04757823 | 0.99531359 |

Energy Dissipation Weighting R (Val) | −0.04673303 | 0.97724645 |

Energy Dissipation Weighting R (Test) | −0.05657500 | 0.97722907 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Moghaddam, M.A.; Ferre, P.A.T.; Ehsani, M.R.; Klakovich, J.; Gupta, H.V.
Can Deep Learning Extract Useful Information about Energy Dissipation and Effective Hydraulic Conductivity from Gridded Conductivity Fields? *Water* **2021**, *13*, 1668.
https://doi.org/10.3390/w13121668

**AMA Style**

Moghaddam MA, Ferre PAT, Ehsani MR, Klakovich J, Gupta HV.
Can Deep Learning Extract Useful Information about Energy Dissipation and Effective Hydraulic Conductivity from Gridded Conductivity Fields? *Water*. 2021; 13(12):1668.
https://doi.org/10.3390/w13121668

**Chicago/Turabian Style**

Moghaddam, Mohammad A., Paul A. T. Ferre, Mohammad Reza Ehsani, Jeffrey Klakovich, and Hoshin Vijay Gupta.
2021. "Can Deep Learning Extract Useful Information about Energy Dissipation and Effective Hydraulic Conductivity from Gridded Conductivity Fields?" *Water* 13, no. 12: 1668.
https://doi.org/10.3390/w13121668