Modeling One-Dimensional Nonlinear Consolidation Problems by Physics-Informed Neural Network with Layer-Wise Locally Adaptive Activation Functions

Abstract

The study on soil consolidation and settlement is of great importance in geotechnical engineering practice. Nowadays, physics-informed neural networks (PINN) are becoming more and more popular in solving geotechnical engineering problems thanks to their meshless, physically constrained, and data-driven nature. Although there have been some successful applications in one-dimensional (1D) consolidation problems in saturated soils, the ability and stability to deal with more complex boundary conditions remain to be tested. In this paper, the effects of activation function and random state on the PINN are investigated for solving two 1D consolidation problems in saturated soils, and the proposed method for inverse modeling of the two 1D consolidation problems. The results show that PINN with layer-wise locally adaptive activation functions improves the convergence speed and prediction accuracy of the PINN for solving the 1D nonlinear soil consolidation problems, and at the same time the robustness of the model to random states. Moreover, the proposed method still converges faster in the inverse modeling of 1D consolidation problems.

1. Introduction

Saturated soft soil will produce excess pore water pressure under loading, and the dissipation of excess pore water pressure causes internal pore shrinkage to produce soil settlement, i.e., consolidation of the soil body. In 1925, Terzaghi [1] established a 1D consolidation theory based on a series of idealized assumptions, such as coefficients of permeability and compression remaining unchanged in the consolidation process, etc. Subsequently, researchers continuously revised its basic assumptions to develop and improve the 1D consolidation theory so that it is closer to the natural properties.

Gibson et al. [2] derived the governing equation of 1D large deformation consolidation of soils and obtained the theory of 1D large deformation consolidation, but due to the highly nonlinear nature of the governing equation, the analytical solution could not be obtained directly. Xie et al. [3] obtained the analytical solution of 1D large deformation consolidation of saturated soil by assuming that the compression coefficient is a constant and the permeability coefficient and the square of the void ratio are proportional to each other. Davis and Ramond [4] obtained a 1D nonlinear consolidation theory for soils based on a linear $e-\mathrm{l}\mathrm{g}{\mathsf{\sigma }}^{\prime }$ relationship by assuming that the coefficient of permeability $k_{\mathrm{v}}$ varies synchronously with the coefficient of volume compressibility $m_{\mathrm{v}}$ during consolidation and that the self-gravitational stress remains constant along depth. Shi et al. [5] established a 1D nonlinear consolidation equation of soil based on a hyperbolic compression model and obtained the analytical solutions. Xie et al. [6] derived an analytical solutions of 1D nonlinear consolidation of single-layer and two-layer soils under the conditions of unipolar loading on the basis of assumptions made by Davis and Ramond. Poskitt [7] utilized the ingress method to solve the problem of nonlinear consolidation of soil when the consolidation coefficient $C_{\mathrm{v}}$ is a variable. Xie et al. [8] obtained a semi-analytical solution to 1D nonlinear consolidation for layered soils under time-varying loading with $C_{\mathrm{v}}$ by using the method of discretization of soil thickness.

It is worth noting that it is difficult to obtain an analytical solution in nonlinear consolidation theory unless the consolidation coefficient $C_{\mathrm{v}}$ is constant. Therefore, in most cases, nonlinear consolidation was usually solved by numerical methods such as the finite element method (FEM), finite difference method (FDM), finite volume method (FVM), etc., in order to obtain a nonlinear consolidated high-accuracy numerical solution. With the increase in available computational resources, traditional numerical methods can effectively solve such highly nonlinear problems. However, for these traditional numerical methods, careful spatio-temporal discretization is essential.

In recent years, PINNs have become increasingly popular in solving partial differential equations (PDEs), which, instead of solving PDEs directly, identify the nonlinear mapping between inputs and outputs through agent modeling [9]. Although ordinary deep neural networks (DNNs) can approximate any continuous function under certain conditions, they usually require a large amount of labeled data to maintain their accuracy [10]. To overcome this drawback, Raissi et al. [11] introduced the concept of PINN and used PINN to solve benchmark PDEs such as the Navier-Stokes equations. PINN is the process of embedding PDEs and known physical knowledge such as initial and boundary conditions in a DNN and evaluating the model on a set of dispersed spatio-temporal points called residual points. The development of PINN has benefited from the progress of deep learning, such as libraries like Pytorch [12] and tensorflow [13], and many code libraries based on PINN have been developed, such as DeepXDE [14] and SciANN [15].

PINN has been proven to be a powerful tool for solving PDEs or determining model parameters in many fields. In fluid mechanics, PINN is used to solve forward and inverse problems such as incompressible flow [16] and turbulent flow [17]. In the field of solid mechanics, PINN is used to solve linear elastic problems [18], nonlinear elastoplastic problems [19], and elastic dynamic problems [20]. In the fields of materials science, PINN is applied to the reconstruction of particles [21] and the simulation of tiny bubbles [22]. In the field of electromagnetics, PINN is applied in the design of various electromagnetic metamaterials [23,24]. In geotechnical engineering, PINN has also been used to solve the Terzaghi consolidation [25,26] and seepage problems in unsaturated soils [27,28]. Lan et al. [29] applied hard constraints on the fixed solution conditions to PINN and solved two 1D nonlinear consolidation problems as well as the estimation of interface parameters.

Compared with traditional numerical methods, the advantages of PINN include being meshless, capable of handling complex boundaries, easy to integrate physical knowledge, easy to integrate observational data, etc. [14,30]. The disadvantages are slow training speed, sensitivity to hyperparameters, unstable convergence, high dependence on random seeds, etc. [28,31]. The activation functions in the aforementioned PINNs for solving the soil consolidation problems are hyperbolic tangent functions with fixed activation slopes. In this paper, we apply a locally adaptive activation function with a scalable parameter, add this parameter to the network for learning, and apply it to 1D nonlinear consolidation models. The paper is organized as follows. Firstly, the principle of PINN is introduced for solving 1D Terzaghi consolidation with continuous drainage boundaries, and the layer-wise locally adaptive activation functions (L-LAAF) are described. Secondly, the performance difference is demonstrated between the PINN with layer-wise locally adaptive activation functions (L-LAAF-PINN) and the baseline PINN for solving two 1D nonlinear consolidation problems. Finally, the advantages of the L-LAAF-PINN are summarized for solving 1D nonlinear consolidation problems.

2. Materials and Methods

2.1. 1D Consolidation by PINN with Layer-Wise Locally Adaptive Activation

2.1.1. 1D Terzagghi Consolidation with Continuous Drainage Boundaries

The traditional 1D Terzaghi consolidation equation is

$${\displaystyle \frac{\partial u}{\partial t}}=C_{\mathrm{v}}{\displaystyle \frac{{\partial }^{2}u}{\partial z^2}}, z\in \left[0,H\right],t>0$$

(1)

where $t$ is the time, $z$ is the depth, $u$ is the excess pore water pressure, $H$ is the soil layer thickness, and $C_{\mathrm{v}}$ is the consolidation coefficient, which is expressed as

$$C_{\mathrm{v}}={\displaystyle \frac{k_{\mathrm{v}}}{m_{\mathrm{v}}{\gamma }_{\mathrm{w}}}}$$

(2)

where $k_{\mathrm{v}}$ is the permeability coefficient, $m_{\mathrm{v}}$ is the coefficient of volume compressibility, and ${\gamma }_{\mathrm{w}}$ is the weight per unit volume of water. The traditional 1D Terzaghi consolidation boundary conditions are either fully drained or fully undrained. However, in the practice, the boundary conditions are between fully permeable and fully impermeable, and all 1D consolidation models in this paper use a time-dependent continuously drained boundary [32]. Figure 1 shows the schematic diagram of the 1D consolidation of a soil layer with a continuously drained boundary under a uniform instantaneous vertical loading $q$.

The boundary conditions for the upper and lower interfaces are taken, respectively.

$$u\left(t,0\right)=q{e}^{-bt}$$

(3)

$$u\left(t,H\right)=q{e}^{-ct}$$

(4)

where $q$ is the top applied load, $b$, $c$ are the drainage parameters at the top and bottom boundaries, and additionally, the initial conditions are set to

$$u\left(0,z\right)=q$$

(5)

In order to speed up the neural network solution, the input and output data of the network need to be converted into dimensionless form in advance [33], and the following normalized parameters are defined:

$$\overline{u}={\displaystyle \frac{u}{q}},Z={\displaystyle \frac{z}{H}},T_{\mathrm{v}}={\displaystyle \frac{C_{\mathrm{v}}t}{H^2}},\alpha ={\displaystyle \frac{b{H}^2}{c_{\mathrm{v}}}},\beta ={\displaystyle \frac{c{H}^2}{C_{\mathrm{v}}}}$$

(6)

Thus, the governing equation of the soil consolidation and fixed solution conditions in dimensionless form can be obtained:

$${\displaystyle \frac{\partial \overline{u}}{\partial T_{\mathrm{v}}}}={\displaystyle \frac{{\partial }^2\overline{u}}{\partial Z^2}}$$

(7)

$$\overline{u}\left(Z,0\right)=1$$

(8)

$$\overline{u}\left(0,T_{\mathrm{v}}\right)=e^{-\alpha T_v}$$

(9)

$$\overline{u}\left(1,T_{\mathrm{v}}\right)=e^{-\beta T_v}$$

(10)

2.1.2. Feed-Forward Neural Network

The next step is to describe how to use PINN to solve the 1D Terzaghi consolidation problem. Firstly, the implementation of the baseline PINN solving consolidation problem begins with constructing a feed-forward neural network (FNN). Let ${\mathcal{N}}^L:{\mathbb{R}}^{D_{in}}\to {\mathbb{R}}^{D_{out}}$ be a FNN with $L+1$ layers and $N_k$ neurons in the $k^{th}$ layer ($N_0=D_{in},N_L=D_{out}$). FNN consists of input layer, hidden layers and output layer. The weights and bias vectors of the $k^{th}$ layer are denoted by ${\mathbf{W}}^k\in {\mathbb{R}}^{N_k\times N_{k-1}}$ and ${\mathbf{b}}^k\in {\mathbb{R}}^{N_k}$, The input vector is denoted by $\mathit{x}\in {\mathbb{R}}^{{\mathit{D}}_{\mathit{i}\mathit{n}}}$, The output vector of the $k^{th}$ layer is denoted by ${\mathcal{N}}^k\left(\mathit{x}\right)$. With a non-linear activation function, σ, applied layer-wise, the FNN can be recursively defined as:

• input layer:

  $${\mathcal{N}}^0\left(\mathit{x}\right)=\mathit{x}$$

  (11)

  the $k^{th}$ hidden layer ($1\le k\le L-1$):

  $${\mathcal{N}}^k\left(\mathit{x}\right)=\sigma \left({\mathbf{W}}^k{\mathcal{N}}^{k-1}\left(\mathit{x}\right)+{\mathbf{b}}^k\right)$$

  (12)

  output layer:

  $${\mathcal{N}}^L\left(\mathit{x}\right)={\mathbf{W}}^L{\mathcal{N}}^{L-1}\left(\mathit{x}\right)+{\mathbf{b}}^L$$

  (13)

Therefore, the predicted value of FNN is given by

$$\widehat{u}\left(\mathit{x};\lambda \right)={\mathcal{N}}^L\left(\mathit{x};\lambda \right)$$

(14)

where $\lambda =\{{\mathbf{W}}^1,\dots {\mathbf{W}}^L,{\mathbf{b}}^1,\dots ,{\mathbf{b}}^L\}$.

Constructing the above FNN requires first initializing the weight and bias parameters of the network, $\lambda$. However, the initialization of weights and biases can seriously affect the network performance, and Glorot et al. [34] proposed the Xavier initialization method, which can initialize the weight vectors according to the number of input and output neurons in each network layer, preventing the network from experiencing the phenomena of gradient explosion and gradient vanishing, and thus improving the network performance. The PINN models in this paper all use the Xavier initialization method to initialize the weight parameters.

2.1.3. PINN for 1D Terzaghi Consolidation Equation with Continuous Drainage Boundaries

The input vector of FNN is $\mathbf{x}(Z,T_{\mathrm{v}})$, where $Z$ is dimensionless depth and $T_{\mathrm{v}}$ is time factors, the output vector is dimensionless excess pore pressure, $\widehat{\overline{u}}\left(\mathbf{x};\lambda \right)$, which can be computed as the partial differential terms in the governing equations, using the automatic derivation (AD) mechanism in the pytorch [12]. These partial differential terms can be used to compute the residuals of the governing equation $\widehat{f}(\mathbf{x};\lambda )$:

$$\widehat{f}\left(\mathbf{x};\lambda \right)≔{\displaystyle \frac{\partial \widehat{\overline{u}}}{\partial T_{\mathrm{v}}}}-{\displaystyle \frac{{\partial }^2\widehat{\overline{u}}}{\partial Z^2}}$$

(15)

The input vector $\mathbf{x}$ comes from the dataset $D$, which includes the upper boundary condition points ${\mathcal{D}}_{ub}$, lower boundary condition points ${\mathcal{D}}_{lb}$, initial condition points ${\mathcal{D}}_{ic}$, measured points obtained from experiments or high-precision numerical simulations ${\mathcal{D}}_m$, and the residual points ${\mathcal{D}}_f$. Among them, the residuals are mostly sampled randomly, and in this paper, we use the Latin hypercube sampling proposed by Helton et al. [35].

The next step is to define the loss function. For a PINN without a measured dataset, the loss function should consist of two main components: (1) the $\overline{u}\left(\mathbf{x}\right)$ and $\widehat{\overline{u}}(\mathbf{x};\lambda )$ for $\mathbf{x}\in {\mathcal{D}}_{ub},{\mathcal{D}}_{lb},{\mathcal{D}}_{ic}$; (2) zero and $\widehat{f}\left(\mathbf{x};\lambda \right)$ for $\mathbf{x}\in {\mathcal{D}}_f$. According to the definition of Raissi et al. [11], the loss function of PINN is the sum of mean square error (MSE) of FNN on ${\mathcal{D}}_{ub},{\mathcal{D}}_{lb},{\mathcal{D}}_{ic}$ and ${\mathcal{D}}_f$:

$$\mathcal{L}\left(\mathcal{D};\lambda \right)=\mathcal{L}\left({\mathcal{D}}_{ub};\lambda \right)+\mathcal{L}\left({\mathcal{D}}_{lb};\lambda \right)+\mathcal{L}\left({\mathcal{D}}_{ic};\lambda \right)+\mathcal{L}\left({\mathcal{D}}_f;\lambda \right)$$

(16)

$$\mathcal{L}\left({\mathcal{D}}_{ub};\lambda \right)={\displaystyle \frac{1}{{|\mathcal{D}}_{ub}|}}{\sum _{x\in {\mathcal{D}}_{ub}}\left|u\left(\mathbf{x}\right)-\widehat{u}\left(\mathbf{x};\lambda \right)\right|}^2$$

(17)

$$\mathcal{L}\left({\mathcal{D}}_{lb};\lambda \right)={\displaystyle \frac{1}{{|\mathcal{D}}_{lb}|}}{\sum _{x\in {\mathcal{D}}_{lb}}\left|u\left(\mathbf{x}\right)-\widehat{u}\left(\mathbf{x};\lambda \right)\right|}^2$$

(18)

$$\mathcal{L}\left({\mathcal{D}}_{ic};\lambda \right)={\displaystyle \frac{1}{{|\mathcal{D}}_{ic}|}}\sum _{x\in {\mathcal{D}}_{ic}}{\left|u\left(\mathbf{x}\right)-\widehat{u}\left(\mathbf{x};\lambda \right)\right|}^2$$

(19)

$$\mathcal{L}\left({\mathcal{D}}_f;\lambda \right)={\displaystyle \frac{1}{{|\mathcal{D}}_f|}}{\sum _{x\in {\mathcal{D}}_f}\left|\widehat{f}\left(\mathbf{x};\lambda \right)\right|}^2$$

(20)

Finally, the network is trained by setting the number of network iterations, and the optimizer is used to minimize the loss function to iteratively update the network’s learnable parameters $\lambda$. In this paper, the Adam optimizer [36], which is based on the stochastic gradient descent (SGD) algorithm, is used to update the parameters, and the learning rate is set to 0.001. Figure 2 illustrates the framework of PINN solving the 1D Terzaghi consolidation equation.

The loss function of the above PINN contains several loss terms, and in order to minimize the loss function, each loss term needs to be weighed; such a process is known as multi-task learning [37]. Simply adding up all the loss terms may lead to an imbalance in the training rate across tasks, which seriously affects the PINN performance, a phenomenon known as gradient pathology [38]. Gradient pathology is usually addressed by specifying the weights of each loss term and weighting the summation [39]:

$$\mathcal{L}\left(D;\lambda \right)=\sum _i{w}_i\mathcal{L}\left(D_i;\lambda \right),i\in \left\{ub,lb,ic,f\right\}$$

(21)

where $w_i$ is the weight for the loss term $L\left(D_i;\lambda \right)$. Methods for determining the optimal weights include manual trial-and-error and adaptive dynamic adjustment. Xiang et al. [40] proposed LBPINN that can dynamically balance the losses, and Chen et al. [31] applied the principle loss function and gradient normalization method in computer vision to dynamically adjust the weights of the loss terms in solving the unsaturated soil infiltration problem using PINN. In this paper, we are not trying to find out the optimal weight configuration but to investigate the possible advantages of the adaptive activation function under the influence of the stochastic state of PINN and different initial loss weight configurations. In the study, we kept $w_{ub}=w_{lb}=w_{ic}$, the loss function eventually reduces to two terms:

$$\mathcal{L}\left(\mathcal{D};\lambda \right)=w_{train}\mathcal{L}\left({\mathcal{D}}_{train};\lambda \right)+w_f\mathcal{L}\left({\mathcal{D}}_f;\lambda \right)$$

(22)

where ${\mathcal{D}}_{train}=\{{\mathcal{D}}_{ub},{\mathcal{D}}_{lb},{\mathcal{D}}_{ic}\}$.

In addition, DNN-based models are subject to various stochasticities during training. Madhuastha et al. [41] found that models under a set of random seeds outperform other models, and PINNs also face this problem. To solve this problem, Bengio et al. [42] used random seeds as additional hyperparameters of the network dynamically tuned to find the model with the best performance.

2.2. Layer-Wise Locally Adaptive Activation Function

Since the optimization parameters depend on the derivatives of the activation function, the activation function $\sigma \left(x\right)$ is actually a basic function in neural networks, has an important role in the training process, and Jagtap et al. [43] proposed that the activation function $\sigma \left(x\right)$ in PINNs is usually used as the following nonlinear functions.

Hyperbolic tangent function:

$$\sigma \left(x\right)=\tanh \left(x\right)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$$

(23)

Sine function:

$$\sigma \left(x\right)=\sin \left(x\right)$$

(24)

Logistic sigmoid function:

$$\sigma \left(x\right)={\displaystyle \frac{1}{1-e^{-x}}}$$

(25)

The choice of the activation function $\sigma \left(x\right)$ depends on the problem to be solved. Jagtap et al. [35] proposed a PINN with a global adaptive function (GAAF-PINN) based on the global adaptive activation function (GAAF) and introduced an additional scalable parameter $na$ in the network, where n is the scaling factor and the hyperparameter $a$ is the slope of the activation function. The results proved that introducing scalable parameter $na$ in the activation function not only accelerates the convergence of the network, but also obtains better prediction accuracy. Subsequently, Jagtap et al. [9] proposed two locally adaptive activation functions: layer-wise locally adaptive activation functions (L-LAAF) and neuron-wise locally adaptive activation functions (N-LAAF). The locally adaptive activation function is implemented by introducing scalable parameters in each network layer or each neuron $na$. Jagtap et al. demonstrated that the local adaptive activation function outperforms the global adaptive activation function in terms of training speed and accuracy. In addition, compared to the fixed activation function, although the locally adaptive activation function adds extra parameters, nevertheless the computational cost of both is comparable as the number of network layers and the number of neurons per layer increase. The code program for solving 1D consolidation problems with the PINN with layer-wise locally adaptive activation functions (L-LAAF-PINN) is shown in Algorithm 1. It is worth noting that the method is applied to the nonlinear layer of the neural network, thus can be flexibly and quickly applied to complex consolidation problems like 2d and 3d consolidation problems. For instance, the transfer of the method can be achieved by adjusting the input and output dimensions of the neural network and the construction of the loss function accordingly.

The PINNs in this paper all use a hyperbolic tangent function that introduces a scalable hyper-parameter $a$ and sets initial $a=0.01$. Figure 3 shows the image of the hyperbolic tangent function with different activation slopes. As the activation function slope $a$ changes, the expression of hyperbolic tangent function is

$$\sigma \left(x\right)={\displaystyle \frac{e^{ax}-e^{-ax}}{e^{ax}+e^{-ax}}}$$

(26)

Algorithm 1 L-LAAF-PINN

$\mathrm{Initialize} \mathrm{network} \mathrm{parameters} \lambda$$\mathrm{and} \mathrm{dataset} \mathcal{D}$

$\mathrm{Initialize} \mathrm{adaptive} \mathrm{activation} \mathrm{slope} a$

$\mathbf{for} t=0$$\mathbf{to} max\_train\_epochs$ do

$\hspace{1em}\hspace{1em}\mathrm{Compute} \mathrm{predicted} \mathrm{excess} \mathrm{pore} \mathrm{water} \mathrm{pressure} \widehat{\overline{u}}$

$\hspace{1em}\hspace{1em}\mathrm{Compute} \mathrm{the} \mathrm{PDE} \mathrm{residual} \widehat{ f}$ using automatic differentiation

$\hspace{1em}\hspace{1em}\mathrm{Compute} \mathrm{loss} \mathrm{function} \mathcal{L}\left(\mathcal{D};\lambda \right)=w_{train}\mathcal{L}\left({\mathcal{D}}_{train};\lambda \right)+w_f\mathcal{L}\left({\mathcal{D}}_f;\lambda \right)$

$\hspace{1em}\hspace{1em}\mathrm{Compute} \mathrm{the} \mathrm{gradient} \mathrm{of} \mathrm{loss} \mathrm{function} {\nabla }_{\mathsf{\lambda }}\mathcal{L}\left(\mathcal{D};\lambda \right)$

$\hspace{1em}\hspace{1em}\mathrm{Update} \mathrm{the} \mathrm{network} \mathrm{parameters} \lambda \left(t\right)\to \lambda \left(t+1\right)$

$\hspace{1em}\hspace{1em}\mathrm{Update} \mathrm{adaptive} \mathrm{activation} \mathrm{slope} a\left(t\right)\to a\left(t+1\right)$

end for

2.3. Experiment

We discuss the performance of the baseline PINN (PINN in brief) and PINN with layer-wise locally adaptive activation functions (L-LAAF-PINN) through two experiments. Experiment 1 is the 1D nonlinear large strain consolidation, Experiment 2 is the 1D nonlinear consolidation, and the performance of L-LAAF-PINN in the inverse modeling of 1D consolidation problems is studied in Experiment 3. Next, the partial differential equations and parameter Settings of these experiments will be introduced.

2.3.1. Experiment 1: 1D Nonlinear Large Strain Consolidation

The assumptions for $m_{\mathrm{v}}$ and $k_{\mathrm{v}}$ made by Xie et al. [3] are adopted:

(1) The volume compression coefficient $m_{\mathrm{v}}$ remains constant during consolidation, i.e.,

$$m_{\mathrm{v}}=-{\displaystyle \frac{1}{1+e}}{\displaystyle \frac{de}{d{\sigma }^{\prime }}}=C$$

(27)

(2) The permeability coefficient $k_{\mathrm{v}}$

$${\displaystyle \frac{k_{\mathrm{v}}}{k_{\mathrm{v}0}}}={\left({\displaystyle \frac{1+e}{1+e_0}}\right)}^2$$

(28)

where ${\sigma }^{\prime }$ is effective stress, $e$ is void ratio, $C$ is constant, $k_{\mathrm{v}0}$ is initial permeability coefficient, and $e_0$ is initial void ratio. Since the large strain consolidation leads to large soil deformation, the Lagrangian coordinate is used for the equation governing the excess pore water pressure $u$:

$$C_{\mathrm{v}0}\left[{\displaystyle \frac{{\partial }^{2}u}{\partial a^2}}+m_{\mathrm{v}}{\left({\displaystyle \frac{\partial u}{\partial a}}\right)}^2\right]={\displaystyle \frac{\partial u}{\partial t}}$$

(29)

where $C_{\mathrm{v}0}$ is initial coefficient of consolidation, and $a$ is depth in the Lagrangian coordinate. The normalized parameters of Equation (6) are still used to rewrite the dimensionless governing equation:

$$\left[{\displaystyle \frac{{\partial }^2\overline{u}}{\partial Z^2}}+{\left({\lambda }_p{\displaystyle \frac{\partial \overline{u}}{\partial Z}}\right)}^2\right]={\displaystyle \frac{\partial \overline{u}}{\partial T_{\mathrm{v}}}}$$

(30)

where ${\lambda }_p=m_{v}q$ is normalized load parameter. In the computation, the following parameters are adopted: $H=10 \mathrm{m}$, $m_v=4\times {10}^{-3} {\mathrm{k}\mathrm{P}\mathrm{a}}^{-1}$, $q=100 \mathrm{k}\mathrm{P}\mathrm{a}$, $k_{v0}={10}^{-9} \mathrm{m}/\mathrm{s}$, $e_0=3.0$, $G_s=2.75$, $\alpha =2$, $\beta =3$, ${\gamma }_w=9.8 \mathrm{k}\mathrm{N}/{\mathrm{m}}^3$.

As for the PINN solver, we fixed the network architecture as [2, 50, 50, 50, 50, 50, 1]. The training epochs were set to 20,000 epochs. The training dataset contains 101 initial condition points, 1001 upper boundary condition points, 1001 lower edge condition points, and 10,000 residual points; the validation dataset is a uniformly distributed set of 10,201 spatio-temporal points. The model performance is assessed by measuring the relative $L_2$ error of the validation dataset, which is defined as follows:

$$relative L_2 error=\sqrt{{\displaystyle \frac{{\sum }_{x\in {\mathcal{D}}_{val}}{\left|u\left(\mathit{x}\right)-\widehat{u}\left(\mathit{x};\lambda \right)\right|}^2}{{\sum }_{x\in {\mathcal{D}}_{val}}{\left|u\left(\mathit{x}\right)\right|}^2}}}$$

(31)

2.3.2. Experiment 2: 1D Nonlinear Consolidation

The empirical equations proposed by MESRI et al. [44] are adopted:

$$e-e_0=c_{\mathrm{c}}\lg \left({\displaystyle \frac{{{\sigma }_0}^{\prime }}{{\sigma }^{\prime }}}\right)$$

(32)

$$e-e_0=c_{\mathrm{k}}\lg \left({\displaystyle \frac{k_{\mathrm{v}}}{k_{\mathrm{v}0}}}\right)$$

(33)

Further adopting the assumptions of Zong et al. [45] that $c_{\mathrm{c}}/c_{\mathrm{k}}=1$, i.e., the decrease in permeability is proportional to the decrease in compressibility, the coefficient of consolidation $c_{\mathrm{v}}$ is expressed as

$$c_{\mathrm{v}}={\displaystyle \frac{k_{\mathrm{v}}}{{\gamma }_{\mathrm{w}}{m}_{\mathrm{v}}}}={\displaystyle \frac{k_{\mathrm{v}0}}{{\gamma }_{\mathrm{w}}{m}_{\mathrm{v}0}}}=c_{\mathrm{v}0}$$

(34)

According to the principle of effective stress for soils:

$${\sigma }^{\prime }=q+{\sigma }_0^{\prime }-u={\sigma }_{\mathrm{f}}^{\prime }-u$$

(35)

where $q$ is the constant load, ${\sigma }_0^{\prime }$ is the initial effective stress, ${\sigma }_{\mathrm{f}}^{\prime }$ is the ultimate effective stress. According to these equations, the governing equation for the 1D nonlinear consolidation can be described as follows:

$${\displaystyle \frac{\partial u}{\partial t}}=c_{\mathrm{v}}\left[{\displaystyle \frac{{\partial }^{2}u}{\partial z^2}}+{\displaystyle \frac{1}{{\sigma }^{\prime }}}{\left({\displaystyle \frac{\partial u}{\partial z}}\right)}^2\right]$$

(36)

The normalized parameters of Equation (6) are still used to rewrite the dimensionless governing equation:

$${\displaystyle \frac{\partial \overline{u}}{\partial T_{\mathrm{v}}}}={\displaystyle \frac{{\partial }^2\overline{u}}{\partial Z^2}}+{\displaystyle \frac{{\sigma }_{\mathrm{f}}^{\prime }-{\sigma }_0^{\prime }}{{\sigma }^{\prime }}}{\left({\displaystyle \frac{\partial \overline{u}}{\partial Z}}\right)}^2$$

(37)

In this computation, the following parameters are adopted: $H=10 \mathrm{m}$, $m_{\mathrm{v}0}=4\times {10}^{-3} {\mathrm{k}\mathrm{P}\mathrm{a}}^{-1}$, ${\sigma }_0^{\prime }=50 \mathrm{k}\mathrm{P}\mathrm{a}$, ${\sigma }_{\mathrm{f}}^{\prime }=150 \mathrm{k}\mathrm{P}\mathrm{a}$, $q=100 \mathrm{k}\mathrm{P}\mathrm{a}$, $k_{\mathrm{v}0}={10}^{-9} \mathrm{m}/\mathrm{s}$, $\alpha =4$, $\beta =2$, ${\gamma }_{\mathrm{w}}=9.8 \mathrm{k}\mathrm{N}/{\mathrm{m}}^3$. As for the PINN solver, the network architecture is setup as [2, 50, 50, 50, 50, 50, 1]. The training epochs was set to 20,000 epochs. The training dataset contains 101 initial condition points, 1001 upper boundary condition points, 1001 lower edge condition points, and 10,000 residual points; the validation dataset is a uniformly distributed set of 10,201 spatio-temporal points.

2.3.3. Experiment 3: Inverse Modeling

In order to further study the application of the proposed L-LAAF-PINN in the inverse modeling for 1D nonlinear consolidation problems, it is respectively assumed that the volume compression coefficient $m_v$ in Experiment 1 and the drainage parameters $\alpha$ and $\beta$ in Experiment 2 are unknown, while the remaining soil parameters are the same. The hyperparameters of the neural network are consistent with those of the forward model. The difference from the forward model is that the measured data items need to be added to the training set. Among them, Gaussian noise with a standard deviation of 0.05 is added to the analytical solution to serve as the measured data, and the spatio-temporal step size for sampling the measured data are set as $dZ=d{T}_{\mathrm{v}}=0.1$.

3. Results

3.1. 1D Nonlinear Large Strain Consolidation (Experiment 1)

3.1.1. Performance Comparisons: Best Model (Experiment 1)

Figure 4 shows the cloud plot of the analytical solution and the cloud plots of the absolute error of the best PINN and best L-LAAF-PINN model with the analytical solution, respectively. It is observed that the absolute error of the L-LAAF-PINN model is significantly smaller than that of the PINN model, and the maximum absolute error is only 0.0026, which is lower than 0.011 by the PINN model and is basically distributed at the upper and lower boundaries close to the initial moment. The best PINN and L-LAAF-PINN models are listed in

Table 1. 1D nonlinear large strain consolidation: summary of the best PINN and L-LAAF-PINN models.

1D Nonlinear Large Strain Consolidation	PINN	L-LAAF-PINN
$\mathrm{R}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e} L_2 \mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}$	$1.1\times {10}^{-3}$	$2.15\times {10}^{-4}$
Maximum Absolute error	$0.011$	$0.0026$
$w_{train}$	$0.5$	$0.9$
Random seed	$6$	$8$

.

Figure 5 shows how the validation loss decreases with training for the best PINN and L-LAAF-PINN models, and it can be seen that the validation loss of the L-LAAF-PINN model decreased faster, the final $L_2$ error and relative $L_2$ error were smaller than that of the PINN model. As shown in Figure 6, when the training proceeds to the 500th epoch, the best L-LAAF-PINN can accurately predict the excess pore water pressure under different time factors ($T_{\mathrm{v}}$ = 0.2, 0.4, 0.6, and 0.8), while the best PINN cannot obtain accurate predictions. As can be seen in Figure 5 and Figure 6, the best L-LAAF-PINN solving the nonlinear large strain consolidation problem converges faster than the best PINN.

Figure 7 shows a schematic of the activation slope $na$ with training time for the best L-LAAF-PINN model. It is observed that at first the activation slope of the second layer is the largest and rises the fastest, and then it starts to decline. After about 10,000 epochs, the activation slope of the first layer is the largest and rises continuously.

3.1.2. Performance Comparisons: Model Stability (Experiment 1)

Figure 8 shows the means and standard deviations of the relative $L_2$ error for the PINN and L-LAAF-PINN models using different random seeds and $w_{train}$, similar to the previous example, the average relative $L_2$ error of the PINN model generally became larger with decreasing $w_{train}$. It is apparent that the relative $L_2$ error means of the L-LAAF-PINN model under all training loss term weights $w_{train}$ were smaller than those of the PINN model, indicating that the layer-wise locally adaptive activation function improves the prediction accuracy of the PINN solver. Furthermore, the error band of the L-LAAF-PINN model was narrower than the PINN model, especially when the weights of the loss terms, $w_{train}$, are small, which suggests that the layer-wise locally adaptive activation function improved the stability of the PINN prediction for the 1D large-strain consolidation problem for random states.

3.2. 1D Nonlinear Consolidation (Experiment 2)

3.2.1. Performance Comparisons: Best Model (Experiment 2)

In Figure 9, the cloud plots of the analytical solution are shown, as well as the cloud plots of the absolute error of the best PINN and best L-LAAF-PINN model with the analytical solution, respectively. It is observed that the absolute error of the L-LAAF-PINN model was significantly lower than that of the PINN model, and the maximum absolute error was only 0.008, which was lower than 0.025 by the PINN model.

Table 2. 1D nonlinear consolidation: summary of the best PINN and L-LAAF-PINN models.

1D Nonlinear Consolidation	PINN	L-LAAF-PINN
$\mathrm{R}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e} L_2 \mathrm{e}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}$	$1.09\times {10}^{-3}$	$2.2\times {10}^{-4}$
Maximum Absolute error	$0.011$	$0.003$
$w_{train}$	$0.8$	$0.9$
Random seed	$6$	$3$

summarizes the details of the best PINN and L-LAAF-PINN models.

Figure 10 shows how the validation loss decreases with training for the best PINN and L-LAAF-PINN models, it can be seen that the validation loss of the L-LAAF-PINN model decreased faster, the final $L_2$ error and relative $L_2$ error were smaller than those of the PINN model. As shown in Figure 11, when the training proceeds to the 500th epoch, L-LAAF-PINN can accurately predict the excess pore water pressure under different time factors ($T_{\mathrm{v}}$ = 0.2, 0.4, 0.6, and 0.8), while the best PINN cannot obtain accurate predictions. It means that L-LAAF-PINN converges faster than PINN, and this advantage has been shown in the previous nonlinear consolidation problem.

Figure 12 shows the change in the activation slope $na$ with training time for the best L-LAAF-PINN model. It can be seen that the activation slope of the first layer kept rising and was the highest of all activation function layers, and the activation slopes of the other layers only rose briefly over a period of epochs and then declined.

3.2.2. Performance Comparisons: Model Stability (Experiment 2)

Figure 13 shows the means and standard deviations of the relative $L_2$ error for the PINN and L-LAAF-PINN models using different random seed and $w_{train}$. Similar to the previous example, the average relative $L_2$ error of the PINN model generally became larger with decreasing $w_{train}$. It is apparent that the relative $L_2$ error means of the L-LAAF-PINN model under all training loss term weights $w_{train}$ were lower than those of the PINN model, indicating that the layer-wise locally adaptive activation function improved the prediction accuracy of the PINN solver. Furthermore, the error band of the L-LAAF-PINN model was narrower than the PINN model, especially when the weights of the loss terms, $w_{train}$, were small, which suggests that the layer-wise locally adaptive activation function improved the stability of the PINN prediction for the 1D large-strain consolidation problem for random states.

3.3. Inverse Modeling (Experiment 3)

3.3.1. 1D Nonlinear Large Strain Consolidation

Figure 14 shows the variation results of $m_v$ inversion through the training process using PINN and L-LAAF-PINN, respectively. It can be seen from the figure that the inversion results of both models can stably converge to the true value, but L-LAAF-PINN converges faster than PINN, especially in the early stage of thetraining process. It indicates that the layer-wise locally adaptive activation functions improve the solution speed of 1D nonlinear large strain consolidation inverse problems.

3.3.2. 1D Nonlinear Consolidation

Figure 15 shows the variation results of the inversion of drainage parameters, α and β, using PINN and L-LAAF-PINN through the training process, respectively. It can be seen from the figure that both PINN and L-LAAF-PINN can accurately invert the drainage coefficients in the upper and lower boundaries; compared with PINN, L-LAAF-PINN converges faster. This still indicates that the layer-wise locally adaptive activation function accelerates the parameter participation speed of 1D nonlinear consolidation problems.

4. Conclusions

In this study, we investigated how the layer-wise locally adaptive activation function affects the performance of PINN for solving 1D consolidation problems, and for this purpose, two 1D consolidation problems are solved by using L-LAAF-PINN with the layer-wise locally adaptive activation function in comparison with baseline PINN. Based on the results obtained, the following conclusions can be drawn:

• For solving 1D nonlinear consolidation problems, the baseline PINN is seriously affected by the loss term weight settings and random states, and the accuracy of the baseline PINN solution decreases as the weights of the training loss terms decrease, and the random states also lead to the instability of the performance of the baseline PINN.
• Regardless of the loss term weight settings, the L-LAAF-PINN model with the layer-wise locally adaptive activation function solves the 1D consolidation problem with significantly better convergence speed and prediction accuracy than the benchmark PINN model.
• The L-LAAF-PINN model with the locally layered adaptive activation functions is more stable than the baseline PINN model in solving 1D consolidation problems for random states, especially in two nonlinear consolidation problems. However, the effect of loss term weight settings on PINN is hardly improved.
• In the inverse modeling of two 1D nonlinear consolidation problems, the convergence speed of L-LAAF-PINN is faster, indicating that the layer-wise locally adaptive activation functions still accelerate the parameter inversion speed in 1D nonlinear consolidation problems.