Enhanced Physics-Informed Neural Networks for Deep Tunnel Seepage Field Prediction: A Bayesian Optimization Approach

Abstract

Predicting tunnel seepage field is a critical challenge in the construction of underground engineering projects. While traditional analytical solutions and numerical methods struggle with complex geometric boundaries, standard Physics-Informed Neural Networks (PINNs) encounter additional challenges in tunnel seepage problems, including training instability, boundary handling difficulties, and low sampling efficiency. This paper develops an enhanced PINN framework designed specifically for predicting tunnel seepage field by integrating Exponential Moving Average (EMA) weight stabilization, Residual Adaptive Refinement with Distribution (RAR-D) sampling, and Bayesian optimization for collaborative training. The framework introduces a trial function method based on an approximate distance function (ADF) to address the precise handling of circular tunnel boundaries. The results demonstrate that the enhanced PINN framework achieves an exceptional prediction accuracy with an overall average relative error of 5 × 10⁻⁵. More importantly, the method demonstrates excellent practical applicability in data-scarce scenarios, maintaining acceptable prediction accuracy even when monitoring points are severely limited. Bayesian optimization reveals the critical insight that loss weight configuration is more important than network architecture in physics-constrained problems. This study is a systematic application of PINNs to prediction of tunnel seepage field and holds significant value for tunnel construction monitoring and risk assessment.

1. Introduction

With the rapid development of underground infrastructure construction worldwide, the prediction and control of groundwater seepage around tunnels have emerged as critical challenges in modern geotechnical engineering [1,2]. Deep tunnels, particularly those constructed in water-rich geological formations, inevitably disturb the surrounding hydrogeological environment. This disturbance leads to complex seepage patterns that can have a significant impact on construction safety, long-term structural stability, and regional ecological systems [3,4]. Accurate prediction of seepage fields around tunnels is therefore essential for optimizing dewatering strategies, designing effective drainage systems, assessing environmental impacts on surrounding aquifers, and developing sustainable groundwater management strategies for tunnel operations [5,6,7].

Traditional approaches to tunnel seepage analysis rely primarily on analytical and numerical methods, each of which possesses distinct advantages alongside inherent limitations. Analytical solutions, as exemplified by the seminal contributions of Harr [8] and Lei [9], along with the comprehensive framework developed by Bear [10], provide valuable insights for idealized conditions but are generally constrained to simplified geometries and the assumption of homogeneous media. Numerical methods, particularly the finite element method (FEM) and the finite difference method (FDM), have become the predominant tools for complex scenarios, as demonstrated in comprehensive reviews by Zienkiewicz et al. [11] and the recent advances documented in specialized journals [12,13]. Although these methods offer flexibility in handling irregular geometries and complex boundary conditions, they typically require extensive mesh generation, encounter computational limitations for large-scale problems, and may suffer from numerical instabilities [14]. Recent coupled hydro-mechanical modeling approaches often demand significant practical engineering [15], while studies have revealed increasing complexity in seepage analysis under multi-stress environments [16] and challenges in porous media flow modeling [17].

Parallel to these developments in hydrogeological modeling, the emergence of machine learning (ML) and artificial intelligence (AI) has transformed scientific computing across numerous disciplines, offering new paradigms for solving complex physical problems [18,19]. Traditional machine learning approaches in groundwater modeling have primarily focused on data-driven methods, including artificial neural network (ANN), support vector machine (SVM), and ensemble methods for parameter estimation and pattern recognition [20,21,22]. Recent studies have demonstrated growing applications of advanced ML techniques in subsurface engineering, including neural networks for subsurface structure characterization [23] and tunnel ground settlement prediction [24]. While these approaches have demonstrated considerable promise in specific applications, they often lack the physical consistency and generalizability required for robust engineering applications, particularly when dealing with scenarios outside the training data domain and with extremely limited data.

This limitation has motivated a paradigm shift towards physics-informed approaches that integrate physical laws directly into machine learning frameworks. Physics-Informed Neural Networks (PINNs), introduced by Raissi et al. [25], represent a groundbreaking advancement that embeds partial differential equations (PDEs) as flexible constraints within the loss functions of neural networks. This innovative approach enables the solution of forward and inverse PDE problems without requiring traditional mesh-based discretization, offering significant advantages in terms of computational flexibility and efficiency [26,27].

The integration of physics-informed machine learning with hydrogeological modeling constitutes a promising direction in computational geosciences, driven by the need to overcome the limitations of traditional approaches while leveraging advances in computational power and algorithmic sophistication [28,29]. Indeed, PINNs have shown particular promise in subsurface flow problems, as evidenced by recent studies such as the parameter learning framework proposed by Tartakovsky et al. [30], the multiphysics data assimilation research conducted by He et al. [31], and comprehensive analyses by Mao et al. [32], all of which demonstrate that competitive accuracy can be achieved with significantly reduced computational requirements compared to traditional finite element methods. These works have thus paved the way for applying physics-informed approaches to more specialized problems in geotechnical engineering, such as the prediction of tunnel seepage field.

Despite this significant progress, several critical gaps remain that limit the practical application of PINNs to complex tunnel seepage problems. Existing PINN implementations often employ standard optimization approaches that may suffer from training instability, particularly for problems with multiple physical scales and complex boundary conditions [33]. Most current implementations rely on uniform sampling strategies that may inadequately capture regions of high solution gradients, such as those typically found near tunnel boundaries where accurate prediction is most critical for engineering applications. Furthermore, while PINNs offer theoretical advantages in handling irregular geometries, practical implementation for tunnel seepage problems requires the sophisticated treatment of circular boundaries and the simultaneous enforcement of multiple types of boundary condition. Traditional soft constraint approaches may compromise accuracy, particularly at material interfaces and geometric discontinuities [34]. Additionally, standard PINN training often relies on single-stage optimization with manually tuned hyperparameters, leading to suboptimal performance and limited robustness across different problem configurations [35].

To address these limitations, this study develops an enhanced PINN framework specifically designed for predicting tunnel seepage field, incorporating several innovative components that contribute to both the fundamental development of physics-informed machine learning and its practical implementation in geotechnical engineering. The proposed methodology introduces a novel three-stage training strategy that combines Adam pre-training with Exponential Moving Average (EMA) for weight stabilization, Adam reinforcement training with Residual Adaptive Refinement with Distribution (RAR-D) for intelligent sampling, and L-BFGS-B fine-tuning for high-precision convergence. The framework further implements geometry-aware trial functions based on approximate distance functions (ADFs) to exactly satisfy Dirichlet boundary conditions, thereby eliminating errors associated with boundary modeling and enhancing training efficiency for complex tunnel geometries. A sophisticated Residual Adaptive Refinement with Distribution (RAR-D) method is employed to dynamically identify and refine regions with high PDE residuals, thus ensuring adequate resolution near tunnel boundaries without the need for uniform grid refinement. Moreover, the integration of Bayesian optimization enables automated hyperparameter tuning, which improves model robustness and reduces manual calibration requirements.

In this study, the analytical solution for the tunnel seepage field is first presented, along with the proposed enhanced PINN framework and its detailed algorithms. Then the configuration of relevant algorithmic parameters is outlined. Subsequently, the results are analyzed and discussed, including the predictive performance of the enhanced PINN model and the influence of measurement point density on its robustness. Finally, conclusions and future research directions are presented.

2. Methodology: Enhanced PINN Framework

2.1. Mathematical Model and Analytical Solution of Tunnel Seepage Field

Tunnel construction in water-rich formations inevitably causes significant changes in the surrounding seepage field. Accurate prediction of seepage influence zones is essential for optimizing monitoring schemes, selecting appropriate construction techniques, and mitigating adverse ecological impacts. Deep tunnel engineering frequently encounters practical problems such as complex geology, large water inflow, and high hydraulic heads. Based on the characteristics of deep tunnel seepage, the paper adopts the following idealized assumptions:

(1)

The surrounding rock is homogeneous and porous, forming a continuous medium;

(2)

Groundwater is incompressible, and the seepage behavior is steady-state.

For two-dimensional steady-state seepage in isotropic media without source or sink terms, the governing equation is given in polar coordinates by

$${\nabla }^{2}h = {\displaystyle \frac{{\partial }^{2}h}{\partial r^2}} + {\displaystyle \frac{1}{r}}{\displaystyle \frac{\partial h}{\partial r}} + {\displaystyle \frac{1}{r^2}}{\displaystyle \frac{{\partial }^{2}h}{\partial {\phi }^2}} = 0$$

(1)

where h is the hydraulic head, r is the radial distance, and $\phi$ is the angular coordinate. The boundary conditions are specified as follows:

$$h(x,y) = \overline{h},\hspace{1em}(x,y)\in {\mathsf{\Gamma }}_{\mathrm{d}}$$

(2)

$$-k(x){\displaystyle \frac{\partial h(x,y)}{\partial n}} = 0,\hspace{1em}(x,y)\in {\mathsf{\Gamma }}_{\mathrm{q}}$$

(3)

where ${\mathsf{\Gamma }}_{\mathrm{d}}$ represents Dirichlet boundary, which comprises the top boundary and tunnel boundary. The tunnel boundary is a fully permeable surface with a constant hydraulic head of 0, while the top boundary is assigned a constant head value of 5, as illustrated in Figure 1. ${\mathsf{\Gamma }}_{\mathrm{q}}$ denotes the Neumann boundary, including the bottom, left, and right boundaries, all of which are impermeable boundaries. ${\displaystyle \frac{\partial h}{\partial n}}$ is the derivative of the head in the direction normal to the boundary, representing zero fluid flux, and $k$ is the permeability coefficient with a value of $k = 1 \times {10}^{-7} \mathrm{m}/\mathrm{s}$.

This study adopts the analytical solution for two-dimensional steady-state groundwater flow around a horizontal tunnel as a benchmark case to evaluate model prediction accuracy. The analytical solution was proposed for the tunnel seepage field in isotropic, homogeneous, semi-infinite aquifers by Lei [9], extending the approximate formula of Polubarinova-Kochina to accurately describe the hydraulic head distribution. The computational domain is an annular region centered on the tunnel. Lei [9] derived the following analytical solution of hydraulic head using the method of images:

$$h_{\mathrm{A}} = {\displaystyle \frac{d + h_{\mathrm{a}} - h_0}{2\ln \left(\frac{D}{R} + \sqrt{{\left(\frac{D}{R}\right)}^2 - 1}\right)}}\ln \left({\displaystyle \frac{x^2 + {\left(y + \sqrt{D^2 - R^2}\right)}^2}{x^2 + {\left(y - \sqrt{D^2 - R^2}\right)}^2}}\right) + d + h_{\mathrm{a}}$$

(4)

where x and y are Cartesian coordinates, R is the tunnel radius, D denotes the depth of the tunnel, $h_{\mathrm{a}}$ is the surface head, $h_0$ is the head at the tunnel boundary, and d represents water depth above ground level.

2.2. Enhanced PINN Framework Based on ADF Trial Functions

Physics-Informed Neural Networks (PINNs), introduced by Raissi et al. [20], embed PDE residuals, initial conditions, and boundary conditions into neural network loss functions. Within this framework, solving PDEs is transformed into a neural network optimization problem. This study utilizes a PINN algorithm using Automatic Differentiation (AD) methods to embed partial differential equations into neural networks through Multi-Layer Perceptrons (MLPs) as the network structure. MLPs consist of multiple fully connected neuron layers, divided into input, hidden, and output layers. The inputs of each neuron in each layer undergo linear combination through weights and biases, and then pass through nonlinear activation functions (σ) to obtain the outputs, which are calculated as follows:

$$\widehat{h}(x\left|\theta \right.) = f\left(x\right) = \sigma ({\displaystyle {\sum }_{i = 1}^n{\omega }_i{x}_i + b})$$

(5)

where $x_i$ represents neuron inputs, $\theta$ denotes the collection of all weight and bias parameters in the neural network, $\widehat{h}(x\left|\theta \right.)$ is the neural network outputs, ${\omega }_i$ represents the weight of the i-th layer, b denotes biases introduced to neurons, and $\sigma$ represents activation functions.

Activation functions primarily enhance the network’s representational capacity and learning capabilities. Currently common activation functions include sigmoid, tanh, ReLU, and Leaky ReLU functions. In PINNs, the infinitely differentiable tanh function is typically used as the activation function, defined as

$$\sigma (x) = \tanh (x) = {\displaystyle \frac{{\mathrm{e}}^x - {\mathrm{e}}^{-x}}{{\mathrm{e}}^x{ + \mathrm{e}}^{-x}}}$$

(6)

When applying PINN algorithms to solve seepage flow velocity, Automatic Differentiation (AD) technology is primarily used to calculate derivatives of network output functions with respect to input variables. AD is a method between symbolic and numerical differentiation that performs symbolic differentiation and then uses the chain rule to obtain desired derivative values. Therefore, compared to numerical differentiation, AD can replace grid-scale difference operations, avoiding computational errors from equation discretization in numerical differentiation, improving calculation accuracy. Since AD is primarily applied to neural network outputs, gradient calculations are exact.

The mean squared error loss functions for the governing equations, initial conditions, and boundary conditions are integrated into a unified neural network loss function. Using AD to construct loss functions for governing equations and boundary conditions, the expressions are

$${\mathcal{L}}_{\mathrm{PDE}} = {\displaystyle \frac{1}{{\tau }_{\mathrm{cp}}}}{\displaystyle \sum _{i = 1}^{N_{\mathrm{cp}}}{\left|h(r_i,{\phi }_i)\right|}^2}$$

(7)

$${\mathcal{L}}_{\mathrm{BC}1} = {\displaystyle \frac{1}{{\tau }_{\mathrm{BC}1}}}{\displaystyle \sum _{i = 1}^{N_{\mathrm{BC}1}}{‖h(r_i,{\phi }_i) - h_{\mathrm{BC}1}(r_i,{\phi }_i)‖}^2}$$

(8)

$${\mathcal{L}}_{\mathrm{BC}2} = {\displaystyle \frac{1}{{\tau }_{\mathrm{BC}2}}}{\displaystyle \sum _{i = 1}^{N_{\mathrm{BC}2}}{‖h(r_i,{\phi }_i) - h_{\mathrm{BC}2}(r_i,{\phi }_i)‖}^2}$$

(9)

$${\mathcal{L}}_{\mathrm{D}} = {\displaystyle \frac{1}{{\tau }_{\mathrm{D}}}}{\displaystyle \sum _{i = 1}^{N_{\mathrm{D}}}{‖h(r_i,{\phi }_i) - h_{\mathrm{D}}(r_i,{\phi }_i)‖}^2}$$

(10)

The total loss function is defined as

$$\mathrm{Loss}(\theta ) = {\lambda }_{\mathrm{P}}{\mathcal{L}}_{\mathrm{PDE}}(\theta ) + {\lambda }_{\mathrm{BC}1}{\mathcal{L}}_{\mathrm{BC}1}(\theta ) + {\lambda }_{\mathrm{BC}2}{\mathcal{L}}_{\mathrm{BC}2}(\theta ) + {\lambda }_{\mathrm{D}}{\mathcal{L}}_{\mathrm{D}}(\theta )$$

(11)

where $(r_i,{\phi }_i)$ represents randomly distributed points within the computational domain; $h(r_i,{\phi }_i)$ is the hydraulic head; $‖·‖$ denotes the Euclidean norm; $h_{\mathrm{BC}1}(r_i,{\phi }_i)$ and $h_{\mathrm{BC}2}(r_i,{\phi }_i)$ represent Dirichlet and Neumann boundary heads, respectively; $h_{\mathrm{D}}(r_i,{\phi }_i)$ represents actual monitoring points’ heads; and ${\tau }_{\mathrm{cp}}$, ${\tau }_{\mathrm{BC}1}$, ${\tau }_{\mathrm{BC}2}$, and ${\tau }_{\mathrm{D}}$ are the training point sets for interior PDE, Dirichlet boundaries, Neumann boundaries, and actual observation points, respectively. These residual points have initial positions and values that follow random distributions. ${\lambda }_{\mathrm{P}}$, ${\lambda }_{\mathrm{BC}1}$, ${\lambda }_{\mathrm{BC}2}$, and ${\lambda }_{\mathrm{D}}$ are weights for adjusting loss functions constructed from governing equations and boundary conditions, which can be adjusted based on loss value feedback during calculation.

The optimization process follows a two-stage strategy to minimize the loss function. First, the Adam optimizer is used in combination with a ReduceLROnPlateau scheduler to dynamically adapt the learning rate. Subsequently, the L-BFGS-B algorithm is applied for high-precision convergence to a local minimum, thereby obtaining optimal neural network parameter $\theta$:

$$(\widehat{\theta }) = \arg \underset{\theta }{\min }\mathrm{Loss}(\theta )$$

(12)

Conventional PINN algorithms typically enforce boundary constraints as soft constraints, which may compromise computational accuracy. To impose Dirichlet boundary conditions exactly, a hard-constraint technique is adopted, drawing on the geometry-aware trial function approach proposed by Sukumar et al. [36]. This method constructs approximate distance functions and neural network approximations to ensure that the boundary conditions are satisfied exactly, thereby eliminating boundary modeling errors. The trial function is constructed as follows:

$$h(r,\phi ) = \varphi (r,\phi )•h_{\mathrm{BC}}(r,\phi ) + g(r,\phi )$$

(13)

$$\varphi (r,\phi ) = {\displaystyle \frac{{\varphi }_{\mathrm{t}}•{\varphi }_{\mathrm{c}}}{\sqrt{{\varphi }_{\mathrm{t}}^2•{\varphi }_{\mathrm{c}}^2 + \epsilon }}}$$

(14)

$$g(r,\phi ) = {\displaystyle \frac{5{\varphi }_{\mathrm{c}}}{{\varphi }_{\mathrm{t}} + {\varphi }_{\mathrm{c}} + \epsilon }}$$

(15)

where $h_{\mathrm{BC}}(r,\phi )$ is the MLP output, $\varphi$ and g are auxiliary functions, ${\varphi }_{\mathrm{t}}$ and ${\varphi }_{\mathrm{c}}$ represent distance functions from computational domain points to the top boundary and tunnel boundary, respectively, and $\epsilon$ = 5 × 10⁻⁵ is a small constant included to prevent division by zero. This trial function mathematically ensures the automatic satisfaction of the Dirichlet boundary conditions on both the top and tunnel boundaries. Consequently, the loss function is simplified to consider the PDE residuals and the monitoring point residuals:

$$\mathrm{Loss}(\theta ) = {\lambda }_{\mathrm{P}}{\mathcal{L}}_{\mathrm{PDE}}(\theta ) + {\lambda }_{\mathrm{BC}2}{\mathcal{L}}_{\mathrm{BC}2}(\theta ) + {\lambda }_{\mathrm{D}}{\mathcal{L}}_{\mathrm{D}}(\theta )$$

(16)

This method avoids the weight balancing issues in soft constraints, improving training efficiency and boundary accuracy. It is especially well-suited for problems involving complex geometric boundaries, such as circular interfaces. Neumann boundary conditions are still imposed as soft constraints through the ${\mathcal{L}}_{\mathrm{BC}2}(\theta )$ term in the loss function.

This study employs the relative error (L₂) as an evaluation metric to assess the accuracy of the computational results, which is defined as

$$L_2 = {\displaystyle \frac{{‖h(r,\phi )-\widehat{h}(r,\phi )‖}_2}{{‖h(r,\phi )‖}_2}}$$

(17)

where $h(r,\phi )$ represents the result of the analytical solution and $\stackrel{⏜}{h}(r,\phi )$ represents the predicted model result. An improved coefficient of determination indicates a closer agreement between the results of the PINN method and the actual solutions.

2.3. Bayesian Optimization and Three-Stage Collaborative Training

In research applying PINN algorithms to solve groundwater seepage problems, the weights of the loss function are crucial for model convergence and prediction accuracy. Traditional manual parameter tuning methods are time-consuming and make it difficult to guarantee optimal solutions. To address this, the paper employs adaptive Bayesian Optimization (BO) methods, with the overall framework illustrated in Figure 2, where PINNs serve as the main component to output neural network parameters. During the progressive optimization, EMA and RAR-D are used to adaptively adjust network weights and sampling point distributions, respectively. Bayesian optimization is then utilized to adjust the loss function weights, thereby improving the efficiency and accuracy of modeling the tunnel seepage field.

To alleviate parameter oscillations that may affect the convergence quality, Exponential Moving Average (EMA) is introduced in the first stage to improve training stability by maintaining weighted historical averages of network parameters. The EMA parameter updates follow the recursive relationship:

$${\theta }_{\mathrm{EMA}}^{(t)} = {\beta }^{(t)}{\theta }_{\mathrm{EMA}}^{(t - 1)} + (1 - {\beta }^{(t)}){\theta }^{(t)}$$

(18)

where ${\theta }^{(t)}$ represents the current model parameters at step t. To adapt to different training stages, the effective decay coefficient ${\beta }^{(t)}$ is calculated as

$${\beta }^{(t)} = \min \left({\beta }_0,{\displaystyle \frac{1 + n_{\mathrm{update}}}{10 + n_{\mathrm{update}}}}\right)$$

(19)

where ${\beta }_0$ = 0.999 is the initial decay coefficient and $n_{\mathrm{update}}$ is the number of EMA updates. Based on the EMA mechanism, stable parameters are supplied to the RAR-D algorithm during the second training stage.

Randomly distributed sampling points may lead to insufficient coverage of high-gradient regions near tunnel boundaries, which compromises the predictive accuracy of the seepage field. Therefore, the Residual Adaptive Refinement (RAR-D) method is introduced to dynamically optimize the distribution of sampling points. RAR-D is an intelligent sampling strategy that dynamically identifies regions with high PDE residuals and subsequently increases the density of collocation points, thereby ensuring adequate numerical resolution in critical regions. For tunnel seepage problems, the PDE residuals are defined as

$$R(x_i) = {\left|{\displaystyle \frac{{\partial }^{2}h}{\partial r^2}} + {\displaystyle \frac{1}{r + \epsilon }}{\displaystyle \frac{\partial h}{\partial r}} + {\displaystyle \frac{1}{r^2 + \epsilon }}{\displaystyle \frac{{\partial }^{2}h}{\partial {\phi }^2}}\right|}^2$$

(20)

where $\epsilon$ is a numerical stability parameter. The Automatic Differentiation technology calculates the derivatives of various orders. The number of new points is controlled at 5% of the total number of sampling points (approximately 250 points) to strike a balance between computational cost and accuracy improvement. The newly added sampling points are incorporated into the training set, after which the PDE residuals and loss functions are recalculated. Finally, the parameters of the PINN model are subsequently updated.

Bayesian optimization is a global optimization method based on probabilistic models that is suitable for optimizing computationally expensive and non-differentiable objective functions. In PINNs, the objective function is defined as the relative error (L₂) of model predictions. Taking loss function weight $({\lambda }_i)$ as an example, BO constructs a probabilistic surrogate model of the objective function through Gaussian Processes (GP):

$$L_2 ~ GP(m(l),k(l,l\prime ))$$

(21)

where GP predicts mean $\mu (\lambda )$ and variance ${\sigma }^2(\lambda )$ of $f(\lambda )$ based on observed data ${({\lambda }_i,f_i)}_{i = 1}^n$. $m(\lambda )$ is the mean function, typically taken as a constant or zero; $k(\lambda ,{\lambda }^{\prime })$ is the covariance kernel, commonly using the Radial Basis Function (RBF) kernel:

$$k(\lambda ,{\lambda }^{\prime }) = {\sigma }_f^2\exp \left( - {\displaystyle \frac{|\lambda -{\lambda }^{\prime }{|}^2}{2l^2}}\right)$$

(22)

where ${\sigma }_f^2$ is the signal variance controlling function amplitude, and $l$ is length scale controlling function smoothness. The GP model is updated via Bayesian inference, which combines the L₂ errors from evaluated points to predict the performance of unevaluated hyperparameters.

The Expected Improvement (EI) acquisition function is typically used to select the next hyperparameter set, balancing exploration of unevaluated regions with exploitation of current optimal regions:

$$EI(x) = E[\max (0,f(\lambda ) - f({\lambda }^{+}))]$$

(23)

where $f(\lambda )$ is the L₂ error and $f({\lambda }^{+})$ is the current best hyperparameter. EI is calculated using analytical formulas:

$$EI(x) = (\mu (\lambda ) - f({\lambda }^{+}))\mathrm{\Phi }(z) + \sigma (\lambda )\varphi (z),\hspace{1em}z = {\displaystyle \frac{\mu (\lambda ) - f({\lambda }^{+})}{\sigma (\lambda )}}$$

(24)

where $\mu (\lambda )$ and $\sigma (\lambda )$ are the mean and standard deviation predicted by the GP, $z(\cdot )$, and $\mathrm{\Phi }(\cdot )$ and $\varphi (\cdot )$ are the cumulative distribution function and probability density function of the standard normal distribution, respectively.

The optimization process begins by initializing with a set of random hyperparameters (typically 5–10 sets) and evaluating the corresponding L₂ errors. Based on GP model and EI acquisition function, the next hyperparameter set is selected. The PINN model is then retrained and evaluated under this new configuration.

3. Experimental Configuration

3.1. Problem Configuration and Analytical Solution

The experimental framework aims to evaluate the performance of the proposed enhanced PINN method by comparing its predictions with established analytical solutions for tunnel seepage problems. The computational domain is defined as a rectangular region in Cartesian coordinates: $x\in [-10,10]$, $\mathrm{y}\in [-20,0]$, representing a typical deep-buried tunnel configuration. The tunnel geometry features a circular cross-section centered at $(0,-5)$ with radius $R = 0.5$, located 5 below the surface (the unit of the model is meters).

A coordinate transformation to polar coordinates $(r,\phi )$ is applied to naturally accommodate the circular tunnel boundary. The radial distance r varies from the tunnel radius to a maximum of 20 m, and the angular coordinate $\phi$ spans the complete $2\mathsf{\pi }$ range. The computational grid employs 800 radial points and 800 angular points, providing high resolution for accurate gradient calculations near the tunnel boundary, where solution gradients are typically highest.

The analytical solution benchmark follows the framework developed by Lei [9], providing exact solutions for steady-state groundwater flow around circular tunnels in semi-infinite aquifers. The parameters for the analytical solution are set as follows: tunnel depth $D = 5.0$ m, permeability coefficient $k = 1 \times {10}^{-7}$ m/s, surface head $h_{\mathrm{a}} = 5.0$ m, tunnel boundary head $h_0 = 0.0$ m, and water depth above ground $d = 0.0$ m. This configuration represents typical conditions encountered in deep tunnel construction projects, offering a rigorous benchmark for assessing the prediction accuracy of the PINN model, as illustrated in Figure 3. The three-dimensional results more clearly show the overall trend in head variation, indicating significant head drops near the tunnel.

3.2. Dataset Configuration

The training dataset is constructed using a multi-component strategy designed to capture the fundamental physical characteristics of tunnel seepage while maintaining computational efficiency. The collocation point generation begins with 5000 uniformly distributed random points within the computational domain, excluding the circular tunnel interior to ensure physical consistency. These points undergo filtering to remove any positions falling within the tunnel radius plus a small tolerance to prevent numerical instability. Boundary condition points are systematically generated to impose prescribed boundary conditions. The Dirichlet boundary points are placed along the surface $(y = 0)$ as Dirichlet₁ with a prescribed head value of 5.0 m, and along the tunnel perimeter as Dirichlet₂ with a prescribed head value of 0.0 m. The Neumann boundary points are distributed along the bottom, left, and right domain boundaries to impose zero-flux conditions representing impermeable boundaries. The selection of monitoring points is randomly generated over the full computational domain.

The proposed three-stage training strategy addresses different aspects of optimization challenges while maximizing convergence efficiency and solution accuracy. Each stage employs different optimization algorithms and techniques that are tailored to specific training objectives.

Stage 1: Adam Pre-training with Exponential Moving Average (EMA)

The first stage is conducted over 300 epochs of Adam optimization to establish stable initial network weights. The Adam optimizer is configured with a weight decay parameter $1 \times {10}^{-4}$ to prevent overfitting while maintaining sufficient model expressivity. The initial learning rate is determined through Bayesian optimization, typically ranging from $1 \times {10}^{-3}$ to $5 \times {10}^{-3}$.

The Exponential Moving Average (EMA) mechanism is employed to stabilize training and reduce parameter oscillations during early stages. The EMA decay rate is set to 0.999 with a warm-up period of 50 epochs to allow for initial parameter stabilization. The EMA parameters are updated with every epoch, providing continuously smoothed weight estimates that enhance convergence stability and final solution quality.

Stage 2: Adam Reinforcement Training with Residual Adaptive Refinement with Distribution (RAR-D)

The second stage extends training to 800 epochs while implementing an RAR-D adaptive sampling mechanism. The learning rate is reduced by 50% from the final stage 1 value to enable fine-tuning of network parameters. The RAR-D algorithm is triggered at specific epochs (100, 200, 300, 400, and 500), systematically refining the training dataset in high PDE residual regions. The choice of a 100-epoch interval balances several practical considerations: allowing sufficient time for network stabilization before introducing sampling perturbations, providing adequate learning cycles for newly added configuration points, and maintaining computational efficiency by controlling refinement frequency.

The RAR-D implementation calculates the PDE residuals for candidate points across the entire computational domain, ranks them according to magnitude, and selects top-ranked locations to add additional collocation points. Each RAR-D activation adds 50 new training points, which are distributed to improve solution accuracy in critical regions while maintaining computational efficiency.

Stage 3: L-BFGS-B High-Precision Convergence Fine-tuning

The final stage employs the Limited-memory Broyden–Fletcher–Goldfarb–Shanno with Bound constraints (L-BFGS-B) algorithm to achieve high-precision optimization. This quasi-Newton method is particularly effective for final convergence stages, providing a rapid convergence to high-accuracy solutions. L-BFGS-B optimization is configured for a maximum of 8000 iterations with a gradient tolerance of $1 \times {10}^{-8}$ and function change tolerance of $1 \times {10}^{-11}$.

The experimental implementation is conducted in a controlled computational environment ensuring result reproducibility and fair comparison. The software environment is built on Python 3.11+ with PyTorch 1.12 as the primary deep learning framework. The Neuromancer library provides physics-informed neural network implementation, while Optuna handles Bayesian optimization procedures. Additional libraries include NumPy 2.0+ for numerical computation, Matplotlib 3.10.3 for visualization, and SciPy 1.13.1 for scientific computing functions.

4. Results and Discussion

4.1. Enhanced PINN Prediction Performance Evaluation

This section validates the performance of the method using 100 random monitoring points. Since the values at the tunnel boundaries are 0, the model’s maximum relative error appears at tunnel boundaries as $6 \times {10}^{-3}$, with an overall average relative error of $5 \times {10}^{-5}$, demonstrating high computational accuracy. As shown in Figure 4, the PINN predictions demonstrate overall trends that closely match the analytical solutions. Figure 4b shows the absolute error spatial distribution with the maximum errors appearing at the left and right boundaries around 0.02. Figure 4c uses a three-dimensional distribution to more intuitively display the error distribution regions, with most central areas being flat and low-lying with errors below 0.01. This reflects the varying performance of PINN methods when handling different types of constraints. In regions with sufficient physical constraints (PDE residuals) and geometric constraints (ADF boundary conditions), models maintain high accuracy even facing complex flow field characteristics.

The spatial variation of prediction errors can be attributed to several interconnected physical and numerical factors. High-error zones are predominantly located in regions with steep hydraulic head gradients, as these areas demand that the neural network capture rapid spatial variations that test the limits of its approximation capability. Although strong gradients are pre-sent near the tunnel boundary, errors remain relatively low due to the exact enforcement of boundary conditions enabled by the ADF trial function method, which guarantees pointwise satisfaction of Dirichlet conditions.

The transition zone between the near-field and far-field exhibits intermediate error levels, reflecting the complex interplay between weakening geometric constraints and maintaining PDE constraint effectiveness. In contrast, far-field regions show elevated relative errors despite lower absolute gradients, primarily due to insufficient physical constraint guidance where PDE residuals data constraints and become weaker. The error distribution also correlates with the underlying flow physics. Near the tunnel, source-dominated radial flow patterns provide strong directional gradients that offer clear learning signals for the neural network. However, far-field regions approach uniform flow conditions, where weak driving forces and minimal gradients offer limited information for effective network training, resulting in degraded prediction accuracy. This physical insight clarifies why adaptive sampling strategies such as RAR-D are highly effective: they dynamically allocate computational resources to regions where pronounced physical gradients furnish optimal training signals, while avoiding unnecessary refinement in areas with inherently weak constraints.

The EMA mechanism significantly improved the stability of the first-stage training. The effective decay rate gradually increased from an initial value of 0.091 to the target value of 0.999, demonstrating the effectiveness of the adaptive decay mechanism. The EMA-stabilized weights provided a reliable initialization for the second-stage RAR-D algorithm.

As shown in Figure 5, the number of sampling points increases over five iterations, with the maximum residual value gradually decreasing from approximately 1 to around 0.1. Figure 5a reveals that the second to fourth RAR-D iterations achieve a substantial and consistent reduction in the maximum residual to 0.1, while the fifth iteration has reached a plateau. This pattern confirms the effectiveness of the iterative refinement strategy. Figure 5b displays that within a 2.5 m proximity to the tunnel, high residuals are concentrated primarily around the tunnel periphery, with particularly dense clustering near the upper tunnel boundary. Figure 5c illustrates how the newly added adaptive sampling points spread outward as the number of iteration rounds increases.

Figure 6 displays the correlations between hyperparameters and prediction errors across 15 Bayesian optimization trials. Points of the same color within each subgraph represent results from the same experiment. The results indicate that the configuration of the loss function weight has a far greater impact on model performance than the parameters of the network structure. The PDE loss weight (scaling_pde) shows a strong negative correlation (−0.675) with error, with an optimal value of 4.903. Scatter plots show that prediction errors decrease significantly as PDE weights increase, indicating that physical constraints indeed need dominant positions in loss functions.

The optimal value of the Dirichlet boundary weight is 5.941, though the correlation is weaker (−0.047), still reflecting the importance of precise boundary conditions. The optimal value for the Neumann boundary weight is only 0.009, which is consistent with its role as a soft constraint. For the neural network structure, the optimal number of layers is seven with a correlation of only 0.177. More notably, the number of neurons shows almost no correlation with error (−0.002), with the optimal value of 120 coinciding exactly with the lower bound of the predefined search range. Similarly, the optimal learning rate value is 0.001, which also demonstrates a very weak correlation (–0.007) with model error.

More neurons provide stronger function approximation capability, and deeper networks can capture more complex nonlinear relationships, but oversized networks suffer from gradient problems, which leads to error accumulation, convergence difficulties, and increased computational costs. The Figure 7 radar chart further reveals parameter variation sensitivity. The left chart shows that scaling_pde has the highest sensitivity (about 60%), which echoes the results of the correlation analysis. Learning_rate sensitivity follows (about 40%), indicating that the training process is sensitive to learning rate settings. After 15 trials, the best error decreased from 0.095 to 0.021, representing a 78% improvement.

For problems with clear physical constraints, ensuring PDE loss dominance in the overall loss is more effective than increasing network scale. This explains why the proposed three-stage training strategy achieves good results: it systematically improves training quality through EMA stabilization, RAR-D sampling improvement, and L-BFGS-B fine optimization, rather than relying solely on more complex network structures.

To quantify the individual contributions of each component in our enhanced PINN framework, we conducted systematic ablation studies comparing four configurations under identical experimental conditions. All experiments used the same network architecture (seven layers, 120 neurons), training data distribution (5000 collocation points), and evaluation metrics. Each configuration was run for 10 independent trials to ensure statistical significance.

Table 1. Relative Error and Computational Efficiency of the Ablation Experiment.

Configuration	Method	L2 Relative Error	Training Time (Seconds)
Config-1	Standard PINN with Adam optimization only	0.019551 ± 0.000648	374.96 ± 3.64
Config-2	Standard PINN with Exponential Moving Average weight stabilization	0.014207 ± 0.000926	377.82 ± 1.79
Config-3	Config-2 + (RAR-D)	0.009327 ± 0.000438	447.59 ± 0.72
Config-4	Full framework including Bayesian hyperparameter optimization	0.006264 ± 0.000374	1082.28 ± 33.43

summarizes the quantitative performance comparison across all configurations. The baseline PINN (Config-1) achieved an L₂ of 0.019551 ± 0.000648, demonstrating the fundamental capability of physics-informed approaches for tunnel seepage problems. The addition of EMA (Config-2) provides a modest accuracy improvement (L₂ = 0.014207 ± 0.000926), while providing essential training stability. The integration of RAR-D adaptive sampling (Config-3) provided a significant performance gain, achieving L₂ = 0.009327 ± 0.000438. The complete framework (Config-4) with Bayesian optimization achieved the highest performance (L₂ = 0.006264 ± 0.000374), validating our hypothesis that automated hyperparameter tuning provides the final optimization layer. The RAR-D component accounts for additional computation (1.2 × baseline), while EMA contributes minimal overhead. While the enhanced framework requires the most additional computational overhead (3× training time compared to baseline), the accuracy gains justify this cost for engineering applications requiring high precision.

The systematic ablation study validates our framework design philosophy, revealing that adaptive sampling (RAR-D) contributes the most significant performance improvement (52% error reduction), while weight stabilization (EMA) provides an essential training foundation and automated hyperparameter optimization delivers the final precision enhancement. Although the complete framework requires approximately three times the baseline training time, this one-time computational investment is justified by the substantial gains in accuracy and eliminates the need for manual hyperparameter tuning across different scenarios.

4.2. Analysis of Measurement Point Density Impact on Model Robustness

To further evaluate the feasibility of the proposed PINN framework in practical engineering applications, this section systematically reduces the number of random measurement points from 50 to 10, and then to 5, analyzing the performance of model predictions and the characteristics of error distribution under conditions of limited monitoring data. The visualization of randomly generated monitoring points is shown in Figure 8. To ensure experimental reproducibility, random seeds were employed for generation. Detailed information for each point can be found in Supplementary Table S1. The points are uniformly distributed throughout the computational domain, with no significant clustering or spatial bias observed. Statistical analysis further confirms this uniformity: the spatial distribution is most even with 50 points (variance = 0.3488). As the number of points decreases to five, the variance increases to 1.1631, indicating a gradual decline in spatial representativeness while overall uniformity is still maintained.

Under conditions of low measurement point density, the magnitude and distribution of relative errors are similar to those with 100 measurement points, with maximum values around $6 \times {10}^{-3}$. Figure 8 shows the distribution of absolute errors under varying configurations of measurement points. With 50 measurement points (Figure 9a), the model exhibits good prediction accuracy, achieving a maximum absolute error of 0.01661 and a relatively uniform spatial distribution of errors. Planar error plots show that regions in the vicinity of the tunnel maintain low error levels. Three-dimensional error plots display relatively smooth overall error terrain without significant error peaks, indicating that the model effectively captures the head distribution throughout the computational domain at this data density. Notably, after reducing the number of monitoring points by 80%, the 10-measurement-point configuration (Figure 9b) still maintains considerable prediction capability with a maximum absolute error of only 0.01309, even slightly lower than in the 50-point case. This phenomenon may stem from the RAR-D algorithm’s adaptive sampling capability, which partially compensates for insufficient measured data by dynamically adding collocation points in high residual regions. However, spatial error distributions begin to exhibit localized error clustering, particularly in areas distant from the tunnel, reflecting subtle shifts in the balance between physical constraints and data fidelity. When the number of monitoring points is reduced to five (Figure 9c), the model’s performance shows an obvious decline, with the maximum absolute error rising to 0.02557. The error distribution exhibits stronger spatial heterogeneity, forming obvious high-error bands in certain regions of the computational domain. Three-dimensional error plots reveal steeper error peaks, indicating that the model is unable to accurately describe head fields in certain regions when data is extremely limited.

As the number of measurement points decreases, the spatial distribution of errors gradually transitions from uniform dispersion to local clustering. In the case with only five measurement points, regions of high error are predominantly concentrated near the edges of the computational domain and in areas far from the tunnel, where physical constraints are weaker and the model relies more heavily on data constraints for learning. In contrast, the vicinity of the tunnel boundary maintains relatively low error levels due to the precise handling of boundaries by the ADF trial function and the RAR-D adaptive densification.

With only 10 monitoring points, the model’s accuracy is comparable to that achieved with 50 points, indicating high data efficiency of the proposed framework. This is of great significance for practical engineering applications, given that monitoring data is often scarce and costly to obtain in the early stages of tunnel construction.

Current seepage prediction methods suffer from fundamental limitations that restrict their practical applicability to complex tunnel engineering scenarios. Classical analytical solutions, while providing exact results under idealized conditions [8,9,10], are inherently constrained to simplified geometries and homogeneous media assumptions that rarely correspond to real-world geological complexity [37,38]. Traditional numerical methods (FEM/FDM), despite their widespread adoption in commercial software, require extensive mesh generation, complete material property specification, and face significant computational scaling challenges for large-scale tunnel systems [39,40]. Data-driven machine learning approaches, though effective at capturing complex nonlinear patterns, often lack physical consistency and exhibit limited generalization beyond their training data, rendering them unreliable for engineering applications with safety-critical requirements [41,42].

In contrast, physics-informed neural networks address these limitations by maintaining exact physical consistency through embedded governing equations, handling complex geometries without mesh dependency, and requiring minimal data input while leveraging physical constraints to enable robust extrapolation. Furthermore, they provide both accurate predictions and physical insights essential for engineering decision-making. The enhanced PINN framework presented in this study extends these advantages through systematic integration of multi-stage training, adaptive sampling, and automated hyperparameter optimization, positioning physics-informed machine learning as a transformative approach that bridges the gap between theoretical rigor and practical engineering applicability that has limited previous approaches.

5. Conclusions

This study addresses critical challenges in physics-informed neural network applications to tunnel seepage prediction, including training instability, boundary handling difficulties, and low sampling efficiency. We developed an enhanced PINN framework integrating Exponential Moving Average weight stabilization, Residual Adaptive Refinement with Distribution adaptive sampling, and Bayesian hyperparameter optimization to systematically overcome these limitations. A systematic ablation study confirmed that each component contributes distinct value: EMA ensures essential training stability, RAR-D delivers the most significant accuracy improvement, and Bayesian optimization achieves optimal performance despite increased variance. The complete framework achieves exceptional prediction accuracy with an overall average relative error of 5 × 10⁻⁵ and demonstrates remarkable data efficiency, maintaining acceptable accuracy even with only five monitoring points. Our Bayesian optimization analysis revealed that loss function weight configuration has a substantially greater impact on model performance than network architecture parameters, challenging conventional assumptions regarding neural network design in physics-constrained problems.

Certain limitations must be acknowledged. Current validation primarily focuses on two-dimensional steady-state problems under ideal homogeneous conditions, with significant challenges arising when extending to complex three-dimensional geometries and transient scenarios. Although Bayesian optimization comprehensively explores the sensitivity of loss function weights, it does not explore systematic analyses of sensitivity to geotechnical parameters such as varying permeability coefficients, tunnel geometries, and burial depths—representing a critical gap in practical engineering deployment. Furthermore, this paper lacks validation with field data, which faces numerous practical challenges: tunnel engineering field monitoring data is often constrained by limitations in spatio-temporal resolution, measurement accuracy issues, geological condition uncertainties, and commercial confidentiality restrictions. Once theoretical reliability is firmly established, field validation will become a core focus of future research.

The enhanced PINN framework demonstrates clear extensibility to more complex engineering scenarios through accommodation of heterogeneous media with spatially-varying permeability fields, anisotropic flow conditions via directional permeability tensors, and transient problems through temporal coordinate extensions. The framework can accommodate heterogeneous media by incorporating spatially varying permeability k (x, y) as learnable parameter fields, with the RAR-D algorithm’s adaptive sampling particularly valuable for capturing sharp material transitions. Anisotropic flow can be addressed by modifying the governing PDE to include directional permeability tensors while preserving the ADF trial function approach for complex geometries. Transient problems require extending the input space to include temporal coordinates and adapting the loss function for initial conditions, with the three-stage training strategy modified to handle temporal evolution patterns. Future research priorities include comprehensive geotechnical parameter sensitivity studies, development of transfer learning strategies for different site conditions, and systematic field validation under actual construction scenarios.

This research systematically addresses core barriers to PINN engineering applications, establishing physics-informed machine learning as a viable methodology that bridges theoretical rigor and engineering applicability. The methodological advances provide valuable guidance for applying physics-informed approaches to other engineering problems with well-defined physical constraints and complex boundary conditions.