A Physics-Informed Surrogate Model for the Bi-Flux Bevilacqua–Galeão Anomalous Diffusion Equation

Abstract

Accurate modeling of bi-flux anomalous diffusion presents significant computational challenges in engineering. This paper investigates the effectiveness of physics-informed neural networks as surrogate models for the bi-flux anomalous diffusion equation. We investigate one-dimensional linear and nonlinear cases. Optimal hyperparameter configurations are determined using a modified differential evolution algorithm, guided by an objective function that leverages a combination of loss values. This optimization approach enables a rigorous evaluation of different neural network setups, providing valuable insights and practical guidance for researchers working with bi-flux anomalous diffusion phenomena. A comparison between physics-informed neural networks and conventional multilayer perceptrons is presented for the analyzed model. Finally, the capability of the best-performing models to act as virtual sensors is evaluated. This work provides guidance on the use of neural networks to efficiently and accurately tackle complex bi-flux anomalous diffusion problems, potentially accelerating research and development in fields where such processes are critical.

1. Introduction

Many transport processes encountered in engineering and the applied sciences depart from the behavior predicted by classical diffusion theory. Phenomena such as contaminant migration in heterogeneous soils, mass transport through porous media, and particle dispersion in biological or anisotropic substrata frequently exhibit anomalous diffusion, in which the mean-squared displacement of the diffusing quantity does not grow linearly in time, giving rise to sub-diffusive or super-diffusive regimes [1]. Fick’s law, expressed as a second-order parabolic equation, cannot reproduce this class of behavior; in particular, it cannot represent the temporary retention of particles that characterizes many non-Fickian processes [2]. By modeling retention through a discrete formulation and taking the continuous limit, ref. [3] obtained an equation whose negative fourth-order term is essential to represent retention. This formulation was later consolidated analytically for reactive media [4] and extended into a bimodal-flux theory in which particles in distinct energy states follow the primary and secondary fluxes, yielding a generalized law for multi-flux processes [5]. We will refer to this model as the Bevilacqua–Galeão model (BG model).

A central modeling choice in the BG framework is the law governing the redistribution parameter $\beta$, which controls the qualitative behavior of the solution. A concentration-dependent (sigmoid) law was adopted by [6], while [7] through a second-moment analysis, showed that the model can reproduce both sub-diffusive and super-diffusive behavior depending on the parameters employed. Time-dependent exponential laws were explored by [2] from energy-transfer principles, revealing richer dynamics such as a hidden internal flux under state migration, while ref. [8] analyzed alternative secondary-flux definitions for anisotropic media and motivated the dependence of $\beta$ on the reactivity coefficient. These studies indicate that constant, spatially varying, and concentration-dependent forms of $\beta$ are all physically admissible, which motivates the parameter configurations considered here.

The identification of the BG model parameters has also been addressed through inverse analysis. Ref. [9] estimated the redistribution and retention parameters using sensitivity analysis to locate optimal measurements and Monte Carlo propagation to quantify uncertainty, an approach later extended with stochastic and Bayesian techniques, including Differential Evolution and the Markov Chain Monte Carlo method [10]. In this context, a modified Differential Evolution algorithm that contracts the search space as the objective function decreases was shown to lower the computational cost of estimation without a significant loss of accuracy [11]. Since the fourth-order operator makes the direct numerical solution costly, particularly when the equation must be solved repeatedly, these inverse studies further motivate the use of an efficient surrogate model.

In recent years, physics-informed neural networks (PINNs) have been increasingly employed to solve partial differential equations. If effective, they could provide a computationally efficient alternative to conventional numerical solvers such as finite element, finite difference, and finite volume methods. Along these lines, many authors have tested the usefulness of PINNs for partial differential equations [12,13,14] including heat conduction equations [15,16], and fourth-order partial differential equations [17,18]. While other studies have focused on improving neural network configurations [19,20,21,22] some studies have investigated the use of metaheuristics to optimize PINNs [23,24]. After carefully implementing PINNs for several well-known partial differential equations, ref. [25] concluded that PINNs have some drawbacks; for example, training can be time-consuming, and because the principles governing optimal network design are still not well established, hyperparameter tuning for stable and efficient convergence remains challenging. In that regard, ref. [26] proposed an automated framework for architecture and hyperparameter selection to enhance PINN performance, but with limited hyperparameters to be optimized.

Building on inverse-problem methodology, and following a strategy similar to that of [26], this work addresses this limitation by optimizing the architecture of a Physics-Informed Neural Network (PINN) designed to serve as an efficient surrogate model for the Bevilacqua–Galeão bi-flux equation. To achieve this, hyperparameter selection is formulated as an optimization problem within an inverse-analysis framework, as is common in many engineering applications [27]. A metaheuristic search strategy is used to identify viable architectures, with the metaheuristic acting as a feasible solution generator [28], and the PINN is trained using high-fidelity synthetic data obtained from the finite difference method.

The results not only highlight the potential of PINNs to reduce computational cost compared with traditional numerical methods but also provide practical guidelines for researchers seeking to leverage machine learning in the modeling of non-standard diffusion phenomena. To demonstrate the practical engineering utility of this optimized surrogate, it is employed to solve a complex inverse problem: estimating the heterogeneous physical properties and reconstructing the spatio-temporal concentration profile of a simulated contaminant spill. The main contributions of this work are:

(i)

A systematic, metaheuristic-driven evaluation of PINN hyperparameter configurations for the fourth-order bi-flux anomalous diffusion equation;

(ii)

A controlled comparison between physics-informed and purely data-driven surrogates sharing the same architecture, initialization, data, and training budget;

(iii)

The demonstration of the optimized surrogate in a practical engineering task, namely the estimation of heterogeneous-medium parameters and the reconstruction of the concentration of a simulated contaminant spill, with an approximately 52% reduction in recurring computational cost.

In Section 2, the main physical problem of interest, the bi-flux Bevilacqua–Galeão anomalous diffusion model, is described and its mathematical formulation is presented.

Section 3 presents the inverse problem methodology used to find the best hyperparameters, which includes a brief explanation of the differential evolution algorithm used.

In Section 4, the results of the study are presented and analyzed. Key findings are illustrated with figures and tables and are followed by a comprehensive discussion.

Finally, Section 5 presents concluding remarks on the results obtained in this research and indicates directions for future work.

2. Formulation of the Direct Problem

2.1. Bi-Flux Bevilacqua–Galeão Anomalous Diffusion Model

One commonly used mathematical equation to represent the balance of mass in a fixed control volume can be written as

$${\displaystyle \frac{\mathsf{\partial }}{\mathsf{\partial }t}}\underset{cv}{\iiint }C(\overrightarrow{x},t)dV=-\underset{cs}{∯}\Phi ·\overrightarrow{n}dS,$$

(1)

where C is the particle concentration at position $\overrightarrow{x}$ and time t, V is the volume, $cv$ is the control volume being analyzed, $\Phi$ is the flux of particles, $\overrightarrow{n}$ is the unit normal to the control surface (positive outwards), S is the surface area and, finally, $cs$ denotes the control surface.

The key point in the bi-flux anomalous diffusion theory is the hypothesis of a composite flux, which combines Fick’s classical diffusion flux and a secondary flux, proportional to the first but in the opposite direction. Here, we define this composite flux as

$$\Phi =-\beta (\overrightarrow{x},t){\Phi }_F+\left(1-\beta (\overrightarrow{x},t)\right){\Phi }_S,$$

(2)

where ${\Phi }_F$ is the Fickian flux of diffusion, defined as

$${\Phi }_F=D\overrightarrow{\nabla }c,$$

and ${\Phi }_S$ is the secondary flux, given by

$${\Phi }_S=\beta (\overrightarrow{x},t)R\overrightarrow{\nabla }(\nabla ·\left(d\overrightarrow{\nabla }c\right)),$$

where $\beta$ represents the fraction of particles that spread according to the classical diffusion formulation, D is Fick’s diffusion coefficient, d is a proportional coefficient, and R is the reactivity coefficient as described in [5,8].

When using Equation (2) with the Gauss divergence theorem applied to Equation (1), together with some algebraic manipulation, we obtain a general 1D bi-flux anomalous diffusion equation, written here as

$${\displaystyle \frac{\mathsf{\partial }c}{\mathsf{\partial }t}}=A_1{\displaystyle \frac{\mathsf{\partial }c}{\mathsf{\partial }x}}+A_2{\displaystyle \frac{{\mathsf{\partial }}^{2}c}{\mathsf{\partial }{x}^2}}+A_3{\displaystyle \frac{{\mathsf{\partial }}^{3}c}{\mathsf{\partial }{x}^3}}+A_4{\displaystyle \frac{{\mathsf{\partial }}^{4}c}{\mathsf{\partial }{x}^4}},$$

(3)

where $A_1$, $A_2$, $A_3$ and $A_4$ are functions that depend on the nature of $\beta$, D, d and R.

An analytical solution for Equation (3) depends strongly on the assumptions adopted and is beyond the scope of the present work. Equation (3) requires four boundary conditions and one initial condition. The first set of boundary conditions used in this study was defined as

$$c(x=0,t)=0,$$

and

$$c(x=L,t)=0,$$

while the second set can be written as

$${\displaystyle \frac{\mathsf{\partial }c}{\mathsf{\partial }x}}{|}_{x=0}=0,$$

and

$${\displaystyle \frac{\mathsf{\partial }c}{\mathsf{\partial }x}}{|}_{x=L}=0,$$

with the following initial condition

$$c(x,t=0)=sin\left(\pi {\displaystyle \frac{x}{L}}\right).$$

To better investigate the PINN for the Bevilacqua–Galeão bi-flux anomalous diffusion phenomenon, three distinct parameter configurations were evaluated. The first case assumed constant coefficients, whereas the second postulated a spatial logarithmic dependence for the parameter $\beta$ and the third assumed a dependence of the concentration on the parameter $\beta$. Despite these constitutive differences, the same initial and generalized boundary conditions were imposed across all three configurations. The aim was to assess the behavior of the PINN when dealing with different governing partial differential equations derived from the same underlying bi-flux theory while trying to cover the main scenarios presented in the literature for the BG model.

2.1.1. The Constant Redistribution

By treating $\beta$, D, d and R as constants, the medium is modeled as homogeneous and time-invariant, meaning the diffusion behavior is decoupled from specific spatial or temporal coordinates.

Consequently, this formulation with the representation adopted in this study, leads to

$$A_1=0,$$

$$A_2=\beta D,$$

$$A_3=0,$$

and

$$A_4=-\beta (1-\beta )Rd,$$

which, when substituted in Equation (3), results in

$${\displaystyle \frac{\mathsf{\partial }c}{\mathsf{\partial }t}}=\beta D{\displaystyle \frac{{\mathsf{\partial }}^{2}c}{\mathsf{\partial }{x}^2}}-\beta (1-\beta )Rd{\displaystyle \frac{{\mathsf{\partial }}^{4}c}{\mathsf{\partial }{x}^4}}.$$

2.1.2. The Logarithmic Law for the Redistribution

One method to characterize the redistribution behavior as a function of the domain, is to postulate the parameter $\beta$ as a logarithmic function of the spatial coordinate x. This spatial dependence is formulated as

$$\beta \left(x\right)=log(x+1),$$

which leads to

$$A_1\left(x\right)=D{\displaystyle \frac{d\beta \left(x\right)}{dx}},$$

$$A_2\left(x\right)=D\beta \left(x\right),$$

$$A_3\left(x\right)=Rd(1-2\beta \left(x\right)){\displaystyle \frac{d\beta \left(x\right)}{dx}},$$

and

$$A_4\left(x\right)=Rd\beta \left(x\right)(1-\beta \left(x\right)).$$

2.1.3. The Sigmoid Law for the Redistribution

This dependence of $\beta$ on the concentration has long been motivated by the authors of the model, but was later analyzed in greater detail in [7], the idea is to define

$$\beta \left(c\right)={\beta }_{max}-{\displaystyle \frac{{\beta }_{max}-{\beta }_{min}}{1+e^{-\gamma (c-c_0)}}},$$

where $c=c(x,t)$. This results in

$$A_1\left(x\right)=D{\displaystyle \frac{\mathsf{\partial }\beta \left(c\right)}{\mathsf{\partial }x}},$$

$$A_2\left(x\right)=D\beta \left(c\right),$$

$$A_3\left(x\right)=Rd(1-2\beta \left(c\right)){\displaystyle \frac{\mathsf{\partial }\beta \left(c\right)}{\mathsf{\partial }x}},$$

and

$$A_4\left(x\right)=Rd\beta \left(c\right)(1-\beta \left(c\right)).$$

2.2. Numerical Solution

The finite difference method requires the definition of a spatial and temporal discretization grid. For the one-dimensional case considered in this study, the domain discretization was analyzed in both space and time, in order to select a grid that is sufficiently small for the purpose of this study.

Reference [29] conducted an important analysis on the truncation order error while choosing the approximation for the derivatives, the results guided the selection of the following terms for the derivatives, starting with the first time derivative Forward-Time Centered-Space implicit approach

$${\displaystyle \frac{\mathsf{\partial }c(x,t)}{\mathsf{\partial }t}}\approx {\displaystyle \frac{{\varphi }_i^{n+1}-{\varphi }_i^n}{\Delta t}},O\left[(\Delta t)\right]$$

where ${\varphi }_i^n$ is the concentration at the i-th spatial node and the n-th time node, i.e., ${\varphi }_i^n=c(i\Delta x,n\Delta t)$. For the spatial derivatives, the scheme used is given as follows

$$\begin{array}{ccc}\hfill & {\displaystyle \frac{\mathsf{\partial }c(x,t)}{\mathsf{\partial }x}}\approx {\displaystyle \frac{{\varphi }_{i-2}^{n+1}-8{\varphi }_{i-1}^{n+1}+8{\varphi }_{i+1}^{n+1}-{\varphi }_{i+2}^{n+1}}{12\Delta x}},\hfill & \hfill O\left[{(\Delta x)}^4\right]\\ \hfill & {\displaystyle \frac{{\mathsf{\partial }}^{2}c(x,t)}{\mathsf{\partial }{x}^2}}\approx {\displaystyle \frac{-{\varphi }_{i-2}^{n+1}+16{\varphi }_{i-1}^{n+1}-30{\varphi }_i^{n+1}+16{\varphi }_{i+1}^{n+1}-{\varphi }_{i+2}^{n+1}}{12\Delta x^2}},\hfill & \hfill O\left[{(\Delta x)}^4\right]\\ \hfill & \hfill & \hfill \\ \hfill & {\displaystyle \frac{{\mathsf{\partial }}^{3}c(x,t)}{\mathsf{\partial }{x}^3}}\approx {\displaystyle \frac{-{\varphi }_{i-2}^{n+1}+2{\varphi }_{i-1}^{n+1}-2{\varphi }_{i+1}^{n+1}+{\varphi }_{i+2}^{n+1}}{2\Delta x^3}},\hfill & \hfill O\left[{(\Delta x)}^4\right]\\ \hfill & \hfill & \hfill \\ \hfill & {\displaystyle \frac{{\mathsf{\partial }}^{4}c(x,t)}{\mathsf{\partial }{x}^4}}\approx {\displaystyle \frac{{\varphi }_{i-2}^{n+1}-4{\varphi }_{i-1}^{n+1}+6{\varphi }_i^{n+1}-4{\varphi }_{i+1}^{n+1}+{\varphi }_{i+2}^{n+1}}{\Delta x^4}},\hfill & \hfill O\left[{(\Delta x)}^2\right]\end{array}$$

The resulting linear system was solved using Gaussian elimination with back substitution.

2.3. The Neural Network Configuration

To establish a robust surrogate model capable of capturing the complex redistribution dynamics of the Bevilacqua–Galeão bi-flux formalism, a Physics-Informed Neural Network (PINN) architecture was implemented based on the approach proposed by [30]. The fundamental topology consists of a Multilayer Perceptron (MLP) with six input variables: the position x, the time t, $A_1$, $A_2$, $A_3$, and $A_4$. The output layer has one node, representing the concentration $c(x,t)$ for the given parameters $A_1,A_2,A_3,A_4$.

However, the internal topology, specifically the hyperparameters and activation mechanisms, was not fixed a priori. Instead, this work proposes an optimization framework for identifying suitable network topologies for bi-flux anomalous diffusion.

The search space $\mathcal{S}$ is defined by the lower and upper bounds of these parameters, as detailed in

Table 1. Hyperparameter search space boundaries.

Hyperparameter	Lower Bound	Upper Bound
Hidden Layers	1	15
Neurons per Layer (Layer Width)	1	512
Learning Rate	${10}^{-5}$	$1.0$
Epochs	10	30,000
Batch Size	2	32,768
Optimizer	0	9
Activation Functions	0	16

. The optimization is driven by a Differential Evolution (DE) metaheuristic, which iteratively evolves the population of network configurations to identify the parameters within the search space $\mathcal{S}$ that minimize the objective function $\mathcal{J}$.

As can be seen from Table 1, this framework integrates a comprehensive dictionary of 17 activation functions, ranging from standard ReLU to functions such as Mish, GELU, and Swish (SiLU). This allows the evolutionary algorithm to identify activation functions capable of accurately representing the bi-flux dynamics.

Figure 1 illustrates the basic neural network structure used.

The training process for each candidate configuration utilizes the Adam optimizer, subject to an early stopping mechanism monitoring the validation loss with a patience parameter equal to 10% of the maximum number of epochs. The performance metric used to guide the evolutionary selection process is the sum of the training and validation losses, ensuring the model generalizes well to unseen data within the Bevilacqua–Galeão framework.

The Artificial Neural Network loss function was customized to account for the partial differential equation; this loss is presented here as

$${\mathcal{L}}_{PINN}={\displaystyle \frac{1}{N}}{\displaystyle \sum _j^N}\left|\right|f\left(x_j\right|\theta )-y_j{\left|\right|}_2^2+{\displaystyle \frac{1}{M}}{\displaystyle \sum _i^M}\left|\right|g(x_i,f\left(x_i\right|\theta )){\left|\right|}_2^2,$$

(4)

where $f\left(x_j\right|\theta )$ is the neural network being evaluated that approximates $c(x_j,t_j)$ and g represents the sum of losses at collocation points plus boundary and initial conditions, that is

$$g(x_i,f\left(x_i\right|\theta ))=\lambda \left({\mathcal{L}}_{\mathrm{pde},\mathrm{i}}+{\mathcal{L}}_{\mathrm{bc},\mathrm{i}}+{\mathcal{L}}_{\mathrm{ic},\mathrm{i}}\right).$$

(5)

All derivatives for ${\mathcal{L}}_{\mathrm{pde}}$ are obtained by automatic differentiation of the network output at the i-th collocation point. The boundary term enforces the two boundary-condition sets, and the initial term enforces $c(x,0)=sin(\pi x/L)$. The PDE residual was evaluated at $8096$ collocation points per epoch, sampled uniformly over the space–time domain, no additional weighting was applied beyond the factor $\lambda$.

Likewise, to isolate and assess the specific advantages of physical regularization, a MLP model is evaluated alongside the PINN. This is achieved methodologically by setting the physics-residual term (g) to zero by defining $\lambda =0$ in the source code, thereby stripping the network architecture of all physical information.

3. Formulation of the Inverse Problem

3.1. Hyperparameter Estimation

In this work, the inverse problem of parameter estimation is formulated as an optimization task. To determine the optimal PINN configuration, we minimize an objective function (${\mathcal{J}}_1$) that quantifies the discrepancy between the model predictions and the reference data. To account for both optimization performance and generalization capability, ${\mathcal{J}}_1$ is defined as the arithmetic mean of the training loss ($T_L$) and the validation loss ($V_L$):

$${\mathcal{J}}_1(T_L,V_L)={\displaystyle \frac{T_L+V_L}{2}}.$$

(6)

Because both the training and validation losses are highly sensitive to the model’s configuration, minimizing ${\mathcal{J}}_1$ effectively translates into a hyperparameter optimization problem. The specific hyperparameters selected for this tuning process are detailed in Table 1.

3.2. Heterogeneous Medium Parameter Estimation

Once the optimal PINN architecture is established, it can be used as a fast forward solver for practical engineering problems. In this study, we apply the surrogate to an inverse parameter-estimation problem in a heterogeneous medium. The objective function ${\mathcal{J}}_2$ was formulated as the residual sum of squares between the measured concentration profiles and the trial values

$${\mathcal{J}}_2(C_{exp},C_{trial})={\displaystyle \sum _{i=0}^N}{(C_{exp,i}-C_{trial,i})}^2,$$

(7)

where N is the number of available data points. In this particular case, the reference data ($C_{exp}$) was synthetically generated using the finite difference method. To simulate low-noise experimental conditions, Gaussian noise with zero mean and a standard deviation of 0.01 was added to this numerical solution. Conversely, the predicted concentrations ($C_{trial}$) were evaluated using the PINN surrogate model based on the candidate parameters $A_1\left(x\right)$, $A_2\left(x\right)$, $A_3\left(x\right)$, and $A_4\left(x\right)$. This strategy was used to reduce the computational cost of the optimization process.

3.3. Normalized Sensitivity Coefficient

The sensitivity coefficient ($X_{A_i}$) is defined in terms of the first-order partial derivative of the observable variable, c, with respect to the parameter of interest, $A_i$. The normalized sensitivity coefficient is used and is defined as

$$X_{A_i}=A_i{\displaystyle \frac{\mathsf{\partial }\mathbf{c}}{\mathsf{\partial }{A}_i}},i=1,2,\dots ,4.$$

(8)

For reliable parameter estimation, the system under consideration must be sufficiently sensitive to the parameters being estimated. In other words, small variations in a given parameter should produce measurable changes in the observable variable, allowing the influence of that parameter on the system response to be assessed. This analysis also indicates which positions and response intervals provide the most favorable conditions for parameter estimation and which parameters can be estimated simultaneously [27]. When two or more parameters are estimated at the same time, their sensitivity coefficients must be linearly independent; otherwise, the inverse problem may not have a unique solution [31].

3.4. Optimization Strategy

Proposed by Storn and Price [32], the Differential Evolution is a heuristic optimization method based on vector operations in which the weighted difference between two vectors is added to a third vector to generate candidate solutions. We summarize our implemented version of this method as follows:

• Generation of an initial random population

  $$q_{k,j}^{t=0}=q_{L,k}+{rand}_{k,j}(q_{U,k}-q_{L,k}),j=1,2,\dots ,N_{pop}$$

  where $q_{U,k}$ and $q_{L,k}$ are the upper and lower bounds of the k-th variable, $ran{d}_{k,j}$ is a random number between 0 and 1, and $N_{pop}$ is the size of the population.
• Mutation operation in order to generate a candidate

  $${\overrightarrow{v_l}}^t={\overrightarrow{q}}_{r0}^t+F({\overrightarrow{q}}_{r1}^t-{\overrightarrow{q}}_{r2}^t)$$

  where F is a perturbation factor, and the vectors ${\overrightarrow{q}}_{r0}^t$, ${\overrightarrow{q}}_{r1}^t$ and ${\overrightarrow{q}}_{r2}^t$ are randomly chosen from within the population and must be distinct from each other, if ${\overrightarrow{q}}_{r0}^t$,${\overrightarrow{q}}_{r1}^t$ or ${\overrightarrow{q}}_{r2}^t$ sizes differ, then a random number is sampled from a uniform distribution between 0 and 1 and if this random number is bigger than the probability of adding an element $p_a=0.6$, the element missing is added into the small vector, this process is repeated until all vectors have the same size for the mutation operation to occur, these changes to the vector are made on a copy of the original vector, not changing the old vector.
• The next step is the crossover operation where the generated vector can be accepted or not depending on the criterion

  $${\overrightarrow{q}}_l^{t+1}=\left\{\begin{array}{c}{\overrightarrow{v_l}}^t,\mathrm{if}CR\le {rand}_{k,l}\hfill \\ {\overrightarrow{q}}_l^t,\mathrm{otherwise}\hfill \end{array}\right.$$

  where $CR$ is the crossover probability.
• Finally, if the new vector ${\overrightarrow{v_l}}^t$ provides a better value for the objective function than vector ${\overrightarrow{q}}_l^t$, the latter is replaced by the former in the next generation, otherwise ${\overrightarrow{q}}_l^t$ remains in the population for one more generation
• Repeat steps 2–4 until a predefined maximum number of generations is achieved.

Each individual (referred to as a gene) is a structure whose elements represent the hyperparameters, according to Table 1.

The idea is that each generation refines the population to converge towards an optimal configuration. After the last generation is achieved, the best-performing individual (the one with the lowest objective function value) is selected as the optimal architecture. The search-space bounds in Table 1 were deliberately broad. In particular, the learning-rate upper bound of 1.0 was chosen so that the metaheuristic itself, rather than prior assumptions, would discard divergent configurations: candidates with excessively large learning rates fail to reduce the validation loss and are naturally eliminated by the selection step, at a modest computational cost. The stopping criterion was fixed to 20 generations with a population of 10 individuals, and each (CR, F) setting was repeated four times under independent random seeds to account for the stochastic nature of the algorithm.

Because the presented algorithm requires numerical inputs, categorical parameters like activation functions were mapped to discrete integer values (

Table 2. Mapping of indices to activation functions in the neural network hyperparameter optimization code.

Index	Activation Function
0	sigmoid
1	relu
2	softmax
3	linear
4	tanh
5	softplus
6	softsign
7	elu
8	selu
9	exponential
10	leaky_relu
11	relu6
12	silu
13	hard_silu
14	mish
15	gelu
16	log_softmax

). This integer encoding allowed the algorithm to seamlessly apply mathematical operations during the mutation and crossover phases.

3.5. Training Data

To train the model, a comprehensive dataset comprising the input vector $(x,t,A_1,A_2,$ $A_3,A_4)$ is required. The data generation process employed a hierarchical sampling strategy, beginning with the establishment of bounding intervals for the fundamental physical parameters. Specifically, the parameter spaces were defined as $R\in [5\times {10}^{-5},2\times {10}^{-4}]$, $d\in [0.005,0.02]$, and $D\in [0.005,0.02]$. Samples for these parameters were then drawn from a uniform distribution within the defined limits. Subsequently, these values were substituted into the governing equations to compute the corresponding derived coefficients $A_1$ through $A_4$, the parameters were then tested to ensure they generated a physically feasible solution (mainly a positivity test) and the accepted dataset was partitioned using a 70/15/15 split for training, validation, and testing, respectively.

4. Results

4.1. Numerical Validation and Verification

As can be seen from Figure 2 the error falls rapidly and levels off for $N_x\gtrsim 700$, allowing us to define $N_x=1101$ with confidence. The same behaviour can be seen for temporal mesh-refinement of the Figure 3 relative $L_2$ error norm as a function of the number of temporal nodes $N_t$, with the spatial grid held fixed. The error decreases monotonically and saturates for $N_t\gtrsim 1000$, which led us to use $N_t=1501$.

4.2. Surrogate Model

All experiments were executed on a Linux-based machine with an AMD Ryzen 9 9950X CPU, an NVIDIA RTX 3060 GPU, and 192 GB of Kingston DDR5 RAM (Fountain Valley, CA, USA). Each differential-evolution search used a population of 10 evolved over 20 generations and was repeated four times for every $(CR,F)$ setting with independent random seeds.

The optimization results are detailed in

Table 3. Table of best genes found for different parameter settings with $g=0$.

			${\mathcal{J}}_1({\mathit{T}}_{\mathit{L}},{\mathit{V}}_{\mathit{L}})$	Hidden Layers	Layer Widths	Learning Rate	Activations	Epochs	Batch Size
CR = 0.5	F = 0.8	#1	0.0118	2	[63, 98]	0.0086	[3, 5]	24,600	3112
		#2	0.0215	1	[36]	0.0069	[3]	15,437	390
		#3	0.0230	1	[173]	0.0073	[3]	23,358	4177
		#4	0.0119	1	[512]	0.0061	[3]	24,749	2639
	F = 0.4	#1	0.0150	2	[8, 3]	0.0127	[5, 3]	25,188	64
		#2	0.0085	2	[74, 109]	0.0089	[11, 8]	23,566	3183
		#3	0.0099	1	[154]	0.0046	[3]	24,485	544
		#4	0.0178	1	[101]	0.0096	[5]	22,718	1157
CR = 0.7	F = 0.8	#1	0.0230	2	[64, 41]	0.5575	[15, 10]	14,910	1617
		#2	0.0157	1	[89]	0.1597	[1]	24,761	1427
		#3	0.0089	2	[36, 117]	0.5295	[7, 3]	23,755	7518
		#4	0.0191	1	[32]	0.1512	[3]	24,210	573
	F = 0.4	#1	0.0205	3	[225, 132, 82]	0.0162	[1, 0, 9]	18,947	3298
		#2	0.0153	1	[122]	0.2593	[3]	14,213	10,101
		#3	0.0086	1	[253]	0.0514	[7]	29,663	256
		#4	0.0165	1	[32]	0.3175	[10]	17,953	728

for the MLP approach with different values of the crossover rate and mutation factor and

Table 4. Table of best genes found for different parameter settings.

			${\mathcal{J}}_1({\mathit{T}}_{\mathit{L}},{\mathit{V}}_{\mathit{L}})$	Hidden Layers	Layer Widths	Learning Rate	Activations	Epochs	Batch Size
CR = 0.5	F = 0.8	#1	0.033	10	[193, 334, 288, 248, 32, 512, 32, 140, 398, 290]	0.0029	[4, 4, 1, 12, 4, 12, 4, 12, 4, 4]	30,000	3681
		#2	0.015	6	[136, 32, 32, 147, 202, 155]	0.0146	[4, 4, 12, 4, 4, 15]	19,771	3468
		#3	0.008	9	[364, 32, 263, 99, 32, 490, 32, 32, 512]	0.0006	[4, 4, 15, 12, 4, 15, 15, 1, 12]	13,640	512
		#4	0.007	9	[86, 32, 32, 296, 508, 32, 257, 32, 475]	0.0011	[4, 1, 4, 1, 12, 4, 15, 4, 12]	16,504	2694
	F = 0.4	#1	0.003	4	[147, 32, 32, 239]	0.0243	[12, 15, 1, 12]	1000	570
		#2	0.039	3	[383, 121, 268]	0.0001	[4, 4, 4]	18,977	7249
		#3	0.018	8	[367, 385, 69, 59, 176, 309, 330, 387]	0.0044	[12, 1, 15, 4, 12, 4, 1, 12]	20,069	14,910
		#4	0.020	3	[337, 417, 125]	0.0125	[12, 1, 15]	26,681	9018
CR = 0.7	F = 0.8	#1	0.010	7	[163, 497, 118, 153, 194, 495, 213]	0.0016	[12, 12, 12, 12, 15, 1, 4]	26,113	512
		#2	0.023	2	[321, 143]	0.0068	[15, 4]	16,864	8987
		#3	0.002	3	[335, 32, 32]	0.0037	[12, 15, 12]	25,354	9283
		#4	0.063	7	[163, 380, 113, 143, 194, 495, 238]	0.0016	[12, 15, 12, 12, 4, 4, 1]	27,025	1861
	F = 0.4	#1	0.020	6	[512, 144, 170, 512, 387, 309]	0.0007	[12, 4, 12, 12, 1, 12]	14,467	4186
		#2	0.015	4	[318, 90, 301, 276]	0.0007	[4, 4, 1, 15]	23,099	7880
		#3	0.004	4	[231, 132, 378, 151]	0.0030	[4, 4, 1, 15]	16,931	9678
		#4	0.003	7	[512, 32, 344, 124, 369, 452, 32]	0.0021	[4, 1, 15, 12, 12, 1, 12]	25,554	15,259

for the PINN approach.

Surprisingly, from Table 3 the Differential Evolution (DE) algorithm consistently favored a simple single hidden-layer architecture over more complex neural network topologies. While this streamlined MLP proved highly efficient, reducing computational costs by 80% for the concentration simulation at t = 4 in comparison with the finite difference method, while maintaining acceptable validation accuracy, its predictive capability was confined to the interpolation domain.

The subsequent optimization results for the PINN framework are summarized in Table 4 and it shows that the evolutionary search favored smooth activation functions: tanh dominated the global distribution.

Figure 4 presents the training history of the best PINN configuration, showing that the loss and each of its four components, the data loss, the PDE residual, and the boundary and initial-condition terms, decrease together over training and settle after roughly ${10}^4$ epochs.

To compare the two approaches under controlled conditions, the best architecture identified by the Differential Evolution search was trained twice using identical settings, namely the same initialization, data, optimizer, and number of epochs. The two runs differed only in the physics weight, with $\lambda =1$ for the PINN and $\lambda =0$ for the purely data-driven MLP.

Figure 5 compares the predictive capabilities of the optimized models when evaluated on input data that was not part of the training data, more specifically the values of R and D were 10% higher than the upper boundary. As the input parameters deviate further from the training bounds, the purely data-driven predictions progressively fail to capture the fundamental physical characteristics of the modeled phenomenon.

To quantify this behavior,

Table 5. RMSE results of best MLP and PINN architectures in two scenarios: Interpolation and Extrapolation.

Model	Regime	Data Size	RMSE
Interpolation	PINN	449,764	$1.2334\times {10}^{-3}$
	MLP	449,764	$1.0806\times {10}^{-3}$
Extrapolation	PINN	449,764	$1.1775\times {10}^{-2}$
	MLP	449,764	$2.9694\times {10}^{-1}$

reports the RMSE of both models with respect to the solver reference, evaluated on the test set (interpolation) and on the extrapolation scenarios. In interpolation, the two models achieve comparable accuracy, with the MLP attaining a slightly lower RMSE than the PINN, an expected consequence of the physics residual acting as a regularizer, which can marginally penalize the in-distribution fit in exchange for better generalization. In extrapolation, however, the RMSE of the MLP increases by over two orders of magnitude, whereas that of the PINN increases by about tenfold.

4.3. Training Stability and Robustness

To evaluate the training stability and robustness of the chosen architecture, the best configuration was probed along two further axes.

Initialization: Retraining under several random seeds produced final errors with a small standard deviation. The relative $L_2$ error varied by under $10\%$ of its mean across seeds, so convergence was reproducible and not the result of a fortunate initialization.

Noise: Adding zero-mean Gaussian noise to the training targets, with standard deviation up to $0.05$, changed the RMSE of the solution only marginally, while a standard deviation greater than $0.1$ led to the non-convergence of the PDE loss.

4.4. Sensitivity Analysis

Figure 6 presents the sensitivity coefficient as a function of position x at several time instants for each coefficient of the BG model.

The low sensitivity to $A_3$ and $A_4$ anticipates that these coefficients are intrinsically harder to recover from concentration data but also indicates the regions where sensors could be placed such as the regions of higher sensitivity.

4.5. Parameter Estimation of a Heterogeneous Medium

The true advantage of a surrogate model becomes evident in scenarios requiring repeated evaluations of Equation (3), such as inverse problems of parameter estimation. As outlined in the previous section, we applied the optimized surrogate to estimate the heterogeneous properties of a medium following a contaminant spill. Our primary objective is to identify the underlying spatial parameters necessary to accurately replicate this transport phenomenon.

For this application, the true heterogeneous medium was simulated assuming a logarithmic redistribution behavior, with $D=0.05$, $R=0.001$, and $d=1$, yielding the reference values for $A_1\left(x\right)$ through $A_4\left(x\right)$. The optimization results, obtained after 30 executions of the Differential Evolution algorithm minimizing ${\mathcal{J}}_2$, are illustrated in Figure 7. The figure compares the algorithm’s mean estimated profiles against the true values for $A_1\left(x\right)$, $A_2\left(x\right)$, $A_3\left(x\right)$, and $A_4\left(x\right)$, together with 95% confidence intervals representing the variability obtained across the 30 executions of the Differential Evolution algorithm.

Notably, the spatial regions adjacent to the domain edges exhibit negligible sensitivity to these parameters, consistent with the sensitivity analysis results in Figure 6. This localized lack of sensitivity is primarily driven by the enforced fixed boundary conditions, and the spatial extent of this boundary influence is clearly visible in the confidence bands shown. The execution time of the optimization loop was also recorded. Using the traditional FDM as the solver, a complete execution of the Differential Evolution algorithm required approximately $1319.11$ s to converge. In contrast, replacing the FDM with the optimized PINN surrogate reduced the runtime of the same optimization process to $635.01$ s. For a fair assessment of this comparison, the one-off training cost of the surrogate must also be considered. Training the best PINN configuration required approximately 3700 s on the hardware. Taking this result in consideration, the surrogate model becomes advantageous whenever more than 5 DE executions are required; in the present study, the 30 executions used to construct the confidence intervals amortize the training cost by a wide margin.

4.6. Reconstruction of the Concentration Curve

In scenarios where the availability of concentration sensors is limited, it is crucial that these sensors are strategically placed in regions where the model exhibits high sensitivity to the target parameters. Although these optimal locations can be identified through sensitivity analysis of Equation (3), as demonstrated by [9], sparse physical data still leave gaps in spatial observation. To address this limitation, the proposed surrogate model serves as a computationally efficient virtual sensor. It provides a low-cost method for reconstructing the full concentration curve in regions lacking physical instrumentation. The mean parameter values estimated via the inverse problem were subsequently used to simulate the time evolution of the concentration profile. Figure 8 compares the concentration generated using the true parameters ($C_{exp}$) with that generated using the estimated parameters ($C_{predicted}$), alongside the absolute error between the two.

5. Concluding Remarks

This work investigated the optimization of Physics-Informed Neural Network architectures for solving the bi-flux Bevilacqua–Galeão anomalous diffusion equation. The Differential Evolution algorithm proved effective in navigating the complex hyperparameter search space, consistently identifying configurations that achieved objective-function values $J_1$ in the range ${10}^{-3}$–${10}^{-2}$. A particularly noteworthy finding is that for the purely data-driven MLP, shallow architectures with one or two hidden layers dominated; for the PINN, the top-performing configurations concentrated between 2 and 7 hidden layers, although competitive solutions with up to 10 layers were also found (Table 4).

However, when tested on out-of-distribution parameters, the purely data-driven did not deliver a result as good as the PINN, demonstrating an inability to extrapolate. This structural limitation indicates the need to embed physical knowledge into the learning process, thereby motivating the transition to a Physics-Informed Neural Network.

In the physics-informed approach, the evolutionary search consistently favored smooth activation functions, such as hyperbolic tangent (tanh), with the best architectures ranging from 2 to 10 hidden layers. A key observation from this optimization process is the direct correlation between physical complexity and network topology. When generalizing across varying redistribution laws, the model required deeper, more complex configurations to capture the underlying physics; conversely, scenarios with constant parameters allowed for simpler architectures. We anticipate that these insights will provide practical, data-driven guidance for researchers developing PINN-based surrogate models for bi-flux anomalous diffusion phenomena.

Furthermore, the practical utility of the optimized PINN was successfully demonstrated through an engineering inverse problem. By using the surrogate as a fast solver, the Differential Evolution algorithm accurately estimated the spatially varying parameters of a heterogeneous medium and successfully reconstructed the complete concentration of a contaminant spill with an acceptable error margin. Notably, using the surrogate model for this task reduced computational time by approximately 52%.

Future work should extend this analysis to two-dimensional domains and investigate the surrogate’s performance under varying physical parameters beyond the training regime. The incorporation of uncertainty quantification mechanisms and the exploration of learning strategies for adapting the optimized architectures to related diffusion phenomena represent promising directions for advancing this research.