Investigating the Number of Monte Carlo Simulations for Statistically Stationary Model Outputs

Abstract

The number of random fields required to capture the spatial variability of soil properties and their impact on the performance of geotechnical systems is often varied. However, the number of random fields required to obtain higher-order statistical moments of model outputs has not yet been studied. This research aims to investigate the number of Monte Carlo simulations needed to achieve stationary higher-order statistics of a pore pressure head in an unsaturated soil slope under steady-state infiltration. The study recommends using at least 500 Monte Carlo samples for the probabilistic analysis of geotechnical engineering models. A more conservative choice for up to second-moment analysis is 1000 samples. The analysis reveals significant variations in skewness, which become stationary for all mesh grids when the number of samples exceeds 15,000. Kurtosis stabilizes only when the number of samples reaches 25,000. The pore pressure head in the unsaturated zone is less uncertain. Additionally, the probability density function of the pore pressure head follows a leptokurtic distribution.

1. Introduction

Probabilistic analysis is a very important topic in engineering modelling [1,2,3]. Geotechnical engineering deals with the behavior and properties of soil and rocks, which are inherently complex and variable materials [4,5,6,7]. Therefore, there is always uncertainty associated with geotechnical engineering analyses and design. By understanding and quantifying these uncertainties and the reliability index, geotechnical engineers can make more informed decisions about design, construction, and risk management.

Geotechnical engineering problems are often complex and involve non-linear relationships between inputs and outputs, making it difficult to obtain explicit relationships or closed-form solutions (e.g., [8,9,10,11]). Therefore, numerical methods, such as finite element or finite difference methods, are commonly used to solve geotechnical engineering problems. Because of the complexity and non-linearity of geotechnical engineering problems, it is often difficult to directly compute the uncertainty and reliability of model outputs. Monte-Carlo-based probabilistic analysis is, therefore, widely used to evaluate the probability of failure or performance of geotechnical structures and systems under uncertain conditions (e.g., [12,13,14]). The Monte Carlo simulation is a computational technique that uses random sampling and probability distributions to simulate the potential outcomes of a complex system or process. In a Monte Carlo simulation, a large number of samples are generated from probability distributions of the uncertain inputs, and the model is evaluated for each set of input samples. The resulting output samples are used to estimate the probability distribution of the model output and the probability of failure or success of the system. This approach allows for the quantification of uncertainty and risk in geotechnical engineering problems, even when explicit relationships between inputs and outputs are difficult to obtain [15].

The convergence of the Monte Carlo method implies obtaining a reasonably accurate estimator. This convergence is guaranteed by the law of large numbers, which indicates that repeated sampling in the Monte Carlo method will result in the average outcome converging towards the expected value. There are currently two widely used approaches to assess the level of convergence.

The first approach is based on the strong law of large numbers and central limit theorem. The formula provided by Fenton and Griffiths [16] to determine the minimum number of simulations needed to achieve a given estimation error with a confidence level (1 − α) is

$$n={\left(\frac{z_{\alpha /2}\sigma }{e}\right)}^2$$

(1)

where z_α/2 is the value of the standard normal variable with a cumulative probability level (1 − α/2), σ is the standard deviation of the mode output for uncertainty analysis, and e is the desired estimation error. For example, if an estimation error is allowed to be as large as 0.1σ (e = 0.1σ) and the confidence level is 95% (α = 0.05), the required minimum number of simulations is 384. Therefore, many studies set the number of 500 for Monte Carlo simulations. It is evident from this that Monte Carlo convergence occurs gradually since to attain ten times the accuracy we would need to amplify our sampling by a hundred times, as per the well-known $\sqrt{N}$-rule.

The second approach is based on the weak law of large numbers. As the sample size grows larger, the sample statistics will tend toward the population statistics. This is a more practical approach for determining the minimum number of simulations required in a Monte Carlo simulation. The method involves monitoring the variation of the statistics of model outputs as the number of simulations increases, and stopping the simulation once the variation becomes stationary. Specifically, the method involves calculating the statistics of interest (such as mean, variance, or quantiles) using a progressively increasing number of simulation samples, and plotting these statistics against the number of samples. As the number of samples increases, the statistics will initially show high variability due to the limited sample size but eventually converge to a stable range as the sample size increases. The minimum number of simulations required to achieve stable results can then be determined by observing the point at which the statistics become stationary.

Up until now, many scholars have conducted probabilistic analysis using Monte Carlo simulations for the interests of geotechnical problems. The random fields are used to represent the spatial variability and uncertainty of soil properties in a Monte Carlo simulation. A random field is a stochastic function that assigns a random value to each point in a spatial domain, such as a soil deposit. The values assigned to neighboring points are correlated, reflecting the fact that soil properties tend to vary smoothly in space. In the literature, there are several examples of researchers employing Monte Carlo simulations to analyze geotechnical problems using different numbers of random fields. Zevgolis and Bourdeau [17] performed Monte Carlo simulations 3000 times for assumed probability distributions of the backfill and foundation material engineering properties. Zhu et al. [18] studied how the variability of the permeability function propagates to the variability of hydraulic conditions based on 1000 random fields. Peng et al. [19] investigated wave-induced oscillatory response in a spatially random heterogeneous porous seabed, considering cross-correlated multiple soil properties based on 500 random fields. Gong et al. [20] probabilistically analyzed tunnel longitudinal performance with 5000 conditional random fields using the Monte Carlo simulation. Liu et al. [21] examined the necessity of three-dimensional analysis when dealing with slope with full randomness in soil properties, using 600 random fields. Chen et al. [22] integrated the Karhunen–Loève expansion method and the coupled Eulerian–Lagrangian method to simulate the large deformation behavior of slopes with spatially varied shear strengths characterized using 200 random fields. Shen et al. [23] investigated the effect of soil spatial variability on failure mechanisms and capacities of strip foundations under uniaxial loading on marine clay deposits with linearly increasing undrained shear strength using 100,000 random fields. Deng et al. [24] conducted a probabilistic analysis of land subsidence due to pumping using Biot’s poroelasticity and 500 random fields. By using random fields to represent soil variability, Monte Carlo simulations can provide a more realistic and comprehensive analysis of geotechnical systems, accounting for the spatial variability of soil properties and its influence on the performance of the system. However, the number of random fields used in Monte Carlo simulations is not universally agreed upon and varies widely, ranging from 100 to 100,000. Determining the number of samples required in a Monte Carlo simulation to achieve convergence is not a straightforward task, and there is no consensus on the best approach [25,26,27]. The choice of the number of random fields depends on various factors such as the specific problem being analyzed, the desired level of detail, and available computational resources. Generally, increasing the number of random fields leads to more precise and reliable results, but it also demands more computational effort and resources. Nonetheless, higher-order statistical moments such as skewness and kurtosis can offer valuable insights into the distribution and shape of the response. These moments can also help to identify possible sources of non-linearity or non-normality in geotechnical behavior. Interestingly, the number of random fields needed to obtain higher-order statistical moments of model outputs has not been explored yet.

It is reported that the models used to simulate the infiltration of unsaturated soil face significant non-linearity issues, primarily due to the exponential changes in the soil–water characteristic curve (SWCC) and permeability function with water content [11]. Therefore, this study investigates the number of Monte Carlo simulations required to achieve stationary higher-order statistics of the pore pressure head based on an unsaturated soil slope under steady-state infiltration, using the second approach that is based on the weak law of large numbers.

2. Stochastic Model of Unsaturated Soil Slope under Infiltration

The governing equation for steady-state infiltration in unsaturated soil is described as [28]

$$\frac{\partial }{\partial x}\left(k_x\frac{\partial H}{\partial x}\right)+\frac{\partial }{\partial z}\left(k_z\frac{\partial H}{\partial z}\right)=0$$

(2)

where H is the total hydraulic head, which is the sum of the elevation head z and pore–water pressure head h. The unsaturated coefficient of permeability is represented by k_x and k_z in the x and z direction, respectively. For simplicity, we assume that the soil permeabilities are identical in both directions, that is k_x = k_z = k.

The exponential form of the unsaturated coefficient of the permeability function proposed by Leong and Rahardjo [29] is used:

$$k=k_s{\left(\frac{{\theta }_w}{{\theta }_s}\right)}^b$$

(3)

where k_s is the saturated coefficient of permeability; θ_w is the volumetric water content; θ_s is the saturated water content; and b is a constant parameter signifying soil type. The volumetric water content θ_w of SWCC summarized in Fredlund [30] is used:

$${\theta }_w={\theta }_s{\left\{\ln \left[\exp (1)+{\left(\frac{u_a-u_w}{a_f}\right)}^{n_f}\right]\right\}}^{-m_f}$$

(4)

where u_a and u_w represent the pore–air pressure and the pore–water pressure, respectively, and a_f, n_f, and m_f are the fitting parameters. For this study, we assume that the soil is sand, and the SWCC parameters are a_f = 5, n_f = 2, and m_f = 1. The permeability function parameter b is set to 3 to represent sand soil.

Figure 1 illustrates a numerical representation of a slope of unsaturated soil subjected to rainfall. The upper boundary is subjected to a vertical flux of q = 2 × 10⁻⁷ m/s to simulate rainfall infiltration. The boundary conditions on the left and right sides are set as constant hydraulic heads of 10 and 20 m, respectively, representing the underground water table. The lower boundary is considered impermeable and no flow boundary condition is applied.

The saturated hydraulic conductivity k_s is taken as a spatially varied soil parameter following log-normal distribution, with a mean of 2 × 10⁻⁵ and standard deviation of 1.6 × 10⁻⁵ m/s [31]. The covariance function C is used to simulate the spatial correlation of log(k_s):

$$C(x_1,x_2)={\sigma }^2{e}^{-{\left[\frac{{(x_1-x_2)}^2}{l_x^2}+\frac{{(z_1-z_2)}^2}{l_z^2}\right]}^{\frac{1}{2}}}$$

(5)

where x₁ = (x₁, z₁) and x₂ = (x₂, z₂) represent two coordinates in the model domain; σ² is the variance of log(k_s); and l_x and l_z are the horizontal and vertical correlation lengths, with l_x set to 50 m and l_z set to 10 m, as reported by Yang et al. [32].

Table 1. Parameters for stochastic numerical model.

Parameters	Definition	Value	Unit
q	Vertical flux	2 × 10−7	m/s
θs	Saturated volumetric water content	0.4
af	SWCC parameter	5
nf	SWCC parameter	2
mf	SWCC parameter	1
b	Permeability function parameter	3
μks	Mean of ks	2 × 10−5	m/s
σks	Standard deviation of ks	1.6 × 10−5	m/s
lx	Correlation length in horizontal direction	50	m
lz	Correlation length in vertical direction	10	m

summarizes the parameters used in this study.

3. Random Field Simulation

This study utilizes the Karhunen–Loève expansion method to generate the spatial random field k_s(x) random field with a mean μ(x) and covariance function C(x₁, x₂). The covariance function C(x₁, x₂) can be decomposed as

$$C(x_1,x_2)={\displaystyle \sum _{i=1}^{\infty }{\lambda }_i{\phi }_i(x_1){\phi }_i(x_2)}$$

(6)

where λ_i and φ_i(x) are the ith eigenvalues and eigenfunctions (eigenvectors) of the covariance function, respectively. By utilizing singular value decomposition, the eigenvalues and eigenfunctions can be determined.

Utilizing the Karhunen–Loève expansion, the spatial random variable k_s(x) can be further decomposed as

$$\log k_s(x)=\mu (x)+{\displaystyle \sum _{i=1}^{\infty }\sqrt{{\lambda }_i}}{\theta }_i{\phi }_i(x)$$

(7)

where θ_i is a Gaussian random variable with θ_i ~ N(0,1) and is independently and identically distributed.

Practically, the spatial variability representing by the random field is approximated by truncating the Karhunen–Loève expansion:

$$\log k_s(x)=\mu (x)+{\displaystyle \sum _{i=1}^n\sqrt{{\lambda }_i}}{\theta }_i{\phi }_i(x)$$

(8)

where the value of n represents the truncation level, which involves disregarding the small-scale variation of the field in the Karhunen–Loève expansion. The selection of the truncation level is dependent on both the covariance function of the random field and the desired accuracy of total variability. Typically, a value of n is chosen to preserve 95% of the total variance (Yang et al., 2019):

$$\frac{{\displaystyle \sum _{i=1}^n{\lambda }_i}}{{\displaystyle \sum _{i=1}^{\infty }{\lambda }_i}}=0.95.$$

(9)

4. Calculation of Statistical Moments

The results of a Monte Carlo simulation are the samples of the model outputs. The mean μ and variance σ² can be estimated to illustrate the central tendency and variability of the stochastic model output.

Skewness is a measure of the degree of asymmetry of a probability distribution around its mean. A negative skewness value indicates that the distribution is skewed to the left, meaning that the tail of the distribution extends more to the left of the mean than to the right. On the other hand, a positive skewness value indicates that the distribution is skewed to the right, meaning that the tail of the distribution extends more to the right of the mean than to the left. In more practical terms, a negative skewness value indicates that there are more extreme values on the left-hand side of the distribution, while a positive skewness value indicates that there are more extreme values on the right-hand side of the distribution. The skewness of a distribution is defined as

$$s=\frac{E{(x-\mu )}^3}{{\sigma }^3}$$

(10)

where x represents the values of the model output; E represents the expectation operator.

Kurtosis is a statistical measure that quantifies the degree of peakedness and tail heaviness of a probability distribution, relative to the normal distribution. The normal distribution has a kurtosis value of 3. Distributions with a kurtosis greater than 3 are more prone to outliers than the normal distribution, while distributions with a kurtosis less than 3 are less prone to outliers. In other words, a high kurtosis value indicates that the distribution has more extreme values (outliers) than the normal distribution, while a low kurtosis value indicates that the distribution has fewer extreme values. The skewness of a distribution is defined as

$$k=\frac{E{(x-\mu )}^4}{{\sigma }^4}.$$

(11)

In this study, the mean, variance, skewness, and kurtosis are used to represent the first, second, third, and fourth statistical moments, respectively.

5. Results and Discussions

5.1. Results of Simulation

In Figure 2, a comparison is made between the simulated pore pressure head from the determinist model and stochastic model. Figure 2a,b displays the pore pressure head of homogeneous and spatially varied k_s, respectively. For comparison purposes, only one realization of spatially varied k_s is generated. Both figures indicate that the matric suction in the shallow depth begins to dissipate. In the unsaturated zone, there are significant differences in the contours of the pore pressure head, which gradually become more apparent as rainfall continues. For spatially varied k_s, the matrix suction is relatively small. Therefore, the spatial variation of k_s has a significant impact on the pore pressure head, highlighting the need to consider the spatial variability of k_s in engineering practice.

5.2. Results of Statistical Moments

Figure 3 displays the variation in the mean pore pressure head (μ) with an increase in the number of samples taken along the A–A′ mesh grid section. It can be observed that the mean pore pressure head becomes stationary quickly. In order to accurately characterize the mean pore pressure head, a total of 100 Monte Carlo samples are sufficient. Figure 4 represents the first moment of the mean pore pressure head in the entire slope, which indicates that there is only a slight difference in the mean pore pressure head in the unsaturated zone. Specifically, the mean suction head (negative pore pressure head) in the unsaturated zone is approximately 3 m, which is comparable to the results obtained from the deterministic model.

Figure 5 displays the variation in the variance (σ²) with an increase in the number of samples taken along the A–A′ mesh grid section. The plot indicates that the variation becomes stable when the number of samples exceeds 500. Figure 6 demonstrates the variance based on different numbers of samples. The distribution of variance based on different numbers of samples is generally similar, with only slight differences in the variance around the right boundary. However, when the number of samples exceeds 1000, this difference becomes insignificant. In geotechnical engineering, researchers typically use the second moment of variance as a useful statistical measure to represent certain aspects of model outputs. Therefore, it is recommended to use at least 500 samples for Monte-Carlo-based probabilistic analysis for geotechnical engineering models, while 1000 samples is a more conservative choice. The suggested number aligns with the estimation of the strong law of large numbers. However, it is important to note that this approach may not be appropriate in all cases, and further research is needed to determine its effectiveness and limitations. Additionally, the variance is larger in areas far from hydraulic boundaries because the fixed hydraulic boundaries constrain the variation of the pore pressure head. Furthermore, it was observed that the pore pressure head in the unsaturated zone experienced less uncertainty, with a variance of around 0.8 m².

Figure 7 displays the variation in skewness with an increase in the number of samples taken along the A–A′ mesh grid section. The plot indicates significant variations in skewness, which become stationary for all mesh grids when the number of samples exceeds 15,000. Figure 8 shows the skewness based on different numbers of samples. For the pore pressure head, the skewness sign is roughly separated by the water table and x = 60 m, which is close to the middle part of the model. This may be due to the constant hydraulic heads on the left and right boundaries, and the skewness sign is hard to change. In the left part of the model, the skewness values are predominantly negative, indicating that the probability density function (PDF) is left-skewed. At the slope toe of the model, the skewness is almost positive and can be up to 1, indicating a significantly right-skewed PDF.

The variation in kurtosis with an increase in the number of samples taken along the A–A′ mesh grid section is presented in Figure 9, indicating significant fluctuations as the number of samples increases. It is only when the number of samples reaches 25,000 that the variation in kurtosis stabilizes. Figure 10 illustrates the kurtosis based on different numbers of samples. The kurtosis of the entire slope is greater than 3, indicating a leptokurtic distribution for the probability density function of the pore pressure head. However, the kurtosis around the underground water table is relatively small, suggesting that the degree of tailedness around the water table is more closely approximated by normal distributions.

6. Conclusions

This study investigates the number of Monte Carlo simulations required to achieve stationary higher-order statistics of a pore pressure head based on an unsaturated soil slope under steady-state infiltration. The major conclusions are summarized below:

• It is recommended to use at least 500 samples for Monte Carlo-based probabilistic analysis for geotechnical engineering models, while 1000 samples is a more conservative choice for up to second-moment analysis.
• The pore pressure head in the unsaturated zone experiences less uncertainty, with a variance of around 0.8 m². The skewness sign is roughly separated by the water table and x = 60 m. The probability density function of the pore pressure head is a leptokurtic distribution.
• Significant variations are observed in skewness, which become stationary for all mesh grids when the number of samples exceeds 15,000. It is only when the number of samples reaches 25,000 that the variation in kurtosis stabilizes.

It is important to note that this study has some limitations. Firstly, the investigation only covers one geotechnical model with one soil type, and the required number of Monte Carlo simulations may vary with different models. Secondly, the convergence assessment is based on the weak law of large numbers, where statistics are only calculated against increased samplings, which may not be mathematically rigorous. Nevertheless, as it is a high non-linear model, the conclusions drawn are representative of engineering practice. Future studies could explore different models and focus on selecting some aggregate quantities of interest, such as L2-norm, while referencing the results of an extremely large number of simulations.