Reliability Analysis of Pile Foundation Using Soft Computing Techniques: A Comparative Study

: Uncertainty and variability are inherent to pile design and consequently, there have been considerable researches in quantifying the reliability or probability of failure of structures. This pa ‐ per aims at examining and comparing the applicability and adaptability of Minimax Probability Machine Regression (MPMR), Emotional Neural Network (ENN), Group Method of Data Handling (GMDH), and Adaptive Neuro ‐ Fuzzy Inference System (ANFIS) in the reliability analysis of pile embedded in cohesionless soil and proposes an AI ‐ based prediction method for bearing capacity of pile foundation. To ascertain the homogeneity and distribution of the datasets, Mann–Whitney U (M–W) and Anderson–Darling (AD) tests are carried out, respectively. The performance of the de ‐ veloped soft computing models is ascertained using various statistical parameters. A comparative study is implemented among reliability indices of the proposed models by employing First Order Second Moment Method (FOSM). The results of FOSM showed that the ANFIS approach outper ‐ formed other models for reliability analysis of bearing capacity of pile and ENN is the worst per ‐ forming model. The value of R 2 for all the developed models is close to 1. The best RMSE value is achieved for the training phase of the ANFIS model (0 in training and 2.13 in testing) and the poorest for the ENN (2.03 in training and 31.24 in testing) model. Based on the experimental results of reli ‐ ability indices, the developed ANFIS model is found to be very close to that computed from the original data.


Introduction
Pile foundations are used in weak soils to increase its bearing capacity and reduction of foundation settlements. Due to scarcity of space and demand to build taller and heavier buildings, study on reliability analysis of bearing capacity of the pile foundations is increasingly attracting attention of geotechnical researchers. Reliability is the probability of performance of the essential function by the system in consideration, effectively for a given period of time under specified conditions [1]. It is also defined as (1-probability of failure). The inability of the system to perform its proposed function adequately under defined conditions, it is said to have reached the state of failure. To ensure safety, necessity of assessing calculated risk has been put forward long back [2] owing to different degrees of uncertainties involved in the geotechnical design. Uncertainties may be classified as inherent variability (uncertainties in material parameters), Measurement variability (uncertainties in loads or equipment errors) and transformation uncertainties (uncertainties in hypothesis or mathematical modeling) [3][4][5]. Observational method [5] does not allow the design to be changed during construction based on observed behavior. To address uncertainties and variability in design, factor of safety (FS) approach has traditionally been used, based on experience but this approach fails to address the problem in rational way. FS approach not only results into conservative analysis but also owns noteworthy probability of failure [6].
Reliability analysis addresses safety in rational manner. Many major uncertainties (e.g., material parameters, loads and pore-water pressures) which are not considered in the FS approach are addressed in Reliability in lucid way. First Order Second moment Method (FOSM) has been effectively used in reliability analysis pile bearing capacity problems [7,8]. However, the method has been found to be very time consuming and requires additional efforts, especially in case of multi-variable relationships, due to large number of calculation performance functions and/or its partial derivatives with respect to the random basic variables. For performance functions other than normal distribution, results tend to be flawed. The multi-tangent-plane surface method, the multi-plane surfaces method [9] and Response Surface Method (RSM) [10][11][12] improves the accuracy but its applicability is limited to nonlinear convex or concave limit state surfaces. Meanwhile, Artificial Neural Networks (ANN) approaches [13,14] have been adequately applied to reliability analysis but its performance cannot be expressed in explicit form and implicit performance function contains partial derivatives which are too intricate to be solved. It results in low generalization capacity, over-fitting problems, arriving at local minima and slow-convergence speed. Generalization performance is greatly improved in case of Support Vector Machines (SVMs) but its major drawback lays in high algorithmic intricacy and enormous memory necessities in comprehensive tasks [15]. Selection of Kernel is another major limitation in SVMs [16]. Fuzzy based reliability analysis [17,18] is too tedious to be formulated due to fuzzy rules and complex membership functions. Moreover, fuzzy system involves large data requirements and expertise to develop [19].
The paper proposes Minimax Probability Machine Regression (MPMR), Emotional Neural Network (ENN), Group Method of Data Handling (GMDH) and Adaptive Neuro-Fuzzy Inference System (ANFIS) based reliability analysis as an alternative to the conventional methods. MPMR is a type of regression model which by applying some bound (±ε) on minimum probability (Ω), maximizes it for all the feasible distributions that comprise the same mean and covariance matrix as the model data [20]. There have been considerable researches applying MPMR in various fields [21][22][23]. ENN is developed from the Emotional Back Propagation (EBP) learning algorithm and simulates emotional parameters and emotional weights for improved learning and decision-making abilities. The use of emotions in machine learning is being extensively researched in recent past [24][25][26][27]. GMDH is self-organizing type neural networks (NN) which has been found to be reliable computational method to replace classical methods [28][29][30]. ANFIS has been applied in various fields in literature [31][32][33], and in the current study its developed for pile bearing capacity detection. Therefore, this study examines and compares MPMR, ENN, GMDH and ANFIS in reliability analysis of pile bearing capacity as a new soft computing approach.

Details of Present Analysis
The bearing capacity of piles (Q) is the addition of soil friction (Qf) caused by the perimeter area and bearing resistance (Qb) transferred by base area to the ground. Note that, the bearing capacity of piles depends on the length ( ) and diameter ( ) of the pile, and soil properties, such as cohesion (C), angle of internal friction (φ), and unit weight (γ) of soils. For pure sand, i.e., for cohesionless soils, C = 0; and thus, the bearing capacity of piles in pure sand is the function of L, D, φ,and γ only. On the other hand, L and D are deterministic parameters which are irrelative in reliability analysis; and therefore, the effect of φ, γ on output parameter (Q) can be expresses as follows [34]: (1) For piles in sands: Here, qb and Ab is the ultimate bearing capacity of the soil and the base area of the pile respectively. Moreover, qf and Af denotes the ultimate friction resistance of the soil per unit area and the perimeter of the pile respectively. Df is the embedded depth of the pile, denotes angle of friction between the pile surface and the surrounding sand and is the average effective over-burden pressure acting along the embedded length of the pile [34].
FOSM approach is extremely powerful probabilistic method to deal with calculations. Most effective application of FOSM is to state the uncertainties in form of reliability index. FOSM uses first order Taylor series approximation to formulate performance functions [35]. If the resistance or load carrying capacity is 'R' and load effect is 'Q'; Performance function (Z) is given by [36]: Steps to evaluate Reliability index [37,38]: (1) State the basic variables (R, Q) in standard non-dimensional form: , (2) Convert the limit state function related by reduced variables which represents a straight line. , (3) The shortest distance to the f(R, Q) from the origin gives reliability index (β) (Figure 1).
(8) Figure 1. Reliability index termed as the shortest distance of limit state function from the origin.

Details of MPMR
MPMR is built improving minimax probability machine classification algorithm (MPMC). It constructs regression function using Mercer Kernels and by putting a direct bound (±ε) on minimum probability and maximizing it [20]. MPMR do not make assumptions respecting the data distribution which leads to lack of validity and generality. The most striking feature of MPMR is that it is able to provide low bound on probability such that the regression model remains in a little required error of the regression function when the covariance and mean matrix of data is known. MPMR algorithm is shown in Figure 2.
Some unknown regression function g: R n →R is used to generate the learning data based on data set (ui,vi); i = 1,2....N. The relationship between ui and vi can be given by: Here, δi represents the fitting error such that The goal of MPMR is to come across a model in which the minimum probability is maximized under the constraint of error ±ε.
v ℊ u The bound on minimum probability (Ω) is defined as below: Similar to the kernel formulation for minimax probability machine classification (MPMC), MPMR formulation is represented as: Here, , , is Mercer's Kernel where ui, i = 1, 2…N. denotes the learning data whereas , ∈ represents output of the MPMR algorithm. Here, W is weight and b is bias.

Details of ENN
ENN is the evolved form of neural network based on emotions aimed at improving the decision-making abilities and learning capacity. Although we don't expect machines to feel emotions and react emotionally, regulatory signals and information signals do travel within them. Emotions can be simulated in machines similar to machine intelligence. It makes the model more user-friendly, quick learning and prompt reacting. The proposed model is rooted in Emotional Back Propagation (EBP) and simulates emotional parameters (such as confidence, stress and anxiety) and emotional weights in emotional neurons. The neural network possesses numerous hormone glands which emits virtual hormones and which in turn affects the performance of the specific nodes of neural system [24].
ENN consists of three layers based on the flow of information e.g., input layer, hidden layer and output layer ( Figure 3) Input layer, hidden layer and output layer contains i, h and j neurons respectively. Here, Xi and Xi and Yi denotes input and output values of i neurons respectively such that Xi = Yi. Input and output values of hidden neurons for the hidden layer are denoted by Xh and Yh respectively and are related by formula: Furthermore, input and output values of hidden neurons for the output layer are denoted by Xj and Yj respectively and are related by formula:

Details of GMDH
GMDH is a self-organizing feed-forward neural network algorithm which have capability of solving very complex non-linear problems having several inputs and one output (Xi,yi) (=1,2,….,M). Neuron layers define the GMDH model. In this algorithm, the effective input variables, the number of layers, neurons within the hidden layers and the optimum model configuration are calculated automatically. Through quadratic polynomials, each neuron layer has many data points connected to each other, and new neurons are created in the process [39,40]. The general structure of the model is depicted in Figures  4 and 5 by Cai et al. [28]. Input parameters are entered in the first layer and some functions are defined. Some parameters are selected and transferred to the next layer and some are left out. The general form of the function in the first layer is given in Equation (13), and the output can be expressed as follows: , , , … . , 1,2,3, … In this modelling, one of the fundamental steps is to obtain the parameters in Equation (14), and the output of the proposed model can be formulated as: For detailed study about the modelling, refer [41].
In each layer, the computational functions are described with the help of neurons. In order to choose parameters, the selection pressure criterion is defined in several layers. If the built function has the desired conditions on the basis of the stated criteria (Equation (15)), it is passed to the next layer or otherwise, left out. The criteria vary in the range [0,1] and can be calculated as follows: Here, for minimum error, α is taken close to one and vice-versa.

Details of ANFIS
A neuro-fuzzy technique called ANFIS is a data driven procedure and fusion of fuzzy interface systems and neural networks [42]. In a neural network platform, the method gives the solution to function approximation problems. A fuzzy inference method consisting of an initial fuzzy model is initially generated here, based on the fuzzy rules derived from the data set of the input output. The neural network is used in the next step to finetune the rules of the initial fuzzy model that was constructed. The input-output pairs defined are generated by using a collection of if-then fuzzy rules. Training phase in ANFIS methodology is very rapid due to hybrid rule of learning. By applying an optimal criterion for data collection, the number of training data used in ANFIS is significantly decreased. Three layers of the neuro fuzzy network's classic construction are: input layer; single hidden layer; and output layer. The input layer normalizes into the lattice structure, while the hidden layer consists of basic functions and specifies their dimension, shape and overlap. The output layer generates network output by summing up the basis function's weighted outputs [43]. If x and y are inputs and A and B are fuzzy sets, the ANFIS structure is depicted in Figure 6. Output is Z = f (x, y). For more detailed study, refer [44]

Details of Data Set
A single pile embedded in pure sandy soil (i.e., C = 0) of diameter 300 mm and length 12 m is taken to explore a new methodology for stability analysis of pile ( Figure 7). The data has been generated randomly in MATLAB using 'randn' command. Specifically, input variables, i.e., φ and γ have been generated to estimate the values of output variable, Q. For this purpose, the range of γ has been taken from 14 KN/m 3 to 21 KN/m 3 and that of φ from 33° to 43° [29]. The descriptive statistics of the input and output parameters are provided in Table 1 and histogram is presented in Figure 8. As can be seen, the φ content mostly covers the range of 0 to 60% and the γ content covers the whole range of 0 and 100%, respectively. The range of output parameter, Q lies between 0 and 80%. 80 data sets were taken and divided into 2 sub-sets for the purpose of investigation.
(1) Testing (TS) dataset: 70% of dataset i.e., 56 was taken to construct the model (2) Training (TR) dataset: To validate the model performance, 30% of dataset i.e., 24 was taken. Following formula was used to normalize the data:

Statistical Parameters
To validate the accuracy of the models used, the widely used multiple statistical approaches have been used which are listed below.
Weighted mean absolute percentage error (WMAPE) WMAPE is a statistical measure of the accuracy of a simulation model. It is an improvement over mean absolute percentage error where weighted errors are calculated ∑ ∑ (19) Nash-Sutcliffe efficiency (NS) [45] NS is the ratio of residual error variance (noise) to measured variance in observed data. NS values less than one are unacceptable. The ideal value of this index is 1. It's an important parameter recommended in literature by various scientists.
Root mean square error (RMSE) RMSE is one of the most commonly used parameters for evaluating error. Its ideal value is zero and a higher value indicates lower performance.

(21)
Variance account factor (VAF) VAF is accounted for among original and predicted values of regression models. Perfect models have a value of 100%.
Coefficient of determination (R 2 ) R 2 gives the proportion of the predicted values in agreement to the observed data.
Adjusted determination coefficient (Adj. R 2 ) Adj. R 2 indicates how well a predictive model predicts the desired output based on the number of input variables used to develop the model. The addition of an insignificant input variable decreases the Adj. R 2 value, while the addition of useful variable increase its value.
Bias factor (BF) Bias factor value of 1 means balanced prediction, a value greater than 1 means over prediction, and a value less than 1 means under prediction. BF 1 (25) RMSE to observation's SD ratio (RSR) RSR [46] is a model evaluation statistic which standardizes RMSE by dividing it with standard deviation. It varies from the ideal value of 0 to large positive values. RSR > 0.7 is desirable result.

∑
Normalized mean bias error (NMBE) [47] NMBE compute the correlation between predicted value and the mean value.
The mean absolute percentage error (MAPE) [48] MAPE relates the residual error for each data point regarding the observed value.

Mean bias error (MBE)
MBE is a statistical parameter used to estimate whether the model is overestimated (MBE greater than one) or underestimated (MBE less than one). Values close to zero indicates perfect agreement, however, overestimated and underestimated values may cancel out, which is a significant drawback of the parameter.

(29)
Willmott's index of agreement (WI) [49] It is a measure of the degree of model prediction error. It varies from 0 to 1, where 0 signifies the null model while 1 confirms perfect agreement.
where di and yi = the observed and forecasted ith value, respectively; dmean = average of observed value; SD is the standard deviation; n = number data samples; and p = model input quantity. Legate and McCabe's index (LMI) [50] For model evaluation purposes, LMI is superior to the R 2 because it is more sensitive to variations in observed and model-simulated means and variances. It may also be overly sensitive due to squared differences of observed and predicted values. The ideal value of the index is 1. Value of zero indicates that the observed mean 0 is as good a predictor as the observed mean. Value of less than zero is undesirable.
t-statistic (t-stat) [51] It was proposed by Stone [51] to be used along with MBE and RMSE to determine if predicted values agree with measured values. t-statistic relates sample mean(s) to the null hypothesis. This parameter inspects predictions of the model at some specific confidence level.

Taylor Diagram
Taylor diagrams [52] are simple graphical representation of how the predicted values of are in correspondence with the observed values and compares the performance of various models used for simulation. It presents statistical comparison of various models in a 2-D graph by plotting standard deviations, correlation coefficient and centered rootmean-square difference (RMS). Standard deviation is denoted by the radial distance from the origin. The RMS error is relational to the distance between observed and simulated fields assessed in the identical units to standard deviation. Correlation coefficient is represented by the azimuthal angle.

REC Curve
Regression Error Characteristic (REC) curve [53,54] gives the graph of error tolerance versus percentage of points that are predicted within the tolerance. The x-axis and the yaxis represent the error tolerance and the accuracy of a regression function respectively.
The area over the REC curve (AOC) gives the approximation of the expected error. Lesser the AOC, better is the performance of the models. Thus, ROC curves allow easy and reliable visual estimate of the performance of the models.

AD Test and M-W Test
To test the normal probability distribution of the datasets, Anderson-Darling (AD) test is carried out. If the p-value is within level of significance (≤0.05), the model is conformed to follow Normal distribution. Mann-Whitney U (M-W) test is carried out to examine the homogeneity in the outcomes of different independent models. The nonparametric test is used to determine whether two independent samples have same distribution. p value > 0.05 rejects null hypothesis.

Rank Analysis
Rank analysis is the easiest and most popular method to conclude the effectiveness of the simulation models and compare their robustness. In this process, the score value is given to the probabilistic parameters, using their ideal values as the benchmark. It depends on the number of models used. The best model results get the highest score and vice versa. Two models with the same results may get the same ranking ratings. A model's overall rank score is determined by summing up the rank scores in all phases (testing and training).

Results and Discussion
MATLAB have been used to construct the program of MPMR, ENN, GMDH and ANFIS. The REC curves, Taylor diagrams are drawn in R-studio. The analysis of M-W test and AD test also is done in R-studio. Figures 9-12 shows the performance of the ENN, MPMR, GMDH and ANFIS model respectively. The value of R for all the models in the both training and testing phases is very close to one, so both the models are robust for predicting the strength of the piles embedded in sandy soil.
The training dataset was used to construct the ENN model, while the testing dataset was used to validate the developed ENN model. The developed ENN model attained 99.9% prediction accuracy in terms of R 2 in both phases (R 2 = 0.999).
In MPMR modelling, a trial-and-error approach was followed to compute the design values of σ and ε. The design values of ε and σ were obtained as 0.004 and 1, respectively. The obtained results show that the developed MPMR model almost 100% accuracy in both phases (training R 2 = 0.99 and testing R 2 = 0.99). The values of RMSE also support the same for the training dataset (RMSE = 0.32), however, a slight deviation was observed in the testing phase (RMSE = 8.29).
The GMDH model is trained with various sets of the number of neurons and number of layers and the best combination, giving the best value of R 2 is taken. For the optimal model, the number of neurons was 20 and the number of layers was 10. The developed GMDH model attained the prediction accuracy R 2 = 1 in both phases. The values of RMSE in the training and phases are obtained as 0.60 and 2.13, respectively.
In ANFIS modelling, the design value of the number of epochs is 3 and the type of membership function used was Sugeno for each input and linear for each output. In the developed model, "and method" is prod and "or method" is probor. In the training phase, the developed ANFIS model attained 100% prediction accuracy with R 2 = 1 and RMSE = 0 while a slight deviation was observed in the testing phase (RMSE = 2.13). Table 2 represents the performance of ENN, MPMR, GMDH and ANFIS models. Comparing the results of the performance parameters with their ideal values, it is ascertained that all the developed models are very reliable for the prediction of the bearing capacity of the piles. However, RMSE value is poorest for ENN in the testing phase (31.24) and best for ANFIS in the training phase (0). RMSE value for both ANFIS and GMDH in testing phase is 2.13, which is slightly better than MPMR (RMSE = 8.29). To compare the performance of the different developed models, we undertake rank analysis ( Table 3). The results conclude that the ANFIS is the best performing model followed by GMDH, MPMR and ENN. The ENN's performance in both testing and training phases is worst among the developed models. The heat map of the rank analysis presents simple understanding of comparison of performance of the developed models.      Table 3. Rank analysis of the developed models. ENN  MPMR  GMDH  ANFIS  TR  TS  TR  TS  TR  TS  TR  TS Total rank  19  21  24  23  29  32  33  33  Finial rank  40  47  61  66 Regression Error Characteristic (REC) curve for both the models, drawn using MATLAB is given in Figures 13-15 represents the area over the REC curve (AOC) values of both the models for both the phases. Values of AOC enumerate the expected error of the regression models. Better models have lower values. The AOC values are quite analogous to each other for both the models. All the models give excellent AOC values and so all the models are robust models for prediction. Comparing the models, ENN's reliability is worst compared to companion models. ANFIS in training phase possess least value of AOC.

S.No. Parameters
For all the models in M-W test, in both testing and training; we are getting p value close to one (Table 3). Thus, all the models are confirmed to follow Normal distribution. For AD test, we get p values close to one, thus null-hypothesis is rejected (Table 4).
Reliability indices are calculated as per Equations (5)-(8) using the predicted outputs for all the models and compared with the reliability index calculated from observed outputs calculated using Equation (1). Reliability index values of both the models ( Figure 16) are very close to the observed value. Hence, both the models are reliable to be used for the reliability analysis of pile foundation.

Conclusions
The paper demonstrates the application of four machine learning algorithms (ENN, MPMR, GMDH and ANFIS) in the reliability analysis of pile foundation embedded in a cohesionless soil. Input parameters are Φ and γ and its effect on output parameter Q has been studied. The soil data has been divided into training (70% of the main dataset) and testing (30% of the main dataset) phases. Various statistical performance parameters have been applied and their results been discussed to ascertain the performance of both the models. The performance of both the models in both the phases has been found to be excellent and on par with each other. The predicted values of β are very close to the observed value; however, that of MPMR, GMDH and ANFIS are better than ENN. ANFIS is concluded to be most reliable model for prediction of bearing capacity of pile foundation. The REC curve was plotted, and the AOC values were excellent. The reliability index values of all the models are very close to that of traditional method. Thus, the paper proposes that both the models are robust models for prediction, however, compared to each other, ANFIS is the best and ENN is the poorest.
It is needless to mention that the outcome of a predictive model using randomly generated datasets may differ from case to case; however, a randomly generated dataset with a predefined range of maximum and minimum values can be quite useful for computational modelling. The models developed in the present study can be used to predict the bearing capacity and its corresponding reliability index for the piles with 450 mm diameter installed in a cohesionless soil medium with a specified range of φ (from 33° to 43°) and γ (from 14 KN/m 3 to 21 KN/m 3 ). Nevertheless, this study may also consider as a reference due to the fact that the nature of soil varies from place to place, and hence may not applicable for those soils which are originated from a completely different environment. In such cases, the concept presented in this study can be considered as a reference to perform the reliability analysis of pile foundations in cohesionless soils.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.