Predicting the Bearing Capacity of Shallow Foundations on Granular Soil Using Ensemble Machine Learning Models

Abstract

Shallow foundations are widely used in both terrestrial and marine environments, supporting critical structures such as buildings, offshore wind turbines, subsea platforms, and infrastructure in coastal zones, including piers, seawalls, and coastal defense systems. Accurately determining the soil bearing capacity for shallow foundations presents a significant challenge, as it necessitates considerable resources in terms of materials and testing equipment, as well as a substantial amount of time to perform the necessary evaluations. Consequently, our research was designed to approximate the forecasting of soil bearing capacity for shallow foundations using machine learning algorithms. In our research, four ensemble machine learning algorithms were employed for the prediction process, benefiting from previous experimental tests. Those four models were AdaBoost, Extreme Gradient Boosting (XGBoost), Gradient Boosting Regression Trees (GBRTs), and Light Gradient Boosting Machine (LightGBM). To enhance the model’s efficacy and identify the optimal hyperparameters, grid search was conducted in conjunction with k-fold cross-validation for each model. The models were evaluated using the R² value, MAE, and RMSE. After evaluation, the R² values were between 0.817 and 0.849, where the GBRT model predicted more accurately than other models in training, testing, and combined datasets. Moreover, variable importance was analyzed to check which parameter is more important. Foundation width was the most important parameter affecting the shallow foundation bearing capacity. The findings obtained from the refined machine learning approach were compared with the well-known empirical and modern machine learning equations. In the end, the study designed a web application that helps geotechnical engineers from all over the world determine the ultimate bearing capacity of shallow foundations.

1. Introduction

A foundation is an important structural component whose purpose is to transfer the loads from the other structural elements above it to the soil beneath. A shallow foundation is a type of foundation used to transfer loads to the subsurface soil, which has sufficient strength to support them [1]. The ultimate bearing capacity of the soil is defined as the highest pressure that a structure can impose on it while avoiding shear failure or significant settlement [2]. Geotechnical engineers give considerable importance to the concept of bearing capacity because it plays a significant role in the design of a foundation. Determining soil bearing capacity is not an easy task due to its reliance on several key factors, such as soil characteristics, foundation depth, and geometric configuration [2]. For this reason, many researchers are involved in this subject, trying their best to find solutions for determining the bearing capacity of shallow foundations.

The prediction of the ultimate bearing capacity of foundations using traditional theories follows the equation introduced by Terzaghi, which is based on a superposition approach [3]. This approach was originally derived from the bearing capacity theory of Prandtl rooted in plasticity theory, to evaluate the punching resistance of a rigid base into a softer material [3]. Terzaghi’s equation for vertically loaded foundations on cohesionless soil is given as follows:

$$q_u=\gamma D{N}_q{F}_{qs}{F}_{qd}+{\displaystyle \frac{1}{2}}\gamma B{N}_{\gamma }{F}_{\gamma s}{F}_{\gamma d}$$

(1)

where B is the foundation width (m), $\gamma$ is the unit weight of soil (kN/m³), D is the depth of embedment (m), $q=\gamma D$ is the overburden pressure at the foundation level, $N_q$ and $N_{\gamma }$ are bearing capacity factors, $F_{qs}$ and $F_{\gamma s}$ are shape factors, and $F_{qd}$ and $F_{\gamma d}$ are depth factors. All the factors are non-dimensional and are used to adjust the base equation for specific conditions. Based on Terzaghi’s study [3], several research efforts evaluated bearing capacity factors, with significant contributions from Bolton and Lau [4], Chen [5], Hansen [6], Meyerhof [7,8], Michalowski [9], and Veic [10]. The investigators used various analytical and numerical techniques, such as Limit Analysis (LA), the Limit Equilibrium Method (LEM), and the Slipline Approach, among others, all under the premise of isotropic soil characteristics. Vešic [10] proposed unique coefficients to address the influences of footing inclinations, load, and soil, yet his shape and depth factors remained similar to those outlined by Hansen [6]. However, determining the various factors that affect the final bearing capacity in the equations remains a complex task. A fundamental method for determining the maximum load capacity of a foundation is to perform on-site tests. Nevertheless, this method incurs significant costs and requires substantial time investment. Despite the usefulness of analytical and numerical methods, they typically require simplifying assumptions regarding soil behavior, geometry, or boundary conditions, which can limit their applicability to real-world problems. In contrast, machine learning (ML) models can uncover complex, nonlinear relationships between input variables and outcomes directly from data, without relying on predefined theoretical formulations. This allows ML models to provide more flexible and often more accurate predictions when trained on high-quality datasets. Additionally, ML methods can be computationally efficient and scalable, making them suitable for rapid assessment and integration into practical engineering tools.

Single learner ML techniques have been applied to a wide range of topics in geotechnical engineering. Examples of these techniques include support vector machines, decision trees, K-Nearest Neighbors, artificial neural networks, Naive Bayes, Linear Regression, and Logistic Regression. These models have been successfully utilized in various applications such as settlement analysis [11,12], rock strength prediction [13,14], tunneling [15,16], landslide identification [17,18], soil liquefaction assessment [19,20], slope stability evaluation [21,22], and deep foundation design [23,24].

In recent studies, scholars have investigated ensemble learning strategies, which involve training multiple weak learners and aggregating their outputs [25,26,27]. The core principle of ensemble learning methodologies involves instructing multiple weak learners on the training dataset and combining their outputs to create a more robust predictive model. Ensemble learning methodologies are predominantly classified into two primary types: bagging and boosting. XGBoost, the Gradient Boosting Decision Tree (GBDT), and AdaBoost represent the three main models of boosting. A random forest is the bagging model that is used most frequently.

The use of ensemble models to forecast the performance of geotechnical engineering elements marks a notable advancement in the discipline. A hybrid model integrating XGBoost with the whale optimization algorithm (WOA) demonstrated superior performance over conventional techniques in estimating the capacity of a concrete pile. This was substantiated through tests conducted on a dataset comprising 472 samples, where the accuracy exhibited notable improvements [28]. Moreover, the study presents [29] an innovative ML approach to accurately predict shallow foundation settlement, combining advanced techniques such as gradient boosting (GB), random forest (RF), support vector machine (SVM), and K-Nearest Neighbor (KNN) with Particle Swarm Optimization (PSO) for improved model performance. By analyzing key factors such as the blow count of the standard penetration test (SPT), the footing width, the embedment ratio, and the applied pressure, the research provides valuable information for geotechnical engineering, along with a practical Excel-based tool to simplify settlement predictions for engineers. Additionally, a CPT-SPT transformation model using XGBoost outperformed traditional and ML models, achieving the highest R² (0.693) and lowest RMSE (5.417) [30]. One study [31] investigated the prediction of clay activity using various soil properties and ML. Gradient Boosting Trees, optimized via Bayesian methods, achieved high accuracy, especially with LL alone (R² = 0.94) or combined with SSA and CEC (R² = 0.96). Moreover, a super-learner model, which consists of Extra Trees Regressor and Gradient Boosting Regressor as base learners and Random Forest Regressor as the meta-learner, was developed to predict the compression index of clay [32]. Trained on a global dataset, the model demonstrated strong performance and was deployed online for practical use in geotechnical engineering. According to another investigation detailed in [33], six ML models were applied to predict the seismic bearing capacity of strip footings in rock masses. The dataset used for training and testing consisted of 960 samples for evaluation. Of the six models analyzed, the XGBoost model performed the best, with the highest accuracy and an R-squared value of 0.999. The work reported in [34] applied random forest regression to predict the bearing capacity of strip footings on layered soil and outperformed the ANN and M5P models. In addition, the authors of [35] examined the shear behavior at the geomembrane–soil interface and showed that coarse sand performed best with respect to the bedding layer in a high-stress environment. Gradient boosting with Bayesian optimization achieved the highest accuracy in predicting the peak friction angles of soils, and this can be used for geosynthetic design.

In terms of predicting the ultimate bearing capacity of a shallow foundation, several single learner ML models have been developed, including relevance vector machine (rvm) and long short-term memory (LSTM) [36]; genetic programming [37,38,39] and artificial neural networks [40,41,42,43,44,45]; M5’ model tree [46]; a Gaussian process regression approach [47]; random forest [48]; multi-expression programming (MEP) [2]; gene expression evolutionary algorithm [49]; and least squares support vector machine [50]. Evaluations of the performance of the newly developed models were performed against traditional bearing capacity equations. The investigation revealed that the accuracy of the novel model exceeded that of the conventional formula.

Our research benefits from the advancement of developing an ensemble ML learning algorithm, and the study uses four ensemble ML models: AdaBoost, LightGBM, GBRT, and XGBoost. These four models have successfully handled problems in the engineering and scientific domains. Despite this, their application in the ultimate bearing capacity of shallow foundations is considered limited. Motivated by this gap, the present work seeks to advance the state of practice by implementing and rigorously evaluating these ensemble algorithms on a comprehensive dataset of shallow foundation cases. To further enhance the robustness and accuracy of the model, we adopted a grid search strategy to systematically identify optimal hyperparameter configurations for each algorithm. This approach ensures peak predictive performance while maintaining generalization. Performance was tested using well-known metrics to ensure that the models met the accuracy, reliability, and generalization requirements. Furthermore, a graphical user interface (GUI) was developed to bridge the gap between advanced ML model development and practical engineering applications. The GUI aims to help geotechnical engineers predict the ultimate bearing capacity more efficiently and effectively and support their on-site decision-making process. By combining the strengths of ensemble learning with accessible computational tools, the proposed method effectively addresses the accuracy and usability requirements of modern geotechnical engineering practice.

2. Materials and Methods

2.1. Research Methodology

The research methodology consists of five interconnected phases (Figure 1). In Phase I, a database was created using data collected from previous studies. This database included five input parameters: unit weight of soil ($\gamma$), foundation width (B), foundation depth (D), internal friction angle ($\phi$), and length-to-width ratio ($L/B$), with the ultimate bearing capacity ($q_u$) as the output variable. Moving to Phase II, the data were first randomly shuffled and then split into 85% for training and 15% for testing. For reproducibility, especially when working with small datasets, a random state value was set. This value controls the randomness and ensures that the same data split is generated each time the code is executed in the Python program (3.7). Four ensemble ML models were developed: Adaptive Gradient Boosting (Adaboost), Gradient Boosting Regression Tree (GBRT), Light Gradient Boosting (LightGBM), and Extreme Gradient Boosting (XGBoost). These models underwent hyperparameter tuning using the GridSearchCV method to identify their optimal configurations. After tuning, the models’ performance was assessed on unseen data using RMSE, MAE, R², and additional evaluation metrics. The most effective model was chosen for subsequent analysis. In Phase III, the importance of the features in this model was evaluated to determine which input variables most significantly impacted the prediction of the ultimate bearing capacity. Phase IV compared the performance of the best model obtained from the previous phase with common empirical methods and existing ML models; this demonstrates its extra accuracy and wider applicability. Finally, Phase V focused on developing a graphical user interface (GUI) that incorporates the best ML model. This tool simplifies the prediction process, making it accessible and practical for engineers in the field.

2.2. Ensemble ML Models Background

2.2.1. Adaptive Boosting

(Adaboost) This technique employs an iterative approach that begins by uniformly weighting all instances within the learning dataset [51]. During subsequent iterations, greater weights are allocated to instances with larger prediction errors, emphasizing these particular observations. The predictive performance of the model can be improved by integrating base estimators, such as the decision trees used in this study, which iteratively refine their predictions to minimize errors identified by earlier decision trees. Eventually, the AdaBoost regressor computes the mean predicted outcome using the subsequent equation:

$$y=\sum \left({\alpha }_i\times h_i\left(x\right)\right)$$

(2)

In the above formula, y denotes the total sum of every decision tree within the ensemble, where ${\alpha }_i$ is the weight given to the i-th decision tree $h_i\left(x\right)$. The observation weights are updated by multiplying by the exponential of the negative error. Consequently, observations with higher prediction errors receive higher weights, while those with lower errors receive smaller weights in the subsequent iteration [52].

2.2.2. Gradient Boosting Regression Tree (GBRT)

Gradient Boosting Regression Tree (GBRT), introduced by Friedman in 2001 [53], evolved from the AdaBoost algorithm with two key adjustments for weak learners. Firstly, decision trees (DTs) are used as weak learners. Secondly, instead of updating the weights based on classification errors, they are adjusted according to the residual errors of the previous learner. GBRT is widely appreciated in ML for regression tasks due to its capacity to manage intricate variable interactions and its adaptability to grasp nonlinear patterns. As illustrated in Figure 2, the training of each subsequent tree incorporates the residual error of its predecessor. After the initial weak learner, each additional tree is built to refine the model’s ability to predict by addressing the weak learner’s residuals. GBRT also provides flexibility in tuning hyperparameters, such as the learning rate and the number and depth of trees, to manage the model’s convergence and training pace.

2.2.3. Light Gradient Boosting Machine (LightGBM)

To enhance computational efficiency without sacrificing model accuracy, Ke et al. (2017) [54] introduced the LightGBM algorithm. Compared to traditional Gradient Boosting Machines (GBMs), LightGBM can achieve training speeds up to 20 times faster. The word “Light” is used in the name of this algorithm because it offers faster performance compared to other boosting methods like XGBoost, which can sometimes have slow training times on large datasets. The main difference between LightGBM and other boosting algorithms is its tree expansion. As we can see in Figure 3, LightGBM uses the leaf-wise strategy. In this strategy, the leaf with the highest loss is chosen in order to minimize additional loss and increase accuracy more effectively than the depth-first strategies of other algorithms. This strategy is different from other boosting algorithms that typically use a level-wise (depth-wise) growth approach. To achieve high speed, LightGBM incorporates (1) exclusive feature bundling (EFB) and (2) gradient-based one-side sampling (GOSS). Employing EFB and GOSS reduces computational complexity, thereby expediting the training of gradient-boosting decision trees.

2.2.4. Extreme Gradient Boosting (XGBoost)

XGBoost, created by Chen et al. [28], is recognized as an extremely effective ML algorithm that optimizes Gradient Boosting Machines. It integrates various advanced strategies to handle high-dimensional data with accuracy and speed. The algorithm employs a compressed column format to efficiently store the input data, thereby reducing sorting costs and accelerating the training process. It also applies randomization to prevent overfitting and increase generalization. Furthermore, XGBoost utilizes distributed and parallel computing to make full use of CPU cores during training and split discovery. These features enhance its scalability and performance, enabling model training on large datasets with numerous features.

2.3. ML Model Development

A comprehensive investigation was carried out to improve the prediction accuracy of the ultimate bearing capacity of shallow foundations by employing cutting-edge ML models. These models were meticulously trained and tested using a broad dataset sourced from prior experimental reports. Due to the task’s complexity and data diversity, selecting and fine-tuning hyperparameters became crucial in crafting effective predictive models. To address this, a systematic search was performed using the gridsearchcv approach (Figure 4) in conjunction with a 5-fold cross-validation to determine the best hyperparameter configurations for each model.

The estimator parameters were adjusted within a broad range, from 10 to 600, to encompass a diverse set of model behaviors. Similarly, the exploration of tree depth within the models extended to a maximum of 20, seeking an equilibrium between detailed learning and generalization. The learning rate, a crucial factor, was varied from 0.0001 to 0.1 to modulate the speed of model adaptation. A rapid learning rate might result in premature convergence, overlooking subtleties, whereas a slower rate could unnecessarily prolong the process and risk convergence to suboptimal solutions.

Table 1. Best hyperparameter settings used for each ML model.

Models	Hyperparameters
	Max Depth	Learning Rate	Number of Estimators
XGBoost	3.0	0.01	500
Adaboost	Not applicable	0.50	50
LightGBM	Not applicable	0.20	600
GBRT	3.0	0.01	600

summarizes the optimal hyperparameter configurations for the four ML models determined via gridserachcv.

2.4. Pearson’s Correlation Analysis

To gain a clearer understanding of the inter-relationships among variables within the dataset, we calculated the Pearson correlation coefficient (PCC). This statistic quantifies the linear dependency between two variables, producing values from −1 to 1. A PCC value of zero indicates the absence of a relationship between the attributes. Typically, a PCC threshold of 0.9 is used to determine if the variables exhibit a strong correlation. Identifying such a robust association can lead to the exclusion of a variable from the dataset to mitigate multicollinearity issues during analysis. The PCC, as a statistical measure, can be derived using the following equation [55]:

$$PCC={\displaystyle \frac{\frac{{\sum }_{i=1}^n(x_i-{\mu }_x)(y_i-{\mu }_y)}{n-1}}{{\sigma }_x{\sigma }_y}}$$

(3)

In the equation stated above, $x_i$ represents an input variable like $L/B$, D, B, $\gamma$, or $\varphi$, while $y_i$ denotes $q_u$. The means of the input variable and $q_u$ are indicated by ${\mu }_x$ and ${\mu }_y$, respectively, and the standard deviations for these are represented by ${\sigma }_x$ and ${\sigma }_y$.

Assessment of an ML model’s performance constitutes a crucial component of the ML modeling workflow. It precisely quantifies the predictive accuracy of the constructed model. In this investigation, three statistical metrics were employed—RMSE, MAE, and R² [56]. The mathematical formulations are as follows:

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{({\widehat{y}}_i-y_i)}^2}{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}}$$

(4)

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{({\widehat{y}}_i-y_i)}^2}$$

(5)

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^n|{\widehat{y}}_i-y_i|$$

(6)

where y represents the true output value, $\widehat{y}$ is the predicted outcome from the model, $\overline{y}$ denotes the mean of all true outputs, and n indicates the total number of data points in the training dataset.

3. Database Used

The dataset employed to tune and assess our ensemble ML models acquired from the literature included a series of 169 shallow foundation bearing capacity tests compiled by Khorrami et al. [46]. This dataset was collected from different studies carried out by researchers such as Golder et al. [57], Cerato and Lutenegger [58], Eastwood [59], Akbas and Kulhawy [60], Muhs and Weiz [61,62], Subrahmanyam [63], Muhs et al. [64], Gandhi [65], Weiz [66], and Briaud and Gibbens [67,68]. The data consist of two types of experimental scales: large-scale and small-scale. The large-scale experiments numbered 102, while the small-scale experiments numbered 65. Additionally, the dataset included experiments with a wide range of values for different factors that affect foundation strength.

According to

Table 2. Statistical overview of the input parameters and the target variable.

Statistical Parameter	Target Variable	Input Predictors
	${\mathit{q}}_{\mathit{u}}$ (kPa)	$\mathit{L}/\mathit{B}$	$\mathit{D}$ (m)	$\mathit{B}$ (m)	$\mathit{\gamma }$ (kN/m3)	$\mathit{\varphi }$ (Degree)
Minimum	14.00	1.000	0.000	0.030	9.850	31.950
Maximum	2847.00	6.000	0.890	3.016	20.800	45.700
Mean	481.53	2.217	0.119	0.532	15.637	39.208

, the maximum soil strength ($q_u$), measured in kN/m², varies from 14.00 kPa to 2847.00 kPa. The internal friction angle ($\varphi$) ranges from 31.950 to 45.700 degrees. The depth of the foundation (D) ranges from 0.000 m to 0.890 m, while the unit weight of the soil ($\gamma$) varies between 9.850 kN/m³ and 20.800 kN/m³. The width of the foundation (B) is between 0.030 m and 3.016 m, and the length-to-width ratio ($L/B$) ranges from 1.000 to 6.000. Some of the tests at the DEGEBO Berlin site were performed in wet conditions, so the unit weights of the submerged soil were used for the calculations [44].

Figure 5 illustrates the data distribution of key variables related to shallow foundations on granular soil. Each variable demonstrates different distribution characteristics. For example, $L/B$ and $q_u$ show a wide range with noticeable outliers, indicating significant variability. In contrast, B and D have compact and consistent distributions, mainly concentrated around their median values. The distributions of $\gamma$ and $\varphi$ are symmetric and continuous, while $q_u$ shows a more dispersed pattern with several extreme values, suggesting a multi-modal trend in certain cases.

Figure 6 illustrates the heatmap that shows the Pearson’s correlation coefficient (PCC) values between the predictors and the output variable. The PCC measures the linear correlation between variables, with values ranging from $-1$ to 1. A PCC of 0 indicates that there is no correlation, 1 represents a strong positive correlation, and $-1$ signifies a strong negative correlation. In ML, a high correlation (e.g., >0.85) between predictors can introduce the problem of multicollinearity, which can negatively impact model performance and interpretability. However, in this case, the correlations between predictors range from $-0.003$ to $0.58$, which is well below the multicollinearity threshold. This suggests that the predictors are sufficiently independent and suitable for use in an ML model without significant risk of redundancy or instability.

4. Model Results

4.1. Optimal Model Results

Figure 7 shows an analysis of the bearing capacity estimated by the algorithms compared to the measured recorded results.

Table 3. Comparative statistical metrics of ML models for ultimate bearing capacity prediction.

Criteria	Phase	LightGBM	GBRT	XGBoost	AdaBoost
MAE (kPa)	Training	73.256	60.162	66.166	126.794
RMSE (kPa)	Training	140.162	125.221	129.731	172.402
R2	Training	0.918	0.935	0.931	0.880
MAE (kPa)	Testing	90.635	78.938	87.868	115.820
RMSE (kPa)	Testing	148.496	133.401	138.515	151.685
R2	Testing	0.817	0.849	0.834	0.817
MAE (kPa)	All	75.930	63.050	69.505	125.106
RMSE (kPa)	All	141.476	126.514	131.121	169.380
R2	All	0.910	0.928	0.923	0.876

presents the MAE, RMSE, $R^2$, and other performance metrics for the total, training, and testing datasets. For the training data, the ML algorithms continuously show higher $R^2$ values, indicating that the models are properly trained. Furthermore, the RMSE and MAE scores for the training data are often lower than those for the testing data. Different ML methods have varying capacities to estimate the bearing capacity of shallow foundations in granular soil. Models are generally seen to have better prediction accuracy if they have higher $R^2$ and lower RMSE and MAE scores.

According to the data from Table 3 and Figure 7, the GBRT model performs better than other ML models on the full dataset in terms of $R^2$, RMSE, and MAE. The GBRT algorithm achieves an excellent performance of 0.928 for $R^2$, 126.514 kPa for RMSE, and 63.050 kPa for MAE. The XGBoost and LightGBM models come second and third, respectively, with RMSE scores of 131.121 kPa and 141.476 kPa and $R^2$ values of 0.923 and 0.910. With $R^2$ and RMSE values of 0.876 and 169.380, respectively, the performance of the AdaBoost model is lower compared to the other models. Among the models that were assessed, the GBRT model provides the most accurate predictions for the total dataset, surpassing competitors such as XGBoost and LightGBM. In contrast, the AdaBoost model has the lowest predictive performance.

To thoroughly assess model performance, one can compare the predictive accuracy of ML models on both the training and testing datasets. The results in Table 3 reveal that the GBRT model exhibited the highest $R^2$ evaluation index for the training set, with a value of 0.935, which indicates a strong correlation between the observed and predicted values. The $R^2$ values of 0.918 and 0.931, respectively, were slightly lower for the LightGBM and XGBoost models. The AdaBoost model demonstrated relatively poorer predictive performance within the training set, obtaining an $R^2$ of 0.88.

The results reveal that although the predictive performance of the developed models on the testing dataset showed relatively lower accuracy compared to the training phase, their overall effectiveness remains notable. The GBRT model demonstrated strong predictive ability, attaining an $R^2$ value of 0.849 in the test dataset. Similarly, the XGBoost model achieved an $R^2$ value of 0.834, reflecting its ability to estimate the target variable. On the other hand, the LightGBM and AdaBoost models exhibited slightly lower predictive performance, both producing an $R^2$ value of 0.817. Notably, the AdaBoost model displayed the smallest variation in $R^2$ between the training and testing phases, with a difference of 0.063, suggesting a balanced trade-off between model generalization and fitting. The observed differences in the $R^2$ values across the GBRT, XGBoost, and LightGBM models were recorded as 0.086, 0.097, and 0.10, respectively, indicating relatively stable performance across the datasets. Furthermore, as illustrated in Figure 7a, the AdaBoost model tends to concentrate its predictions for the testing set around 250 kPa, despite the actual values ranging between 0 and 500 kPa. This clustering effect indicates a limitation in AdaBoost’s ability to capture variability in the lower range of bearing capacity. In comparison, other ensemble models, such as GBRT, XGBoost, and LightGBM, provide more accurate and dispersed predictions, reflecting their enhanced capability in modeling complex relationships.

In conclusion, the findings of the comparison of the prediction models demonstrate that the GBRT model can accurately forecast the bearing capacity of shallow foundations. Its performance evaluation index scores are the highest when predicting training and test data and all data. Although the XGBoost and LightGBM models are less accurate than the GBRT prediction model, they are still capable of making accurate predictions. Ultimately, the performance of the AdaBoost model in this context is relatively weak. These findings suggest that the GBRT model stands out as the most effective ML model to estimate the bearing capacity of shallow foundations.

The performance of the GBRT bearing capacity prediction model was evaluated using a Taylor diagram [69] (Figure 8) along with the other tree-based ensemble learning algorithms. This graphic provides a thorough visual evaluation of each model’s ability to represent correlation and variability with respect to observed data. The standard deviation, RMSE, and Pearson correlation coefficient are the three main metrics used in the Taylor diagram. The perfect agreement between the model predictions and the actual data is represented by the ideal position on the diagram (shown by the red star) (RMSE = 0; correlation coefficient = 1). Models with better correlation coefficients and lower RMSE values are regarded as more accurate when they are situated closer to the “Obs” line, which is represented by a dark dashed line in the plot. The Taylor diagram for this assessment illustrates that all the ML models exhibit high accuracy, as they are positioned close to the “Measured” point on the chart. This indicates that the hyperparameters of these ML models have been effectively tuned to maximize their predictive performance. Among the models, GBRT stands out as the most accurate, being closest to the observed data point, which serves as a benchmark of precision. This reliability highlights the potential of these models to improve accuracy and overcome limitations in predicting bearing capacity.

4.2. Feature Importance Analysis

The Permutation Importance (PI) technique was used to perform a sensitivity analysis on the GBRT model in order to investigate the contribution of each input variable. This approach determines the relevance of a feature by permuting its values and analyzing the resulting impact on the model’s performance. If a variable is inconsequential, altering its values should not significantly affect the model’s accuracy. In contrast, if the variable is critical to the predictive capacity of the model, the permutation is expected to result in more errors. The PI score is calculated on the basis of the difference in the model’s performance before and after the permutation. Moreover, this method provides valuable insights into the model’s characteristics and can help identify potential issues such as multicollinearity and overfitting. The importance of the variable was identified as B (m), D (m), $\varphi$ (degree), $\gamma$ (kN/m³), and $L/B$. Among these, the width of the foundation (B) was shown to be the most influential factor in predicting the bearing capacity of shallow foundations, followed by depth (D), the angle of internal friction ($\varphi$), unit weight of soil ($\gamma$), and the length-to-width ratio ($L/B$), as illustrated in Figure 9. This finding underscores the significance of accurately measuring and managing these parameters in real-world applications to ensure reliable predictions of bearing capacity. Such insights can aid civil engineers in designing more efficient foundations, reducing the need for extensive on-site measurements and optimizing construction processes.

The dominance of foundation width (B) as the most influential factor aligns with conventional geotechnical theory, where the width directly affects the load-bearing area and influences stress distribution beneath the footing. The depth (D) and internal friction angle ($\varphi$) are also critical, as they contribute to overburden pressure and soil shear resistance. Unit weight ($\gamma$) and length-to-width ratio (L/B), while important, had a comparatively smaller impact within this dataset. This may be due to lower variability or interaction effects not being captured independently in the model. It is important to note that feature importance rankings can vary depending on the machine learning algorithm used, the method of importance calculation (e.g., permutation vs. gain-based), and the characteristics of the dataset, including the presence of multicollinearity. Our results are specific to the GBRT model trained on this particular dataset. Similar considerations are discussed in the work of Li et al. (2024) [70], which offers a broader methodological context for interpreting feature importance in geotechnical ML applications.

4.3. Comparing with Single Learner ML Models

This section utilized Support Vector Regression (SVR) and K-Nearest Neighbors (KNNs) models to compare their performance with the GBRT model developed in our study. The same procedure of data splitting and grid search hyperparameter tuning was also used to find optimal model parameters. For the SVR model, the hyperparameters used to optimize the model were (rbf and linear), the regularization parameter (C), epsilon, and gamma. On the other hand, the KNN parameters were the number of neighbors (n_neighbors), the weighting scheme (uniform or distance), and the distance metric ($p=1$ for Manhattan and $p=2$ for Euclidean).

The performance of the final models was evaluated using RMSE and $R^2$ based on the entire dataset. As we can see in

Table 4. Model performance comparison.

Model	RMSE (kPa)	${\mathit{R}}^2$
GBRT	126.514	0.928
KNN	133.36	0.9104
SVR	363.59	0.404

, the GBRT model demonstrated the best overall performance. Its RMSE is 126.514 kPa, with an $R^2$ of 0.928. The KNN model, on the other hand, also performed well, with an RMSE of 130.36 kPa and an $R^2$ of 0.9234, indicating a strong predictive capacity. In contrast, the SVR model showed a higher RMSE (363.59 kPa) and a lower $R^2$ (0.404), suggesting a relatively poor fit to the data.

The superior performance of GBRT is due to its ensemble learning approach, which combines multiple weak learners sequentially and handles the complex nonlinear relationship and interaction between the variables.

4.4. Reliability Analysis

This section provides a detailed examination of the reliability of our ML model and compares it with the pre-existing formulas used to estimate the maximum load-bearing capacity of shallow foundations. Shahnazari and Tutunchian [38] formulated a specific equation through the application of multigene genetic programming, a technique within the domain of soft computing. This particular equation is used to assess the maximum load that shallow foundations can support when located on granular soil types, which is delineated in Equation (7).

$$q_u=2\times {10}^{-12}{\varphi }^7(B+D){\left({\displaystyle \frac{L}{B}}+\varphi \right)}^2+{\displaystyle \frac{{10}^{-8}{D}^2{\varphi }^6{(B-\gamma )}^2}{\frac{L}{B}}}$$

(7)

Sadrossadat et al. [37] conducted a study in which they developed an equation to determine the ultimate bearing capacity using the linear genetic programming (LGP) technique. This specific formulation is detailed in the study and is denoted as per Equation (8).

$$P_u=\varphi \left(\left(\gamma +\varphi +{\displaystyle \frac{3.95{(\varphi -35)}^2}{L/B}}\right)D^2+2.5\left(B(\varphi -35)+1\right)\right)$$

(8)

In another study, Zhang [2] formulated two different equations to predict the bearing capacity of shallow foundations. The derivation of these equations utilized two advanced methodologies: Multiple Nonlinear Regression (MLNR) and Multigene Expression Programming (MEP). When subjected to comparative analysis, both predictive models demonstrated comparable levels of precision. As a result, for the sake of maintaining simplicity within our comparative analysis, we elected to employ the MLNR equation, specifically designated as Equation (9).

$$\begin{array}{cc}\hfill q_u=& 4073.9-17754.2\varphi +154\gamma -108.2{\displaystyle \frac{L}{B}}+3389.2D-24.4B\hfill \\ \hfill & +17459.3{\varphi }^2+4.6{\gamma }^2+32.7{\left({\displaystyle \frac{L}{B}}\right)}^2\hfill \\ \hfill & +428.9D^2-21.6B^2-327.6\varphi \gamma +247.9\varphi {\displaystyle \frac{L}{B}}\hfill \\ \hfill & -4051.6\varphi D+2961.6\varphi B-17.3\gamma {\displaystyle \frac{L}{B}}\hfill \\ \hfill & +22.0\gamma D-82.7\gamma B+14.5{\displaystyle \frac{L}{B}}D-59.5L-501.1DB\hfill \end{array}$$

(9)

It is worth noting that the previously mentioned ML equations outperformed the Terzaghi [3], Meyerhof [7], Hansen [6], and Vesic [10] equations. Figure 10 shows a broad spectrum of predictions from existing equations, with some, such as those proposed by Shahnazari and Tutunchian, showing a tendency towards conservative estimates, as reflected by the data points located in the underestimation zone. However, ML models demonstrate significantly more stable performance, with their median values closely aligning with the ideal line (predicted/actual = 1). This alignment indicates balanced accuracy, with minimal tendencies toward overestimation or underestimation. The proximity to the zero line underscores two key attributes of reliable predictive models: low bias and low variance. The GBRT model stands out for its exceptional accuracy, as evidenced by its narrow interquartile range, indicating predictions that are closely aligned with the actual values. This tight range within the optimal limits further highlights the GBRT model’s superior predictive reliability compared to both of the other ML models and previous equations. Since our model outperforms the ML equations mentioned earlier, it consequently also outperforms the Terzaghi [3], Meyerhof [7], Hansen [6], and Vesic [10] equations.

The performance of ML models and empirical formulas was quantitatively evaluated using the Demerit Points Classification (DPC) framework [71], as outlined in

Table 5. Demerit Point Classification based on the ratio ($P_{u\mathrm{pre}}$/$P_{u\exp }$).

Classification	Ratio Range (${\mathit{P}}_{\mathit{u}\mathbf{pre}}$/${\mathit{P}}_{\mathit{u}\mathbf{exp}}$)	Demerit Points
Extremely Conservative	<0.50	2
Conservative	$0.50\le \mathrm{Ratio}<0.85$	1
Appropriate and Safe	$0.85\le \mathrm{Ratio}<1.15$	0
Dangerous	$1.15\le \mathrm{Ratio}<2.00$	5
Extremely Dangerous	≥2.00	10

. The DPC results for each model, shown in

Table 6. Performance comparison of models based on reliability metrics using a demerit points system.

Model	Sum of Penalty Points	Ratio ≥ 2	1.15 ≤ Ratio < 2	0.85 ≤ Ratio < 1.15	0.5 ≤ Ratio < 0.85	Ratio < 0.5
GBRT	217	4	30	112	19	4
LightGBM	241	7	31	117	12	2
XGBoost	272	7	36	106	18	2
Shahnazari and Tutunchian [38]	288	4	27	52	59	27
Sadrossadat et al. [37]	314	4	44	76	36	9
Zhang and Xue [2]	319	1	48	64	43	13
Adaboost	577	27	55	55	32	0

, are calculated by first multiplying the number of samples within each category by their corresponding demerit points and then summing the obtained values. Generally, ML ensemble models achieve total demerit scores of less than 300 except of Adaboost model, whereas empirical formulas tend to exceed this threshold. With only 40% of its predictions categorized as ‘Appropriate and safe,’ the Zhang [2] equation has less performance that previous development of ML other formulas. The Gradient Boosting Regression Tree (GBRT) model achieves the lowest total demerit score of 217 among the ML models, with 70% of its predictions falling under the ‘Appropriate and safe’ category. These observations confirm the enhanced reliability and accuracy of ensemble ML models, with GBRT demonstrating superior performance over other approaches.

4.5. Developing a Web-Based Prediction Application

Finally, to bridge the gap between end-users with limited technical expertise and the complexity of our Gradient Boosting Regression Tree (GBRT) model, we developed a user-friendly interface. This interface enables users to interact with the model’s features effortlessly, without requiring extensive technical knowledge. The user interface was built using the Streamlit framework and incorporates ML to predict the bearing capacity of shallow foundations. Leveraging Python’s capabilities, we utilized the Streamlit library, alongside essential tools such as Joblib for loading a pre-trained scaler and Pickle for model persistence. The hyper-tuned GBRT model powers the interface, processes the input data, and generates accurate predictions. The interface features a clean and intuitive design that allows users to easily input key attributes of the dataset. This application allows civil engineers to predict the bearing capacity of shallow foundations without the need for on-site measurements, significantly reducing both time and costs. The link for the app is as follows: (https://foundation-evbmfjrfy5wrtyznfph4vb.streamlit.app, accessed on 12 August 2025). The app is designed to predict the ultimate bearing capacity of a shallow foundation. For users who want to obtain the allowable bearing capacity, a factor of safety can be used based on the recommendations from Whitlow [72], Craig [1], and Tomlinson [73]. Using this approach can help account for uncertainties, including potential prediction errors, and ensure safe design in engineering practice. These factors of safety can range between 2 and 3, as indicated by the previous references.

It is important to emphasize that the proposed ML model is not intended to replace established geotechnical design procedures or empirical formulas, particularly in unfamiliar or critical design scenarios. The applicability of the developed web application is restricted to the range of the independent variables used in the training dataset. Users must ensure that all input parameters fall within these limits and that the soil conditions match those on which the model was trained (i.e., granular soils without additional treatments such as tillage or compaction processes not captured in the dataset). Sound engineering judgment remains essential. Predictions generated by the model should be interpreted as supportive guidance and not as a substitute for conventional geotechnical analysis. Applying the model outside its validated domain may lead to unreliable or unsafe results. Future work may aim to extend the model’s robustness across broader soil types and field conditions by incorporating additional training data and hybrid modeling techniques.

5. Limitations and Future Studies

Future studies should focus on standardizing and expanding datasets through new experimental investigations conducted under consistent conditions to improve applicability and reliability. The predictive framework for shallow foundation bearing capacity could be further improved by incorporating additional soil and foundation data, such as settlement characteristics, shear strength, and deformation behavior. In this study, ensemble methods were successfully used to achieve high predictive accuracy, and the importance of permutation characteristics was used to identify critical input variables. Building on these advancements, future work could explore more sophisticated deep learning models to further enhance prediction capabilities and interpretability. Additionally, improving the GUI to support batch data processing and provide advanced visualization features would make the tool more practical and user-friendly. Furthermore, incorporating optimization techniques and conducting sensitivity analyses could enable the fine-tuning of foundation design parameters, thereby enhancing performance and safety. Future research can build on the findings of this study by addressing these challenges, paving the way for more adaptable, accurate, and practical tools to estimate and design the bearing capacity of shallow foundations.

6. Conclusions

This research investigated the suitability of state-of-the-art ML algorithms for estimating the maximum load-bearing capacity of shallow foundations on granular soils. The study was carried out in five stages, where a comprehensive database was examined and multiple ensemble ML algorithms were assessed. The resulting outcome is an accessible web application that can be used straightforwardly by geotechnical engineers. The subsequent findings were deduced:

• Research demonstrates the significant capability of ensemble techniques, particularly GBRT, LightGBM, XGBoost, and AdaBoost, to improve the precision of estimating shallow foundation bearing capacities. The GBRT algorithm exhibited the highest accuracy, with an $R^2$ value of 0.935 and an RMSE of 125.221 kPa in the training phase, as well as an $R^2$ of 0.849 with an RMSE of 133.401 kPa for testing. These results indicate that GBRT consistently outperforms the other models across multiple performance metrics. Such improvements can lead to enhanced reliability in geotechnical design calculations.
• The development of a web-based predictive application that utilizes the GBRT model with optimized hyperparameters marks significant progress in integrating complex ML models into regular design workflows. Available on a cloud platform, it eliminates computational and compatibility constraints, allowing engineers worldwide to make swift decisions via a universally accessible and user-friendly interface.