A Novel Hybrid Soft Computing Model Using Random Forest and Particle Swarm Optimization for Estimation of Undrained Shear Strength of Soil

Abstract

Determination of shear strength of soil is very important in civil engineering for foundation design, earth and rock fill dam design, highway and airfield design, stability of slopes and cuts, and in the design of coastal structures. In this study, a novel hybrid soft computing model (RF-PSO) of random forest (RF) and particle swarm optimization (PSO) was developed and used to estimate the undrained shear strength of soil based on the clay content (%), moisture content (%), specific gravity (%), void ratio (%), liquid limit (%), and plastic limit (%). In this study, the experimental results of 127 soil samples from national highway project Hai Phong-Thai Binh of Vietnam were used to generate datasets for training and validating models. Pearson correlation coefficient (R) method was used to evaluate and compare performance of the proposed model with single RF model. The results show that the proposed hybrid model (RF-PSO) achieved a high accuracy performance (R = 0.89) in the prediction of shear strength of soil. Validation of the models also indicated that RF-PSO model (R = 0.89 and Root Mean Square Error (RMSE) = 0.453) is superior to the single RF model without optimization (R = 0.87 and RMSE = 0.48). Thus, the proposed hybrid model (RF-PSO) can be used for accurate estimation of shear strength which can be used for the suitable designing of civil engineering structures.

1. Introduction

In civil engineering, the shear strength of the soil is an essential engineering property in the foundation design and stability analysis of all major construction projects such as dams, bridges, highways and road, railway lines, jetties, underground structures, and high-rise buildings [1,2]. It is well-known that the shear strength of soil is governed by interlocking between soil particles, frictional resistance, and cohesion of soil particles. Soil is a complicated material containing soil particles of different sizes and minerals, water, air, and void. The shear strength of soil is influenced by soil constituents, specific gravity, void ratio, moisture content (liquid and plastic limits), clay content, stress history, and relative density. The shear strength parameters are usually determined in the laboratory using direct shear test, unconfined compression test, and triaxial compression test and also in the field by shear vane test, Standard Penetration Test (SPT) and in situ-shear test [3,4]. In addition to these tests, the estimation of the shear strength of soil from other indirect methods is needed for quick and reliable results. Many researchers have attempted to estimate the shear strength of soil using different alternative methods [3,5,6,7,8,9,10]. The shear strength of unsaturated soil can also be predicted using the empirical correlation function [10,11], using soil-water retention curve [10,12,13].

Nowadays, soft computing techniques of machine learning (ML) or artificial intelligence (AI) have been widely used in many scientific, medical, and engineering fields including geotechnical engineering [14,15,16,17,18,19,20,21,22,23,24,25,26]. Sharma et al. [27] used Artificial Neuron Network (ANN) in estimating elasticity modulus of soil. Kalkan et al. [28] used Adaptive Neuro Fuzzy Inference System (ANFIS) and ANN methods for the prediction of compressive strength of compacted granular soils. They concluded that the ANFIS model gave a promising solution for predicting the compressive strength of the compacted soil. Other researchers have used several algorithms of ML namely Support Vector Regression (SVR) and its hybridization with Particle Swarm Optimization (PSO-SVR) [29] using some basic parameters such as water content, clay content, consistency limits, etc. [30]. Pham et al. [31] also developed and used Parsimonious Network based on a Fuzzy Inference System (PANFIS) hybrid ML model in the prediction of soil shear strength.

Random forest (RF) first introduced by Breiman for solving regression, unsupervised learning, and classification problems [32,33], is a powerful ML technique which has been applied successfully in many classification problems including geotechnical engineering [29]. However, for excellent RF modeling, fine-tuning of its hyperparameters is required by optimization algorithms. There are some good optimization algorithms that are usually employed in solving geotechnical engineering problems such as Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Ant Colony Optimization (ACO), Firefly Algorithm (FA), and Artificial Bee Colony (ABC). Out of these algorithms, the PSO is one of the most popular optimization techniques that has been mostly used in geotechnical engineering including slope stability and foundation engineering [27,34,35,36,37].

The main objective of the present study is to develop a novel hybrid soft computing model RF-PSO using goodness of individual models, namely RF and PSO, for the quick and better estimation of undrained shear strength of soil, based on the basic soil parameters such as moisture content, clay content, consistency limits, and Atterberg limit. The novelty of this study is that a hybrid model PSO-RF was developed the first time for better estimation of the shear strength of soil from basic parameters. For this, one of the national highway projects, Hai Phong-Thai Binh of Vietnam, was selected as a study area considering its importance and availability of the sufficient soil mechanics data for the development and validation of the model. Pearson correlation coefficient (R) and Root Mean Square Error (RMSE) methods were used to validate the model and sensitivity analysis was carried out to analyze the relationship between shear strength and influencing factors.

2. Case Study and Data Collection

2.1. Description of the Study Area

The study area is located along alignment of Hai Phong-Thai Binh coastal national highway, Vietnam. The total length of the highway project is approximately 29.7 km, which covers 20.782 km towards Hai Phong city side and 8.928 km towards Thai Binh province side (Figure 1). In this route, construction of eight bridges will be implemented. Two bridges on this alignment are large bridges with the length of 2 km and 1 km will cross over Van Uc and Thai Binh river, respectively. A total of 127 soil samples in this study were collected from the construction project of Hai Phong-Thai Binh coastal national highway for the estimation of soil shear strength and model study. The soil investigations carried out in this project included the following field tests: Standard Penetration Test (SPT), boring test, shear vane test, and laboratory tests (direct shear test and triaxial compression test) for the determination of engineering properties of soil [3,4]. However, only data of direct shear test with Undrained and Unconfined (UU) scheme [38] was used for this modeling study.

2.2. Data Used

2.2.1. Output (Undrained Shear Strength of Soil)

Output of this study is the undrained total normal shear strength parameter of soil. Shear strength of soil is defined as the maximum resistance per unit of soil that can mobilize to resist the shear stress causing the sliding failure in any plane of a soil mass. Shear strength of soil is known as an important parameter which is used in the design and analysis of stability problems of civil engineering structures. The sliding failure is related to both normal and shear stress, thus shear strength is considered as a linear function of normal and shear stress [38]. The undrained total normal shear strength (${\tau }_f$) is determined using the following equation.

$${\tau }_f=c+\sigma \tan \phi$$

(1)

where, c, ϕ, σ are the cohesion, internal friction angle, and normal stress, respectively.

In this study, direct shear tests with UU scheme was carried out to determine the values of undrained total normal shear strength for the modeling. Initial analysis of data used is presented in

Table 1. Initial analysis of data used in this study.

No	Parameters	Min Values	Max Values	Mean Values	Standard Deviation
1	Clay content (%)	1.00	47.5	25.72	10.172
2	Water content (%)	23.04	70.74	48.3	11.73
3	Specific gravity	2.67	2.72	2.69	0.01
4	Void ratio	0.63	1.92	1.36	0.31
5	Liquid limit (%)	26.08	79.76	53.34	13.39
6	Plastic limit (%)	15.36	40.48	28.38	5.01
7	Undrained total normal shear strength (kG/cm2)	0.29	0.57	0.41	0.06

.

2.2.2. Input Variables

Input data used in the model study include physical properties of soil: clay content, specific gravity, void ratio, and Atterberg limit (liquid limit and plastic limit). According to Das and Sobhan [38], the size of the clay particles are less than 0.002 mm. Some researchers have considered clay size range to be less than 0.002 to 0.005 mm [31,38]. The amount of clay particles directly affects the Atterberg limits such as liquid and plastic limits. The soil that contains more clay content could result in high plasticity when it absorbs water, thus leading to a decrease in shear strength parameters such as cohesion c and internal friction angle ϕ. The amount of clay can be determined from the grain size test [39].

Water content has an important role in influencing many soil properties such as strength, elasticity, and hydraulic conductivity [38]. Water content of the soil is governed by environment condition such as temperature, groundwater level, and humidity. With the increase of water content, cohesion between soil particles reduces; thus, the shear strength also decreases. It is well-known that the shear strength of soil is strongly affected by the moisture change, especially soil containing high clay minerals. Thus, in this research, we considered water content as one of the important input parameters for predicting the shear strength of soil. Water content is defined by the ratio between the mass (weight) of water divided to the mass (weight) of dried soil (i.e., soil particle) in a given volume of soil [39].

Specific gravity is a term that is used to compare how much lighter or heavier solid material (soil particles) is compared to water [38]. It is a ratio between the density of solid phase and water density. It is known that specific gravity is related to the density of minerals of soil: if the specific gravity is large, the soil will be denser with higher shear strength. Thus, the specific gravity must be considered as the main factor in predicting shear strength. In the laboratory, the specific gravity is directly determined by measuring density of soil samples using density bottle and Pycnometer methods [39]. Void ratio of soil not only influences the hydraulic conductivity but also affects significantly the shear strength of the soil. When soil has a large void ratio, the shear strength of soil will be small, because the higher void ratio could have a higher moisture content.

Liquid limit is an important parameter which influences the shear strength. The shear strength decreases with the increase of liquid limit [38]. Liquid limit is determined as the moisture content; at that point soil transfers from the plastic to the liquid state. Plastic limit is one of the important factors affecting the soil strength. It is defined as the water content point at which soil transfers states from semisolid to plastic [38]. These Atterberg limits can be determined by Atterberg tests in laboratory [38].

3. Methods Used

3.1. Random Forest

Random forest (RF) is a powerful method that was first introduced by Breiman for solving regression, unsupervised learning, and classification problems [32,33]. It has been applied in many fields including geotechnical engineering with high performance results [29]. The RF has some advantages such as high accuracy performance with complicated datasets with small calibrating and variables with high noises [40,41]. For the classification problem, Bagging technique is employed in order to arbitrarily choose the variable candidates from the entire dataset for calibrating models [15]. In this research, an Out-Of-Bag (OOB) sample and two kinds of errors (namely decrease in precision and reduction in Gini) were calculated because these errors can be adopted to rank and select variables [42,43]. Regarding each variable, when the variable values are transposed across the OOB observations, the function decides the error of prediction model [44].

3.2. Particle Swarm Optimization (PSO)

The PSO is a computational method which is a form of evolution algorithm such as GA and ant colony algorithm used in the optimization problem, initially proposed by Eberhart and Kennedy [45]. This algorithm differs from the GA as it focuses more on the interaction between individuals in a population to explore search space. The PSO is a result of modeling bird flying to find food; thus, it uses swarm intelligence. This algorithm is a powerful technique that has been employed commonly for optimization problems in many fields, especially in geotechnical engineering [14,31,46]. The PSO works with a random population of particles, in which each particle is considered as a given approach to seek a solution for solving the problem. The PSO includes a group of particles, each particle in a group moves indiscriminately in a research space and is affected by surrounding position during movement [47,48]. The result of each particle position is affected by its knowledge and the knowledge of its neighbors. Thus, it can be said that in a swarm the knowledge of other particles can influence the searching method of a particle. For each iteration, the position of each particle is upgraded considering its current position and velocity [35,49]. The next swarm was established in accordance with the updated particle positions considering their own best position (P_best) in search space and the whole swarm best position (G_best). The particle position and velocities are computed as follows:

$$V_i^{t+1}=w{V}_i^t+m_1{n}_1(p_{best,i}^t-Y_i^t)+m_2{n}_2(g_{best,i}^t-Y_i^t)$$

(2)

$$Y_i^{t+1}=Y_i^t+V_i^{t+1}$$

(3)

where $V_i^t$ and $V_i^{t+1}$ denote velocities of particle i at iteration t and t+1, respectively; whereas, $Y_i^t$ and $Y_i^{t+1}$ represent positions of particle i at repetition t and t+1; w, m_1, and m₂ correspond to the cognitive, social effect, and inertia parameters, respectively; n₁ and n₂ indicate arbitrary numbers with the range of [0, 1]; $p_{best,i}^t$ and $g_{best,i}^t$ symbolize the best position of particle i and swarm, respectively.

The best position of particle and swarm in the following iteration is defined as follows:

$$p_{best,i}^{t+1}=\{\begin{array}{l}{Y}_i^{t+1},h\left(Y_i^{t+1}\right)<h\left(p_{best,i}^t\right)\\ p_{best,i}^t,h\left(Y_i^{t+1}\right)\ge h\left(p_{best,i}^t\right)\end{array}$$

(4)

$$g_{best}^{t+1}=\arg \min \left\{h\left(p_{best,0}^{t+1}\right),\dots ,h\left(p_{best,ns}^{t+1}\right),h\left(g_{best}^t\right)\right\}$$

(5)

where ns indicate the summation of particles in a swarm.

3.3. Dataset Splitting

In the modeling, clay content, moisture content, specific gravity, void ratio, liquid limit, and plastic limit were used as input, whereas the undrained total normal shear strength of soil was used as an output. In other words, there were six inputs and one output for each instance in the dataset. All variables were normalized into (0, 1) range based on their maximum and minimum values.

For supervised learning, the dataset of 127 soil samples needs to be split into two parts. The first part is used for model training and hyperparameter tuning, which is known as the training set. The second part is known as the testing set used for model verification. As the size of the training set will have an important influence on the performance of ML modeling [49], to ascertain the best training set size (TSS) for the preparation of shear strength dataset, the TSS was changed from 30% to 90%. The RF performance was calculated with the default hyperparameters from Scikit-learn [50]. For each TSS, 100 RF models were built and the average performance was calculated to reduce the randomness in random splitting.

It is important to note that the performance on the training set was calculated using 5-fold CV validation. In terms of the performance on the testing set, the whole training set was first used to train the RF model, with which the prediction on the testing set was obtained. In other words, the prediction on the training set was obtained using the RF model trained with part of the training set, compared with the whole training set during the prediction on the testing set. Thus, the prediction on the testing set is usually better than that on the training set if the same prediction method is used [49].

3.4. Modeling and Hyperparameters Tuning

In the current study, the RF is utilized to model the nonlinear relationship from the inputs to the output. In order to obtain the expert performance, the hyperparameters of RF were tuned using the PSO. Five important hyperparameters were tuned as suggested in the literature [51].

Table 2. Hyperparameters description and their tuning range.

No	Hyperparameters	Explanation	Range
1	Max_depth	The maximum depth of DTs.	1–20
2	Min_samples_split	The minimum number of samples for the split.	2–10
3	Min_samples_leaf	The minimum number of samples at the leaf node.	1–10
4	Max_DT	The maximum number of RT models in the ensemble	1–1000
5	Max_features	The number of features considered during the selection of the best splitting	0.4–1

summarizes the tuned hyperparameters, the definition, and their tuning ranges. For each set of hyperparameters, 10 RF models were built to reduce the randomness of random splitting. The number of particles was set to be 100 in the PSO. Moreover, the maximum iteration, inertia parameter, the cognitive influence parameter, and the social influence parameter were set to be 50, 0.7298, 1.49618, and 1.49618, respectively. All parameter setting in the PSO was determined by trial tests [52,53,54,55].

During the hyperparameters tuning, the training performance from 5-fold CV was used as the fitness function of the PSO. Each set of hyperparameters was represented by a particle in the PSO. With the iteration of PSO, particle positions would be updated to maximize the fitness value and the hyperparameters were optimized accordingly. The optimum hyperparameters were selected after the PSO. Finally, the RF model with the optimum hyperparameters was verified on the testing set.

3.5. RF Model Assessment

The RF model assessment of this study was carried out using Pearson correlation coefficient (R) and Root Mean Square Error (RMSE) criteria, which are the most common indicators for validation and comparison of machine learning models [22,56,57,58]. Basically, R presents the correlation between the actual and predicted outputs. Values of R range from −1 to 1, and higher absolute values of R close to 1 indicate better prediction accuracy and vice versa. While RMSE measures the average squared difference between actual and predicted outputs [59,60,61,62]. Lower value of RMSE indicates better performance of the model. These indicators are expressed in equations as follows [63,64,65,66,67]:

$$R=\frac{{\displaystyle {\sum }_{i=1}^m\left(S{S}_{coi}-\overline{S{S}_{co}}\right)\left(S{S}_{aci}-\overline{S{S}_{ac}}\right)}}{\sqrt{{\displaystyle {\sum }_{i=1}^m{\left(S{S}_{coi}-\overline{S{S}_{co}}\right)}^2{\left(S{S}_{aci}-\overline{S{S}_{ac}}\right)}^2}}}$$

(6)

$$\mathrm{RMSE}=\sqrt{\frac{1}{m}{\displaystyle \sum _{i=1}^m{\left(S{S}_{coi}-S{S}_{aci}\right)}^2}}$$

(7)

where $S{S}_{coi}$ and $\overline{S{S}_{co}}$ denote the output value of the sample ith and the output average value of the sample calculated according to the model, respectively; $S{S}_{aci}$ and $\overline{S{S}_{ac}}$ indicate the actual value of the sample ith and the actual average values, respectively; m is the summation of samples.

4. Results and Discussion

4.1. Influence of Training Set Size (TSS)

From Figure 2, it can be seen that training performance progressively increased with the increase of TSS. Moreover, the standard deviation from the RF models was decreased with increasing TSS. To be more specific, the average R value was increased from 0.71 to 0.84 when the TSS was increased from 30% to 90%. Instead, the SD was decreased from 0.11 (TSS = 30%) to 0.03 (TSS = 90%). Above results demonstrate that the training performance was improved and became more stable with the increase of TSS.

In terms of the testing performance, it was increased from 0.79 to 0.87 when the TSS was increased from 30% to 80%. After that, the testing performance was decreased to 0.86 when the TSS was further increased to 90%. Standard Deviation (SD) was evidently increased from 0.08 to 0.12 when the TSS was increased to 80% to 90%. The increase of SD indicates that the RF prediction was negatively influenced when the TSS surpassed 80%. Since the testing performance represents the generalization capability of ML models, 80% was selected to be the best TSS in this study.

4.2. Hyperparameters Tuning

Figure 3 illustrates the highest R value ever found by the PSO with iterations. It can be seen that the highest R value was progressively increased with the iteration of PSO. The highest R was 0.83 at the first iteration, which was increased to 0.88 at the 50th iteration. The optimum RF hyperparameters were determined to be n_estimator = 935, max_depth = 14, min_sample_split = 2, min_samples_leaf = 1, max_features = 0.648.

4.3. Predictive Capability of the Models

Figure 4 presents a visual comparison of the UU based experimental and predicted results from a representative RF model. In this case, the representative RF model was selected since its performance was similar to the average performance from the RF models (Figure 3). The R value was 0.87 on the training set and 0.90 on the testing set. It can be seen that there was a good agreement between the experimental and predicted shear strength values, implying the robustness of RF modeling.

Figure 5 demonstrates performance comparison between the RF model with default hyperparameters and the optimum hyperparameters from the PSO (RF-PSO). To obtain more representative results, 100 RF models were constructed and the average performance was compared. It can be seen that the performance of RF modeling was improved after PSO hyperparameters tuning. On the training set, the R value was increased from 0.86 to 0.87 after the PSO hyperparameters tuning. At the same time, SD was decreased from 0.028 to 0.027, indicating more stable RF modeling. Similar results can be observed on the testing set, where the R value was increased (0.87 to 0.89) and SD was decreased (0.060 to 0.057). Similarly, the values of RMSE of RF-PSO on both training (0.487) and testing (0.453) is lower than those of sing RF model (0.517 for training and 0.48 for testing) (

Table 3. Predictive capability of the models using RMSE criteria.

No	Models	RMSE
		Training	Testing
1	RF	0.517	0.480
2	RF-PSO	0.487	0.453

). These results confirmed the feasibility of PSO in improving the performance of RF modeling.

In general, both hybrid model RF-PSO and single RF model performed well for the prediction of undrained shear strength of soil but hybrid model RF-PSO outperforms single RF model. The results are suitable as the RF can measure data structures and classify data, which help focus important variables and remove similar variables. It is not sensitive to unit differences, pointing out that there is no need for a preprocessing process [68]. In addition, the PSO is effectively applied to address the problem of complex optimization as it is automatically to search for optimization solutions and it can easily perform with good efficiency [69]. Thus, it is confirmed that the PSO is an effective optimization technique in improving performance of the RF model.

In general, the proposed RF-PSO model can be used for quick, better prediction of the undrained total normal shear strength of different types of soil. Performance of this model might be different and improved depending on type of soil. One of the advantages of application of hybrid machine learning model (RF-PSO) is that it can handle the big and complicated data. Thus, large or big data can also be analyzed with this model. It is proposed that researchers use large data, depending on the availability, for future model studies.

4.4. Sensitivity Analysis of Input Parameters

A sensitivity analysis was carried out to evaluate the importance of input parameters for the modeling using partial dependence plots, which is an efficient way to investigate the relationship between inputs and output. Details are available in the published work [70]. Figure 6 shows the relative importance of the inputs to the output. It can be seen that the moisture content was the most significant variable for the shear strength of soil, which achieved an average importance score of 0.337. The void ratio ranked the second with an average importance score of 0.271, followed by liquid limit (0.166), clay (0.109), and plastic limit (0.093). The specific gravity achieved the smallest average importance score (0.025), indicating that it had the lowest influence on the undrained total normal shear strength of soil.

Figure 7 illustrates the partial dependence of the output to the inputs. As shown, the shear strength had an overall negative correlation with clay, moisture content, specific gravity, and void ratio. Moreover, the variation of shear strength was less significant with the variation of specific gravity compared with the variation of clay, moisture content, and void ratio. This result indicates the specific gravity has a relatively lower influence on the undrained shear strength, which agrees well with the important score results (Figure 6). The undrained shear strength decreased first and then increased with the increasing liquid limit. Finally, the undrained shear strength increased with the increase of plastic limit.

In general, out of the input factors, moisture content is considered as the most important factor affecting the undrained shear strength of soil. This is reasonable as the water reduces the friction and cohesion between the soil particles; thus, increase of moisture content leads to decrease of shear strength of soil [31,71].

5. Conclusions

In this study, a novel hybrid machine learning namely RF-PSO model, which is a combination of RF and PSO models, was proposed and applied to estimate the undrained shear strength of soil for the design purpose of the construction projects. In total, the experimental results of 127 samples were used to create datasets for validating and training models. A statistical measure such as R was used to validate and compare the models. The results show that performance of the models improved and stabilized from 0.79 to 0.84 with the increase of training dataset size from 30% to 80%. Performance of the RF-PSO hybrid model is best with R = 0.89 and RMSE = 0.453, followed by RF with R = 0.87 and RMSE = 0.48 in the estimation of soil shear strength.

In addition, the sensitivity analysis using partial dependence plots was carried out to evaluate the importance of input parameters in the model study. Results show that moisture content is considered as the most important parameter for modeling of prediction of the undrained shear strength though other parameters considered are also important.

In this study it is seen that that the proposed hybrid model RF-PSO is capable of predicting shear strength of undrained soil quickly with basic soil properties in a better way for use in the design of civil engineering structures. A limitation of this study is the number of samples tested from one of the highway projects of Vietnam. It would be better to use large/big data from other projects to confirm its wider applicability.