Application of Artiﬁcial Intelligence to Determined Unconﬁned Compressive Strength of Cement-Stabilized Soil in Vietnam

: Cement stabilized soil is one of the commonly used as ground reinforcement solutions in geotechnical engineering. In this study, the main object was to apply three machine learning (ML) methods namely gradient boosting (GB), artiﬁcial neural network (ANN) and support vector machine (SVM) to predict unconﬁned compressive strength (UCS) of cement stabilized soil. Soil samples were collected at Hai Duong city, Vietnam. A total of 216 soil–cement samples were mixed in the laboratory and compressed to determine the UCS. This data set is divided into two parts of the training data set (80%) and testing set (20%) to build and test the model, respectively. To verify the performance of ML model, various criteria named correlation coefﬁcient (R), mean absolute error (MAE) and root mean square error (RMSE) were used. The results show that all three ML models were effective methods to predict the UCS of cement-stabilized soil. Amongst three model used in this study, optimized ANN model provided superior performance compare to two others models with performance indicator R = 0.925, RMSE = 419.82 and MAE = 292.2 for testing part. This study can provide an effective tool to quickly predict the UCS of cement stabilized soil with high accuracy.


Introduction
Ground under the foundation is an important part, which has the effect of bearing most or all of the load on the building. The presence of soft soil layers under the foundation can cause problems for buildings [1]. However, in recent decades, the urban population is increasing rapidly, increasing the need for infrastructure, so soft ground areas are also studied for the construction of buildings. These soils are often characterized by high plasticity, high void ratio and low strength [2]. Soft soil can be reinforced by various methods depending on specific conditions [3], such as mechanically stabilized earth (MSE) embankments [4], granular or sand compaction piles [5], vertical drains [6] and the lime/cement deep mixing method [7]. In another study, YI Oh and EC Shin used pile reinforcement revetments and ground net reinforcement on soft ground to reduce deflection settlement [8]. Among the soft soil reinforcement methods, the cement-stabilized sandy soil method has been used for many years [9,10]. The cementation of sandy soil can increase the hardness, shear strength and compressive strength of the material [9]. Many researchers have investigated the mechanical properties of cement-treated soil by various methods. For example, Changizi and Haddad [11] ran a series of unconfined compression tests and direct shear tests, their studies have shown that the unconfined compressive strength (UCS) and soil adhesion will increase when the nanosilica content increases. In addition, using the unconfined compression tests and the California bearing ratio (CBR) tests, Ghasabkolaei et al. [12] and Choobbasti et al. [13] also concluded similarly about the positive relationship between UCS value and nanosilica content that is in the composition of cement. In many other studies, the authors also build experimental models to predict the strength of cement stabilized soft ground [14].

Experimental Measurement of UCS of Cement-Stabilized Soil
In this study, a database containing 216 soil samples were collected at Tran Nguyen Han street, Thanh Binh district, Hai Duong city, Vietnam ( Figure 1). Soil samples were collected at two −2.0 m and −4.0 m depth, in three boreholes. Those boreholes were made with 10 m distance from each other. Soil samples were collected by pressing a cylinder into the soil block at the boreholes, and then covered with a moisture-proof bag to keep original water content ( Figure 2). Collected soil were stored and transported by boxcar from the collecting site to the laboratory.  Soil parameters was analyzed, evaluated and processed through specific stages: moisture content tests; wet density tests; mix soil cement; forming and curing of specimen and then unconfined compression tests were performed using specialized equipment ( Figure 3). Experimental results obtained the relationship of axial stress-strain curves (Figure 4) and the UCS of the samples were defined as destructive compressive axial stress. An example of axial stress-strain curves are given in Appendix B ( Figure A1).
It is important to note that three kinds of general cement in the north of Vietnam were used for the cement mixing test, including Vissai cement, Chinfon Hai Phong cement and

Experimental Measurement of UCS of Cement-Stabilized Soil
In this study, a database containing 216 soil samples were collected at Tran Nguyen Han street, Thanh Binh district, Hai Duong city, Vietnam ( Figure 1). Soil samples were collected at two −2.0 m and −4.0 m depth, in three boreholes. Those boreholes were made with 10 m distance from each other. Soil samples were collected by pressing a cylinder into the soil block at the boreholes, and then covered with a moisture-proof bag to keep original water content ( Figure 2). Collected soil were stored and transported by boxcar from the collecting site to the laboratory.  Soil parameters was analyzed, evaluated and processed through specific stages: moisture content tests; wet density tests; mix soil cement; forming and curing of specimen and then unconfined compression tests were performed using specialized equipment ( Figure 3). Experimental results obtained the relationship of axial stress-strain curves (Figure 4) and the UCS of the samples were defined as destructive compressive axial stress. An example of axial stress-strain curves are given in Appendix B ( Figure A1).
It is important to note that three kinds of general cement in the north of Vietnam were used for the cement mixing test, including Vissai cement, Chinfon Hai Phong cement and Soil parameters was analyzed, evaluated and processed through specific stages: moisture content tests; wet density tests; mix soil cement; forming and curing of specimen and then unconfined compression tests were performed using specialized equipment ( Figure 3). Experimental results obtained the relationship of axial stress-strain curves ( Figure 4) and the UCS of the samples were defined as destructive compressive axial stress. An example of axial stress-strain curves are given in Appendix B ( Figure A1).
It is important to note that three kinds of general cement in the north of Vietnam were used for the cement mixing test, including Vissai cement, Chinfon Hai Phong cement and Nghi Son cement. A cement mixing ratio (i.e., the amount of cement in q m 3 of the mixture) was applied to 100 kg/m 3 , 150 kg/m 3 and 200 kg/m 3 . The specimens were cured during 7 days and 28 days at the indoor and outdoor condition. With the indoor curing condition, the specimens were covered with a sealant to prevent water evaporation from the specimen and the specimen was placed in the curing container. On the other hand, outdoor curing of specimens were placed in a room without any controlled intern of temperature and humidity, and without the sealant. the specimens were covered with a sealant to prevent water evapo men and the specimen was placed in the curing container. On th curing of specimens were placed in a room without any controlled and humidity, and without the sealant.

Data Preparation
It is known that the UCS values depend on a large number study, factors that are important in the UCS of the soil-cement m were selected. The soil type (denoted as S) seem to influence the U determines the soil's grain composition [22]. Moisture content of so Mc) also play an important role in the UCS detecting [23,24]. The w noted as We) should also be considered as it determines the natura to mixing [24]. The soil sampling depth (denoted as D) should also when it affects the soil condition [25]. We cannot fail to mention Axial stress (kPa) Axial strain (%) curing of specimens were placed in a room without any controlled i and humidity, and without the sealant.

Data Preparation
It is known that the UCS values depend on a large number o study, factors that are important in the UCS of the soil-cement mi were selected. The soil type (denoted as S) seem to influence the UC determines the soil's grain composition [22]. Moisture content of soil Mc) also play an important role in the UCS detecting [23,24]. The we noted as We) should also be considered as it determines the natural to mixing [24]. The soil sampling depth (denoted as D) should also b when it affects the soil condition [25]. We cannot fail to mention th (denoted as Ac) used for mixing when it determines the adhesion b [26]. Factors related to the sample after mixing such as: specimen d

Data Preparation
It is known that the UCS values depend on a large number of parameters. In this study, factors that are important in the UCS of the soil-cement mixture determination were selected. The soil type (denoted as S) seem to influence the UCS of the mixture as it determines the soil's grain composition [22]. Moisture content of soil samples (denoted as Mc) also play an important role in the UCS detecting [23,24]. The wet density of soil (denoted as We) should also be considered as it determines the natural state of the soil prior to mixing [24]. The soil sampling depth (denoted as D) should also be taken into account when it affects the soil condition [25]. We cannot fail to mention the amount of cement (denoted as Ac) used for mixing when it determines the adhesion between soil particles [26]. Factors related to the sample after mixing such as: specimen diameter (denoted as Di); specimen length (denoted as L); specimen area (denoted as A); specimen volume (denoted as V); mass of specimen (denoted as M) and density of specimen (denoted as De) might also affect much to the prediction [27,28]. In addition, a number of other important factors to consider were the curing condition (denoted as Cc) and the curing period (denoted as Cp) that affect the rate of strength development of the cement. Finally, the type of cement (denoted as T) was also taken into consideration when the quality of cement in Vietnam appears to be uneven among each other. The UCS of the cement-soil mixture (denoted as q u ) was the single output. Detailed statistics of the parameters used in the study are presented in Table 1. An example of this data set is given in Appendix A (Table A1).

Gradient Boosting (GB)
Gradient boosting (GB) is a machine learning algorithm developed by scientist Jerome Friedman [29]. GB is suitable for regression and classification problems [30], has high adaptability and is able to model feature parent and inherently perform feature selection [31]. In it, many decision trees are planted sequentially using information from existing trees [32]. Each tree (weak learner) was added to improve previous learners, forming an additive model [33]. That helps the prediction become more accurate (good learner model), the process ends when no further improvement is possible [32]. So, by continuously adjusting and optimizing the weak learner's weight in order for it to be a good learner, in which the learner's weights are optimized by gradient descent of the loss function [30]. Mathematically the model can be viewed as [34]: where F(x) is the output model, γ i is the learner's weight of iteration i th , h i (x) is weak learner of iteration i th and M is the number of iterations.

Artificial Neural Network (ANN)
The artificial neural network is a form of artificial intelligence that mimics the behavior of the human brain and nervous system [35]. ANN can also be learned through the gradient descent of the loss function, namely the back propagation method [36]. ANN is a powerful tool for predicting non-linear problems [37]. The non-linear mapping reinforces the linkages between the input data and the output data [38]. The operational structure of ANN has been described in many studies [39,40]. Multilayer perceptron is the most widely used ANN type.
It consists of an input data layer, an output data layer and in the middle is one or more hidden layers [36]. The model complexity is determined by the number of nodes of hidden layers. The input weight matrix is used to link the input layer and the hidden layer. The output weight matrix is used to link the hidden layer and the output layer [41]. The output values are compared with the expected values in the set in the training data, errors are calculated and returned to the network. The connection weight is automatically adjusted until the minimum error target is reached [38]. The architecture of ANN model was illustrated in Figure 5. more hidden layers [36]. The model complexity is determined by hidden layers. The input weight matrix is used to link the inpu layer. The output weight matrix is used to link the hidden layer an The output values are compared with the expected values in the errors are calculated and returned to the network. The connection adjusted until the minimum error target is reached [38]. The arch was illustrated in Figure 5.

Support Vector Machine (SVM)
SVM is a popular learning model first proposed by Vapnik (1 of the support vector machine algorithm is to find a hyper-plane space that distinctly classifies the data points. To separate two lay tiple hyper-plane can be selected. However, the most suitable pl maximum margin (i.e., the maximum distance between data poin has the optimization goal of minimizing structural risk [43], mini and model complexity [20]. SVM exhibits a significant improve which is often used. Use when the target variable involves catego amount of data and non-linearity, multiple input parameters [43, ing data set: x y x y x R y r where x is the input and y is the output. R n is the N-dimensional v one-dimensional vector space [21]. The final equation of SVM can where β and * β are the Lagrangian multipliers and nsv is the

Support Vector Machine (SVM)
SVM is a popular learning model first proposed by Vapnik (1995) [42]. The objective of the support vector machine algorithm is to find a hyper-plane in a multidimensional space that distinctly classifies the data points. To separate two layers of data points, multiple hyper-plane can be selected. However, the most suitable plane is the one with the maximum margin (i.e., the maximum distance between data points of both layers). SVM has the optimization goal of minimizing structural risk [43], minimizing predictive error and model complexity [20]. SVM exhibits a significant improvement in functionality, which is often used. Use when the target variable involves categorical data [42], a small amount of data and non-linearity, multiple input parameters [43,44]. Consider the training data set: where x is the input and y is the output. R n is the N-dimensional vector space and r is the one-dimensional vector space [21]. The final equation of SVM can be written as: where β i and β * i are the Lagrangian multipliers and nsv is the number of support vectors. K(x i x j ) is the kernel function [45].

Hyperparameters Tuning with the Random Search (RS) Method
In machine learning, hyper-parameters are valuable parameters used to control the learning process, for example the number of neurons in the hidden layer of the ANN model, the kernel type of SVM or the number of trees in the GB model. It must be asserted that the hyper-parameters control the behaviors of training algorithms and has a significant effect on the performance of machine learning models [44]. In addition, it is difficult to compare models once they are not optimized or compare an optimal model to a suboptimal one.
In general, there are different ways to optimize a math problem, for example grid search (GS) and random search (RS) [45,46] or use some global optimization algorithm such as Bayesian optimization [44], genetic algorithm [47] and particle swarm optimization [48]. Among those algorithms, GS and RS showed simple algorithm and good performance [45,46,49]. The two algorithms have different strategies for searching in hyperparameters space.
The comparison between the GS and RS method was shown in Figure 6. It can be seen that GS looks for different hyper-parameter combinations in order while the RS chooses the hyper-parameter randomly within the permitted range. Bergstra and Bengio [46] have shown that RS is more interesting than GS in the case of some machine learning algorithms on some data sets. The RS method can significantly reduce the number of solutions to seek before there is a high probability to find the most accurate model, on the contrary, it can be seen that to find a good solution, the GS method must have the thick mesh and takes more resources to deal with. In order to compare the performance of different machine learning algorithms, only RS technique is chosen to find the optimal hyper-parameters for the construction of ML models. model, the kernel type of SVM or the number of trees in the GB model. It must be asserted that the hyper-parameters control the behaviors of training algorithms and has a signifi cant effect on the performance of machine learning models [44]. In addition, it is difficul to compare models once they are not optimized or compare an optimal model to a subop timal one.
In general, there are different ways to optimize a math problem, for example grid search (GS) and random search (RS) [45,46] or use some global optimization algorithm such as Bayesian optimization [44], genetic algorithm [47] and particle swarm optimiza tion [48]. Among those algorithms, GS and RS showed simple algorithm and good perfor mance [45,46,49]. The two algorithms have different strategies for searching in hyper-pa rameters space.
The comparison between the GS and RS method was shown in Figure 6. It can be seen that GS looks for different hyper-parameter combinations in order while the RS chooses the hyper-parameter randomly within the permitted range. Bergstra and Bengio [46] have shown that RS is more interesting than GS in the case of some machine learning algorithms on some data sets. The RS method can significantly reduce the number of so lutions to seek before there is a high probability to find the most accurate model, on the contrary, it can be seen that to find a good solution, the GS method must have the thick mesh and takes more resources to deal with. In order to compare the performance of dif ferent machine learning algorithms, only RS technique is chosen to find the optimal hyper parameters for the construction of ML models.

K-Fold Cross Validation
K-folding cross validation (CV) is a common technique in machine learning, used during model training and editing, helping to avoid over fitting with the final model Fig  ure 7 showed flowchart of the 5 fold cross-validation technique. In this technique, the training data was divided into five folding sets. The training will be done in five iteration and for each time 4 folds will be used for training and the remainder fold will be used fo verification. The performance of the model was the average of the performance of the five iteration results.

K-Fold Cross Validation
K-folding cross validation (CV) is a common technique in machine learning, used during model training and editing, helping to avoid over fitting with the final model Figure 7 showed flowchart of the 5 fold cross-validation technique. In this technique, the training data was divided into five folding sets. The training will be done in five iterations and for each time 4 folds will be used for training and the remainder fold will be used for verification. The performance of the model was the average of the performance of the five iteration results.

Performance Indicators
In this paper, to evaluate the accuracy of the model, the author used three indicators accounting for the error between the actual and predicted values were used, namely the

Performance Indicators
In this paper, to evaluate the accuracy of the model, the author used three indicators accounting for the error between the actual and predicted values were used, namely the mean absolute error (MAE) [51], root mean square error (RMSE) [52] and correlation coefficient (R) [50]. R measures the correlation between the predicted value and the actual value, valid in the range [−∞, 1]. The model will have high accuracy when R approaches 1. Conversely, low RMSE and MAE show better accuracy of the proposed ML algorithms. More specifically, the ideal values of RMSE and MAE are 0. On the other hand, RMSE calculates the squared root average difference, whereas MAE calculates the difference between the predicted and actual values. These values can be calculated using the following equations: where k is the number of the observations, p o and p o are the measured and mean measured values of the critical; p t and p t are the predicted and mean predicted values, respectively.

Hyperparameters Tuning Results
In this section, three ML models including GB, ANN and SVM were developed to predict the USC of the cement-stabilized soil. The hyper-parameters range of those ML models was also given in Table 2. To prepare the data for the hyper-parameters tuning process, the initial data set was random divided into two sets, including the training set (80%) and testing set (20%). To avoid data leakage, ML models were evaluated based on data from the 5 fold CV technique, which mean testing data was hidden in this step.  In the process of hyper-parameter tuning, the model with the best R performance indicator was selected as the final model and the model's hyper-parameters were considered the optimum hyper-parameters. A summary of the optimal hyper-parameters of each model was presented in Table 3. It can be seen that all three models showed good performance after hyper-parameter optimization when the R criterion was above 0.87. The hyper-parameters combined quite complexly to create the best model. In the GB model, the higher learning rate seemed to bring better performance when in the ANN model, Adam was the best training algorithm for this data set. Furthermore, the SVM model with the kernel type of the radial basis function gives better performance than the sigmoid function. Besides, the lower the gamma on SVM model, the lower the performance. Out of the three models, ANN and GB showed outstanding performance compared to the SVM model. To be more specific, the best R criteria of the ANN and GB model was 0.93 and 9.29 respectively compared to 0.871 of the SVM model.

Comparison of GB, ANN and SVM
From a statistical standpoint, the randomness in the data set needed to be carefully considered when comparing models. In this section, to compare the performance of the three optimized models, 300 samplings were performed taking into account the random division between training set and testing set. In these samplings, the training and test set sizes were kept the same, however the index number of the training and test data were randomly selected in the original data set. The models would be built on the training set and then validated on the testing set. Figures 8-10 showed a density curve of the performance results after 300 samplings on the training set and testing set. The summary of the performance indicators of each models was presented in Tables 4-6. It can be seen that the values of R of all three models showed a strong prediction UCS of cement-stabilized soil as the values of R were in the range of 0.9-1 on the training set and in the range of 0.8-1 on the testing set. The values of RMSE were in the range of 150-350 (kPa) on training set and in the range of 200-650 (kPa) on the testing set while the value of MAE varied from 50 to 250 (kPa) on the training set and from 100 to 400 (kPa) on the testing set.
It also can be seen that out of the three models, the ANN model gave outstanding performance, reflected in the average of all performance indicators, namely R = 0.925, RMSE = 419.82 and MAE = 292.2 on testing set. The GB and SVM models showed equal performance when the GB model had better performance at R but worse at RMSE and MAE. To be more specific, GB had the average performance indicator of R = 0.912, RMSE = 446.79 and MAE =319.23 while SVM model had the average criteria R = 0.903, RMSE = 446.67 and MAE = 309.76 on the testing set. In addition, the minimum and maximum values of the performance indicators of the ANN modet all allowed it to outperform the other models, proving that the model was more stable.
performance, reflected in the average of all performance indicators, namely R = 0.925, RMSE = 419.82 and MAE = 292.2 on testing set. The GB and SVM models showed equal performance when the GB model had better performance at R but worse at RMSE and MAE. To be more specific, GB had the average performance indicator of R = 0.912, RMSE = 446.79 and MAE =319.23 while SVM model had the average criteria R = 0.903, RMSE = 446.67 and MAE = 309.76 on the testing set. In addition, the minimum and maximum values of the performance indicators of the ANN model all allowed it to outperform the other models, proving that the model was more stable.   RMSE = 419.82 and MAE = 292.2 on testing set. The GB and SVM models showed equal performance when the GB model had better performance at R but worse at RMSE and MAE. To be more specific, GB had the average performance indicator of R = 0.912, RMSE = 446.79 and MAE =319.23 while SVM model had the average criteria R = 0.903, RMSE = 446.67 and MAE = 309.76 on the testing set. In addition, the minimum and maximum values of the performance indicators of the ANN model all allowed it to outperform the other models, proving that the model was more stable.

Predictability of Models
In this section, the results of typical ML models were presented. All three models showed good prediction when the linear fit almost overlapped with the best fit on both the training set and testing set ( Figure 11). Out of the three models, ANN showed the best performance when all prediction points on the training and testing set were almost closest to the perfect fit. Based on the analysis results, it can be confirmed that the ML models were successful in predicting UCS of the cement-stabilized soil and optimized ANN was the most suitable model for this data set. Table 7 presented some previous research results on ML applications in determining USC of some soil type. The results of the present study and previous studies show the expected effect of the ML technique in determining the UCS of soils with most of the R reaching between 0.8 and 0.95 on the testing data set. However, due to the use of different data sets, the comparison among these results make no sense. A project that combines datasets from different studies is needed to create a large database for building generalized models in the UCS prediction of soil reinforcement.
In this section, the results of typical ML models were presented. All three models showed good prediction when the linear fit almost overlapped with the best fit on both the training set and testing set ( Figure 11). Out of the three models, ANN showed the best performance when all prediction points on the training and testing set were almost closest to the perfect fit. Based on the analysis results, it can be confirmed that the ML models were successful in predicting UCS of the cement-stabilized soil and optimized ANN was the most suitable model for this data set.  Table 7 presented some previous research results on ML applications in determining USC of some soil type. The results of the present study and previous studies show the expected effect of the ML technique in determining the UCS of soils with most of the R reaching between 0.8 and 0.95 on the testing data set. However, due to the use of different data sets, the comparison among these results make no sense. A project that combines datasets from different studies is needed to create a large database for building

Feature Importance Analysis
The GB algorithm allows estimating the importance of input features. In fact, the GB algorithm included many decision trees and for each tree, the feature importance of an input variable was calculated as the fraction of samples that will traverse a node that splits based on that variable. The mean score of all trees then decided the important index of each features. The important index scores will be in the range [0, 1] and the higher scores the more important the feature.
The result shown on Figure 12: It can be seen that amongst the 14 input variables that used to detect the UCS of cement-stabilized soil, the wet density (We) and the amount of cement (Ac) was the most important features, which score an average sensitive index of 0.7 and 0.212, respectively. From a soil mechanic point of view, We affects the unit weight of soil or decides particle density while the amount of cement (Ac) decides the cohesion between the soil particles, so both of them play an important role in predicting the UCS of cement-stabilized soil. The variables M, Mc and De were ranked as the third to the fifth important predictors with an average sensitive index, ranging from 0.12 to 0.085. The other variables such as D, Cc, A, V and S had a lower sensitive index, ranging from 0.051 to 0.026, indicating that they are not affected much by the regression result. Remain features include T, Cp, Di and L, which had an important index lower than 0.006, showing that they nearly did not affect the prediction result.

Conclusions
The main aim of this study was to develop three machine-learning methods to predict the USC of cement-stabilized soil. The models were optimized by the RS technique to find out the best architecture including some hyper-parameters that had a significant effect on the performance of machine learning models.

Conclusions
The main aim of this study was to develop three machine-learning methods to predict the USC of cement-stabilized soil. The models were optimized by the RS technique to find out the best architecture including some hyper-parameters that had a significant effect on the performance of machine learning models.
The results showed that all three optimized machine-learning model including GB, ANN and SVR had an impressive ability in predicting the USC of cement-stabilized soil with R criteria ranging from 0.85 to 1. Besides, 300 simulations including randomization of data between the training set and the testing set were conducted. It can be seen that, among the three models used in this study, the ANN model had superior performance compared to the other two models on both training and testing training, represented in the average performance index of 300 simulations, specifically R = 0.98, RMSE = 231.2 and MAE =115.29 for the training set and R = 0.925, RMSE = 419.82 and MAE = 292.2 for the testing set.
In addition, the feature important index analysis by the GB model showed that between 14 input variables, the wet density (We) and the amount of cement (Ac) was the most important features, which play an important role in predicting of the UCS of cement-stabilized soil.
The results of this study indicated that machine learning methods, especially the ANN model, can be an effective tool for quickly predicting UCS of cement stabilized soils with excellent performance.