A Kernel Extreme Learning Machine-Grey Wolf Optimizer (KELM-GWO) Model to Predict Uniaxial Compressive Strength of Rock

Abstract

Uniaxial compressive strength (UCS) is one of the most important parameters to characterize the rock mass in geotechnical engineering design and construction. In this study, a novel kernel extreme learning machine-grey wolf optimizer (KELM-GWO) model was proposed to predict the UCS of 271 rock samples. Four parameters namely the porosity (P_n, %), Schmidt hardness rebound number (SHR), P-wave velocity (V_p, km/s), and point load strength (PLS, MPa) were considered as the input variables, and the UCS is the output variable. To verify the effectiveness and accuracy of the KELM-GWO model, extreme learning machine (ELM), KELM, deep extreme learning machine (DELM) back-propagation neural network (BPNN), and one empirical model were established and compared with the KELM-GWO model to predict the UCS. The root mean square error (RMSE), determination coefficient (R²), mean absolute error (MAE), prediction accuracy (U₁), prediction quality (U₂), and variance accounted for (VAF) were adopted to evaluate all models in this study. The results demonstrate that the proposed KELM-GWO model was the best model for predicting UCS with the best performance indices. Additionally, the identified most important parameter for predicting UCS is the porosity by using the mean impact value (MIV) technique.

1. Introduction

Uniaxial compressive strength (UCS) is one of the most important parameters for determining the behavior of intact rocks, both in geotechnical and mining engineering [1,2,3,4,5,6]. Initially, the UCS was obtained mainly through laboratory uniaxial compression tests using the standards proposed by the American Society for Testing and Materials (ASTM) or the International Society for Rock Mechanics (ISRM) [7,8,9,10]. Nevertheless, the laboratory experiments of UCS have been controversial for three reasons, including time-consuming, cost-ineffective, and rock sample quality-dependent [2,11,12,13]. Therefore, it is practically meaningful and scientifically significant to develop economical and effective but robust methods to obtain UCS to meet the needs of engineering practices and research.

Several scholars used parameters to establish single regression formulas to predict UCS, e.g., the Schmidt hammer rebound number (SHR), P-wave velocity (V_p), and point load strength (PLS) [11,14,15,16,17,18,19]. Nevertheless, these empirical relationships between a single parameter and UCS are not satisfactory in rock engineering [20,21]. To tackle this problem, multiple regression analysis (MR) has been developed to predict UCS considering at least two parameters related to the rock properties (

Table 1. Multiple regression analysis (MR) for predicting UCS.

References	Equation	Rock Type	Performance
[2]	UCS = 0.0065Vp + 1.468BPI + 4.094PLS + 2.418TS − 225	WE, FR, THR	RMSE = 15.62
[12]	UCS = −6.479 + 3.425BPI + 0.639CPI + 7.889PLS	MU	R2 = 0.87
[20]	UCS = −595.303 − 442.363Vp + 45.338Vp2 − 6.1 Pn + 0.52 Pn2 + 28.314 (PLS − 4.06PLS)2 + 115.822SH − 2.007SH2	TR	R2 = 0.64
[21]	UCS = 0.386EH + 39.268r − 1.307 Pn − 246.804	SA, LI, DO, GR, GRA	RMSE = 2.91
[23]	UCS = 0.88r2.24SH0.22CI0.89	IG, SR	R = 0.55
[25]	UCS = 0.48SH + 1.863PLS + 248WC + 7.972Vp − 23.859	GY	RMSE = 7.332
[29]	UCS = exp (0.011BPI + 0.065PLS + 0.029SH + 0.000012Vp + 2.157)	CL, MU	R2 = 0.91
[33]	UCS = 15.14UW + 2.88SHR − 446.3	MA, DO, LI, TR	R2 = 0.79
[35]	UCS = −120.912 − 2.036Vp + 31.064PLS	TR	RMSE = 9.43
[65]	UCS = 0.25EH + 18.14r − 0.75 Pn − 15.47GS − 21.55RT	SA, LI, DO, GR, GRA	R2 = 0.90
[66]	UCS = 0.89SH + 131.PLS − 1.68Vp − 35.9	MA, LI, DA	RMSE = 11.38
[67]	UCS = 5.734Vp + 10.876TS − 2.408PLS − 10.029	TR, LI, DI	R = 0.90
[68]	UCS = −2.572 Pn + 23.665PLS + 41.654PR + 12.197r − 0.001Vp − 11.813	PYR	RMSE = 11.40
[69]	UCS = −153.61 Pn + 0.010Vp + 7.111PLS	GR	RMSE = 13.81
[70]	UCS = 1.277SH + 2.186BPI + 16.41PLS + 0.011Vp − 82.436	GS, WS, BS, GY, SM	RMSE = 10.80
[71]	UCS = −350.784 − 1.825 Pn + 82.749r + 5.708SHR	GRA, GA	R2 = 0.89
[72]	UCS = −22.1 + 0.4SHR + 0.0093Vp + 3.9PLS	GR	R2 = 0.79
[73]	UCS = 2.411u + 0.004Vp + 4.322TS + 2.583E − 49.700 UCS = 0.0003u3.099Vp0.172TS0.206E0.393	WCJR	R2MLRA = 0.853 R2MNRA = 0.855

Note: EH: Equotip hardness; u: unit weight; r: density; P_n: porosity; GS: grain size; RT: rock type; V_p: P-wave velocity; BPI: block punch index; PLS: point load strength; TS: tensile strength; SH: Schmidt hammer number; CI: cone indenter hardness; WC: water content; PR: Poisson’s ratio; CPI: cylinder punch index; UW: unit weight; SHR: Schmidt hardness rebound number; E: elastic modulus; SA: sandstone; LI: limestone; DO: dolomite; GR: granite; GA: gabbro; GRA: granodiorite; WE: weak; FR: fractured rock; THR: thin-bedded rocks; MA: marble; DA: dacite; IG: igneous; SR: sedimentary rocks; GY: gypsum; TR: travertine; DI: dolomitic limestone; CL: claystone; MU: mudstone; PYR: pyroclastic rocks; GS: grainstone; WS: wackstone/mudstone; BS: boundstone; SM: silty marl; WCJR: weakly cemented Jurassic rocks; R²: coefficient of determination; RMSE: root mean square error; R: Pearson correlation coefficient; MLRA: multiple linear regression analysis; MNRA: multiple nonlinear regression analysis.

). It is found that the prediction accuracy for most of the multiple regression equations is not sufficient. Furthermore, some research proved that the prediction performance of artificial intelligent models is better than that of traditional statistical methods [22]. Therefore, an increasing number of research seeks to predict UCS using artificial intelligence (AI) technologies, including artificial neural network (ANN) [20,23,24,25,26], fuzzy inference system (FIS) [27,28,29,30], support vector machine (SVM) [31,32,33,34], random forest (RF) [35,36], adaptive neuro-fuzzy inference system (ANFIS) [12,37,38,39], multi-layer perceptron (MLP) [32,40], gene expression programming or gene programming (GEP or GP) [13,21,41,42], and genetic algorithms (GA) [43].

Compared to other AI methods, the extreme learning machine (ELM) was rarely reported to predict UCS of rock samples [44]. On the contrary, ELM has been widely used to solve a variety of engineering prediction problems such as backbreak, flyrock, and penetration rate [45,46,47,48,49,50]. ELM was firstly developed by [51] based on a special neural network architecture, namely the single-layer feed-forward neural network (SLFN). To improve the prediction accuracy of ELM, Huang et al. [52] developed a kernel-based ELM (KELM) model which is less affected by collinearity. Nevertheless, the single AI model sometimes falls into a local minimum and results in poor prediction performance [53,54,55]. The meta-heuristic algorithm based on biological behavior was used as an effective method to solve optimization problems [19,56,57,58,59,60,61,62,63,64].

A novel model consisting of KELM and the grey wolf optimizer (GWO) was developed to predict UCS in this study. Besides, the other five models were also implemented to predict UCS of rock samples and compared their results with the prediction performance of the KELM-GWO model. These five models included four AI models and one empirical model, i.e., ELM, KELM (initial), deep extreme learning machine (DELM), back-propagation neural network (BPNN), and an empirical formula.

2. The Novel KELM-GWO Model for Estimating the Uniaxial Compressive Strength

2.1. Kernel Extreme Learning Machine (KELM)

The kernel extreme learning machine (KELM) was modified by Huang et al. [52] based on the ELM. The conventional ELM model has a single-hidden-layer feed-forward neural network (SLFN) architecture, which can be written as follows:

$$\mathrm{F}(x_i)={\displaystyle \sum _{i=1}^m{\beta }_{i}g\left(w_i·x_i+b_i\right)=o_i} i=1, 2, 3,\dots ,N$$

(1)

where $\mathrm{F}(x_i)$ represents the ith output of the ELM. ${\beta }_i$ and $w_i$ represent the output weight vector and the input weight vector of the ith neuron in the hidden and input layer, respectively. $b_i$ shows the bias of the ith neuron in the hidden layer. N represents the number of samples and m represents the number of neurons in the hidden layer. On the basis of the activation function a(x), Equation (1) can be described by the following mapping matrix:

$$T=H·B={\left[\begin{array}{c}{t}_1^T\\ ⋮\\ t_n^T\end{array}\right]}_{N\times m}H(\begin{array}{c}h(x_1)\\ ⋮\\ h(x_N)\end{array})={\left[\begin{array}{ccc}a(w_1·x_1+b_1)& \cdots & a(w_m·x_1+b_m)\\ ⋮& \cdots & ⋮\\ a(w_1·x_N+b_1)& \cdots & a(w_m·x_N+b_m)\end{array}\right]}_{N\times m}B=={\left[\begin{array}{c}{\beta }_1^T\\ ⋮\\ {\beta }_m^T\end{array}\right]}_{m\times n}$$

(2)

where T, H, and B represent the target output matrix, the feature mapping matrix, and the output weight matrix, respectively. t_i represents the ith output vector, h(x) represents the feature mapping in the hidden layer, and n shows the number of neurons in the output layer.

To obtain the best ELM in the training phase, the least square solution of the linear equation must be estimated as follows:

$$\Vert H(w_1,\dots ,w_m,b_1,\dots ,b_m)\widehat{B}-T\Vert =\underset{{\beta }_i}{\min }\Vert H(w_1,\dots ,w_m,b_1,\dots ,b_m)B-T\Vert$$

(3)

To avoid ELM collinear problems, a penalty term C and a unit matrix I were proposed by Huang et al. [52] to optimize Equation (3) based on the ridge regression method and the Tikhonov regularization idea. Therefore, the least square solution of the output weight can be rewritten as:

$$\widehat{B}=H^T{(H{H}^T+\frac{C}{I})}^{-1}T$$

(4)

Then, the kernel function K () was introduced to replace the unknown of the feature mapping function h(x) in the initial ELM:

$${\mathsf{\Omega }}_E=h\left(x_i\right)h\left(x_j\right)=K\left(x_i,x_j\right)$$

(5)

Hence, Equation (1) can be changed to by combining Equations (4) and (5):

$$\mathrm{F}(x)=h(x)H^T{(\frac{C}{I}+H{H}^T)}^{-1}T=\left[\begin{array}{c}K(x,x_1)\\ ⋮\\ K(x,x_N)\end{array}\right]·{(\frac{C}{I}+{\mathsf{\Omega }}_E)}^{-1}T$$

(6)

The kernel functions have important influence on the KELM model performance. Therefore, choosing the right kernel function is the first step to solve a particular problem. Wang et al. [74] reported seven kernel functions for the KELM, such as the Gauss kernel function, the linear kernel function, the polynomial kernel function, the Fourier kernel function, etc. In fact, the Gauss kernel function is one of the most popular kernel equations [75], which is also called the radial basis function (RBF) and can be written as follows:

$$K(x,y)=exp(-\upsilon {\Vert x-y\Vert }^2)$$

(7)

where $\upsilon$ is the kernel width of RBF.

2.2. Grey Wolf Optimizer (GWO)

The grey wolf optimizer (GWO) is a meta-heuristic optimization algorithm developed by Mirjalili et al. [76], which was inspired by the predation behavior of grey wolves. GWO is characterized by a simple algorithm structure and an easy implementation, which can be applied to solve the optimization problem by adjusting the population size [77]. Nevertheless, the hunting abilities of grey wolves depend on their hierarchical social relationships. There are four classes of wolves from top to bottom in the group of grey wolves, namely alpha, beta, delta, and omega. Alpha is responsible for the development of the whole group, including hunting, resting, food distribution, etc. Beta is responsible for conveying instructions given by the alpha to other wolves and reporting to the alpha the feedback information of other wolves; who is second only to the alpha. Delta is mainly responsible for the peripheral work of the group, such as detecting prey and protecting the other wolves. If the behavior of the delta is not in the population interest, they can be demoted to the lowest class, called omega. Based on the hierarchy and division of labor in this strict social bond, the hunting behavior of grey wolves is meticulous and efficient. The detailed description was introduced in [78].

In the hunting process, the grey wolves can recognize prey by searching and tracking and surround the prey from different directions to prevent it from escape. This behavior is called encircling, which can be expressed mathematically as follows:

$$P_w(t+1)=P_p(t)-A·D D={\displaystyle |}C·P_p(t)-P_w(t){\displaystyle |}$$

(8)

where P_w and P_p represent the positions of the grey wolf and the prey, respectively. t describes the current iteration; D indicates the distance between a grey wolf and prey. A and C represent coefficients, which can be calculated by the following formula:

$$A=2a·r_1-a C=2·r_2$$

(9)

where a represents a convergence factor that decreases from 2 to 0 as the number of iterations increases. r₁ and r₂ indicate two random numbers in the range of [0, 1].

The wolves begin to move towards prey after encircling. As shown in Figure 1a, this behavior is carried out under the leadership of the alpha, with the beta and delta following the alpha to participate in the hunting occasionally. Thus, the positions of different wolves during the movement can be expressed by the following formulas:

$$D_{\alpha }=\left|C_1·P_{\alpha }(t)-P_{\omega }\right|,D_{\beta }=\left|C_2·P_{\beta }(t)-P_{\omega }\right|,D_{\delta }=\left|C_3·P_{\delta }(t)-P_{\omega }\right|$$

(10)

$$P_1=P_{\alpha }(t)-A_1·D_{\alpha },P_2=P_{\beta }(t)-A_2·D_{\beta },P_3=P_{\delta }(t)-A_3·D_{\delta },P_{\omega }(t+1)=\frac{1}{3}{P}_1+\frac{1}{3}{P}_2+\frac{1}{3}{P}_3$$

(11)

where $D_{\alpha }$, $D_{\beta }$, and $D_{\delta }$ represent the distance between the alpha ($\alpha$), beta ($\beta$), delta ($\delta$), and omega ($\omega$), respectively. P₁, P₂, and P₃ describe the current positions of the alpha ($\alpha$), beta ($\beta$), and delta ($\delta$), respectively. $P_{\omega }$ and $P_{\omega }(t+1)$ show the current and final position of the omega, respectively. Although the grey wolves have captured prey, they may give up and move on to better ones. This behavior is called exploration or exploitation (see Figure 1b) and can be controlled by A. If A > 1, the grey wolves choose to leave the prey. On the contrary, the grey wolves attack the prey.

2.3. Novel Hybrid KELM-GWO

In the development of the initial KELM model, choosing the right hyperparameters combination of C and $\gamma$ had a great impact on the model performance, especially being that these parameters were not sensitive to data in the training phase [79]. Whereas, the GWO algorithm was developed to optimize similar problems by considering the number of wolves [77]. Therefore, the GWO algorithm was used to select the suitable kernel parameters (C and $\gamma$) of the KELM model for the UCS prediction in this study. To this end, 70% of the dataset was used for training the KELM-GWO model and the rest of the data (30%) was regarded as a testing set to evaluate the prediction performance of the trained model. Before running the training and testing processes, the dataset was normalized to −1 and 1 by using the MinMax scaling method. Subsequently, the initial KELM can be developed with random kernel parameters; then, the function of the GWO algorithm was to find the optimal kernel parameters that met the target prediction accuracy with different numbers of populations and iterations. The mean squared error (MSE) was applied to evaluate the predictive power of the model. In other words, the kernel parameters corresponding to the minimum value of MSE are the best for predicting UCS. The framework of the proposed KELM-GWO model to estimate UCS of rock is illustrated in Figure 2.

$$MSE=\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(y_i-y_i^{*}\right)}^2}$$

(12)

where n is the number of samples; y_i and $y_i^{*}$ are the actual and predicted values.

In order to compare the performance of UCS predicted by the KELM-GWO model and other AI models or empirical equations, such as ELM, initial KELM, DELM, BPNN, and an empirical formula were developed in this study. The principle of DELM and BPNN can found in the literature [26,48,80,81,82,83,84,85].

3. Dataset

In this study, there are 271 pieces of data of UCS obtained from various rock samples by Dehghan et al. [20], Armaghani et al. [69], and Mahmoodzadeh et al. [34], including travertine, claystone, granite, schist, sandstone, travertine, limestone, slate, etc. On the basis of previous works conducted on the UCS prediction as shown in Table 1, four parameters, namely, the porosity (P_n, %), the Schmidt hardness rebound number (SHR), the P-wave velocity (V_p, km/s), and the point load strength (PLS, MPa), were considered as input variables for the UCS (MPa) of rock samples prediction in this study. The detailed violin plots and correlation matrix of the dataset are illustrated in Figure 3 and Figure 4. The violin plots are used to show the data distributions and their probability densities. This is a type of chart which combines the features of a box chart and a density chart. The thick black bar in the middle indicates the interquartile range, while the white dot shows the median. Figure 3 clearly shows the data details of the input and output parameters, such as the minimum, minimax, and median. As can be seen in Figure 4, the relationships between their variables are nonlinear, and the three variables except P_n are positively correlated with UCS. Observing the histograms, the distributions of all variables are skewed and are bimodal skewed distributions. Accordingly, all variables need to be normalized before being applied to the models.

4. Performance Indices for the Assessment of Models

In this study, the root mean square error (RMSE), the determination coefficient (R²), the mean absolute error (MAE), the prediction accuracy (U₁), the prediction quality (U₂), and the variance accounted for (VAF) were used as performance indicators to evaluate the reliability and accuracy of all AI models and empirical formula for predicting UCS. These indicators are defined as follows [47,49,86,87,88,89,90,91,92,93,94,95]:

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(UC{S}_{o,i}-UC{S}_{p,i}\right)}^2}}$$

(13)

$${\mathrm{R}}^2=1-\frac{{\left[{\displaystyle \sum _{i=1}^n(UC{S}_{o,i}-UC{S}_{p,i})}\right]}^2}{{\left[{\displaystyle \sum _{i=1}^n(UC{S}_{o,i}-\overline{UC{S}_o})}\right]}^2}$$

(14)

$$\mathrm{MAE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{\displaystyle |}UC{S}_{o,i}-UC{S}_{p,i}{\displaystyle |}}$$

(15)

$${\mathrm{U}}_1=\frac{RMSE}{\sqrt{\frac{1}{n}{\displaystyle {\sum }_{i=1}^{n}UC{S}_{o,i}{}^2}}+\sqrt{\frac{1}{n}{\displaystyle {\sum }_{i=1}^{n}UC{S}_{p,i}{}^2}}}$$

(16)

$${\mathrm{U}}_2=\frac{{\displaystyle {\sum }_{i=1}^n{(UC{S}_{o,i}-UC{S}_{p,i})}^2}}{{\displaystyle {\sum }_{i=1}^{n}UC{S}_{o,i}{}^2}}$$

(17)

$$\mathrm{VAF}=\left[1-\frac{\mathrm{var}(UC{S}_{o,i}-UC{S}_{p,i})}{\mathrm{var}(UC{S}_{o,i})}\right]\times 100$$

(18)

where n represents the number of rock samples in the training or testing phase, $UC{S}_{o,i}$ and $\overline{UC{S}_o}$ are the observed values and mean observed values of UCS, respectively, $UC{S}_{p,i}$ and $\overline{UC{S}_p}$ are predicted values and mean predicted values of UCS, respectively.

5. Developing the Models for Predicting UCS

In this study, five AI models (i.e., ELM, KELM, KELM-GWO, DELM, and BPNN) and one empirical model have been developed to predict UCS of rock samples. In what follows, the development of the models is presented and discussed comprehensively.

5.1. ELM

To develop a suitable ELM model for UCS prediction, an SLFN architecture should initially be created with a certain number of neurons in a single hidden layer. For this purpose, the number of neurons was set in the range of 20–150 and 14 ELM models were proposed in this study. RMSE and R² were used to select the optimal number of hidden layer neurons, which means that the number of neurons corresponding to the model with the lowest value of RMSE and the highest values of R² is the best model for predicting the UCS. As can be seen in

Table 2. Performance evaluation of ELM models with different number of neurons in the hidden layer.

Model No.	Neurons of Hidden Layer	RMSE		R2
		Training	Testing	Training	Testing
1	20	28.7283	27.4770	0.6786	0.7050
2	30	24.0466	21.9013	0.7480	0.8126
3	40	22.5454	21.5436	0.8021	0.8186
4	50	22.1986	21.7206	0.8081	0.8157
5	60	22.3844	21.3123	0.8049	0.8225
6	70	19.6458	22.7176	0.8497	0.7983
7	80	19.9072	25.3655	0.8457	0.7486
8	90	17.6883	31.2900	0.8782	0.7683
9	100	18.7275	25.3018	0.8634	0.7499
10	110	17.5671	23.6957	0.8798	0.7806
11	120	18.8266	27.1939	0.8620	0.7110
12	130	18.6357	34.9914	0.8648	0.5216
13	140	16.8519	34.6346	0.8894	0.5313
14	150	17.1542	34.7546	0.8714	0.5285

Note: Line in bold represents the better solution.

, in model number 5 with 60 neurons in the hidden layer, the lowest RMSE and the highest R² were obtained in the testing phase, even if it was not optimal in the training phase. Therefore, an SLFN architecture with 60 neurons in the hidden layer was considered for the ELM model to predict UCS in the following study.

5.2. KELM

The KELM model is an improved version of the ELM model to predict UCS by using kernel functions. Therefore, the numbers of neurons in the hidden layer no longer need to be changed, but different kernel parameters were chosen to predict UCS. In particular, the values range of the kernel parameters should depend on the specific problem, such as the range of [2⁻²⁰, 2²⁰] being used by Zhu et al. [79] and the range of [2⁻⁸, 2⁸] being considered by Baliarsingh et al. [96]. To determine these parameters, the C and $\gamma$ were the same and changed in the range [2⁻², 2⁹]. The performance of the KELM models with different kernel parameters was evaluated in terms of RMSE and R² in the training and testing phase, as shown in Figure 5. As can be seen in this figure, the KELM model with the lowest RMSE and the highest R² is considered as the best model, which has the best kernel parameters of C and $\gamma$ being 2⁰.

5.3. KELM-GWO

Although the KELM model with the kernel parameters C and $\gamma$ of 2⁰ show a good predictive performance, it is not practical to manually test every possible parameter combination due to the large range of [2⁻², 2⁹]. Therefore, the modified KELM with the new kernel parameters was obtained by using the GWO algorithm, and a reasonable comparison between the KELM and novel hybrid KELM-GWO could be conducted. It is interesting to note that only the numbers of grey wolves should be tuned in the GWO algorithm to optimize the other models [77]. To this purpose, the populations of grey wolves were considered to be equal to 25, 50, 75, 100, 150, and 200 for 600 iterations in this study. Meanwhile, the MSE is used as the stopping condition to select the optimal population, as shown in Figure 6. As can be seen in this figure, the numbers of populations have no effect on the fitness values after 500 iterations. The ideal KELM-GWO model with the lowest MSE was reached at the population of 75, where it has the best kernel parameters of C = 254.63 and $\gamma$ = 0.52, respectively. Therefore, the final KELM-GWO model considering C = 254.63 and $\gamma$ = 0.52 was used for predicting UCS of rock samples.

5.4. DELM

Similar to other neural network models, the performance of the DELM model is also controlled by the numbers of hidden layers and the corresponding number of neurons. For this purpose, 3, 4, and 5 multi-hidden layers with different numbers of neurons (5, 10, and 15) were considered to predict UCS. To verify the performance of all DELM models, the RMSE and R² were also recorded in the training and testing phases, as shown in

Table 3. Performance evaluation of DELM models with various numbers of hidden layers.

Model Multi-Hidden Layers	RMSE		R2
	Training	Testing	Training	Testing
5-5-5	31.1924	30.0702	0.6211	0.6467
10-10-10	28.4753	27.6212	0.6843	0.7019
15-15-15	28.5459	27.6791	0.6827	0.7006
5-5-5-5	34.0340	32.3176	0.5490	0.5919
10-10-10-10	35.5967	33.8719	0.5066	0.5517
15-15-15-15	32.3059	30.5482	0.5936	0.6354
5-5-5-5-5	32.1756	30.6333	0.5969	0.6333
10-10-10-10-10	34.0784	33.2670	0.5478	0.5676
15-15-15-15-15	40.7977	39.3185	0.3519	0.3959

Note: Line in bold represents the better solution.

. As can be seen in this table, in the DELM model with numbers of 10-10-10 neurons in three hidden layers, the lowest value of RMSE and the highest value of R² in the training and testing phase were obtained and recorded, respectively. Therefore, the DELM model considering the numbers of 10-10-10 neurons in three hidden layers was selected to predict UCS in this study, as illustrated in Figure 7.

5.5. BPNN

BPNN is a typically and widely used neural network model for prediction problems in geotechnical engineering [80,81]. To develop an effective BPNN model for predicting UCS of rock samples, the numbers of hidden layers and neurons should be determined to prevent overfitting and reduce the computation time. Therefore, double hidden layers with different neurons were developed; the RMSE and R² were recorded as evaluation indicators to compare the performance of all models.

Table 4. Performance evaluation of BPNN with various neurons of two hidden layers.

Model	Hidden Layers		RMSE		R2
	1	2	Training	Testing	Training	Testing
1	2	2	24.7580	20.6986	0.7613	0.8326
2	2	4	20.3518	17.7909	0.8387	0.8763
3	2	6	21.6890	24.4836	0.8168	0.7658
4	2	8	20.0367	17.5967	0.8437	0.8790
5	2	10	25.3568	21.7866	0.7496	0.8145
6	4	4	20.4635	18.0377	0.8369	0.8729
7	4	6	23.7747	21.6982	0.7799	0.8160
8	4	8	19.9445	17.2805	0.8602	0.8833
9	4	10	20.0098	17.9689	0.8441	0.8738
10	6	6	24.7290	20.3359	0.7619	0.8384
11	6	8	19.2109	17.1627	0.8563	0.8849
12	6	10	22.4402	18.1689	0.8039	0.8710
13	8	8	19.7370	17.5722	0.8633	0.8793
14	8	10	22.5188	19.8386	0.8025	0.8462
15	10	10	22.7941	19.1961	0.7941	0.8560

Note: Line in bold represents the better solution.

shows a summary of the obtained values of RMSE and R² in the training and testing phases. As can be seen in this table, model number 11, i.e., two hidden layers with 6 and 8 neurons, has shown the lowest value of RMSE in both the training and testing phases; the highest value of R² in the testing phase. Therefore, model number 11 was considered as the final BPNN model to predict UCS, as depicted in Figure 8.

5.6. Empirical Model

Multivariate regression analysis (MRA) is a useful tool to create an empirical model for predicting UCS, as summarized in Table 1. MRA is made of various parameters to achieve the best-fit equation through performing the least squares fit, which can be described as a multivariate equation of first order by Armaghani et al. [97]. Therefore, an empirical formula, Equation (19), was proposed to predict UCS based on the four input variables considered in this study.

$$UCS=-128.37-3.34\times \mathrm{PLS}-0.776\times {\mathrm{P}}_{\mathrm{n}}+0.0212\times {\mathrm{V}}_{\mathrm{p}}+2.789\times \mathrm{SHR}$$

(19)

6. Results and Discussion

The ELM, KELM, KELM-GWO, DELM, BPNN models, and the empirical equation were run to predict UCS after ensuring that the optimal involving parameters and the performance of all the six models were evaluated in terms of six statistical indices (i.e., RMSE, R², MAE, U₁, U₂, and VAF) previously mentioned in this study. The results of the performance indices in the training and testing phases of all models were represented in

Table 5. Comparison of the performances of all models (training phase).

Model	Performance (Training)
	RMSE	R2	MAE	U1	U2	VAF (%)
ELM	22.3844	0.8049	15.9951	0.1041	0.0443	80.4891
KELM	22.2684	0.8069	15.9380	0.1038	0.0442	80.7237
KELM-GWO	17.2176	0.8846	12.0577	0.0798	0.0259	88.4566
DELM	28.4753	0.6843	22.4743	0.1306	0.0677	69.1651
BPNN	19.2109	0.8563	13.6784	0.0905	0.0344	85.9230
Empirical	26.3309	0.7300	19.8876	0.1223	0.0610	73.0331
Model	Performance (Testing)
	RMSE	R2	MAE	U1	U2	VAF (%)
ELM	21.3123	0.8225	17.0308	0.1038	0.0452	82.4108
KELM	17.5050	0.8803	12.4699	0.0854	0.0308	88.1894
KELM-GWO	14.7327	0.9152	11.4315	0.0706	0.0259	91.5207
DELM	27.6213	0.7019	22.9152	0.1332	0.0730	70.3401
BPNN	17.1627	0.8849	11.9449	0.0945	0.0305	89.4529
Empirical	24.0864	0.7733	18.2310	0.1181	0.0595	77.6171

Note: Line in bold represents the better solution.

. As can be seen in this table, the KELM-GWO model was the best model for predicting UCS with the best performance indices in both the training (RMSE: 17.2176; R²: 0.8846; MAE: 12.0577; U₁: 0.0798; U₂: 0.0259; VAF: 88.4566%) and testing phases (RMSE: 14.2176; R²: 0.9152; MAE: 11.4315; U₁: 0.0706; U₂: 0.0259; VAF: 91.5207%). Nevertheless, it is important to note that not all AI models perform better than the empirical formulas. It should be noted that the DELM model is worse than the empirical formula in both training and testing phases with an RMSE of 28.4753–27.6213, R² of 0.6843–0.7019, MAE of 22.4743–22.9152, U₁ of 0.1306–0.1332, U₂ of 0.0677–0.0730, and VAF of 69.1654–70.3401%. The reason for this result is that the DELM lacks a reverse fine-tuning process compared with a traditional deep learning algorithm. Although more hidden layers are added, excessive input weights aggravate the training load and result in a low prediction performance. After the KELM-GWO model, the BPNN, KELM, and ELM have a close competition in both the training and testing phases.

To further compare the predictive performance of the six models, the regression diagrams of all the models in the training and testing phases are demonstrated in Figure 9. As can be seen in this figure, the horizontal and vertical axes represent the observed values of UCS measured in laboratory experiments and the predicted values proposed by AI models and the empirical equation in this study, respectively. Particularly, the data point will appear on the black diagonal line in each diagram when the predicted value is equal to the observed value. Meanwhile, other radial lines with 10% and 30% are second and third criteria to indicate deviations from the predicted values. Accordingly, the more predicted values appear within the 10% and 30% lines and the closer they are to the black diagonal line, the better the prediction performance of the model. As can be realized, the KELM-GWO model not only has the least predicted values outside the 30% line, but also has the most predicted values close to the black diagonal line in both the training and testing phases. After the KELM-GWO model, BPNN, KELM, and ELM have a similar prediction performance. Nevertheless, there are more predicted values of UCS away from the black diagonal in the empirical and DELM model.

To clearly show the performance of the six models in the training and testing phases, the error histograms were described in Figure 10. The horizontal axis and the vertical axis represent the error between the predicted value and its total percentage (%), respectively. That is to say, increasing the tall error bars close to the origin, increases the prediction accuracy of the model. As can be seen in this figure, the percentage of the lowest error is highest in the KELM-GWO model and the lowest in the DELM model.

Figure 11 shows the error curves of the six models in the testing phase. There are two types of curves in each diagram: the black curve represents observed values, and the other colored curves represent predicted values for each model. Taken as a whole, the predictive performance of each model in the testing phase is close. Nevertheless, magnified local errors can indicate differences in the predictive performance of all the models. As can be seen in these diagrams, there are smaller predicted errors from rock samples of No. 65 to No. 81 in the KELM-GWO model than the other five models, which means that the KELM-GWO model is more suitable for predicting UCS than other models.

The graphical Taylor diagrams can be a concise tool for evaluating the performance of the models more comprehensively, as shown in Figure 12. A complete Taylor diagram consists of three parts: standard deviation (blue circles), RMSE (green circles), and correlation coefficient (black dotted lines). The RMSE and correlation coefficient of the observed value was 0 and 1; the standard deviation, RMSE, and correlation coefficient of all models can be calculated based on their predicted values. Accordingly, the highest similarity to the observed data corresponded to the best prediction model. As can be seen in the diagrams, the KELM-GWO model was the best model with the closest position to the observed values in both the training and testing phases. After this model, the ranking of the prediction performance is BPNN, KELM, ELM, empirical, and DELM.

To compare and assess the prediction performance with the proposed model, the current and previous works are listed in

Table 6. Comparison between the current and previous works by using AI algorithms for UCS of rock prediction.

References	AI Models	Input Parameters	No. of Dataset	Performance
[20]	ANN	PLS, Vp, SHR, Pn	30	R2 = 0.93
[21]	GP	Pn, r, Vp	72	RMSE = 12.3
[24]	ANN	r, CC, QC	138	R2 = 0.76
[25]	ANN	Pn, SD, SH, PLS	39	R2 = 0.93
[28]	FIS	SDI2, SDI4, CLC	65	RMSE = 2.767
[29]	FIS	BPI, PLS, SHR, Vp, Pn, r	60	R2 = 0.98 RMSE = 8.21
[30]	FIS	SH, r, Pn	75	R2 = 0.9437
[31]	SVM	Pn, Vp, SD	47	R2 = 0.7712
[32]	SVM	Vp, Pn, SHR	85	R2 = 0.9516 RMSE = 2.14
[33]	SVM	Vp, SHR, CSS	90	R2 = 0.867
[34]	SVM	SH, Pn, Vp, PLS	170	R2 = 0.9363 RMSE = 1.097
[35]	RF	PLS, Pn, Vp, SHR	30	R2 = 0.93
[36]	RF	Vp, SH, Pn, PLS	93	R2 = 0.488 RMSE = 8.071
[40]	MLP	RC, r, Pn, Vp, WA, PLS	197	R2 = 0.90 RMSE = 0.289
[41]	GEP	UPV, WA, Dd, Sd, Bd	167	R2 = 0.877
[43]	GA	QC, r, Pn, CI, SSH	44	R2 = 0.63
[101]	ANN	WS, r, Pn	83	R2 = 0.96
[102]	ANN	Vp, r, Pn	105	R2 = 0.95
This study	ELM KELM KELM-GWO DELM BPNN	SH, Pn, Vp, PLS	271	R2ELM = 0.8225 R2KELM = 0.8803 R2KELM-GWO = 0.9152 R2DELM = 0.7019 R2BPNN = 0.8849

Note: QC: quartz content; SH: shore hardness; SHR: Schmidt hardness rebound number; CI: cone indenter hardness; CC: concavity/convexity; SD: slake durability; SDI: slake durability index; SDI₂: two-cycle slake durability index; SDI₄: four-cycle slake durability index; CLC: clay content; CSS: cubic sample sizes; RC: rock class; WA: water absorption; WS: water saturation; UPV: ultrasound pulse velocity; Dd: dry density; Sd: saturated density; Bd: bulk density; SSH: shore scleroscope hardness; GWO: grey wolf optimizer.

. As can be seen in this table, the KELM model was almost never used to predict UCS and the KELM-GWO model proposed in this study is better than most of the single AI models. Although the performance results of several studies are better than this paper, the volume of data limits the difficulty of applying these models to the UCS prediction of other rocks. In addition, different prediction results can be obtained by using different input parameter combinations based on the same model. Therefore, it is necessary to obtain the importance of the different input parameters for the UCS prediction in this study.

The mean impact value (MIV) has been reported to analyze the importance of input variables in prediction problems [98,99,100]. The core of the MIV technique is enlarging and reducing the input variable with the same ratio. Then, two new kinds of datasets were used to train and test based on the proposed models. Accordingly, two new outputs were obtained and the errors between them were named as impact values (IVs) of the input variable. Finally, the mean of those IVs (MIV) can be calculated and the largest MIV of the variable represents the highest importance for predicting UCS. The results of importance analysis for input variables are illustrated in Figure 13. As can be seen in this figure, the most important input variable is n with an MIV of 15.47; the remaining order of importance for the others is PLS of 10.74, SHR of 9.50, and V_p of 8.47.

7. Conclusions and Summary

The UCS is a parameter of great importance for designing geotechnical and mining works such as tunneling, underground excavation, rock slope stability, and dam construction. Therefore, five AI models (ELM, KELM, KELM-GWO, DELM, and BPNN) and an empirical equation were proposed to predict UCS based on the 271 rock samples. The results showed that the KELM-GWO model was the best model for predicting UCS with the best performance indices in both the training (RMSE: 17.2176; R²: 0.8846; MAE: 12.0577; U₁: 0.0798; U₂: 0.0259; VAF: 88.4566%) and testing phases (RMSE: 14.2176; R²: 0.9152; MAE: 11.4315; U₁: 0.0706; U₂: 0.0259; VAF: 91.5207%). It is also verified that GWO is an effective algorithm to improve the prediction performance of the KELM model. The porosity is the most important parameter for predicting UCS by using the MIV technique; the remaining order of importance for the others is point load strength of 10.74, Schmidt hardness rebound number of 9.50, and P-wave velocity of 8.47. This paper highlights the performance advantages of hybrid algorithms and the use of integrated large databases, but the parameter diversity considered in this paper limits the prediction performance to a certain extent. We believe that on the premise of fusing more reference data, increasing effective input parameters as much as possible is helpful to improve the predictive performance of the hybrid model.