Developing Hybrid DMO-XGBoost and DMO-RF Models for Estimating the Elastic Modulus of Rock

Abstract

Accurate estimation of the elastic modulus (E) of rock is critical for the design of geotechnical projects such as mining, slopes, and tunnels. However, the determination of rock mechanical parameters usually involves high budget and time requirements. To address this problem, numerous researchers have developed machine learning models to estimate the E of rock. In this study, two novel hybrid ensemble learning models were developed to estimate the E of rock by optimizing the extreme gradient boosting (XGBoost) and random forest (RF) algorithms through the dwarf mongoose optimization (DMO) approach. Firstly, 90 rock samples with porosity, dry density, P-wave velocity, slake durability, and water absorption as input indicators were collected. Subsequently, the hyperparameters of XGBoost and RF were tuned by DMO. Based on the optimal hyperparameters configuration, two novel hybrid ensemble learning models were constructed using the training set (80% of the data). Finally, the performance of the developed models was evaluated by the coefficient of determination (R² score), root mean squared error (RMSE), mean absolute error (MAE), and variance accounted for (VAF) on the test set (20% of the data). The results show that the DMO-RF model achieved the best comprehensive performance with an R² score of 0.967, RMSE of 0.541, MAE of 0.447, and VAF of 0.969 on the test set. The dry density and slake durability were more influential indicators than others. Moreover, the convergence curves suggested that the DMO-RF model can reduce the generalization error and avoid overfitting. The developed models can be regarded as viable and useful tools in estimating the E of rock.

1. Introduction

Elastic modulus (E) is one of the key parameters characterizing the properties of rock, and the accurate estimation of the E of rock is essential for the safe construction of geotechnical engineering [1,2,3,4,5]. The E of rock can be determined by situ and laboratory tests, and the test process should comply with a series of operating specifications. However, it is sometimes difficult to extract high-quality core specimens from fragile, weak, and stratified rock masses. Additionally, the actual testing process is often limited by time and cost constraints [6,7,8]. Therefore, it is essential to develop economical, non-destructive, and indirect methods to estimate the E of rock.

In the early stage of the development of rock mechanics, several scholars attempted to estimate the E of rock by some easily accessible rock indexes, such as the point-load index, slake durability index, Schmidt hammer rebound number, and P-wave velocity [9,10,11,12]. As a result, several empirical formulas between E and the above indexes were established. For example, Sachpazis [13] calculated a correlation coefficient of 0.78 between E and the Schmidt hammer rebound number in carbonate rocks. Lashkaripour [14] analyzed the correlation between the porosity and E of mudrocks. Yasar and Erdogan [15] determined the statistical relations between the hardness and E of rocks. Moradian and Behnia [16] derived fit equations for E and the P-wave velocity of sedimentary rocks by using an ultrasonic test. Armaghani et al. [17] proposed several simple and multiple regression equations to calculate the E of granite rocks. Shan and Di [18] developed formulas to calculate the E of multiple-jointed basalt rocks by laboratory tests. Karakus et al. [19] adopted a multiple regression model to analyze the elastic properties of intact rock. However, the estimation results of these empirical formulas are only applicable to specific rock data and do not reach the generalization purpose.

On the other hand, in recent years, machine learning (ML) approaches have garnered considerable interest in geotechnical engineering because of their powerful nonlinear data processing capabilities [20,21,22]. With the accumulation of data, some researchers have attempted to develop ML models to estimate the E of rock. For instance, Armaghani et al. [17] proposed an adaptive neuro-fuzzy inference system (ANFIS) for predicting unconfined compressive strength (UCS) and E of granite rocks. Cao et al. [23] hybridized an extreme gradient boosting machine (XGBoost) with the firefly algorithm to estimate the E and UCS of granite rock. Meng and Wu [24] combined the experimental, numerical, and random forest (RF) methods to predict the UCS and the E of frozen fractured rock. Abdi et al. [25] investigated the feasibility of tree-based techniques, including RF, Adaboost, XGBoost, and Catboost models, in predicting the E of weak rock. Acar and Kaya [26] adopted the least square support vector machine (LS-SVM) model to predict the E of weak rock. Several scholars [27,28,29] adopted different artificial neural network models (ANN) to predict the E of different types of rocks, respectively. More related works on the estimation of E using ML methods and details are tabulated in

Table 1. Related works on E estimation using ML approaches.

Year	References	Models	Inputs	Number of Samples
2010	Majdi and Beiki [30]	GA-ANN	UCS, GSI, RQD, ρ, n, NJ	120
2010	Dehghan et al. [29]	ANN	Vp, Is(50), Rn, n	30
2011	Khandelwal and Singh [31]	ANN	UCS, BTS	120
2012	Ocak and Seker [32]	ANN	UCS, γ	195
2015	Armaghani et al. [17]	ANFIS	ρ, Vp, content of Qtz, Kpr, Plg, and Bi	45
2018	Bejarbaneh et al. [33]	ANN	Is(50), Vp, Rn	96
2018	Saedi et al. [34]	ANFIS	BPI, BTS, Vp, Is(50)	120
2018	Rezaei [35]	MFIS	H, ρ, n, DI	50
2019	Yang et al. [36]	Bayesian	IS(50), Rn, Vp, n, UCS	71
2020	Acar and Kaya [26]	LS-SVM	Vp, γ, Is(50), BTS	575
2022	Cao et al. [23]	XGBoost-firefly	ρ, Vp, content of Qtz, Kpr, Plg, Bi	45
2022	Pappalardo and Mineo [27]	ANN	n, γ, Vp, Edyn, UCS	/
2023	Meng and Wu [24]	RF	PF, SF, IAF, TF, NF	/
2023	Abdi et al. [25]	RF	n, ρ, Vp, Id2, Abs	90

Note: GSI is geological strength index; RQD is rock mass quality designation; GA is genetic algorithm; BTS is Brazilian tensile strength; γ is unit weight; ρ is density; n is porosity; NJ is number of joints per meter; V_p is P-wave velocity; I_S(50) is point load index; R_n is the Schmidt hammer rebound number; Qtz is quartz; Kpr is alkali feldspar; Plg is plagioclase; Bi is biotite; BPI is block punch index; H is depth of coring; DI is durability index; E_dyn is dynamic elastic modulus; Id₂ is slake durability; Abs is water absorption; PF is persistence factor of ice-filled fractures; SF is spacing between fractures; IAF is inclination angle of fractures; TF is thickness of fractures; NF is number of fractures.

.

In comparison to other methods, ML approaches can yield dependable results by establishing the nonlinear relationship between input and output variables [37,38]. It is a promising method for estimating the E of rock. Recently, the XGBoost and RF algorithms have shown great potential to improve prediction accuracy and have been successfully applied in many fields, such as electricity consumption forecasting [39,40], infectious disease prediction [41,42], mining maximum subsidence prediction [43,44], and heavy metal contamination prediction [45,46]. XGBoost and RF are two efficient ensemble methods that combine multiple homogeneous weak learners in certain ways to reduce overfitting. However, the ML algorithms require a proper hyperparameter setting to improve the accuracy. As a novel heuristic optimization algorithm, the dwarf mongoose optimization (DMO) algorithm has proven to have a strong global search capability and high search efficiency in solving optimization problems [47]. Therefore, it may be an efficient approach to optimize the hyperparameters of XGBoost and RF models. The available studies show that the DMO algorithm has not been integrated with the XGBoost and RF models to estimate the E of rock.

This study aims to investigate the feasibility of XGBoost and RF algorithms optimized by the DMO approach to estimate the E of rock. First, a dataset comprising 90 specimens with five indicators was compiled from available rock mechanics data. Next, the DMO-XGBoost and DMO-RF models were developed. The proposed models can be used to estimate the E of rock easily by avoiding the complexity of core specimen preparation in the laboratory.

2. Methodology

2.1. XGBoost Algorithm

XGBoost is a boosting ensemble algorithm with a decision tree (DT) as the base learner that improves the classical gradient boosting decision tree (GBDT) algorithm [48]. It is designed using an efficient, flexible, and portable distributed gradient boosting framework. Compared with GBDT, the calculation speed is faster while retaining excellent performance. The flowchart of this algorithm is indicated in Figure 1. The core principle of XGBoost is to build DTs one after another and train the subsequent DT with the residuals of the previous ones. The values computed by all the constructed DTs are integrated to achieve a better result. It has been regarded as an advanced evaluator with ultra-high performance in both classification and regression [21]. The XGBoost algorithm is explained as follows:

$${\widehat{y}}_i^{(t)}={\displaystyle \sum _{k=1}^t{f}_k\left(x_i\right)=}{\widehat{y}}_i^{(t-1)}+f_t\left(x_i\right)$$

(1)

where ${\widehat{y}}_i^{(t)}$ denotes the final model, ${\widehat{y}}_i^{(t-1)}$ is the previous model, $x_i$ represents the features corresponding to the sample i, $f_t\left(x_i\right)$ is the newly generated DT model, and t is the total number of DT models.

The objective function of the ensemble algorithm is an alternative. In order to avoid overfitting and improve iteration efficiency, XGBoost introduces model complexity to measure the computational efficiency [49]. Accordingly, the objective function of XGBoost can be given by:

$$Ob{j}^{\left(t\right)}={\displaystyle \sum _{i=1}^{t}l\left(y_i,{\widehat{y}}_i^{(t)}\right)}+{\displaystyle \sum _{i=1}^t\Omega \left(f_i\right)}$$

(2)

where $y_i$ is the actual value, $l\left(y_i,{\widehat{y}}_i^{(t)}\right)$ is a convex loss function describing how well the model fits with training data, and $\Omega \left(f_i\right)$ is the penalty term for regularization to avoid overfitting.

Gradient statistics are applied to the loss function, which effectively eliminates constant parameters. The simplified objective function is obtained as follows:

$$Ob{j}^{\left(t\right)}={\displaystyle \sum _{i=1}^t\left[g_i{f}_t\left(x_i\right)+\frac{1}{2}{h}_i{f}_t^2\left(x_i\right)\right]}+\Omega \left(f_t\right)$$

(3)

where $g_i=\partial {\widehat{y}}_i^{(t-1)}l\left(y_i,{\widehat{y}}_i^{(t)}\right)$ and $h_i={\partial }^2{\widehat{y}}_i^{(t-1)}l\left(y_i,{\widehat{y}}_i^{(t)}\right)$ are the first and second order gradient statistics of the loss function, respectively. The penalty term $\Omega \left(f_t\right)$ is evaluated by:

$$\Omega \left(f_t\right)=\gamma T+\frac{1}{2}\alpha \left|w\right|+\frac{1}{2}\lambda {‖w‖}^2$$

(4)

where α and λ are regularization parameters, whose default values are 1 and 0, respectively, T is the number of leaf nodes controlled by the parameter γ, and w denotes the corresponding weight of the leaf nodes.

2.2. Random Forest Algorithm

RF is a bagging ensemble algorithm with DT as the base learner [50]. Its principle diagram is shown in Figure 2. Different DTs are constructed to set up a forest by extracting sample and feature subsets from the original data. It is a stochastic process that ensures the independence of each DT, which improves the generalization capacity and avoids overfitting [51]. Moreover, all constructed DTs tend to grow freely without pruning, which makes the construction process of each DT fast and the computational efficiency of RF improved. By integrating the results generated by all constructed DTs, the voting (for classification problems) or averaging (for regression problems) approach is used to determine the final results [21]. According to the type of output variable in this paper, the RF algorithm is explained as follows:

$$\stackrel{-}{y}(x)=\frac{1}{T}{\displaystyle \sum _{i=1}^T{f}_i(x)}$$

(5)

where $\stackrel{-}{y}(x)$ represents the predicted value of the RF model, $f_i(x)$ is the predicted value of each DT, and T represents the number of constructed DTs.

2.3. Dwarf Mongoose Optimization

The DMO algorithm is a swarm intelligence optimization algorithm that simulates the seminomadic life of dwarf mongoose in nature. The dwarf mongooses are known for foraging and scouting as a unit, and the DMO algorithm finds the optimal solution by simulating their social behavior. The DMO algorithm divides the swarm into three groups: the alpha group, the scout group, and the babysitter group [47,52,53].

Firstly, the alpha group leads the scout group to forage and scout for a new sleeping mound, leaving the babysitters group to take care of the young dwarf mongooses in the old sleeping mound. Secondly, when a food source is not found, the alpha and scout groups will go back to the old sleeping mound to exchange members with the babysitter group. Thirdly, the alpha group continues to go out foraging and scouting for new sleeping mounds. The family returns intermittently to exchange babysitters and repeats the cycle until a food source is found.

The process of DMO for hyperparameter tuning can be broken down into the following steps [47]:

(1) Initialization: The DMO algorithm starts by initializing the population of solutions as follows:

$$X=\left[\begin{array}{ccccc}{x}_{1,1}& x_{1,2}& \dots & x_{1,d-1}& x_{1,d}\\ x_{2,1}& x_{2,2}& \dots & x_{2,d-1}& x_{2,d}\\ ⋮& ⋮& x_{i,j}& ⋮& ⋮\\ x_{n,1}& x_{n,2}& \dots & x_{n,d-1}& x_{n,d}\end{array}\right]$$

(6)

where X is the set of current candidate populations, n denotes the population size, and d is the dimension of the problem. These solutions represent different sets of hyperparameters for the XGBoost and RF models.

(2) Objective function: The DMO algorithm requires an objective function that quantifies the performance of each solution. In this study, the average root mean squared error (RMSE) values of five-fold cross-validation (CV) were employed as the fitness function. The foraging behavior of the dwarf mongoose is mimicked by dividing the population into alpha, scout, and babysitter groups. The probability of each individual becoming alpha is computed by:

$$\alpha =\frac{fi{t}_i}{{\displaystyle \sum _{i=1}^{n}fi{t}_i}}$$

(7)

where fit_i denotes the fitness value of the i-th individual.

(3) Behavior of dwarf mongooses: In the optimization of the DMO process, these dwarf mongooses work together to find food and a new sleeping mound. The movement of dwarf mongooses is determined by the leader of its subgroup. Their movement is based on the position of the alpha and the current position of the solution, which is calculated as follows:

$$M={\displaystyle \sum _{i=1}^n\frac{X_i\times s{m}_i}{X_i}}$$

(8)

where X_i is the position of the alpha group, and sm_i denotes the current position of the solution.

(4) Cooperative search: DMO encourages a cooperative search among the solutions in the population. This means that solutions collaborate and share information to improve the overall performance of the population. In the context of hyperparameter optimization, it implies that solutions with promising hyperparameters influence or guide other solutions towards better hyperparameter configurations.

(5) Communication and information sharing: In DMO, solutions share information about their performance and hyperparameters with others. This communication helps the population converge toward better solutions.

(6) Update hyperparameters: Based on the cooperative search and shared information, the hyperparameters of each solution are updated.

(7) Fitness evaluation: After updating the hyperparameters, the fitness of each solution is re-evaluated using the objective function. Solutions that perform better are favored and contribute more to the cooperative search process.

(8) Termination: The algorithm terminates when a stopping criterion is met.

(9) Final solution selection: Once the optimization process concludes, the solution with the best-performing hyperparameters, according to the objective function, is selected as the final model configuration.

2.4. Model Evaluation Metrics

In this study, four metrics were employed to assess the performance of the proposed models, including the coefficient of determination (R² score), the RMSE, the mean absolute error (MAE), and the variance accounted for (VAF). RMSE and VAF are maximization performance metrics, while MAE and VAF are minimization performance metrics [54].

R² score represents the proportion of the squared correlation between the predicted and actual values of the target variable, which can be calculated by:

$$R^2=1-\frac{{\displaystyle \sum _i{\left(y_i-{\widehat{y}}_i\right)}^2}}{{\displaystyle \sum _i{\left(y_i-{\overline{y}}_i\right)}^2}}$$

(9)

where $y_i$ indicates the actual value, ${\widehat{y}}_i$ indicates the predicted value of the model, and ${\overline{y}}_i$ represents the average of the actual values.

RMSE represents the standard deviation of the fitted error between the predicted value and the actual value, which can be calculated by:

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}}$$

(10)

where n indicates the total number of samples.

VAF characterizes the prediction performance by comparing the standard deviation of the fitted error with the standard deviation of the actual value, which is defined as:

$$\mathrm{VAF}=\left[1-\frac{\mathrm{var}\left(y_i-{\widehat{y}}_i\right)}{\mathrm{var}\left(y_i\right)}\right]\times 100$$

(11)

where var indicates the variance.

MAE represents the average error between predicted value and actual value, which can be calculated by:

$$\mathrm{MAE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left|y_i-{\widehat{y}}_i\right|,}$$

(12)

2.5. Proposed Approach

In this study, two novel hybrid ensemble learning models were proposed to estimate the E of rock by optimizing the XGBoost and RF models through the DMO algorithm. The structure of the proposed approach is indicated in Figure 3. Firstly, a database including 90 rock samples with five indicators was established. Secondly, 80% of the samples were utilized for training, while the remaining 20% were reserved for testing [55]. Thirdly, to validate the superiority of the proposed hybrid ensemble learning models, a comparison against default XGBoost and RF models was performed. Finally, the R² score, RMSE, MAE, and VAF were adopted to evaluate the performance of models on the test set.

The XGBoost and RF algorithms were implemented on the Python library “sci-kit-learn” [56], and the DMO algorithm was implemented on the Python library “MELPY” [57,58]. All experiments were processed using a Windows 10, 64-bit computer with 8 Gb of RAM running an Intel^® Core™ i7-9700F CPU @ 3.00 GHz × 2.

3. Data and Variables

According to the work of Abdi et al. [25], four types of rocks, including marl, siltstone, claystone, and limestone, were collected and cored in the laboratory to obtain standard core samples. A total of 90 rock samples were obtained for conducting physical tests and developing E-estimation models. Among the 90 samples, each set of data contains five indicators, including porosity (A₁), dry density (A₂), P-wave velocity (A₃), slake durability (A₄), and water absorption (A₅). Among them, porosity (A₁) is an important factor in determining the strength and deformation behavior of rock, and water absorption (A₅) is related to porosity. Dry density (A₂) and P-wave velocity (A₃) are two common petrophysical properties related to the E of rock. Yagiz et al. [28] found that slake durability (A₄) cycles have a significant effect on the prediction of UCS and modulus of elasticity for carbonate rocks. It is important that these five indicators can be conveniently measured in the laboratory, and therefore, they were selected as input indicators. The detailed descriptions of these indicators are displayed in

Table 2. Descriptions of input indicators.

Indicator	Description
Porosity (%)	The ratio of void space to the total volume of rock.
Dry density (g/cm3)	The mass per unit volume of a rock sample without any water content.
P-wave velocity (m/s)	The speed at which compressional waves travel through a rock sample.
Slake durability (%)	A measure of the resistance of a rock sample to disintegration when exposed to wetting and drying cycles.
Water absorption (%)	The amount of water that a rock sample can absorb when it is immersed in water for a specified period of time.

, and the statistics of the dataset are illustrated in

Table 3. Statistical values of the dataset.

Indicators	Minimum	Median	Maximum	Mean	Standard
A1 (%)	5.44	21.32	56.55	22.69	9.59
A2 (g/cm3)	1.61	2.06	2.98	2.17	0.24
A3 (m/s)	1011.53	2075.05	3250.45	1989.66	541.78
A4 (%)	22.75	80.52	96.54	74.87	16.40
A5 (%)	2.51	10.59	26.54	11.62	5.24
E (GPa)	1.12	3.59	13.23	4.50	2.92

.

The violin plots of the five indicators and E are shown in Figure 4. These plots are a combination of box plots and density plots, offering insights into the overall distribution of the dataset. In each violin plot, the white dot at the center represents the median of the samples, the upper and lower extents of the thick black line denote the third and first quartiles of the samples, the top and bottom of the thin black line indicate the upper and lower adjacent values, and the black dots are outliers. From Figure 4, it can be seen that the distribution of all the data is relatively balanced, but there are some outliers, which indicates the complexity of estimating the E of rock.

To visualize the distribution of this dataset, the correlation pair plots and heatmap of the Pearson correlation coefficient between indicators and E are displayed in Figure 5. The results show that A₂, A_3, and A₄ are positively correlated with E, while others are negatively correlated. Among them, the strongest correlation was found between A₂ and E with an absolute value of the correlation coefficient of 0.78, and the weakest correlation was found between A₁ and E with an absolute value of the correlation coefficient of 0.28.

4. Results and Analysis

4.1. Results of Hyperparameters Tuning

In contrast to other optimization methods, swarm intelligence algorithms need to determine the population size and the number of iterations, which directly affect the running time of the models and the ability to tune the optimal hyperparameters [59]. In this study, each model performed 100 iterations at population sizes of 10, 20, 30, 40, 50, 75, and 100, and the average RMSE values of five-fold CV were employed as the fitness function. The fitness curves and the running time of all models are indicated in Figure 6. With an increase in the number of iterations, the average fitness value gradually reaches a stable point. The minimum fitness value corresponds to the optimal population size of the models. However, an excessively large population size will significantly prolong the running time, which is not conducive to the practical application of these models in engineering [60,61]. Therefore, the best trade-off between performance and efficiency was achieved when the population sizes of the DMO-RF and DMO-XGBoost models were 75 and 20, respectively. After tuning the population size, the hyperparameters of the RF and XGBoost models were tuned based on the optimization algorithm. The scope, interval values, and ultimate optimization results of the hyperparameter values are presented in

Table 4. Scope and interval values of DMO-RF and DMO-XGBoost models.

ML Algorithms	Hyperparameters	Scope of Values	Interval of Values	Optimal Values
DMO-RF	n_estimators	(10, 300)	10	10
	max_depth	(1, 20)	1	4
	min_samples_split	(2, 10)	0.001	5
DMO-XGBoost	n_estimators	(10, 300)	10	12
	max_depth	(1, 15)	1	5
	learning_rate	(0.001, 1)	0.001	1
	reg_alpha	(0.001, 1)	0.001	1
	reg_lambda	(0.001, 1)	0.001	0.636

.

4.2. Model Comparison and Evaluation

To further compare the estimation performances of the two proposed hybrid ensemble learning models with the default XGBoost and RF models, the scatter plots of the four models in the training and testing phases are presented in Figure 7. The vertical and horizontal axes represent predicted and actual values of E, respectively. The diagonal dashed line represents the ideal regression line, representing that the predicted and actual values are equal. Models with more data points lying on this line exhibit higher predictive performance. The solid blue lines are the 10% boundaries, which were set to better observe the distribution of points and compare models. The green and red scatters represent the training data points and test data points, respectively. At the same time, the values of the evaluation metrics (R², RMSE, MAE, and VAF) of each model in the training and testing phases are recorded in

Table 5. Performance comparison of different models.

Models	R2	RMSE	MAE	VAF
Training
RF	0.964	0.541	0.368	0.965
XGBoost	0.999	0.001	0	0.999
DMO-RF	0.968	0.513	0.353	0.968
DMO-XGBoost	0.999	0.016	0.012	0.999
Testing
RF	0.913	0.886	0.748	0.920
XGBoost	0.795	1.357	0.994	0.816
DMO-RF	0.967	0.541	0.447	0.969
DMO-XGBoost	0.935	0.763	0.674	0.936

The results presented in bold denote the best values.

. Some observations can be obtained from Figure 7 and Table 5. First, in the training phase, the XGBoost and DMO-XGBoost models performed better than the RF and DMO-RF models without the phenomenon of the green scatters outside the 10% boundaries, and the default XGBoost model performed the best in all evaluation metrics. Second, in contrast to the training phase, the predictive performance of each model decreased during the testing phase, but the DMO approach improved the performance of the default XGBoost and RF models in the testing phase. Third, in the testing phase, the DMO-RF and DMO-XGBoost models had fewer red scatters outside the 10% boundaries than the default RF and XGBoost models, and the DMO-RF model performed best in all evaluation metrics.

5. Discussion

5.1. Comparison with Different Classical Hybrid Models

To verify the effectiveness of the proposed DMO algorithm, the classical simulated annealing (SA) and Bayesian optimization (BO) algorithms were introduced to optimize the RF and XGBoost models for comparison [62,63,64,65]. The evaluation metrics of the SA-RF, SA-XGBoost, BO-RF, and BO-XGBoost models in the training and testing phases are shown in

Table 6. Performance comparison of different hybrid models.

Models	R2	RMSE	MAE	VAF
Training
SA-RF	0.949	0.646	0.491	0.949
SA-XGBoost	0.997	0.029	0.022	0.997
BO-RF	0.950	0.644	0.478	0.950
BO-XGBoost	0.996	0.183	0.138	0.996
Testing
SA-RF	0.933	0.774	0.641	0.935
SA-XGBoost	0.918	0.858	0.681	0.921
BO-RF	0.933	0.773	0.654	0.936
BO-XGBoost	0.953	0.648	0.559	0.954

.

To compare the developed models more visually, the Taylor diagram was plotted, as seen in Figure 8. A complete Taylor diagram consists of three components: standard deviation (SD, black and green dashed line), root mean square deviation (RMSD, red dashed line), and correlation coefficient (CC, blue dashed line) [66]. The red pentacle in Figure 8 was the reference point, which represents the actual E values, and the scatters represent the different prediction models. The scatter with the closest distance to the reference point (with the lowest RMSD value) corresponds to the optimal prediction model. The SD, RMSE, and CC values of all scatters are calculated in

Table 7. Taylor diagram values of all models.

	Reference	DMO-RF	DMO-XGBoost	SA-RF	SA-XGBoost	BO-RF	BO-XGBoost
SD	2.998	2.7	2.999	2.519	2.513	2.671	2.562
CC	0	0.988	0.968	0.977	0.968	0.981	0.975
RMSD	1	0.530	0.757	0.764	0.843	0.641	0.758

. It can be seen that the DMO-RF model was the best model that was closest to the reference point with an RMSD of 0.530, followed by the BO-RF, DMO-XGBoost, BO-XGBoost, SA-RF, and SA-XGBoost models.

5.2. Comparison with Other ML Models

To confirm the performance of the developed models, three other ML models were introduced for comparison, namely the SVM, decision tree (DT), and multilayer perceptron neural network (MLPNN). The hyperparameters of these models were default. In addition, Adaptive boosting machine (Adaboost) and Category gradient boosting machine (CatBoost) methods were developed by Abdi et al. [25] on the same dataset. Furthermore, to visually compare the seven ML models, a scoring system was implemented to give corresponding scores to each model [67]. The values and scores of all models on four evaluation metrics are calculated in

Table 8. Performance comparison of different ML models on test set.

Models	R2	Score	RMSE	Score	MAE	Score	VAF	Score	Total Score
Adaboost	0.841	5	1.188	5	0.920	5	0.849	5	20
CatBoost	0.784	4	1.382	4	1.075	4	0.788	4	16
SVM	0.737	3	1.410	3	1.132	3	0.745	3	12
DT	0.688	1	1.679	1	1.321	1	0.701	1	4
MLPNN	0.721	2	1.473	2	1.163	2	0.722	2	8
DMO-RF	0.967	7	0.541	7	0.447	7	0.969	7	28
DMO-XGBoost	0.935	6	0.763	6	0.674	6	0.936	6	24

. It can be seen that the overall score ranking for all models is DMO-RF (28) > DMO-XGBoost (24) > CatBoost (20) > Adaboost (16) > SVM (12) > MLPNN (8) > DT (4).

5.3. Relative Importance of Indicators

In this study, the relative importance of each indicator was determined by combining the DMO-RF and DMO-XGBoost models with the permutation feature importance technique, which is indicated in Figure 9 [68,69]. Based on the DMO-RF model, the rank of the indicators’ importance was A₂ > A₄ > A₃ > A₅ > A₁. According to the DMO-XGBoost model, the rank of the indicators’ importance was A₂ > A₄ > A₅ > A₃ > A₁. The results were consistent with the calculations of the feature importance analysis module built into the XGBoost and RF models. Apparently, two indicators were the most important: A₂ (dry density) and A₄ (slake durability). The results can be used as a reference for developing more reliable E estimation models of rock in the future.

5.4. Overfitting Validation

Furthermore, to further analyze whether the proposed DMO-RF and DMO-XGBoost models suffered from overfitting or underfitting, the convergence curves are plotted in Figure 10. The horizontal axis represents the number of samples, while the vertical axis represents the R² score obtained through a five-fold CV. It can be seen that the DMO-RF and DMO-XGBoost models tended to converge as the sample size increased. However, the DMO-RF model performed with less error between the training and test sets than the DMO-XGBoost model, which indicates that the proposed DMO-RF model effectively mitigated the issue of generalization error and exhibited a certain degree of resistance against overfitting.

5.5. Limitations

Although the proposed hybrid ensemble approaches obtained excellent results in estimating the E of rock, there are still some limitations to consider:

(1) The dataset used in this study is relatively small, and the specimens in the original dataset were composed of four types of weak rocks, including marl, siltstone, claystone, and limestone. There is no consideration of other types of rock, such as granite, basalt, and other hard rocks. This situation might lead to the presence of a sampling bias that might affect the generalizability of the proposed ML models. Therefore, it is crucial to expand the database by collecting various types of rock cases to increase the quantity and quality of the dataset.

(2) More indicators should be considered. Although the five indicators in this study affect the E of rock significantly, other factors such as depth of coring, mineral composition, grain size distribution, and RQD index also have an effect on the E of rock. It is valuable to explore the influences of these indicators on the E of rock estimation.

Although the aforementioned limitations exist, the developed DMO-XGBoost and DMO-RF models in this study could be considered a feasible and practical tool for estimating the E of rock. Compared to traditional expensive and time-consuming laboratory testing methods for rock mechanical parameters, the proposed models have potential advantages in terms of cost savings and time efficiency. Geotechnical engineers can significantly reduce the need for costly laboratory testing, providing project budget savings while enabling faster and more effective assessment of geologic and geotechnical risks. In addition, the developed models could be extended to estimate other geotechnical parameters, such as UCS, rock shear strength, and rock brittleness, et al.

6. Conclusions

E is one of the important parameters in rock mechanics. Accurately estimating the E of rock is significant for the design and construction of geotechnical projects. In this study, DMO-XGBoost and DMO-RF models were developed to estimate the E of rock. The effectiveness of the proposed models was verified using a database including 90 rock samples with five indicators. To avoid overfitting or selection bias, the five-fold CV method was combined with the DMO algorithm to tune the hyperparameters on the training set (80% of the dataset). The R² score, RMSE, MAE, and VAF were adopted to evaluate the performance of models on the test set (20% of the dataset). In addition, two default ensemble models (XGBoost and RF) were introduced and compared with the proposed two hybrid models. Overall, the DMO algorithm improved the predictive performance of the default XGBoost and RF models, and the DMO-RF model performed best with an R² of 0.967, RMSE of 0.541, MAE of 0.447, and VAF of 0.969 on the test set. Furthermore, the classical SA and BO algorithms were introduced to optimize the RF and XGBoost models for comparison, and the Taylor diagram was plotted to determine the comprehensive rank, which was DMO-RF > BO-RF > DMO-XGBoost > BO-XGBoost > SA-RF > SA-XGBoost. The permutation feature importance technique revealed that dry density and slake durability were more influential than other indicators in the evaluation results. Based on the convergence curves, it was verified that the DMO-RF model can reduce the generalization error and avoid overfitting.

In future work, a larger dataset containing higher-quality data should be established to improve the estimation performance. In conclusion, the proposed DMO-RF model in this paper can be considered a viable and useful tool in estimating the E of rock and can be recommended for the application of other geotechnical parameter estimations, such as UCS, rock shear strength, and rock brittleness, amongst others.