Hybrid Machine Learning Model for Predicting Shear Strength of Rock Joints

Abstract

The accurate prediction of joint shear strength is crucial for rock mass engineering design and geological hazard assessment. However, traditional machine learning (ML) models often suffer from local optima and limited generalization ability when dealing with complex nonlinear problems, thereby compromising prediction accuracy and stability. To address these challenges, this study proposes a hybrid ML model that integrates a multilayer perceptron (MLP) with the slime mold algorithm (SMA), termed the SMA-MLP model. While MLP exhibits strong nonlinear mapping capability, SMA enhances its training process through global optimization and parameter tuning, thereby improving predictive accuracy and robustness. A dataset with five input variables was constructed to evaluate the performance of the SMA-MLP model comprehensively. The proposed model was compared with other ML models. The results indicate that SMA-MLP outperforms these models in key metrics such as the root mean squared error (RMSE) and the correlation coefficient (R²), achieving an R² of 0.97 and an RMSE as low as 0.10 MPa on the test set. Furthermore, feature importance analysis reveals that normal stress has the most significant influence on joint shear strength. This study demonstrates the superiority of SMA-MLP in predicting joint shear strength, highlighting its potential as an efficient and accurate tool for rock mass engineering analysis and providing reliable technical support for geological hazard assessment.

1. Introduction

A rock mass is a combination of intact rock blocks separated by rock joints [1,2]. Generally, a rock mass consists of intact rock blocks separated by discontinuous rock joints [3,4]. With the increase in natural resource mining engineering and underground engineering, hazards caused by joint shear slip exist widely in geotechnical engineering excavation [5,6]. For example, Figure 1 illustrates a case of a rock slope in Jiangxi Province, China, characterized by numerous irregular rock blocks and rock joints. Shear failure of these rock joints occurs under normal boundary conditions, which poses a serious threat to pedestrians and vehicles [7,8]. Consequently, accurate determination of the shear strength of rock joints is crucial for mitigating geological hazards and preventing engineering failures [9,10].

Since the 1960s, the estimation of joint shear strength has remained a critical research topic in the field of civil engineering. The Mohr–Coulomb strength model, which posits that there is a linear relationship between shear strength and the normal stress acting on rock joints, has served as a foundational framework. For sawtooth joints, Patton [11] established a bilinear strength model considering undulation angles. Under different shear displacement rates, Li et al. [12] modified the Patton model to obtain a shear strength model of sawtooth joints considering the shear rate. Ladanyi and Archambault [13] integrated the effects of friction, dilatancy, cohesion, and rock bridge strength, deriving a comprehensive shear strength model for joints containing rock bridges. Since there had been a turning point in the strength curve in the Patton model, Jager [14] established a new negative exponential strength model in order to further obtain a smooth shear strength curve. Barton [15] pioneered the incorporation of the joint roughness coefficient (JRC), joint wall strength, and basic friction angle into shear strength analysis. Similar to the practice in the literature [12], Wang et al. [16] improved the Barton shear strength model based on the direct shear test results of a series of rough joints. Xie et al. [17] introduced a novel approach by distinguishing between micro-roughness and macro-undulation in joint profiles, leading to an enhanced shear strength model. Recently, the shear strength characteristics of rock joints have been explained and interpreted through the frameworks of disturbed state concept theory and renormalization group theory [18,19,20]. However, excessive attention to mechanical mechanisms can lead to extremely complex mechanical models, which greatly limit their applicability to practical engineering. In addition to these studies, scholars have also proposed a series of theoretical or empirical models to study the shear strength of rock joints. Classic models include the Barton model [21], the Kleepmek model [22], the Zhao model [23], and the Grasselli model [24]. A brief review of the existing strength models can be found in References [25,26]. It is noteworthy that empirical regression-based strength models frequently exhibit significant deviations due to data limitations and material heterogeneity.

The rapid advancement of computer technology has ushered civil engineering into a new era. Machine learning (ML) models have emerged as powerful tools for solving complex engineering problems without requiring explicit knowledge of the underlying physical and mathematical mechanisms [27,28]. Extensive studies have demonstrated that ML models can not only complement but also replace certain traditional methods, offering a more efficient approach to capturing complex nonlinear relationships between input and output variables [29,30,31]. Under specific experimental conditions, ML models can serve as an alternative to traditional analytical models [32]. In geotechnical engineering, numerous successful applications of ML have been reported. For example, Fathipour-Azar [33] proposed a shear strength model based on a stacked ensemble of the support vector machine (SVM), M5P model tree, and random forest (RF) algorithms. Babanouri and Fattahi [34] employed a search algorithm-enhanced support vector regression model to predict joint shear strength. They noted that, compared to the Barton model, this model shows significant superiority in predicting joint shear strength. Recently, based on tree-based models and convolutional neural networks, Chen et al. [35] developed a new ML model for predicting joint shear strength. Recently, artificial neural networks, particularly the multilayer perceptron (MLP), have gained attention due to their powerful ability to model nonlinear relationships between input features and output responses. MLP is composed of interconnected layers of neurons, where each neuron applies an activation function to a weighted sum of its inputs. The network iteratively adjusts its weights through the backpropagation algorithm to minimize the prediction error. This flexible structure enables MLP to capture complex patterns and interactions that are often difficult to represent using traditional statistical models. For a comprehensive background of the mathematical and computational principles of MLP, readers are referred to Arena et al. [36], Murlidhar et al. [37], and Babanouri et al. [38]. Despite these advancements, existing ML models still face limitations, including suboptimal accuracy and limited generalization capabilities, highlighting the need for further research.

Leveraging the robust nonlinear modeling capabilities of MLP and the global optimization strengths of the slime mold algorithm (SMA), this study proposes a novel hybrid machine learning model, termed the SMA-MLP model. A dataset for joint shear strength consisting of five independent variables was established. The robustness and predictive performance of the proposed model were comprehensively evaluated using four different performance evaluation indicators. Furthermore, a comparative analysis was conducted to benchmark the predictive performance of the SMA-MLP model against other machine learning models.

2. Methodology

2.1. Multilayer Perception (MLP)

The multilayer perceptron (MLP) has been widely applied in the field of machine learning, demonstrating remarkable performance in classification, regression, pattern recognition, and signal processing tasks [39,40,41]. Its simple yet flexible architecture makes it an ideal choice for both research and practical applications, serving as a fundamental model in deep learning frameworks.

MLP is a feedforward artificial neural network composed of an input layer, one or more hidden layers, and an output layer, as illustrated in Figure 2. Each layer consists of multiple neurons, and full connectivity between the layers is established through weights and biases, enabling information propagation and processing within the network [42]. MLP relies on nonlinear activation functions to introduce nonlinearity, allowing the network to learn and represent complex relationships. Without these activation functions, each layer’s output would merely be a linear combination of the previous layer’s input, making it incapable of modeling intricate data patterns. Additionally, MLP employs the backpropagation algorithm combined with gradient-based optimization methods to update network weights, minimizing the loss function and enhancing learning efficiency. The multilayer architecture and nonlinear activation functions of MLP enable hierarchical feature extraction and composition, facilitating the transformation from low-level to high-level representations and effectively capturing complex patterns and dependencies in the data. This capability makes MLP highly applicable in high-dimensional data modeling, time series forecasting, and image recognition. Despite its strong feature learning ability and good generalization performance, MLP’s effectiveness heavily depends on the proper configuration of network architecture (e.g., number of hidden layers and neurons) and hyperparameters (e.g., learning rate, regularization parameters). Improper hyperparameter selection may lead to overfitting or underfitting, thereby affecting the model’s generalization ability on test data. Therefore, when implementing MLP, hyperparameter optimization techniques are often employed to identify optimal model parameters, enhancing model stability and predictive accuracy. More details of MLP are detailed in references [43,44].

2.2. Slime Mold Algorithm (SMA)

In recent years, metaheuristic optimization techniques that simulate natural phenomena such as biological evolution and collective behavior to find the optimal solution or parameter combinations have gained widespread attention [45,46]. These techniques improve the predictive accuracy and generalization ability of ML models by adjusting their hyperparameters [47]. Considering the multivariable and nonlinear nature of the experimental data, as well as the time required for hyperparameter tuning, the slime mold algorithm (SMA) was ultimately chosen for parameter optimization.

Originally proposed by Li et al. [48], SMA is a novel optimization algorithm based on swarm intelligence. It optimizes the objective function by simulating the behavioral characteristics of slime mold during the foraging process [49]. SMA has several advantages, including fewer parameters, fast convergence, and strong global optimization capabilities. To maximize food acquisition, slime mold dynamically adjusts its exploration direction, establishing the shortest and most optimal path to connect to the food. Figure 3 shows a schematic diagram of the foraging path of slime molds.

The mathematical model of the slime mold foraging behavior is as follows:

$$X\left(t+1\right)=\left\{\begin{array}{lll}{X}_g\left(t\right)+vb \cdot & \left[W\cdot X_A\left(t\right)-X_B\left(t\right)\right]\hfill & r_1<p\\ vc\cdot X\left(t\right)& r_1\ge p\end{array}\right.$$

(1)

where r₁ is a random number uniformly distributed in the range [0, 1], vc is a parameter linearly decreasing from 1 to 0, and vb is a random number in the range [−a, a]. X_g(t) represents the position of the best fitness at the t-th iteration, while X_{(t + 1)} and X_(t) denote the positions of the slime mold at the (t + 1)-th and t-th iterations, respectively. X_A(t) and X_B(t) represent the positions of two randomly selected slime molds at the t-th iteration, and W represents the weight of the slime mold.

The parameters a, p, and weight W are calculated according to Equations (2)–(5), respectively.

$$p=\tanh \left|S\left(i\right)-DF\right|$$

(2)

$$\mathrm{a}=arc\tanh \left[-\left({\displaystyle \frac{t}{\max t}}\right)+1\right]$$

(3)

$$\mathrm{W}\left(SmellIndex(i)\right)=\left\{\begin{array}{cc}1+r_2\cdot \log \left({\displaystyle \frac{bF-S(i)}{bF-\mathrm{w}F}}+1\right)& condition\\ 1-r_2\cdot \log \left({\displaystyle \frac{bF-S(i)}{bF-\mathrm{w}F}}+1\right)& others\end{array}\right.$$

(4)

$$SmellIndex=Sort(S)$$

(5)

where S(i) represents the fitness of the i-th individual, DF denotes the currently obtained optimal fitness value, and maxt is the maximum number of iterations. r₂ is a random number uniformly distributed in the range [0, 1], and condition indicates the individuals ranked in the top half based on fitness. bF and wF represent the best and worst fitness values obtained during the current iteration, respectively. SmellIndex refers to the fitness sequence, where the minimum values are arranged in ascending order.

The SMA position update formula is as follows:

$$X\left(t+1\right)=\left\{\begin{array}{ll}rand\left(UB-LB\right)+LB& rand<\mathrm{z}\\ X_g\left(t\right)+vb\cdot \left[W\cdot X_A\left(t\right)-X_B\left(t\right)\right]& r_1<p\\ vc\cdot X\left(t+1\right)& r_1\ge p\end{array}\right.$$

(6)

where UB and LB represent the upper and lower bounds of the search space, respectively. The term rand denotes a random number uniformly distributed within the range [0, 1]. The parameter z defines the probability of an individual performing a global search. According to the literature [48], the algorithm achieves optimal performance when z is set to 0.03. More details of SMA are detailed in references [50,51].

2.3. SMA-MLP Model

The powerful modeling capabilities, flexibility, and broad applicability of MLP make it a crucial tool in ML methods. Particularly when traditional ML approaches struggle to effectively model nonlinear problems, MLP often demonstrates superior performance. However, MLP typically relies on gradient descent algorithms during training, which may lead to local optima and degrade model performance. Additionally, MLP suffers from issues such as overfitting and insufficient global optimization capabilities. SMA, with its robust global search and parameter optimization capabilities, can significantly address these shortcomings. Therefore, this study introduces SMA to optimize MLP, constructing a hybrid ML model (the SMA-MLP model). This combination not only enhances MLP’s optimization efficiency but also reduces the complexity of manual parameter tuning, enabling the MLP model to adapt more effectively to complex problems.

Figure 4 illustrates the modeling process of the SMA-MLP framework. By optimizing the connection weights and biases of MLP, SMA enhances the model’s predictive accuracy. Specifically, SMA leverages its strong global search capability to prevent MLP from getting trapped in local optima during training. Additionally, it autonomously adjusts the hyperparameters and weight distributions, reducing the need for manual tuning. This optimization strategy not only improves the generalization ability of MLP but also enhances model stability and computational efficiency. Therefore, the integration of MLP’s powerful nonlinear modeling capacity and SMA’s global optimization capability results in an efficient and flexible hybrid modeling approach.

3. Dataset Description and Processing

3.1. Data Processing

The shear strength of rock joint is influenced by multiple factors, including the mineral composition of the rock, loading conditions, and joint surface morphology [52]. However, due to the inherent complexity of rock mass structures and the limitations of field data collection [53], it is challenging to comprehensively account for all the potential influencing factors [54]. Consequently, studies on joint shear strength typically focus on a selection of key parameters. Previous research has extensively reported that joint shear strength is closely related to parameters such as the joint roughness coefficient (JRC), normal stress (σ_n), basic friction angle (φ_b), Young’s modulus (E), and uniaxial compressive strength (σ_c). For instance, as shown in Figure 5, Babanouri and Fattahi [34,55] conducted a series of direct shear tests on replicas of natural rock joints. Their laboratory experiments systematically investigated the effects of these factors on joint shear strength. Based on these findings, this study compiles the experimental data obtained by Babanouri and Fattahi [34,55] to develop a dataset for developing ML models. The dataset employs JRC, σ_n, φ_b, E, and σ_c as input variables, with joint shear strength (τ_p) as the output variable. And finally 84 sets of valid data were obtained. The establishment of this dataset not only enhances the accuracy of the shear strength predictions but also provides valuable references for subsequent numerical simulations and engineering applications. Detailed information regarding the direct shear tests can be found in references [34,55].

Statistical analysis was conducted to examine the distribution characteristics of the dataset, and a violin plot was generated accordingly, as shown in Figure 6. The box line box in the violin plot represents data from the lower quartile to the upper quartile, the black line extending from it represents data from 1.5 times the quartile distance, and the outer shape represents the kernel density estimator. As can be seen from Figure 6, there are no outliers or few outliers in the dataset, indicating that the dataset is reasonably constructed. When a dataset contains a large number of rows and columns, data analysis becomes increasingly complex. Therefore, descriptive statistical methods are commonly used to analyze the dataset based on parameters such as mean, confidence interval, kurtosis, minimum value, maximum value, and standard deviation.

Table 1. Description of variable categories and statistical analysis.

Variables	Skewness	Kurtosis	Coefficient of Variation	Minimum	Median	Maximum
σn (MPa)	0.035	−1.310	0.4724	0.57	1.3	2.5
σc (MPa)	−0.209	−1.593	0.48818	8	37.37	52.505
E (GPa)	−0.015	−1.483	0.42237	2.88	7.54	11.91
φb (°)	0.395	−1.889	0.09737	28	28	34
JRC	0.229	−1.392	0.49666	4.1	12.4	18.9
τp (MPa)	0.511	−0.262	0.46326	0.37	1.08	2.54

provides the descriptive statistics of 84 rock samples. As shown in Table 1, the dataset encompasses a broad range of rock properties, including JRC (4–20), σ_n (0.57–2.5 MPa), φ_b (28–34°), E (2.88–11.91 GPa), and σ_c (8–52.5 MPa), indicating its robustness and representativeness.

Training a machine learning model on the entire dataset can result in overfitting. Therefore, it is common practice to divide the dataset into two parts: one for training the model and the other for evaluating its predictive performance [56,57]. For instance, as described in references [58,59], the dataset is randomly split into a training set and a test set at a ratio of 4:1. Similarly, the collected dataset in this study was processed in the same manner. Specifically, the training set contains 67 data points, while the test set includes 17 data points. The machine learning model is first trained on the training set and then evaluated on the test set to assess its predictive capabilities. After dividing the training set and the test set, the data in the dataset is regularized according to the Equation (7). This translates features at different scales to the same scale so that ML models can learn better.

$$\overline{X}={\displaystyle \frac{X-\mu }{\sigma }}$$

(7)

where X and $\overline{X}$ denote the original and normalized values, respectively, while σ and µ are the standard deviation and mean of the original data, respectively.

After normalizing the dataset, a five-fold cross-validation strategy as presented in Figure 7 was employed to further assess the generalization capability and stability of the proposed model.

The training set was used to perform the five-fold cross-validation, enabling model parameter optimization and performance evaluation across multiple data partitions. Specifically, the training set was equally divided into five subsets. In each iteration, four subsets were used as the learning subset, and the remaining subset was used for validation. This process was repeated five times to ensure that each subset was used once as the validation subset. The performance of the machine learning (ML) model in each fold was evaluated using performance evaluation indicators (P_i) such as the root mean squared error (RMSE) and the correlation coefficient (R²). The final predictive performance was determined by averaging the P_i across all five folds, as defined in Equation (8). This cross-validation method effectively reduces the randomness introduced by arbitrary training–validation splits and provides a more robust and objective assessment of the model’s predictive capability.

$$\overline{P}={\displaystyle \frac{1}{5}}{\displaystyle \sum _{i=1}^5{P}_i}$$

(8)

3.2. Performance Evaluation Indicators

To provide a robust and comprehensive evaluation of the machine learning model’s predictive capability, four commonly adopted performance evaluation indicators are utilized, including the root mean squared error (RMSE), the mean absolute error (MAE), the coefficient of determination (R²), and the variance accounted for (VAF) [60]. R², also known as the coefficient of determination, quantifies the strength of the linear relationship and assesses the goodness of fit between the model’s predictions and actual observations [61,62]. VAF indicates the proportion of the variance in the actual values explained by the model’s predictions, measuring how well the model captures the overall trend of the data [63]. RMSE is the square root of the mean squared difference between the predicted and actual values, and it quantifies the magnitude of the model’s prediction error [64,65]. MAE is the average of the absolute differences between the predicted and actual values, providing a measure of the model’s average prediction error [66,67]. The mathematical expressions for these performance evaluation indicators are provided in Equations (9)–(12).

$$R^2=1-{\displaystyle \frac{{\displaystyle \sum _{i=1}^K{\left(y_i^{obs}-y_i^{pre}\right)}^2}}{{\displaystyle \sum _{i=1}^K{\left(y_i^{obs}-E\left[y^{mean}\right]\right)}^2}}}$$

(9)

$$VAF=\left[1-{\displaystyle \frac{\mathrm{var}\left(y_i^{obs}-y_i^{pre}\right)}{\mathrm{var}\left(y_i^{obs}\right)}}\right]\times 100\%$$

(10)

$$RMSE=\sqrt{{\displaystyle \frac{1}{K}}{\displaystyle \sum _{i=1}^K{\left(y_i^{obs}-y_i^{pre}\right)}^2}}$$

(11)

$$MAE={\displaystyle \frac{1}{K}}{\displaystyle \sum _{i=1}^K\left|y_i^{obs}-y_i^{pre}\right|}$$

(12)

where y_i^obs denotes the observed values, y_i^pre refers to the predicted values, and K is the total number of samples.

4. Results and Analysis

4.1. Results of the Proposed Hybrid ML Model

After completing the hyperparameter tuning, the SMA-MLP model can be applied to predict joint shear strength in the dataset. To facilitate comparison, this section also presents the prediction results of the original MLP model (i.e., the MLP model without SMA optimization). Figure 8 illustrates the comparison between the measured joint shear strength values from the direct shear tests and the predicted results from the two ML models. The closer the data points are to the actual measurements, the better the predictive performance of the model. Perfectly overlapping data points indicate a complete match between the predicted and experimental results. As shown in Figure 8, under the same training and test sets, the SMA-MLP model demonstrates superior predictive performance compared to the original MLP model, with smaller discrepancies between the predicted values and experimental data. This indicates that SMA effectively enhances the predictive capability of the original MLP model, making it more reliable and accurate in forecasting the joint shear strength.

As shown in

Table 2. Prediction performances for each model.

Model	Dataset	RMSE	MAE	R2	VAF
SMA-MLP	Training set	0.01819	0.01232	0.99863	99.864%
	Test set	0.09699	0.06677	0.96869	97.145%
MLP	Training set	0.03322	0.02306	0.99583	99.583%
	Test set	0.14861	0.12334	0.89092	89.943%

, the comparison of the performance evaluation indicators between the two models further verifies the impact of SMA on the predictive capability of the MLP model. The results indicate that the optimized SMA-MLP model exhibits superior predictive performance and higher computational efficiency across multiple performance evaluation indicators, demonstrating the significant role of SMA in enhancing the predictive ability of the MLP model. Specifically, on the test set, the R² of the SMA-MLP model reaches 0.96869, indicating an excellent fit between the predicted and actual values. Moreover, compared to the unoptimized MLP model, the SMA-MLP model achieves 0.09699 and 0.06677 in RMSE and MAE, respectively, significantly minimizing prediction errors. The SMA-MLP model outperforms the MLP model across all four key performance evaluation indicators, highlighting its superior generalization capability and robustness in joint shear strength prediction. Furthermore, the SMA-MLP model effectively captures the complex nonlinear relationships of joint shear strength, resulting in more accurate and stable predictions. Given its exceptional predictive performance, this model serves as a reliable approach for joint shear strength prediction, providing valuable insights for rock mass engineering design and geological hazard assessment.

4.2. Feature Importance Analysis

In ML models, feature importance analysis not only helps identify key influencing factors but also uncovers potential relationships among variables [68,69]. By leveraging advanced algorithms, feature selection can be further performed to eliminate redundant variables and enhance the predictive capability of the ML model [70,71]. In the field of civil engineering, this approach significantly improves the accuracy of joint shear strength predictions, optimizes design parameters, and ultimately provides a robust scientific basis for engineering decision-making. This section employs feature importance analysis to determine the relative importance of input variables, such as the JRC, normal stress, and Young’s modulus, in influencing joint shear strength.

Feature importance scores quantify the contribution of different input variables in an ML model, with higher scores indicating a greater influence on shear strength. Figure 9 presents the computed feature importance scores, where σ_n has the highest score (0.86), followed by JRC (0.16), σ_c (0.12), E (0.12), and φ_b (0.01). Notably, φ_b exhibits the lowest feature importance score (0.01), suggesting that among the five input parameters, it has the least impact on joint shear strength. It is important to note that feature importance scores may vary depending on the dataset and the evaluation method used. Therefore, the ranking of feature importance scores obtained in this paper is specific to the dataset constructed in Section 3.1 and should not be considered a general guideline.

Feature importance analysis provides clear guidance for the subsequent optimization of ML models, helping to identify key variables that require prioritization or more precise measurement. This facilitates more efficient and targeted experimental design. In practical applications, with the rapid development of big data and artificial intelligence technologies, the prospects for applying feature importance analysis in civil engineering are increasingly promising [72]. Under complex geological conditions, feature importance analysis bridges experimental design with actual needs. This approach offers scientific support for structural design and stability assessment, significantly enhancing the rationality and reliability of engineering solutions [73]. It is worth noting that future work could consider incorporating the SHapley Additive exPlanations (SHAP) method to enhance feature importance analysis. SHAP is an emerging model-agnostic interpretation method based on cooperative game theory, which can consistently quantify the contribution of each input feature to individual predictions. While SHAP was not applied in this study due to practical constraints, its potential to overcome the inherent bias of traditional feature importance techniques is acknowledged. Therefore, in future research, we will prioritize the integration of SHAP to improve model transparency and interpretability in line with recent advances in geotechnical AI modeling [74].

4.3. Comparison with Previous Models

In predictive modeling studies using machine learning, categorical boosting (CatBoost), random forest (RF), ridge regression (Ridge), and backpropagation neural network (BPNN) are among the most commonly used models. To evaluate the predictive performance of the SMA-MLP model, this study compares its performance with these classical machine learning models, as illustrated in Figure 10. The results indicate that the proposed model achieves the highest trend evaluation scores and the lowest prediction error metrics, demonstrating superior predictive performance. Specifically, in terms of test set accuracy, the SMA-MLP model attains a coefficient of determination of 0.97, significantly surpassing the highest R² achieved by other ML models. Additionally, while the RMSE of other models ranges from 0.27 MPa to 0.42 MPa, the SMA-MLP model achieves an RMSE of only 0.1 MPa. This remarkably low RMSE value suggests that the proposed model exhibits higher accuracy in joint shear strength prediction. In summary, due to its enhanced ability to capture the complex nonlinear relationships between input and output variables, the SMA-MLP model outperforms other ML models in terms of prediction accuracy.

4.4. Discussion on Data Processing Method

The data normalization method has a significant impact on both the predictive accuracy and training stability of machine learning models. In general, normalization not only eliminates the magnitude discrepancies inherent in raw data, but also markedly improves the computational efficiency of learning algorithms. Therefore, this study investigates the influence of different normalization strategies on the prediction performance of the SMA-MLP model. Three commonly used normalization techniques were selected for comparison:

(i)

Z-score normalization (Equation (7)), which standardizes the data to have zero mean and unit variance;

(ii)

Min–Max normalization (Equation (13)), which linearly scales features into the [0, 1] interval;

(iii)

Arctangent normalization (Equation (14)), which applies a nonlinear transformation to compress input values into the (−1, 1) interval to suppress the influence of outliers.

$$\overline{X}={\displaystyle \frac{X-X_{min}}{X_{max}-X_{\min }}}$$

(13)

$$\overline{X}=2{\displaystyle \frac{\mathit{arctan}X}{\pi }}$$

(14)

where X and $\overline{X}$ denote the original and normalized values, respectively, while X_max and X_min denote the maximum and minimum values of the original dataset, respectively.

Figure 11 presents the distributional characteristics of the input and output variables under three normalization strategies. As shown, the Z-score normalization defined by Equation (7) transforms each variable to have zero mean and unit variance, while preserving the original relative variability and outlier patterns. This method is particularly suitable for models that are sensitive to the covariance structure of the input data. The Min–Max normalization described in Equation (13) linearly scales all features into the range [0, 1], enhancing the comparability between variables, and is especially appropriate for models—such as neural networks—that are sensitive to feature magnitudes. However, this method is highly sensitive to extreme values, which may compress the main data distribution and cause most samples to cluster around the center, potentially masking meaningful distributional information. In contrast, the Arctangent normalization method (Equation (14)) demonstrates stronger compressive capacity through nonlinear transformation, mapping data into the (0, 1) interval while significantly suppressing high-magnitude values. This makes it more robust against outliers and better suited to datasets with long-tailed distributions. In summary, each normalization technique has its own strengths in terms of scale unification and outlier resistance. The choice of method should be based on the characteristics of the dataset and the specific requirements of the predictive model.

Each normalization method was independently applied to the SMA-MLP model. After preprocessing with each normalization method, the same modeling procedure was applied. Each model was trained using an identical training set and evaluated on the same testing set. The predictive results under the three normalization schemes on the test set are illustrated in Figure 12. It is worth mentioning that the prediction result of Equation (7) is the model prediction result shown in Section 4.1. It can be observed from Figure 12 that the Z-score normalization yielded the highest overall prediction accuracy, as indicated by the highest R² value and the lowest RMSE. This superior performance may be attributed to its ability to maintain a balanced dynamic range of input features while preserving the statistical distribution of the data, which facilitates stable gradient propagation and effective parameter learning in MLP networks.

The Min-Max normalization also shows higher prediction accuracy, but the prediction accuracy is obviously lower than Z-score normalization. This can be explained by its sensitivity to extreme values—when the input feature distribution is skewed or includes outliers, the rescaling becomes uneven. In contrast, the Arctangent normalization resulted in a substantial drop in performance. This degradation is likely due to its nonlinear compression, which reduces the contrast between features and effectively obscures meaningful input signals during training. As a convergent, nonlinear transformation, Arctangent normalization diminishes inter-feature distinctions, making it difficult for the model to extract and optimize relevant patterns, ultimately leading to inferior prediction results. In summary, Z-score normalization is demonstrated to be the most appropriate data processing technique for the SMA-MLP model in this study, owing to its consistency, robustness, and superior generalization performance.

4.5. Limitations

This study developed a novel ML model for predicting joint shear strength, achieving promising prediction results. It has fully demonstrated the applicability of ML techniques in joint shear strength prediction. Despite the positive findings, it is essential to recognize the limitations that should be addressed in future work.

(1)

ML techniques extract knowledge from input data in datasets and construct corresponding nonlinear functions for prediction [75]. Due to differing underlying mechanisms, the output results of the trained models may vary across different datasets. The hyperparameters of the models trained on different datasets are inherently distinct. Therefore, to ensure optimal predictive accuracy in practical scenarios, it is advisable to retrain the model individually for each specific dataset, taking into account the unique geological, mechanical, and statistical characteristics inherent in that dataset. Such tailored training allows the model to better capture domain-specific patterns and minimizes potential biases arising from data heterogeneity.

(2)

High-quality datasets are fundamental to ensuring the predictive accuracy of ML models [76]. Input features have an important influence on the prediction accuracy of ML models. The datasets collected in Section 3 can be enriched by introducing new data to improve the model’s universality and prediction accuracy. It is worth mentioning that when combining data from different sources, it is necessary to pay attention to possible differences between measurements that may lead to bias or inconsistencies in the data. Additional features may need to be added to the combined dataset to ensure data consistency and integrity. In ML models, reasonable input feature selection and processing can significantly improve the performance of the model [77]. In addition, the size and diversity of the dataset are also important factors affecting the generalization ability of the model. Therefore, in the future, we plan to expand the dataset and improve the applicability of ML models by diversifying data sources and further optimizing input characteristics.

(3)

In subsequent studies, the application of this model to predict other properties of rock joints, such as shear stiffness and peak dilatancy angle, will be further explored. At the same time, the research scope will gradually expand to the engineering scale and will be combined with numerical simulation methods such as the simplified finite element model (FEM) [78] to evaluate the applicability and robustness of this method in a wider range of application scenarios. Therefore, it is necessary to combine more advanced machine learning algorithms to optimize model hyperparameters to improve the accuracy and generalization ability of model predictions.

(4)

The uncertainty inherent in ML models is another aspect that warrants attention. The main sources of uncertainty in this study include (i) data-level uncertainty caused by measurement errors, sampling bias, and the heterogeneity of geological conditions; (ii) algorithmic stochasticity stemming from random initialization and evolutionary processes during model optimization; (iii) sensitivity to hyperparameters and structural configurations of the hybrid ML model; and (iv) limited generalization capacity when applied to unseen or out-of-distribution datasets. To mitigate these uncertainties, future research should consider incorporating uncertainty quantification strategies—such as ensemble methods, bootstrap resampling, or Bayesian frameworks—to explicitly estimate and reduce the variance in model predictions. This will enhance the reliability and interpretability of the developed models when applied in practical engineering settings.

In recent years, although the research on soft intelligence technology has been deepening, and the related theories and methods are becoming more and more mature, there are still many challenges in the application of these technologies in solving practical engineering problems [79]. These challenges include high-dimensional data processing in complex systems, real-time response requirements, and stringent requirements for system stability and security [80]. Therefore, when developing intelligent systems in the future, it is necessary to focus on combining industry needs to build efficient and reliable intelligent solutions.

5. Conclusions

This paper proposes a hybrid machine learning model (SMA-MLP) that integrates the MLP with the SMA for predicting the shear strength of rock joints. The incorporation of SMA enhances the global optimization capability of MLP model, effectively mitigating local optima and overfitting issues, thereby improving prediction accuracy and generalization performance. The model was systematically validated using the constructed dataset. The experimental results demonstrate that SMA-MLP model outperforms other machine learning models in performance evaluation indicators, achieving an R² of 0.97 and reducing RMSE to 0.10 MPa on the test set. Furthermore, feature importance analysis indicates that normal stress has the most significant influence on joint shear strength. This paper confirms the superior performance of SMA-MLP model in shear strength prediction, offering a reliable methodological framework for rock engineering design and geological hazard assessment.