1. Introduction
Pillars are essential structural units to ensure mining safety in underground hard rock mines. Their functions are to provide safe access to working areas, and support the weight of overburden rocks for guaranteeing global stability [
1,
2]. Instable pillars can cause large-scale catastrophic collapse, and significantly increase safety hazards of workers [
3]. If a pillar fails, the adjacent pillars have to bear a larger load. The increased load may exceed the strength of adjacent pillars, and lead to their failure. Like the domino effect, rapid and large-scale collapse can be induced [
4]. In addition, with the increase of mining depth, the ground stress is larger and pillar instability accidents become more frequent [
5,
6,
7]. Accordingly, determining pillar stability is essential for achieving safe and efficient mining.
In general, there are three types of methods to determine pillar stability in hard rock mines. The first one is the safety factor approach. The safety factor indicates the ratio of pillar strength to stress [
8]. Using this method, three parts, namely, pillar strength, pillar stress, and safety threshold, should be determined. For the calculation of pillar strength, many empirical methods have been proposed, such as the linear shape effect, power shape effect, size effect, and Hoek–Brown formulas [
9]. For the determination of pillar stress, tributary area theory, and numerical modeling technology are the main approaches [
10]. Based on the pillar strength and stress, the safety factor can be calculated. A larger safety factor means the pillar is more stable. Theoretically, the safety threshold is assigned as 1. If the safety factor is larger than 1, the pillar is stable, otherwise it is unstable [
11]. Considering the possible deviations of this method, the safety threshold is generally larger than 1 to ensure safety in practice. Although the safety factor method is simple, the unified pillar strength formula and safety threshold are still not determined.
The second one is the numerical simulation technique. Numerical simulation methods have been widely used because the complicated boundary conditions and rock mass properties can be considered. Mortazavi et al. [
12] adopted fast Lagrangian analysis of continua (FLAC) approach to study the failure process and non-linear behavior of rock pillars; Shnorhokian et al. [
13] used FLAC
3D to assess the pillar stability based on different mining sequence scenarios; Elmo and Stead [
14] integrated finite element method (FEM) and discrete element method (DEM) to investigate the failure characteristics of naturally fractured pillars; Li et al. [
15] used rock failure process analysis (RFPA) to evaluate the pillar stability under coupled thermo-hydrologic-mechanical conditions; Jaiswal et al. [
16] utilized boundary element method (BEM) to simulate asymmetry in the induced stresses over pillars; and Li et al. [
17] introduced finite-discrete element method (FDEM) to analyze the mechanical behavior and failure mechanisms of the pillar. In addition, some scholars combined numerical simulation and other mathematical models to study pillar stability. Deng et al. [
18] used FEM, neural networks, and reliability analysis to optimize pillar design; and Griffiths et al. [
19] adopted random field theory, elastoplastic finite element algorithm and Monte-Carlo approach to analyze pillar failure probability. Using numerical simulation techniques, the complex failure behaviors of pillars can be modeled and the failure process and range can be obtained. However, because of the complicated nonlinear characteristics and anisotropic nature of rock mass, the model inputs and constitutive equations are not easy to be accurately determined [
20]. Therefore, the reliability of results obtained by this method is limited.
The third one is the machine learning (ML) algorithm. Big data has been proved to be useful for managing emergencies and disasters [
21]. With the accumulation of pillar stability cases, it provides researchers the opportunity to develop advanced predictive models using ML algorithms. Tawadrous and Katsabanis [
22] used the artificial neural networks (ANN) to analyze the stability of surface crown pillars; Wattimena [
23] introduced the multinomial logistic regression (MLR) for pillar stability prediction; Ding et al. [
24] adopted a stochastic gradient boosting (SGB) model to predict pillar stability, and found that this model achieved a better performance than the random forest (RF), support vector machine (SVM), and multilayer perceptron neural networks (MPNN); Ghasemi et al. [
25] utilized J48 algorithm and SVM to develop two pillar stability graphs, and obtained acceptable prediction accuracy; and Zhou et al. [
9] compared the prediction performance of pillar stability using six supervised learning methods, and revealed that SVM and RF performed better. Although these ML algorithms can solve pillar stability prediction issues to some extent, none can be applied to all engineering conditions. To date, there is not a uniform algorithm under the consensus of the mining industry.
Compared with traditional safety factor and numerical simulation methods, ML approach can deeply find implicit relationships between variables, and well handle nonlinear problems. Therefore, it is a promising method for predicting pillar stability. In addition to these above ML algorithms, the gradient boosting decision tree (GBDT) method has shown great prediction performances in many fields [
26,
27,
28]. As one of the ensemble learning algorithms, it integrates multiple decision trees (DTs) into a strong classifier using the boosting approach [
29]. DTs refer to a ML method using tree-like structure, which can deal with various types of data and trace every path to the prediction results [
30]. However, DTs are easy to overfit and sensitive to noises of the dataset. Through the integration of DTs, the overall prediction performances of GBDT become better because the errors of DTs are compensated by each other. Under the framework of GBDT, extreme gradient boosting (XGBoost) [
31] and light gradient boosting machine (LightGBM) [
32] have been proposed recently. They have also received wide attentions because of their excellent performances. Particularly, these three algorithms work well with small datasets. Additionally, overfitting, which means the model fits the existing data too exactly but fails to predict the future data reliably, can be avoided to some extent [
30]. However, to the best of our knowledge, GBDT, XGBoost, and LightGBM algorithms have not been used to predict pillar stability in hard rock mines.
The objective of this study is to predict hard rock pillar stability using GBDT, XGBoost, and LightGBM algorithms. First, pillar stability cases are collected from seven hard rock mines. Next, these three algorithms are applied to predict pillar stability. Finally, their comprehensive performances are analyzed and compared with the safety factor approach and other ML algorithms.
2. Data Acquisition
The existing hard rock pillar cases are obtained as supportive data for the establishment of prediction model. In this study, 236 samples were collected from seven underground hard rock mines [
8,
10,
33]. These mines include the Elliot Lake uranium mine, Selebi-Phikwe mine, Open stope mine, Zinkgruvan mine, Westmin Resources Ltd.’s H-W mine, Marble mine, and Stone mine. The pillar stability data is shown in
Table 1 (the complete database is available in
Table S1), where
indicates the pillar width,
indicates the pillar height,
indicates the ratio of pillar width to pillar height,
indicates the uniaxial compressive strength, and
indicates the average pillar stress.
From
Table 1, it can be seen that each sample contains five indicators and the level of pillar stability. These five indicators have been widely used in a large amount of literature [
9,
10,
24,
33]. The advantages are two-fold: (1) They can reflect the necessary conditions of pillar stability, such as pillar size, strength, and load; (2) their values are relatively easy to be obtained. The descriptive statistics of each indicator are listed in
Table 2. Besides, according to the instability mechanism and failure process of pillars, the level of pillar stability is divided into three types: stable, instable, and failed. Among them, the stable level indicates that the pillar shows no sign of stress induced fracturing, or only has minor spallings which do not affect pillar stability; the instable level indicates that the pillar is partially failed, and has prominent spallings, but still possesses a certain supporting capacity; the failed level indicates that the pillar is crushed, and has pronounced openings of joints, which shows a great collapse risk. The distribution of these three pillar stability levels is described in
Figure 1.
The box plot of each indicator for different pillar stability level is shown in
Figure 2. According to
Figure 2, some meaningful characteristics can be found. First, all indicators have several outliers. Second, the pillar stability level is negatively correlated with
and
(the Pearson correlation coefficients are −0.381 and −0.266, respectively), while is positively correlated with
(the Pearson correlation coefficient is 0.433). However, there are no obvious correlations with
and
(the Pearson correlation coefficients are −0.121 and 0.079, respectively). Third, the distances between upper and lower quartiles are different for the same indicators in various levels. Fourth, there are some overlapping parts for the range of indicator values in different levels. Fifth, as the median is not in the center of the box, the distribution of indicator values is asymmetric. All these phenomena illustrate the complexity of pillar stability prediction.
6. Discussions
Although the pillar stability can be well predicted using GBDT, XGBoost, and LightGBM algorithms, the prediction performance for different instable level was not the same. The reason may be two-fold. One is that the number of samples for instable level is smaller than other two levels. Another is that the discrimination boundary of instable level is more uncertain, which would influence the quality of data. As data-driven approaches, the prediction performances of GBDT, XGBoost, and LightGBM algorithms are greatly affected by the number and quality of supportive data. Therefore, compared with other two levels, the prediction performance for instable level was worse.
According to the analysis of indicator importance,
and
were the most important indicator. The reason may be that the pillar stability is greatly affected by the external stress conditions and inherent dimension characteristics. According to the field experience, the spalling phenomenon of pillar was more common in deep mines because of high stress, and the pillar with small
value was more vulnerable to be damaged. Based on the importance degrees of indicators, some measures can be adopted to improve pillar stability from two directions. One is reducing pillar stress, such as the adjustment of excavation sequence and relief of pressure [
32]. Another is optimizing pillar parameters, such as the increase of pillar width and ratio of pillar width to pillar height.
To further illustrate the effectiveness of GBDT, XGBoost, and LightGBM algorithms, the safety factor approach and other ML algorithms were adopted as comparisons.
First, the safety factor approach was used to determine pillar stability. According to the research of Esterhuizen et al. [
33], the pillar strength can be calculated by
Based on Equation (20) and the given pillar stress, the safety factor can be determined by their quotient. From the work of Lunder and Pakalnis [
45], when
, the pillar was stable; when
, the pillar was unstable; and when
, the pillar was failed. According to this discrimination method, the prediction accuracy was 0.6610. From the work of González-Nicieza et al. [
8], when
, the pillar was stable; when
, the pillar was unstable; and when
, the pillar was failed. Base on this classification method, the prediction accuracy was 0.6398. Therefore, the prediction accuracy of safety factor approach was lower than that of the proposed methodology. It can be inferred that this empirical method has difficulty in obtaining satisfactory prediction results on these pillar samples.
On the other hand, some ML algorithms, such as RF, adaptive boosting (AdaBoost), MLR, Gaussian naive Bayes (GNB), Gaussian processes (GP), DT, MLPNN, SVM, and k-nearest neighbor (KNN), were adopted to predict pillar stability. First, some key hyperparameters in these algorithms were tuned, and the optimization results were listed in
Table 6. Afterwards, these algorithms with optimal hyperparameters were used to predict pillar stability based on the same dataset. The accuracy of each algorithm was shown in
Figure 12. It can be seen that the accuracies of these algorithms were all lower than those of GBDT, XGBoost, and LightGBM algorithms. The accuracies of most algorithms (except for GP) were lower than 0.8, whereas the accuracies of GBDT, XGBoost, and LightGBM algorithms were all higher than 0.8. It demonstrated that compared with other ML algorithms, GBDT, XGBoost, and LightGBM algorithms were better for predicting pillar stability in hard rock mines.
Although the proposed approach obtains desirable prediction results, some limitations should be addressed in the future.
- (1)
The dataset is relatively small and unbalanced. The prediction performance of ML algorithms is heavily affected by the number and quality of dataset. Generally, if the dataset is small, the generalization and reliability of model would be influenced. Although GBDT, XGBoost, and LightGBM algorithms work well with small datasets, the prediction performances could be better on a larger dataset. In addition, the dataset is unbalanced, particularly for samples with instable level. Compared with other levels, the prediction performance for the instable level is not good. This illustrates the adverse effects of imbalanced data on results. Therefore, it is meaningful to establish a larger and more balanced pillar stability database.
- (2)
Other indicators may also have influences on the prediction results. Pillar stability is affected by numerous factors, including the inherent characteristics and external environments. Although the five indictors adopted in this study can describe the necessary conditions of pillar stability to some extent, some other indicators may also have effects on pillar stability, such as joints, underground water, and blasting disturbance. Theoretically, the joints and underground water can affect the pillar strength, and blasting disturbance can be deemed as a dynamic stress on the pillar. Accordingly, it is significant to analyze the influences of these indicators on the prediction results.
7. Conclusions
To ensure mining safety, pillar stability prediction is a crucial task in underground hard rock mines. This study investigated the performance of GBDT, XGBoost, and LightGBM algorithms for pillar stability prediction. The prediction models were constructed based on training set (165 cases) after their hyperparameters were tuned using the five-fold CV method. The test set (71 cases) were adopted to validate the feasibility of trained models. Overall, the performances of GBDT, XGBoost, and LightGBM were acceptable, and their prediction accuracies were 0.8310, 0.8310, and 0.8169, respectively. By comprehensively analyzing the accuracy and macro average of precision, recall and F1, the rank of overall prediction performance was GBDT > XGBoost > LightGBM. According to the precision, recall and F1 of each stability level, the prediction performance for stable and failed levels was better than that for instable level. Based on the importance scores of indicators from these three algorithms, the average pillar stress and ratio of pillar width to pillar height were the most influential indicators on the prediction results. Compared with the safety factor approach and other ML algorithms (RF, AdaBoost, LR, GNB, GP, DT, MLPNN, SVM, and KNN), the performances of GBDT, XGBoost, and LightGBM were better, which further verified that they were reliable and effective for the pillar stability prediction.
In the future, a larger and more balanced pillar stability database can be established to further illustrate the adequacy of these algorithms for the prediction of stable, instable, and failed pillar levels. The influences of other indicators on the prediction results are essential to be analyzed. The methodology can also be applied in other fields, such as the risk prediction of landslide, debris flow, and rockburst.