Machine Learning Prediction and Interpretability Analysis of Coal and Gas Outbursts

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This manuscript addresses the critical safety challenge of "coal and gas outburst" in underground mining through machine learning modeling and interpretability analysis. 
1. Figure 1" is used both for statistical data on page 2 and for the geological map on page 3. Please recheck all figure numbers to ensure they follow a continuous sequence (e.g., Figure 1, Figure 2, etc.) throughout the manuscript.
2. The dataset consists of only 60 samples. For complex models like XGBoost and BP, there is a substantial risk of overfitting. The authors should clarify whether techniques like Cross-Validation or Leave-One-Out Cross-Validation (LOOCV) were used to ensure model generalization.
3. Table 4 lists 11 feature variables ($X_1$ to $X_{11}$) 20, but the abstract and analysis focus on "four main indicators"21. Please provide a clear definition for each variable ($X_1$–$X_{11}$, e.g., burial depth, gas pressure, etc.) in the text or table caption and explain the mapping between these 11 features and the four main factors.
4. Random Forest" is abbreviated as "RT" in the abstract and model selection section. It is more standard to use "RF". Please unify this throughout the paper.
5. There are minor grammatical issues in the abstract (e.g., "...identified four main geological..."). Consider revising to "This study identified four main geological..." or "Four main geological indicators were identified...".
6. SHAP identifies "geological structure" as the top feature. I suggest expanding the discussion on how this finding can be practically applied on-site. For instance, how should mine safety protocols prioritize monitoring based on these feature weightings?
7. Since the paper is submitted to Sustainability, the authors should explicitly link coal mine safety with social/environmental sustainability and resource recovery in the conclusion to better align with the journal's focus.

Author Response

Comments 1: [Figure 1" is used both for statistical data on page 2 and for the geological map on page 3. Please recheck all figure numbers to ensure they follow a continuous sequence (e.g., Figure 1, Figure 2, etc.) throughout the manuscript.]

Response 1: [Thank you for pointing this out. Due to an oversight on my part, the former No. 11 mine thrust fault was erroneously abbreviated as “F1” in the text, and the “F1 fault” was omitted in Figure 2(b). Accordingly, Figure 2(b) has been modified to include the annotation of the F1 fault, and the abbreviation has been corrected to “F1 fault” in the text. Mention exactly where in the revised manuscript this change can be found – page 2, and line 75.]

Comments 2: [The dataset consists of only 60 samples. For complex models like XGBoost and BP, there is a substantial risk of overfitting. The authors should clarify whether techniques like Cross-Validation or Leave-One-Out Cross-Validation (LOOCV) were used to ensure model generalization.]

Response 2: [Thank you for pointing this out. We employed 5‑fold stratified cross‑validation, and the corresponding results have been added to Table 7. Furthermore, in response to the comments from the other two reviewers regarding the nonlinear characteristics of coal and gas outbursts, the mutual information method was incorporated during feature selection, considering both linear and nonlinear relationships. Consequently, the final set of input indicators was updated. Although the AUC value decreased slightly, the standard deviation derived from cross‑validation became smaller. Therefore, the updated set of predictive indicators was ultimately adopted. Mention exactly where in the revised manuscript this change can be found – page11, and line 247.]

Comments 3: [Table 4 lists 11 feature variables ($X_1$ to $X_{11}$) 20, but the abstract and analysis focus on "four main indicators"21. Please provide a clear definition for each variable ($X_1$–$X_{11}$, e.g., burial depth, gas pressure, etc.) in the text or table caption and explain the mapping between these 11 features and the four main factors.]

Response 3: [Thank you for pointing this out. We agree with this comment. Each indicator was categorized under the following four factors. X1 belong to in-situ stress factors, X2, X3, X4, X5 belongs to geological structural factors, X6, X7, X8 belongs to coal properties factors, and X9, X10, X11 belong to coal seam gas factors. Mention exactly where in the revised manuscript this change can be found – page 9-10, and line212-214.]

Comments 4: [Random Forest" is abbreviated as "RT" in the abstract and model selection section. It is more standard to use "RF". Please unify this throughout the paper.]

Response 4: [Thank you for pointing this out. Due to an oversight, "RF" was mistakenly labeled as "RT" in the original manuscript. All relevant instances have been corrected throughout the text. Mention exactly where in the revised manuscript this change can be found – page 1, and line17; page 6, and line158; page 11, and Table 7.] “[updated text in the manuscript if necessary]”

Comments 5: [There are minor grammatical issues in the abstract (e.g., "...identified four main geological..."). Consider revising to "This study identified four main geological..." or "Four main geological indicators were identified...".]

Response 5: Agree. We have revised this passage to: Four main geological indicators were identified by examining the attributes of these fac-tors and their association to outburst intensity. Mention exactly where in the revised manuscript this change can be found – page 1, and line 14-15.]

Comments 6: [SHAP identifies "geological structure" as the top feature. I suggest expanding the discussion on how this finding can be practically applied on-site. For instance, how should mine safety protocols prioritize monitoring based on these feature weightings?]

Response 6: [Thank you for pointing this out. We have incorporated this content into the discussion section as follows: In accordance with the Coal Mine Safety Regulations, geological exploration and assessment must be strengthened. During both the construction and production phases, if significant changes in geological conditions occur, such as encountering unknown faults or folds, timely supplementary exploration should be conducted. Meanwhile, by monitoring changes in parameters such as gas concentration, emission rate, and mining-induced stress in geologically anomalous zones, advanced warning of outburst risks near geological structural belts can be achieved. Mention exactly where in the revised manuscript this change can be found – page number13, and line 289-295.]

Comments 7: [Since the paper is submitted to Sustainability, the authors should explicitly link coal mine safety with social/environmental sustainability and resource recovery in the conclusion to better align with the journal's focus.]

Response 7: [Thank you for pointing this out. We have added a sentence to both the Abstract and Conclusions sections to emphasize the manuscript's relevance to sustainable development, as follows: Coal and gas outbursts constitute a major hazard for mining safety, which is critical for the sustainable development of China’s energy industry. Mention exactly where in the revised manuscript this change can be found – page 1, and line 10-11; The prevention of coal and gas outbursts is critical for mining safety and, by extension, energy sustainability. Thus, enhancing the accuracy of outburst prediction is of paramount importance. Mention exactly where in the revised manuscript this change can be found – page 13, and line 311-313.]

Reviewer 2 Report

Comments and Suggestions for Authors

Many machine learning models operate as ‘black boxes’, exhibiting poor interpretability and making their decision-making logic difficult to comprehend. To improve model explainability, interpretative methods are introduced, aiming to provide technical support and reference for preventing and controlling this hazard.

1.The emphasis on "model interpretability" in the introduction has practical significance, but it does not clearly explain the differences between this study and existing interpretability research. It is suggested to clearly state the contribution of this study at the end of the introduction, such as the first systematic application of SHAP for multi model comparison and global feature interpretation in gas outburst prediction.

2.Feature selection is based solely on Pearson correlation coefficient, without considering non-linear relationships or multicollinearity. Suggest adding feature importance ranking for verification.

3.The division method between the training set and the test set (such as whether it is random or divided by time) is not explained, and it is not mentioned whether cross validation is used. The model evaluation process is not transparent.

4.The description of hyperparameter optimization is relatively vague and does not list the key parameter ranges that each model can search for through BO, lacking reproducibility. Some latest research work related with this topic can be referred. Disaster-causing mechanism of spalling rock burst based on folding catastrophe model in coal mine. Study on the energy evolution mechanism and fractal characteristics of coal failure under dynamic loading.

5.The SHAP analysis section only displays the importance of global features and does not provide local explanatory examples (such as the SHAP contribution map of a high-risk sample), resulting in insufficient depth of explanation.

6.The criticism of the issue of "model complexity" in the conclusion section is reasonable, but it does not propose a modeling approach that prioritizes lightweight or interpretability.

Author Response

Comments 1: [The emphasis on "model interpretability" in the introduction has practical significance, but it does not clearly explain the differences between this study and existing interpretability research. It is suggested to clearly state the contribution of this study at the end of the introduction, such as the first systematic application of SHAP for multi model comparison and global feature interpretation in gas outburst prediction.] Response 1: [Thank you for your positive assessment of our research. We have duly acknowledged your valuable suggestions and incorporated them at the end of the Introduction section. Mention exactly where in the revised manuscript this change can be found – page 2, and line 57-58.] Comments 2: [Feature selection is based solely on Pearson correlation coefficient, without considering non-linear relationships or multicollinearity. Suggest adding feature importance ranking for verification.] Response 2: [Thank you for pointing this out. Mutual information method was incorporated during feature selection, considering both linear and nonlinear relationships. Consequently, the final set of input indicators was updated. Although the AUC value decreased slightly, the standard deviation derived from cross‑validation became smaller. Therefore, the updated set of predictive indicators was ultimately adopted. Mention exactly where in the revised manuscript this change can be found – page10-11, Table 5 and line 224-232,235.] Comments 3: [The division method between the training set and the test set (such as whether it is random or divided by time) is not explained, and it is not mentioned whether cross validation is used. The model evaluation process is not transparent.] Response 3: [Thank you for pointing this out. We employed 5‑fold stratified cross‑validation, and the corresponding results have been added to Table 7. Mention exactly where in the revised manuscript this change can be found – page11, and line 246. We have added Section 3.2 Hyperparameter tuning to explain the division method between the training set and the test set.Mention exactly where in the revised manuscript this change can be found – page11, and line239.] Comments 4: [The description of hyperparameter optimization is relatively vague and does not list the key parameter ranges that each model can search for through BO, lacking reproducibility. Some latest research work related with this topic can be referred. Disaster-causing mechanism of spalling rock burst based on folding catastrophe model in coal mine. Study on the energy evolution mechanism and fractal characteristics of coal failure under dynamic loading.] Response 4: [Thank you for pointing this out. We have added Section 3.2 Hyperparameter tuning to list the key parameter ranges that each model can search for through BO. Mention exactly where in the revised manuscript this change can be found – page 11, and Table 6. Thank you for recommending the two latest articles. We have incorporated them into our content to enhance the overall quality of our reasarch. 28.   Cao, Z.; Zhang, S.; Xue, Y.; Wang, Z.; Du, F.; Li, Z.; Huang, C.; Wang, S.; Yu, Y.; Wang W.; Zhai, M. Disaster-causing mechanism of spalling rock burst based on folding catastrophe model in coal mine. Rock Mech. Rock Eng. 2025, 1-14. https://doi.org/10.1007/s00603-025-04497-6. 29.   Lin, H.; Zhang, W.; Guo, S.; Zhang, X.; Wang, L.; Zhang, Z. Study on the energy evolution mechanism and fractal characteristics of coal failure under dynamic loading. ACS omega 2025, 10(45), 54710-54719. https://doi.org/10.1021/acsomega.5c08162. Mention exactly where in the revised manuscript this change can be found – page 6, and line 137; page 15, and line 393-397.] Comments 5: [The SHAP analysis section only displays the importance of global features and does not provide local explanatory examples (such as the SHAP contribution map of a high-risk sample), resulting in insufficient depth of explanation.] Response 5: [Thank you for pointing this out. While considering the feature importance rankings, we elucidate the role each feature plays in the risk level of coal and gas outbursts—whether it is positively contributing or negatively impacting. Analysis results indicate that as the geological structure characteristic value increases, the SHAP value also increases, with the positive contribution rising, indicating a heightened risk level. Mention exactly where in the revised manuscript this change can be found – page12, and line 262-264; As shown in Figure 8. (b), as the coal hardiness coefficients characteristic value increases, the SHAP value decreases, indicating a greater negative impact and a reduced risk level. All other characteristics have a positive effect on coal and gas outbursts risk level. Mention exactly where in the revised manuscript this change can be found – page12, and line 274-277.] Comments 6: [The criticism of the issue of "model complexity" in the conclusion section is reasonable, but it does not propose a modeling approach that prioritizes lightweight or interpretability.] Response 6: [Thank you for pointing this out. This also highlights the shortcomings in our research and represents areas that require further exploration in subsequent studies.]

Reviewer 3 Report

Comments and Suggestions for Authors

The paper submitted for review, entitled: Machine learning prediction and interpretability analysis of coal and gas outbursts, concerns a very important issue that is deadly dangerous for workers: gas and coal outbursts in hard coal mines.

The paper has a very apt choice of research problem. The authors address a real, significant, and still unresolved problem of predicting gas and coal outbursts. They focused on geological structure as a key predictive factor, which is consistent with classical mining knowledge and empirical literature. They analyzed the geological structure of the region (faults, folds, tectonic stress, tectonic coal) with machine learning tools, thanks to the selection of algorithms (KNN, BP, RF, SVM, XGBoost), which is standard but justified – it allows for a reliable comparison, thus skillfully combining an engineering approach with ML. The authors used the SHAP method, which should be clearly recognized as a strength of the work. The authors not only demonstrated high prediction accuracy, but also attempted to interpret the results, which is still rare in ML work in mining. The result obtained indicates geological structure as the most important feature, which is consistent with domain knowledge, thus increasing the credibility of the model.

However, the paper has limitations and methodological flaws. The main issue is the very small sample size. The dataset includes only 60 observations from a single mine. With such a small sample size, there is a high risk of overfitting, and the AUC = 0.97 may be overestimated. Thus, the generalization of the results to other mines is questionable. The authors did not perform a sensitivity analysis or validation between other mines. There is also a strong aggregation of static data. All input features are static or quasi-static (geology, structure, coal parameters). Dynamic factors, which in practice often act as direct precursors to emissions, are missing. Feature reduction was based solely on Pearson's correlation. Variables were eliminated based solely on linear correlation, which ignores nonlinear interactions and may remove variables that are relevant in the presence of other features. This is a simplified approach, especially in the context of using XGBoost.

A key observation is the lack of analysis of the mine's "own background." The authors did not take into account one of the fundamental preventive approaches, including continuous monitoring of gas concentrations (CH₄, CO₂) and analysis of temporal variability of concentrations, rather than just maximum values.

In mining practice, deviations from the background are important, not absolute values in a single measurement. It is particularly important from a methodological point of view that the authors do not consider the fact that the rock mass works continuously, regardless of the work being carried out, and that at night or during periods without mining, the following can be observed: gas emissions, microseismicity, and acoustic emissions from the rock mass.  The determination of the "mine's own background" (baseline), i.e.

• gas,
• acoustic,
• microseismic,

is used in many countries as an early warning tool.

Are such approaches used in practice? Yes, and have been for years, although often outside the mainstream of ML: microseismic monitoring of the rock mass (AE, MS) to identify stress concentration zones, continuous monitoring of methane emissions with trend and fluctuation analysis, analysis of acoustic signals in excavations as an indicator of increasing instability.

The authors do not refer to literature on acoustic emission, microseismic monitoring, or the concept of "quiet periods" as a point of reference. This is a serious methodological oversight, especially in a work that aspires to be a decision-making tool.

The study focuses exclusively on static geological and gas-related indicators, ignoring approaches based on continuous monitoring. In underground coal mines, rock mass is constantly active, even during non-operational periods. In practice, continuous monitoring of gas concentration, acoustic emissions, or microseismic background analysis during periods of inactivity (e.g., during night shifts) is commonly used to identify areas with an increased risk of explosion. The failure to take such basic background signals into account is a methodological limitation of the proposed forecasting framework.

It is particularly commendable that the authors do not use self-citations, which increases the scientific and ethical credibility of the work.

 

Author Response

Response to Reviewer 3 Comments Thank you very much for taking the time to review our manuscript entitled “Machine learning prediction and interpretability analysis of coal and gas outburst” (ID: sustainability-4070765). Those comments are all valuable and very helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied comments carefully and have made correction which we hope meet with approval. Revised portion are marked in red in the paper. The main corrections in the paper and the responds to the reviewer’s comments are as flowing: Comments 1: [The main issue is the very small sample size. The dataset includes only 60 observations from a single mine. With such a small sample size, there is a high risk of overfitting, and the AUC = 0.97 may be overestimated. Thus, the generalization of the results to other mines is questionable. The authors did not perform a sensitivity analysis or validation between other mines.] Response 1: [Thank you for pointing this out. We employed 5‑fold stratified cross‑validation, and the corresponding results have been added to Table 7. Furthermore, in response to the nonlinear characteristics of coal and gas outbursts, the mutual information method was incorporated during feature selection, considering both linear and nonlinear relationships. Consequently, the final set of input indicators was updated. Although the AUC value decreased slightly, the standard deviation derived from cross‑validation became smaller. Therefore, the updated set of predictive indicators was ultimately adopted.   Considering the nonlinearity of coal and gas outbursts, the mutual information (MI) method is adopted to capture nonlinear relationships among features. The calculation formula is as follows:   (14) As shown in Table 5, the absolute values of Pearson correlation coefficients and mutual information for all obtained features are normalized via Z-score normalization. A comprehensive score for each feature is then calculated through weighted fusion, using the following formula:   (15) where α + β = 1. Given the nonlinear of outbursts, it is assumed that α = 0.3 and β = 0.7. Table 5. Feature Importance Score. Feature X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 Pearson 0.14 0.15 0.15 0.14 0.23 0.09 0.15 0.33 0.11 0.2 0.27 MI 0.04 0.39 0.38 0.41 0.49 0.15 0.37 0.41 0.07 0.13 0.29 Score -0.83 -0.07 -0.09 -0.13 0.89 -1.11 -0.11 1.71 -1.07 -0.08 0.91 Table 7 summarizes the performance of all models 5-fold stratified cross-validation. As shown in Table 7, XGBoost achieved the highest test AUC (0.90) and Accuracy (0.94), followed closely by SVM. Table 7. Summary of model performance. Models AUC Accuracy Precision Recall F1 score AUC (5-CV ± SD) KNN 0.82 0.87 0.85 0.84 0.85 0.818 ± 0.013 BP 0.84 0.90 0.91 0.88 0.89 0.834 ± 0.015 RF 0.77 0.80 0.82 0.84 0.78 0.765 ± 0.017 SVM 0.85 0.91 0.89 0.89 0.9 0.846 ± 0.012 Mention exactly where in the revised manuscript this change can be found –page10-11, Table 5 and line 224-232,235; page11, and line 246.
Comments 2: The authors do not refer to literature on acoustic emission, microseismic monitoring, or the concept of "quiet periods" as a point of reference. This is a serious methodological oversight, especially in a work that aspires to be a decision-making tool. The study focuses exclusively on static geological and gas-related indicators, ignoring approaches based on continuous monitoring. In underground coal mines, rock mass is constantly active, even during non-operational periods. In practice, continuous monitoring of gas concentration, acoustic emissions, or microseismic background analysis during periods of inactivity (e.g., during night shifts) is commonly used to identify areas with an increased risk of explosion. The failure to take such basic background signals into account is a methodological limitation of the proposed forecasting framework.]
Response 2: [Thank you for pointing this out. We have provided an explanation in the discussion section and cited relevant literature.  (1) This study is confined to static features and does not incorporate dynamic features, such as microseismic [53], acoustic emission [54], electromagnetic radiation [55], ground penetrating radar [56], and seismic wave [57] computed tomography. Future research should integrate these dynamic features with static ones to enhance the scientific validity and predictive accuracy of the framework. 1.          Chen, J.; Zhu, C.; Du, J.; Pu, Y.; Pan, P.; Bai, J.; Qi, Q. A quantitative pre-warning for coal burst hazardous zones in a deep coal mine based on the spatio-temporal forecast of microseismic events. Process Saf. Environ. Prot. 2022, 159, 1105-1112. https://doi.org/10.1016/j.psep.2022.01.082. 2.          Li, J.; Hu, Q.; Yu, M.; Li, X.; Hu, J.; Yang, H. Acoustic emission monitoring technology for coal and gas outburst. Energy Sci. Eng. 2019, 7(2), 443-456. https://doi.org/10.1002/ese3.289. 3.          Liu, Q.; Qiu, L.; Ma, Y.; Song, D.; Yan, M.; Wang, W.; Liu, J.; Qie, L.; Jia, Q.; Liao, P. Research on outburst risk prediction method for coal roadway excavation process based on resistivity-electromagnetic radiation detection. J. Appl. Geophys. 2025, 106011. https://doi.org/10.1016/j.jappgeo.2025.106011. 4.          Xu, X.; Peng, S.; Yang, F. Development of a ground penetrating radar system for large-depth disaster detection in coal mine. J. Appl. Geophys. 2018, 158, 41-47. https://doi.org/10.1016/j.jappgeo.2018.07.006. 5.          Qiu, L.; Zhu, Y.; Liu, Q.; Guo, M.; Song, D.; Wang, A. Response law and indicator selection of seismic wave velocity for coal seam outburst risk. Adv. Geo-Energy Res. 2023, 9(3), 198-210. https://doi.org/10.46690/ager.2023.09.07. Mention exactly where in the revised manuscript this change can be found –page13 and line 298-302; page16, and line447-458. .

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

I have carefully reviewed the authors' responses to the reviewers' comments and the corresponding revisions made to the manuscript. I am very satisfied with the revisions provided in response to the points I raised as Reviewer . The authors have addressed all of my concerns thoroughly and thoughtfully. The clarifications added to the text, the additional analysis where requested, and the improved discussion have significantly strengthened the manuscript. The manuscript is now methodologically sound, the results are clearly presented, and the conclusions are well-supported by the evidence. It makes a valuable contribution to the field. Therefore, I recommend acceptance of the manuscript in its current form.

Reviewer 2 Report

Comments and Suggestions for Authors

Accept