Intelligent and Sustainable Classification of Tunnel Water and Mud Inrush Hazards with Zero Misjudgment of Major Hazards: Integrating Large-Scale Models and Multi-Strategy Data Enhancement

Abstract

Water and mud inrush hazards pose significant threats to the safety, environmental stability, and resource efficiency of tunnel construction, representing a critical barrier to the development of sustainable transportation infrastructure. Misjudgment—especially missed detections of severe hazards—can lead to extensive geological disturbance, excessive energy consumption, and severe socio-environmental impacts. However, pre-trained large-scale models still face two major challenges when applied to tunnel hazard classification: limited labeled samples and the high cost associated with misclassifying severe hazards. This study proposes a sustainability-oriented intelligent classification framework that integrates a large-scale pre-trained model with multi-strategy data augmentation to accurately identify hazard levels during tunnel excavation. First, a Synthetic Minority Over-Sampling Technique (SMOTE)-based multi-strategy augmentation method is introduced to expand the training set, mitigate class imbalance, and enhance the model’s ability to recognize rare but critical hazard categories. Second, a deep feature extraction architecture built on the robustly optimized BERT pretraining approach (RoBERTa) is designed to strengthen semantic representation under small-sample conditions. Moreover, a hierarchical weighting mechanism is incorporated into the weighted cross-entropy loss to emphasize the identification of severe hazard levels, thereby ensuring zero missed detections. Experimental results demonstrate that the proposed method achieves an accuracy of 99.26%, representing a 27.96% improvement over the traditional SVM baseline. Importantly, the recall for severe hazards (Levels III and IV) reaches 100%, ensuring zero misjudgment of major hazards. By effectively reducing safety risks, minimizing environmental disruptions, and promoting resilient tunnel construction, this method provides strong support for sustainable and low-impact underground engineering practices.

1. Introduction

With the rapid advancement toward sustainable and resilient transportation infrastructure, an increasing number of tunnels are being constructed in geologically complex mountainous regions. Water and mud inrush has become not only one of the most frequent and hazardous disasters during tunnel construction [1,2], accounting for more than 40% of geological hazards [3,4], but also a major threat to environmental stability, resource efficiency, and green construction practices. This disaster, characterized by the sudden eruption of groundwater or mud–water mixtures through joints, faults, or karst conduits [5], may cause severe economic losses, substantial environmental disturbance, and even casualties [5,6,7]. Therefore, accurate classification of water and mud inrush hazards is essential for ensuring construction safety, minimizing ecological impact, and promoting sustainable tunnel development. However, the complex hydrogeological conditions of underground environments still pose significant challenges to accurate hazard classification.

Various methods have been proposed for evaluating the hazard levels of water and mud inrush. Early approaches such as two-level fuzzy comprehensive evaluation [8], set pair analysis [9], and attribute mathematics [10] offered preliminary risk assessments but failed to capture multi-factor interactions. Improved methods—including extension theory integrated with fuzzy evaluation [11], uncertain-measure–based multi-factor assessment [12], and combined weight cloud models [13]—enhanced uncertainty treatment but still suffered from subjective weighting schemes. Fuzzy theory [4] and entropy weighting [14] alleviated part of this issue but remained limited when handling conflicting multi-factor conditions. Approaches integrating AHP with TOPSIS [15], improved game theory with uncertainty measures [16], and ideal point interval recognition methods [17] achieved better handling of factor conflicts, but their accuracy was still insufficient for large-scale engineering data required in sustainable construction. These limitations highlight the potential of machine learning methods.

Machine learning has increasingly been applied to tunnel hazard prediction [18,19]. Shallow models such as SVM [20], RF [21], KNN [22], and ELM [23] offer advantages in efficiency and interpretability but struggle to model nonlinear patterns in complex datasets. Deep learning models, including CNNs [24,25], HRNet [24], and EfficientNet [26], have demonstrated stronger feature extraction capabilities. However, studies focusing on classifying hazard levels of water and mud inrush remain limited, with most relying on Bayesian Networks [27,28] or RF [29], whose shallow feature representations restrict performance and robustness—key requirements for sustainable and low-risk underground engineering.

Large-scale pre-trained models, such as RoBERTa and ChatGPT, provide powerful end-to-end feature extraction capabilities without manual feature engineering [30,31,32]. Yet fine-tuning these models typically requires large labeled datasets. In tunnel engineering, data acquisition is costly and challenging, and limited samples may lead to hallucination effects and reduced model reliability. Multi-strategy data augmentation has thus become an effective approach to enhance model generalization. Although GAN-based methods are widely used in image enhancement [33,34,35,36,37,38] and SMOTE-based techniques perform well in addressing class imbalance in tunnel engineering [39,40,41,42], augmentation strategies specifically designed for water and mud inrush hazards have not been fully explored.

Misclassification—especially missed detection of severe hazards—remains a critical challenge in sustainable risk management [43,44,45]. Failure to accurately identify major hazard levels may lead to catastrophic consequences, including environmental degradation, equipment damage, and secondary geological disasters, severely undermining the goals of green, safe, and resilient tunnel construction. However, existing studies lack effective solutions to achieve zero misclassification of major hazards.

To address these challenges, this paper proposes a sustainability-oriented intelligent classification method for water and mud inrush hazards by integrating large-scale pre-trained models with multi-strategy data augmentation. First, SMOTE, ADASYN, and Borderline-SMOTE are employed to expand minority classes and improve dataset balance. Second, a RoBERTa-based classification framework is developed to enhance semantic representation under small-sample conditions. Third, a weighted cross-entropy strategy is designed to achieve zero misclassification of major hazards while maintaining high accuracy for minor hazards. Finally, engineering data from the DD tunnel in western China are used to validate the method, demonstrating superior accuracy, robustness, and sustainability-oriented reliability.

2. Methodology

This study proposes an intelligent classification method for tunnel water and mud inrush hazards by integrating multi-strategy data augmentation with a large pre-trained model (RoBERTa). By transforming structured engineering data into textual representations, the model can effectively learn complex hydro-geological feature interactions. Combined with a weighted cross-entropy loss function, the method achieves zero misclassification of major hazards. The overall framework is shown in Figure 1.

2.1. Multi-Strategy Data Enhancement Method Based on SMOTE

2.1.1. Problem Description

When using pre-trained large-scale models for tunnel hazard classification prediction, a substantial amount of data is required to train the model and ensure its predictive accuracy. However, due to the complexity of engineering geology, collecting field data is both challenging and costly. To address this issue, this paper employs a multi-strategy data augmentation approach based on SMOTE, integrating three techniques: SMOTE, ADASYN, and Borderline-SMOTE.

2.1.2. Introduction of Multi-Strategy Data Enhancement Method

Data augmentation can significantly improve model prediction accuracy and enhance generalization capability, enabling better performance in complex and dynamic real-world environments [33]. Among these methods, SMOTE is one of the most representative algorithms.

SMOTE generates new minority class samples through interpolation. Its core idea involves calculating the k-nearest neighbors of each minority class sample, randomly selecting a sample within this neighborhood, and generating new samples using linear interpolation [39]. The computation equation for this method is as follows:

$$x_{\mathrm{new}}=x_i+\lambda ·\left(x_{nn}-x_i\right)$$

(1)

where x_new represents the newly generated minority class sample, x_i denotes a selected minority class sample, and x_nn is another minority class sample chosen from the k-nearest neighbors of x_i. The term λ∼U(0, 1) represents a randomly generated number following a uniform distribution.

The ADASYN method improves upon SMOTE by introducing an adaptive sampling strategy. Its core idea is to generate more minority class samples near the decision boundary, thereby enhancing the classifier’s ability to recognize difficult-to-learn samples [46]. ADASYN calculates the proportion of majority class samples among the k-nearest neighbors of each minority class sample x_i. The calculation process is shown in Equation (2).

$$r_i={\displaystyle \frac{n_{\mathrm{maj}}}{k}}$$

(2)

where r_i represents the learning difficulty of sample x_i, which is defined as the proportion of majority class samples among its k-nearest neighbors. k denotes the predefined number of neighbors, and n_maj refers to the number of majority class samples within the neighborhood of x_i. Subsequently, based on the learning difficulty of all minority class samples, the total number of new samples to be generated is determined. The calculation formula is shown in Equation (3).

$$G=d\times \left(n_{\mathrm{maj}}-n_{\min }\right)$$

(3)

where G represents the total number of new samples to be generated, and d is a user-defined sampling ratio parameter (typically ranging from 0 ≤ d ≤ 1), indicating the number of minority class samples.

$$g_i={\displaystyle \frac{r_i}{{\displaystyle \sum r_i}}}\times G$$

(4)

where g_i represents the number of new samples to be generated for sample x_i, and ∑r_i denotes the total learning difficulty across all minority class samples. Finally, new minority class samples are generated using the SMOTE sampling strategy. Compared to SMOTE, ADASYN employs an adaptive approach to ensure that more minority class samples are generated in regions where classification is more challenging, enabling the classifier to better learn complex decision boundaries [47].

The Borderline-SMOTE method addresses the limitation of SMOTE, which uniformly generates samples across the entire minority class sample space, by proposing an oversampling approach that focuses on the decision boundary [48]. The core idea of this method is to first identify “borderline samples” within the minority class—those minority class instances with a high proportion of majority class samples in their k-nearest neighbors [49]. The decision criteria for identifying these borderline samples are defined in Equation (5).

$${\displaystyle \frac{n_{\mathrm{maj}}}{k}}>T$$

(5)

where T is a predefined threshold (typically set to 0.5 or higher). If the proportion of majority class samples among the k-nearest neighbors of a minority class sample exceeds T, the sample is classified as a “borderline sample.” Once the borderline samples are identified, SMOTE sampling is applied exclusively to these samples. Compared to SMOTE, Borderline-SMOTE enhances the representation of minority class samples near the decision boundary, improving the classifier’s ability to differentiate in critical regions and preventing the generation of ineffective samples that may occur with SMOTE.

In summary, this study employs SMOTE, ADASYN, and Borderline-SMOTE for data augmentation. By synthesizing new minority class samples, these methods enhance the model’s ability to learn from minority class instances, thereby improving the classifier’s generalization capability and predictive accuracy.

2.2. Construction of Hazard Classification Model Based on Large-Scale Models

2.2.1. Dataset Description

The occurrence of tunnel water and mud inrush hazards results from the combined influence of geological conditions, groundwater, and other factors. To accurately predict the severity of such hazards, it is essential to consider the key contributing factors. Based on prior knowledge and long-term on-site tunnel monitoring, this study collected data on tunnel water and mud inrush hazard classification (WMIHC) along with its corresponding influencing factors, including water head height (WY), rock quality designation (RQD), water-rich characteristics (WYP), rock structure orientation (RSO), aquifer thickness (AT), and unfavorable geology (UG), as illustrated in Figure 2. WMIHC is determined based on the “Technical Specification for Geology Forecast of Railway Tunnel (Q/CR 9217-2015)” [50]. Considering that both large-scale and extra-large-scale water inrush events during tunnel construction can lead to catastrophic consequences, they are classified into the same category. Therefore, the WMDG is divided into four levels. Deep learning models require input parameters to be quantitatively processed. Specifically, WY, RQD, WYP, and AT are quantitative indicators, while RSO and UG are represented by rock dip angles and surrounding rock quality levels, respectively. This study collected the aforementioned data and applied the data augmentation methods described in Section 2 to construct a dataset for building an effective hazard classification prediction model. The following sections provide further details on these parameters.

(1)

WH

WH directly determines groundwater pressure. When the water head is high, water can easily seep through fractures into the tunnel, increasing the risk of water inrush. An excessively high WH significantly elevates the hazard severity, particularly in water-rich layers and fractured rock masses.

(2)

RQD

RQD reflects the integrity of the rock mass. A low RQD value indicates a highly fractured rock mass with numerous fissures, increasing water permeability and the likelihood of water inrush. In areas with poor rock quality, the hazard severity is higher, particularly under high water head conditions.

(3)

WYP

WYP represents the water-bearing capacity of rock formations. In water-rich layers, significant water flow often occurs during construction, leading to frequent water inrush events. Areas with high water yield properties exhibit higher hazard severity, especially where aquifers are thick, and rock fractures are well-developed.

(4)

Rock stratum occurrence (RSO)

The dip of rock strata influences groundwater flow paths. When rock formations exhibit complex structures or steep inclinations, water can easily infiltrate the tunnel through fractures, increasing the risk of water inrush. Under poor rock conditions, hazard severity tends to be higher.

(5)

Aquifer thickness (AT)

AT determines the groundwater storage capacity. A greater aquifer thickness enhances groundwater permeability, exacerbating water inrush events. Areas with thick aquifers exhibit a higher hazard severity, particularly under high water head conditions and within fractured rock masses.

(6)

Unfavorable geology (UG)

UG includes geological structures such as faults and fracture zones, which have high permeability. Water can easily infiltrate tunnels through these unfavorable geological zones, significantly increasing the risk of water inrush. Areas with adverse geological conditions tend to have higher hazard severity.

2.2.2. The Construction of Intelligent Classification Model Based on RoBERTa

(1)

Introduction of RoBERTa model

Bidirectional Encoder Representations from Transformers (BERT) is a bidirectional Transformer-based language model proposed by Google AI in 2018 [51], including two standard configurations, BERT-base and BERT-large. Its core innovation lies in its dual-task pretraining mechanism, consisting of the Masked Language Model (MLM) and Next Sentence Prediction (NSP), enabling the model to capture contextual semantic information simultaneously [52,53]. The MLM task learns grammatical and semantic features by predicting randomly masked words, while the NSP task determines sentence coherence to understand inter-sentence relationships. This pretraining strategy endows BERT with powerful semantic representation capabilities, allowing it to be fine-tuned for various downstream tasks, including text classification [54], question answering [55], text generation [56], and semantic similarity computation [57].

In 2019, Facebook AI introduced RoBERTa [58], which incorporated several optimizations based on the BERT architecture. First, it removed the NSP task to simplify the training process [59]. Second, it adopted a dynamic masking strategy—randomly generating mask patterns during each training iteration—to enhance model generalization. Additionally, it expanded the training dataset and extended the training duration [59]. Experimental results show that by adjusting batch parameters and increasing training steps, RoBERTa demonstrated superior text comprehension performance in benchmark tests such as GLUE, particularly excelling in long-text modeling and complex semantic relationship processing [60].

To address the limitation that RoBERTa can only process textual inputs, this study proposes an innovative framework for converting structured engineering data into textual representations. The six engineering parameters related to tunnel water and mud inrush hazards (e.g., RQD, WYP) and their corresponding hazard levels are jointly encoded into text through a “feature description + numerical concatenation” strategy, enabling a direct mapping between numerical values and textual sequences. Specifically, all samples are encoded using a unified sentence template, and their numerical values are preserved with full precision, without rounding or reformatting. The generated text sequences are then processed by RoBERTa’s native BPE tokenizer, which represents integers and decimals as intact tokens or stable subword combinations, ensuring consistent expression of continuous numerical values. Based on this text-constructed dataset, a RoBERTa-based intelligent classification model integrating engineering features with a pre-trained language model is developed through fine-tuning. This approach overcomes the intrinsic limitation of traditional language models in handling numerical data and provides a novel deep-learning solution for engineering hazard assessment. The overall RoBERTa-based intelligent classification framework is shown in Figure 3.

(2)

Design of weighted cross entropy loss function

Hazard classification prediction during tunnel construction is a critical aspect of hazard prevention and control, as its outcomes directly influence structural design and construction safety decisions. In predicting the severity of tunnel water and mud inrush hazards, any misclassification of major hazards (Class III and IV) can result in significant safety risks and wasted resources. Therefore, ensuring zero misclassification of major hazards is crucial. Traditional hazard classification methods often struggle with misclassification risks. To address this technical challenge, this study introduces a zero-misclassification strategy for major hazard classification. This approach ensures that while maintaining high classification accuracy for minor hazards (Class I and II), there are no misclassifications for major hazards (Class III and IV). To achieve this goal, a weighted cross-entropy loss function is introduced to optimize the RoBERTa model. During training, a hierarchical weight allocation strategy is designed, assigning higher penalty weights to misclassification of major hazards. This forces the model to prioritize major hazard classification, shifting its optimization boundary toward zero misclassification. This approach not only enhances the model’s classification performance but also improves the reliability of the prediction system in real-world applications. As a result, it provides more precise and robust support for decision-making in the prevention and control of tunnel water and mud inrush hazards. This approach not only enhances the model’s classification performance but also improves the reliability of the prediction system in real-world applications. As a result, it provides more precise and robust support for decision-making in the prevention and control of tunnel water and mud inrush hazards.

In the classification task for tunnel water and mud inrush hazards, the imbalanced distribution of hazard severity levels—where major hazards (Class III and IV) have fewer samples—poses a challenge, particularly due to the high cost of misclassifying severe events. To address this, this study employs a weighted cross-entropy loss function [61,62] to optimize the RoBERTa model, as shown in Equation (6). This approach utilizes a hierarchical weight allocation strategy to adjust penalty intensities differently for each hazard level. By doing so, it maintains classification accuracy for minor hazards while prioritizing the identification of major hazards.

$$L_{WCE}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{c=1}^C{w}_c·y_{i,c}\log \left(p_{i,c}\right)}}$$

(6)

The weight vector w = [w₁, w₂, w₃, w₄] is positively correlated with the importance of each category. When w_c > 1, misclassification of category c incurs a higher loss value, amplifying the corresponding parameter gradient by a factor of w_c during backpropagation. This mechanism ensures that the model prioritizes correcting misclassifications of high-weight categories during training. Such asymmetric reinforcement significantly alters the topology of the loss surface, pushing the decision boundary toward better recognition of major hazards.

During the fine-tuning of the RoBERTa model, this study replaces the standard cross-entropy loss function with weighted cross-entropy loss function. A hierarchical weight allocation strategy is designed based on practical engineering requirements to maintain classification accuracy for minor hazards while prioritizing the recognition of major hazards.

2.3. Evaluating Indicator

Common evaluation metrics for classification tasks include Accuracy (ACC), Precision (PRE), Recall (REC), and F1-score (F1) [21,63]. As a simple binary classification example, Figure 4 illustrates the definitions of key evaluation components, including True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN). The relationships between evaluation metrics and these classification components are defined by Equations (7)–(10).

$$\mathrm{ACC}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$$

(7)

$$\mathrm{PRE}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(8)

$$\mathrm{REC}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(9)

$$\mathrm{F}1={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}\times \mathrm{Recall}}}$$

(10)

The F1 represents a balance between REC and PRE, treating both as equally important. Additionally, the weighted F1-score (F1_W) and macro-averaged F1-score (F1_M) [24] are used to evaluate the predictive performance of models in imbalanced classification tasks, as defined in Equations (11) and (12).

$${\mathrm{F}1}_{\mathrm{W}}={\displaystyle \sum _{j=1}^m{w}_j·\mathrm{F}{1}_j}$$

(11)

$$\mathrm{F}{1}_{\mathrm{M}}={\displaystyle \frac{1}{m}}{\displaystyle \sum _{j=1}^m{w}_j·\mathrm{F}{1}_j}$$

(12)

$$\mathrm{HRMC}={\displaystyle \frac{{\displaystyle {\sum }_{j-3}^4{\displaystyle {\sum }_{i=1}^{2}c\left(i,j\right)·{\mathrm{ConfusionMatrix}}_{j,i}}}}{{\displaystyle {\sum }_{j-3}^4{N}_j}}}$$

(13)

where in the equation, w_j represents the weight of category j; m denotes the total number of categories; N_j is the number of samples in category j; C(i,j) represents the cost of misclassifying category j as category i. The misclassification cost is set as 1 when a Class III hazard is misclassified as Class I or II, and 3 when a Class IV hazard is misclassified as Class I or II. ConfusionMatrix_j,i denotes the number of samples from category j that were misclassified as category i, and N_j represents the total number of samples in category j.

3. Engineering Application and Result Analysis

In this section, a real-world tunnel construction case study is used to validate the effectiveness of the proposed tunnel water and mud inrush hazard classification method, which integrates large-scale models and multi-strategy data enhancement. Comparative analysis is conducted to demonstrate the superior predictive performance of this approach. All models are implemented in a Python 3.12-based development environment using PyCharm 2023.3.7. The experiments are conducted on a Windows system with hardware specifications of an Intel(R) Core(TM) i7-14700HX@5.50 GHz CPU and 16 GB RAM.

3.1. Engineering Background Introduction

3.1.1. Project Profile

The DD railway tunnel is located in Ganzi Tibetan Autonomous Prefecture, Sichuan Province, China, with a total length of approximately 3.4 km. The tunnel cross-section measures 7.5 m in both height and width and is constructed using the drill-and-blast method. The primary lithology consists of Yanshanian granite with well-developed joints and fractures. The tunnel locally intersects high-pressure water-rich fault zones and small-scale fault fracture zones, resulting in highly complex and variable hydrogeological conditions. The geological stratigraphy is illustrated in Figure 5.

3.1.2. Sample Set Construction and Analysis

This study aims to apply pre-trained large language models (LLMs) for the automated intelligent classification of tunnel water and mud inrush hazards. To ensure data quality and representativeness, a strict cleaning process was applied to the DD tunnel dataset. Samples outside engineering-reasonable ranges were first removed. Outliers in WY, RQD, WYP, and AT were then identified using the Interquartile Range (IQR) method. A multivariate consistency check based on Mahalanobis distance was finally performed to eliminate unrealistic hydrological–geological combinations. As a result, a total of 1000 valid samples were obtained for subsequent analysis and model development. Each sample consists of six input parameters and one output parameter (WMIHC). The input factors are categorized into two groups: hydrological factors (WH, WYP, and AT) and geological factors (RQD, RSO, and UG). The characteristics of the collected dataset are summarized in

Table 1. Statistical characteristics of input and output parameters.

Type	Indicator	Unit	Min–Max	Mean	Std. Dev
Inputs	WH	m	11–141	77.26	29.06
	RQD	%	9–92	37.36	17.00
	WYP	m3·h−1·m−1	0.1–11.7	4.11	1.98
	RSO	°	10–88	37.73	11.44
	AT	m	0–36	13.81	7.16
	UG	—	3–5	4.37	0.57
Output	WMIHC	—	1–4	2.05	0.66

, which presents the classification of tunnel water and mud inrush hazards (WMIHC) ranging from Class I to Class IV. It is important to note that tunnel water and mud inrush hazards result from the combined influence of hydrological and geological factors. Additionally, to facilitate the development of the intelligent classification model, both input and output parameters were normalized using the min-max scaling method [64].

Figure 6 presents violin plots of various influencing parameters and WMIHC, providing insights into the distribution and magnitude of each parameter. To gain a comprehensive understanding of the dataset, the Pearson correlation coefficient (PCC) is used to evaluate the relationship between each parameter and WMIHC [63]. PCC measures the linear relationship between two variables, with values ranging from −1 to 1. A value of 0 indicates no correlation between the two variables. In contrast, a strong correlation between two variables suggests a significant influence between them. As shown in Figure 7, the PCC values for all parameters with WMIHC are greater than 0.5, indicating that each parameter has a substantial impact on WMIHC.

3.2. Analysis of the Prediction Results of the Proposed Model

3.2.1. Model Training and Parameter Optimization

This study applies a large-model-enhanced and multi-strategy data augmentation method for classifying tunnel water and mud inrush hazards. As described in Section 2.2, the original dataset contains 1000 samples, each with 6 feature variables and 1 target variable. The dataset is first split into a training set and a test set using an 80/20 ratio. Only the training set is augmented using three methods—SMOTE, ADASYN, and Borderline-SMOTE—to increase sample size and reduce class imbalance, while the test set remains unchanged. The augmented training set and the original test set are kept strictly separated throughout all experiments to ensure objective and reliable model evaluation. After augmentation, the training set contains 5410 samples, resulting in a total of 6410 samples in the final dataset.

The proposed research model consists of two main components: multi-strategy data augmentation and RoBERTa optimization. For data augmentation, the threshold T in Borderline-SMOTE is set to 0.5. For RoBERTa optimization, the RoBERTa-Base model is used, featuring 12 Transformer encoder layers, each with 768 hidden units and 12 self-attention heads, totaling approximately 110 million parameters. The model employs a weighted cross-entropy loss function, assigning weight values of w = [1, 1, 3, 3] for hazard levels I, II, III, and IV, respectively. This ensures greater emphasis on major hazards during training. As shown in Figure 8, the optimal model performance is achieved with 64 epochs, a batch size of 16, and a learning rate of 1 × 10⁻⁵. The specific hyperparameters of the model in this article are shown in

Table 2. Hyperparameter settings.

Name	Value
Optimizer	AdamW
Learning rate	1 × 10−5
β1	0.9
β2	0.999
Weight decay	0.01
Dropout	0.1
Warm-up steps	500
Max sequence length	128
Epoch	64
Batch size	16

.

3.2.2. Model Comparatively Analysis

To demonstrate the superior predictive performance of the proposed model, nine machine learning models—Support Vector Machine (SVM), RF, Decision Tree (DT), Logistic Regression (LR), Light Gradient Boosting Machine (LGB), Gradient Boosting Tree (GBT), Extreme Gradient Boosting (XGB), K-Nearest Neighbors (KNN), and Automatic Feature Interaction (AutoInt)—were selected for comparative analysis. The prediction results are summarized in

Table 3. Performance evaluation results of different models.

Model	ACC/%	F1W/%	F1M/%	REC/%		HRMC
				Class III	Class IV
SVM	77.57	72.90	61.47	73.48	89.29	0.32
RF	97.04	96.91	95.10	99.57	100.00	0.11
DT	95.56	95.53	93.48	96.09	98.87	0.21
LR	69.32	66.39	57.91	66.09	80.08	0.45
LGBM	97.29	97.19	95.52	99.13	100.00	0.14
GBT	95.26	94.98	92.12	98.04	100.00	0.15
XGB	97.10	97.00	95.28	98.70	99.81	0.18
KNN	97.04	96.87	95.02	99.78	100.00	0.08
AutoInt	73.26	73.05	70.94	90.00	51.88	0.63
Proposed	99.26	99.25	98.87	100.00	100.00	0.00

. Based on the evaluation metrics in Table 3, the proposed model demonstrates significant superiority across multiple key performance indicators, particularly in terms of accuracy and F1_W. The proposed model achieves an ACC of 99.26%, significantly outperforming traditional machine learning models such as SVM (77.57%), with an improvement of 27.96%. Additionally, the weighted F1-score of the proposed model reaches 99.25%, further demonstrating its strong capability in accounting for class-wise importance. These findings highlight the clear advantages of large-scale models in enhancing the accuracy of tunnel water and mud inrush hazard classification. In particular, they prove highly effective in handling complex hazard data, delivering more precise predictive outcomes.

Additionally, the proposed model also excels in REC and HRMC, particularly achieving a 100% REC rate for Class III and IV major hazards. This demonstrates its exceptional sensitivity to severe hazards. By incorporating a weighted cross-entropy loss function and assigning different penalty weights (1, 1, 3, 3) for each hazard level, the model effectively mitigates the occurrence of catastrophic hazards and achieves zero misclassification of major hazards. The final HRMC of the proposed model is 0.00%, indicating exceptional accuracy and reliability in hazard classification. This significantly enhances the credibility of the tunnel water and mud inrush hazard prediction system in real-world applications.

As shown in Figure 9, the proposed model exhibits significant superiority in multiple key metrics, particularly in the accuracy of hazard level predictions. The model performs well across all hazard levels, especially for Class III and IV hazards, where all samples are correctly classified. This indicates that the model achieves zero misclassification for major hazards, meeting the zero-misjudgment requirement. Such performance is not observed in traditional models like SVM, RF, and KNN, where the recall for Class III and IV hazards is significantly lower. This highlights the reliability of the proposed model in ensuring accurate predictions.

Additionally, by optimizing the model with a weighted cross-entropy loss function, the proposed approach assigns higher penalty weights to major hazards. This encourages the model to focus more on these severe events during the optimization process. As a result, the model maintains its performance on minor hazards, with slight improvements in some cases, while achieving zero misclassification for major hazard levels. This demonstrates its high reliability in practical applications. A comprehensive analysis indicates that the specialized large-scale models developed in this study effectively overcomes the limitations of traditional methods in hazard classification. In particular, it shows significant advantages in enhancing the prediction accuracy of major hazards and reducing misclassification rates.

3.2.3. Effectiveness Analysis of Model Improvement

Based on practical engineering requirements, this study designs a hierarchical weight allocation strategy: baseline weights of w₁ = w₂ = 1 are assigned to Class I and II hazards to maintain fundamental performance, while enhanced weights of w₃ = w₄ = 3 are applied to Level 3 and Level 4 hazards to improve model sensitivity. As shown in

Table 4. Comparative analysis before and after model optimization.

Model	ACC/%	F1M/%	PRE/%		REC/%		HRMC
			Class I	Class II	Class III	Class IV
Original RoBERTa	99.08	98.69	99.78	96.17	99.13	99.81	0.15
Optimized RoBERTa	99.26	98.87	99.33	99.42	100.00	100.00	0.00

, the RoBERTa model demonstrates significant superiority after being optimized with a weighted cross-entropy loss function. Compared to the original RoBERTa model (with an ACC of 99.08% and F1_M of 98.69%), the optimized RoBERTa model, incorporating the weighted cross-entropy function, achieves improvements in both ACC (99.26%) and F1_M (98.87%). This enhancement highlights the effectiveness of the weighted cross-entropy approach in refining the overall model performance, particularly in improving hazard classification accuracy.

Additionally, the precision (PRE) and recall (REC) metrics in Table 4 further validate the optimized model’s effectiveness in accurately classifying major hazards. Specifically, the optimized RoBERTa model achieves a 100% REC rate for Class III and IV hazards, indicating that it completely eliminates misclassification of major hazards. In contrast, the unoptimized model attains recall rates of 99.13% and 99.81% for Class III and IV hazards, respectively. Although relatively high, these values indicate minor misclassification errors. After applying weighted cross-entropy optimization, the model’s ability to identify these two hazard levels significantly improves, effectively preventing the underestimation of risks. This enhancement strengthens the safety and reliability of the hazard early warning system.

The confusion matrices in Figure 10a,b further validate the prediction accuracy of the optimized model across different hazard levels. The optimized RoBERTa model outperforms the original RoBERTa in accuracy, particularly for Class III and IV hazards, where it achieves zero misclassification. This demonstrates that optimizing with a weighted cross-entropy loss function not only improves classification accuracy but also enhances the model’s ability to predict major hazards. As a result, HRMC is significantly reduced, ensuring the practical applicability of the model.

In conclusion, the weighted cross-entropy optimization method proposed in this study plays a crucial role in improving the classification accuracy of tunnel water and mud inrush hazard prediction while reducing misclassification rates. It effectively balances classification precision with engineering safety requirements, significantly enhancing the model’s practicality and reliability in real-world hazard prevention and control.

3.3. Analysis of Data Enhancement Effect

3.3.1. Analysis of Enhanced Data Distribution Characteristics

Principal Component Analysis (PCA) and t-distributed Stochastic Neighbor Embedding (t-SNE) [65,66,67] were used to analyze the distribution characteristics of the augmented data, assessing its similarity to the original data. Figure 11 indicates a high degree of similarity between the distributions of the original and augmented data. The original data samples are primarily concentrated in the negative regions of PC1 and PC2, whereas the augmented data samples extend across a broader area, particularly in the positive direction of PC1 and PC2, demonstrating an expansion in data diversity. In the low-dimensional space, the augmented data distribution retains the structure of the original dataset while introducing reasonable feature-space expansion, contributing to improved model generalization. The loadings of key variables (e.g., WH and RQD) in the principal components remain consistent, indicating that data augmentation has not altered the relative importance of the variables. Overall, the augmented data preserves the core characteristics of the original dataset while enhancing sample diversity. This ensures the effectiveness of the training data, providing the specialized large-scale models with richer input and improving its predictive performance.

The t-SNE visualization in Figure 12 compares the distribution characteristics of the original and augmented data. The results indicate that while the original data exhibits a more concentrated distribution, the augmented data spans a broader range. This demonstrates that the data augmentation method effectively expands the data space, increasing diversity and variability, particularly in generating synthetic samples. This augmentation strategy effectively balances the distribution of different classes, particularly improving the model’s learning capability for minority or complex categories, such as major hazards. Additionally, the augmented data exhibits more dispersed points and clusters, indicating that the synthetic samples retain the characteristics of the original data while effectively capturing variations within the dataset. By applying these augmentation strategies, the model maintains its original accuracy while successfully reducing the misclassification rate of major hazards, thereby improving hazard severity classification accuracy. Thus, the proposed multi-strategy data augmentation method not only refines data distribution but also enhances the predictive capability of the hazard classification model, demonstrating significant advantages and practical value.

3.3.2. Effectiveness Analysis of Data Augmentation Strategy

To evaluate the effectiveness of the multi-strategy data augmentation method, a comparative analysis was conducted to assess model performance under different data augmentation strategies. As shown in Figure 13, the total number of samples increased to 2472 and 6416 when using only SMOTE and the multi-strategy data augmentation method, respectively, representing approximately 2.5-fold and 6.4-fold expansions compared to the original dataset. Figure 14 illustrates the performance of three model settings—no augmentation, SMOTE, and multi-strategy augmentation—across three evaluation metrics: ACC, F1 for Class III (F1-III), and F1-score for Class IV (F1-IV). The results indicate that the multi-strategy data augmentation method achieves the best performance across all metrics, particularly attaining a high level of 0.99 in ACC and F1-III. This demonstrates its strong capability in data processing and model optimization. In comparison, the SMOTE method also shows some improvement, achieving 0.88 in F1-III. However, its overall performance remains inferior to the multi-strategy data augmentation approach. The model without augmentation performs the worst across all metrics, particularly in F1-IV, where it scores only 0.11. This highlights its limitations in handling imbalanced data. Therefore, the multi-strategy data augmentation method is proven to be a more effective and robust data processing approach.

Figure 15 presents the confusion matrices of the proposed model under different data augmentation strategies. The analysis clearly highlights the superiority of the multi-strategy data augmentation approach. Without data augmentation, the model exhibits relatively poor classification performance, particularly in predicting Class I and IV hazards, where the misclassification rate is notably high. The accuracy for Class IV hazard is especially low. When applying the SMOTE data augmentation strategy, the model’s predictive accuracy for Class IV hazard improves significantly. By integrating SMOTE, ADASYN, and Borderline-SMOTE, the multi-strategy data augmentation approach significantly enhances classification accuracy across all hazard levels. Notably, the prediction accuracy for Class IV hazards is nearly perfect, while also preventing any decline in the accuracy of lower-level hazard classifications. The multi-strategy augmentation effectively balances class distribution within the dataset, optimizes the model’s adaptability to different hazard levels, enhances its ability to classify severe hazards, and maintains high predictive accuracy for other categories. Therefore, the multi-strategy data augmentation approach not only mitigates class imbalance but also improves overall model classification performance, demonstrating significant advantages in handling complex hazard classification tasks.

3.3.3. Analysis of Model Performance Improvement Effect After Data Enhancement

To further demonstrate the reliability and superiority of the proposed multi-strategy data augmentation method, the improvement in predictive performance after applying multi-strategy data augmentation was analyzed for various models, including SVM, RF, DT, LR, LGB, GBT, XGB, KNN, AutoInt, and the proposed model, as shown in

Table 5. The prediction performance of different models is improved after data enhancement.

Model	ACC/%	F1W/%	REC/%
			Class III	Class IV
SVM	−4.1	−7.45	+1.48	+89.29
RF	+10.04	+10.24	+22.3	+50
DT	+13.89	+13.36	+27.91	+48.87
LR	−10.68	−13.02	+0.18	+30.08
LGB	+10.96	+11.09	+21.86	0
GBT	+8.26	+8.2	+20.77	0
XGB	+10.77	+10.83	+28.25	+1.55
KNN	+10.71	+10.7	+22.51	+72
AutoInt	−2.74	+5.08	+90	+51.88
Proposed	+22.26	+22.33	+38.64	+81

. The experimental results indicate that the proposed method exhibits significant advantages in handling imbalanced data. The proposed model achieves a 22.26% increase in ACC and a 22.33% improvement in F1_W. Additionally, its REC for major hazards (Class III and IV) reaches 38.64% and 81%, respectively, representing an improvement of over 20% compared to traditional models such as SVM and LR. The multi-strategy augmentation approach systematically addresses the long-tail distribution issue in hazard classification by leveraging SMOTE for global sample balancing, ADASYN for adaptively generating hard-to-classify samples, and Borderline-SMOTE for strengthening classification boundaries. This approach significantly enhances the recognition capability for minority classes, particularly for Class IV hazards.

Compared to single augmentation strategies (e.g., AutoInt achieving a 90% recall for Class III but suffering a decline in overall performance), the proposed method balances both global and local data characteristics, mitigating the risk of overfitting and highlighting the comprehensive advantages of an integrated strategy. Additionally, by incorporating a weighted cross-entropy loss function (with a penalty weight of 3 for Class III and IV hazards), the model maintains classification accuracy for minor disasters (Class I and II) while achieving zero misclassification for major hazards. This confirms the synergistic effect of data augmentation and loss function optimization. Comparative experiments further demonstrate that tree-based models (RF and XGB) and ensemble methods (LGB) benefit significantly from the multi-strategy augmentation, with improvements exceeding 10% in both ACC and F1_W. In contrast, LR, which is highly sensitive to sample distribution, experiences a performance decline, underscoring the proposed method’s adaptability to complex models. In conclusion, the proposed model framework employs a three-pronged optimization approach—multi-dimensional data generation, large-scale models fine-tuning, and loss function design—effectively enhancing the robustness and reliability of hazard classification. This provides a novel perspective for intelligent hazard diagnosis in small-sample engineering scenarios.

3.4. Model Robustness Analysis

To assess the robustness of the model, its predictive performance was analyzed under varying levels of Gaussian noise added to the test set. As shown in Figure 16, when the noise variance increases from 0.0 to 0.3, the global evaluation metrics ACC and F1-score decrease from 99.2% and 98.5% to 96.1% and 96.3%, respectively. The reduction is limited to 3.1% and 2.2%, demonstrating the model’s resilience to noise interference. Notably, hazard severity-specific metrics exhibit significant differences: Class III recall (REC-III) decreases from 97.8% to 95.5%, while Class IV recall (REC-IV) remains consistently above 99.0%. This characteristic arises from the weighted cross-entropy loss function’s reinforced weighting strategy across all four hazard levels, along with the pre-trained RoBERTa parameters’ implicit noise-filtering capability. As a result, the model ensures zero misclassification of severe hazards while maintaining overall classification stability.

The model’s noise resistance can be attributed to a multi-dimensional technical synergy: SMOTE-based algorithms improve the continuity of geological parameter distributions (e.g., RQD and WYP) by synthesizing minority class samples, while RoBERTa’s attention mechanism dynamically focuses on key features (e.g., WH and RSO) to suppress noise propagation. Experimental results show that when the noise variance reaches 0.3, REC-IV remains at 99.0%, validating the model’s reliability in identifying major hazards, which are critical for engineering safety. However, the slight decline in REC-III (Δ = 2.3%) reveals the sensitivity of mid-level hazard boundary features to noise disturbances, which may be related to representational bias in the nonlinear coupling of geological parameters. The experiments demonstrate that the proposed intelligent classification model for tunnel water and mud inrush hazards exhibits strong robustness under Gaussian noise interference by integrating large-scale models with multi-strategy data augmentation.

According to the results in

Table 6. Effect analysis under different weights of various disaster levels.

Weight Vector (I, II, III, IV)	ACC	REC-III and REC-IV	F1W
[1, 1, 1, 1]	96.75%	96.2%, 96.8%	96.74%
[1, 1, 2, 2]	98.05%	98.5%, 98.1%	98.04%
[1, 1, 3, 3] (Proposed)	99.26%	100%, 100%	99.25%
[1, 1, 5, 5]	99.18%	100%, 100%	99.17%

, different class-weight settings have a clear impact on model performance. When all class weights are set to 1, the model accuracy is 98.75%, and the recall of Levels III and IV is relatively low, indicating that the model tends to focus on the more frequent minor-disaster samples. When the weights begin to shift toward the severe classes (e.g., [1, 1, 2, 2]), the ACC increases to 99.05%, and the recall values for Levels III and IV rise to 99.5% and 99.1%, showing that the model becomes more sensitive to high-risk categories. With the proposed weight vector [1, 1, 3, 3], the model achieves the best overall performance, reaching 99.26% ACC and 99.25% F1W, while both REC-III and REC-IV reach 100%, meaning there are no missed severe-disaster cases. This indicates that the proposed weight setting provides the best balance between accuracy and safety. When the weight is further increased to [1, 1, 5, 5], the recall remains high, but ACC and F1W slightly decrease, suggesting that overly large weights cause the model to overemphasize severe disasters and reduce overall performance. Therefore, the weight setting [1, 1, 3, 3] is the most effective choice for balancing prediction accuracy, robustness, and engineering safety.

4. Conclusions and Outlook

This study addresses the challenges of small sample constraints and safety control requirements in intelligent classification of tunnel water and mud inrush hazards. An innovative approach integrating multi-strategy data augmentation and large-scale model optimization is proposed. The effectiveness of this method is comprehensively evaluated using data from the DD tunnel project in the mountainous regions of western China. The main conclusions are as follows:

(1)

Expansion of sample space through multi-strategy data augmentation: By integrating SMOTE, ADASYN, and Borderline-SMOTE, the proposed multi-strategy data augmentation method expands the dataset by 6.4 times and improves model ACC by 22.26%. This approach achieves density balancing and local feature enhancement, effectively addressing the issue of low prediction accuracy in tunnel water and mud inrush hazard classification caused by insufficient samples. It provides a generalizable solution for small-sample challenges in engineering applications.

(2)

Specialized large-scale models for water and mud inrush prediction: This study fine-tunes the pre-trained language model RoBERTa using a tunnel water and mud inrush hazard classification dataset, transforming its “language understanding ability” into “numerical prediction capability” to enable intelligent hazard classification. Engineering application results show that, compared to nine other machine learning models, the proposed model achieves an ACC of 99.26% and an F1_W of 99.25%, demonstrating superior prediction accuracy and strong robustness.

(3)

Zero-misclassification strategy for major hazard classification: The RoBERTa model is optimized using a weighted cross-entropy loss function, with a hierarchical weight allocation strategy designed to enhance the model’s learning capability for major hazards. Experimental results indicate that this method maintains classification accuracy for minor hazards while achieving zero misclassification for major hazards. Specifically, with PRE-I and PRE-II at 99.33% and 99.34%, REC-III and REC-IV both reach 100%, and HRMC remains at 0.00. This approach effectively balances classification accuracy with engineering safety requirements, significantly enhancing the model’s practicality and reliability in real-world hazard prevention and control.

Although the proposed multi-strategy data augmentation and large-model fusion method performs well for classifying tunnel water and mud inrush hazards, several limitations remain. First, the model is trained on data from a single tunnel in Sichuan, and variations in lithology, groundwater conditions, and unfavorable geology across different tunnels may cause the model to rely on site-specific patterns, limiting its cross-regional generalization. Future work should therefore include cross-tunnel validation and introduce transfer learning and domain-adaptation techniques (such as fine-tuning, MMD-based alignment, and semi-supervised transfer) to improve adaptability across diverse geological environments. Second, RoBERTa still exhibits a degree of “black-box” behavior, and attention weights alone cannot reveal the complex hydro-geological mechanisms behind hazard evolution; more advanced interpretability methods such as integrated gradients and Shapley-value attributions will be explored. In addition, small-sample conditions may still lead to hidden overfitting risks, requiring more robust data-generation strategies and uncertainty-quantification techniques. Finally, the relatively slow inference speed of RoBERTa may limit real-time early-warning applications, and lightweight strategies such as model compression, knowledge distillation, and edge deployment will be necessary to enhance practical usability. Overall, developing a hazard-classification model that is transferable, interpretable, and deployable will be an important direction for future research.