Intelligent Prediction of Blast Furnace Permeability by Integrating Extreme Gradient Boosting and Light Gradient Boosting Machine ^†

Abstract

We developed an ensemble model based on the fusion of Extreme Gradient Boosting and Light Gradient Boosting Machine. 20 key parameters, including hourly charging rate, blast volume, and blast pressure, are selected as input features to construct and train the ensemble model for predicting the blast furnace permeability index. The results show that the model achieves a root mean squared error of 0.0868, a mean absolute error of 0.0708, and a coefficient of determination of 0.9602, confirming its excellent predictive accuracy.

1. Introduction

Blast furnace operational stability is essential in determining efficiency, energy consumption, and safety in steel production. The permeability index, reflecting internal gas flow distribution, is also vital for optimal operation [1]. Its accurate prediction remains challenging due to complex process dynamics featuring strong nonlinearity, hysteresis, and multi-variable coupling within harsh environments. While traditional models struggle with these complexities, data-driven approaches have emerged as effective solutions [2,3,4], spanning conventional machine learning, deep learning, and hybrid strategies [5].

Recent advances in blast furnace permeability prediction include several data-driven approaches. Su et al. [6] developed a Wavelet Transform-Principal Component Analysis-Multi-layer Extreme Learning Machine model with wavelet denoising and principal component analysis for improved generalization. Yu et al. [7] combined the Least Squares Support Vector Machine with mean shift clustering (mean absolute error (MAE) = 0.6877, coefficient of determination (R²) = 0.9215). Jiang et al. [8] identified Support Vector Regression and Gaussian Process Regression as top performers via multi-method comparison and Shapley Additive Explanation analysis. Zhao et al. [9] employed multi-criteria feature selection for Extreme Gradient Boosting (XGBoost) prediction (a 94.27% accuracy within ±1.5% error). Liu et al. [10] proposed Variational Mode Decomposition-Particle Swarm Optimization-Back Propagation with signal decomposition for 3% accuracy gain. Luo et al. [11] designed a Variational Mode Decomposition-Error Compensation-Temporal Convolutional Network-Gated Recurrent Unit model handling non-stationarity and time-lag effects with high industrial reliability.

Although hybrid modeling predominates and XGBoost/Light Gradient Boosting Machine (LightGBM) demonstrate superior feature processing capabilities [12,13], their synergistic potential in blast furnace applications remains underdeveloped. We established an integrated framework incorporating: (1) Spearman correlation-based feature selection; (2) complementary XGBoost + LightGBM architecture leveraging their respective strengths in nonlinear modeling and high-dimensional processing; and (3) weighted prediction fusion to enhance time-series forecasting generalization.

Using industrial blast furnace data with comprehensive process parameters, we implement preprocessing and stacking integration. The resulting ensemble effectively captures permeability dynamics across operational conditions, demonstrating substantial potential for intelligent control system implementation.

2. Methodology

2.1. Data and Preprocessing

January–June 2024 operational data were obtained from a blast furnace, collected hourly (23 parameters, 4368 observations each, totaling 100,464 data points). The preprocessing pipeline comprises (1) comprehensive data cleaning addressing missing values (forward-backward/sliding-window imputation), noise (median filtering), and outliers (Isolation Forest detection with window-mean correction); (2) dataset partitioning (70%/20%/10% for training/validation/testing), with test set excluded from normalization to ensure unbiased evaluation; (3) Z-score standardization was applied to numerical features using Equation (1).

$$x^{’}={\displaystyle \frac{x-\mu }{\delta }},$$

(1)

where x′ is the normalized value, x is the observed value, μ is the population mean, and δ is the population standard deviation.

2.2. Feature Selection

To address the parameter coupling and nonlinearity in blast furnace processes, this study employed Spearman correlation analysis [11] for feature selection. Variables with correlation coefficients exceeding the 0.9 threshold [14] (e.g., blast volume vs. bosh gas volume; blast pressure vs. pressure difference) were eliminated to reduce redundancy. The final feature set comprised 20 key parameters (excluding the target variable), with the refined dataset containing 87,360 instances as validated by the correlation heatmap in Figure 1.

2.3. XGBoost + LightGBM Model

2.3.1. XGBoost Model

XGBoost is a machine learning algorithm based on Gradient Boosting Decision Trees (GBDT), which sequentially integrates multiple weak learners (typically decision trees) to gradually optimize the objective function. The algorithm is based on an additive model, constructing a new tree in each iteration to fit the residual between the current model prediction and the true value, as shown in Figure 2. The input for each subsequent tree is the difference between the sum of all previous trees’ predictions and the true value, and the final output of the model is the weighted sum of all tree predictions.

In the model construction process [12], for a given dataset $\mathrm{D}=\left\{(x_i,y_i)\right\}(i=\mathrm{1,2},...,n)$, containing n samples, after t rounds of iteration, the model’s prediction output is expressed as

$${\widehat{y}}_i^{\left(t\right)}={\displaystyle {\sum }_{k=1}^t}{f}_k\left(x_i\right),$$

(2)

where ${\widehat{y}}_i^{\left(t\right)}$ represents the predicted value of the i-th sample after the t-th iteration, $f_k$ represents the k-th decision tree, and $x_i$ represents the feature vector of the i-th sample. XGBoost’s regularized objective function includes a loss term and a regularization term.

$${\mathrm{O}\mathrm{b}\mathrm{j}}^{\left(\mathrm{t}\right)}={\displaystyle {\sum }_{i=1}^n}L(y_i,{\widehat{y}}_i^{\left(t\right)})+{\displaystyle {\sum }_{k=1}^t}\mathsf{\Omega }\left(f_k\right),$$

(3)

where $L(y_i,{\widehat{y}}_i^{\left(t\right)})$ is the loss function, measuring the difference between the predicted value and the true value; $\mathsf{\Omega }\left(f_k\right)$ is the regularization term for the k-th tree. The specific form of the regularization term is

$$\mathsf{\Omega }(f_k)=\alpha ·T+{\displaystyle \frac{1}{2}}\beta ·{\displaystyle {\sum }_{j=1}^T}{w}_j^2,$$

(4)

where T represents the number of leaf nodes in the tree, w_j represents the weight value of the j-th leaf node, α is the penalty coefficient controlling the number of leaf nodes, and β is the L2 regularization coefficient controlling the weight values. During the optimization process, a second-order Taylor expansion is employed by XGBoost to approximate the objective function. At the t-th iteration, the objective function is approximated as

$${Obj}^{\left(t\right)}\approx {\displaystyle {\sum }_{i=1}^n}\left[g_i{f}_t(x_i)+{\displaystyle \frac{1}{2}}{h}_i{f}_t^2(x_i)\right]+\mathsf{\Omega }(f_t),$$

(5)

where $g_i={\displaystyle \frac{\partial L(y_i,{\widehat{y}}_i^{(t-1)})}{\partial {\widehat{y}}_i^{(t-1)}}}$ is the first-order gradient of the loss function, $h_i={\displaystyle \frac{{\partial }^{2}L(y_i,{\widehat{y}}_i^{(t-1)})}{{\partial }^2{\widehat{y}}_i^{(t-1)}}}$ is the second-order gradient of the loss function. In terms of tree structure optimization, define $I_{\mathrm{j}}=\left\{i|\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e} i \mathrm{i}\mathrm{s} \mathrm{a}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{e}\mathrm{d} \mathrm{t}\mathrm{o} \mathrm{l}\mathrm{e}\mathrm{a}\mathrm{f} \mathrm{n}\mathrm{o}\mathrm{d}\mathrm{e}j\right\}$ as the sample set of leaf node j. The optimal leaf node weight is calculated as

$$w_j^{*}=-{\displaystyle \frac{{\sum }_{i\in I_j}{g}_i}{{\sum }_{i\in I_j}{h}_i+\beta }}.$$

(6)

The corresponding optimal objective value is

$${Obj}^{*}=-{\displaystyle \frac{1}{2}}{\displaystyle {\sum }_{j=1}^T}{\displaystyle \frac{{\left({\sum }_{i\in I_j}{g}_i\right)}^2}{{\sum }_{i\in I_j}{h}_i+\beta }}+\alpha ·T,$$

(7)

where Obj* is the value sought by the model, and a smaller value (i.e., objective function) is better.

2.3.2. LightGBM Model

LightGBM is an efficient implementation of GBDT. It optimizes the training process through three key technologies: Gradient-based One-Side Sampling (GOSS), Exclusive Feature Bundling (EFB), and the Histogram algorithm [13]. GOSS is based on the observation that samples with larger gradients contribute more to the information gain. It retains the top a × 100% of high-gradient samples and random samples b × 100% of low-gradient samples, calculating the information gain ${\widehat{V}}_j\left(d\right)$ with weight compensation.

$${\widehat{V}}_j(d)={\displaystyle \frac{1}{n}}\left[{\displaystyle \frac{{({\sum }_{x_i\in A_l}{g}_i+\frac{1-a}{b}{\sum }_{x_i\in B_l}{g}_i)}^2}{n_l^j\left(d\right)}}+{\displaystyle \frac{{({\sum }_{x_i\in A_r}{g}_i+\frac{1-a}{b}{\sum }_{x_i\in B_r}{g}_i)}^2}{n_r^j\left(d\right)}}\right]$$

(8)

where $g_i$ is the gradient of the sample, A_l, A_r, B_l, and B_r are the left and right subsets of sets A and B after splitting at threshold d for feature j, respectively, and $n_l^j\left(d\right)$ and $n_r^j\left(d\right)$ are the corresponding sample counts.

EFB is applied to high-dimensional sparse data as it bundles exclusive features (i.e., features that are rarely non-zero simultaneously). Suppose the feature set is partitioned into K bundles B₁, B₂, …, B_K, each bundle satisfies

$$\sum _{f\in B_k}\mathsf{I}(x_{i,f}\ne 0)\le 1,\forall i=\mathrm{1,2},...,n,$$

(9)

where k is the bundle index, B_k is the set of the k-th feature bundle, and f is the feature index, $\mathsf{{\rm I}}(\cdot )$ is the indicator function, which takes the value 1 when the condition inside the parentheses is satisfied, otherwise 0.

The Histogram Algorithm is used to discretize continuous features into k bins to build a feature histogram.

$$\mathrm{H}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{o}\mathrm{g}\mathrm{r}\mathrm{a}\mathrm{m}\left(j\right)=\left\{({\mathrm{b}}_1,{\mathrm{S}}_1),({\mathrm{b}}_2,{\mathrm{S}}_2),...,({\mathrm{b}}_{\mathrm{k}},{\mathrm{S}}_{\mathrm{k}})\right\},$$

(10)

where b_i is the bin boundary, S_i = {(G_i, H_i)} contains the gradient statistics of samples falling into that bin, G_i = ∑g is the sum of gradients, and H_i = ∑h is the sum of second-order gradients.

2.3.3. XGBoost + LightGBM Ensemble Model

To combine the advantages of both algorithms, this study adopts a weighted average ensemble strategy. The final prediction ${\widehat{y}}_{\mathrm{ensemble}}$ of the ensemble model is the arithmetic mean of the XGBoost predictions ${\widehat{y}}_{\mathrm{xgb}}$ and the LightGBM prediction ${\widehat{y}}_{\mathrm{lgb}}$

$${\widehat{y}}_{\mathrm{ensemble}}=\alpha ·{\widehat{y}}_{\mathrm{xgb}}+(1-\alpha )·{\widehat{y}}_{\mathrm{lgb}}$$

(11)

In this study, an equal weight configuration α = 0.5 is used to ensure model robustness and simplicity. Theoretically, the effectiveness of this ensemble model stems from variance reduction theory. Assume the prediction errors of the base models are

$${\widehat{y}}_{\mathrm{xgb}}=y+{\delta }_{xgb},{\widehat{y}}_{\mathrm{lgb}}=y+{\delta }_{lgb},$$

(12)

where the expectation of the error terms is zero, and $\mathrm{E}\left[{\delta }_{xgb}\right]=\mathrm{E}\left[{\delta }_{lgb}\right]=0$. The variance of the ensemble prediction error is

$$Var({\delta }_{\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{e}\mathrm{m}\mathrm{b}\mathrm{l}\mathrm{e}})=w_{xgb}^2{\sigma }_{xgb}^2+w_{lgb}^2{\sigma }_{lgb}^2+2w_{xgb}{w}_{lgb}{\rho \sigma }_{xgb}{\sigma }_{lgb},$$

(13)

where ${\sigma }_{xgb}^2$ and ${\sigma }_{lgb}^2$ represent the variances of the prediction errors of XGBoost and LightGBM, respectively, and ρ represents the correlation coefficient between the prediction errors of the two models. When ρ ≤ 0, the prediction error variance of the ensemble model is less than the average variance of the individual models, thus achieving a variance reduction effect.

Due to the fundamental differences in the algorithm implementations of XGBoost and LightGBM, that is, the former uses pre-sorting and level-wise growth, while the latter is based on the histogram algorithm and leaf-wise growth, resulting in different feature processing and sampling strategies. This means that when one model makes a prediction error on certain samples, the other model is likely to provide a correct prediction, thereby enhancing the model’s generalization ability and prediction stability through an ensemble.

2.4. Model Performance Evaluation Metrics

Multiple is a commonly used evaluation metric to assess model performance. Root mean square error (RMSE) reflects the overall level of prediction error; MAE is used to measure the absolute size of prediction errors and is robust to outliers; R² is used to evaluate the goodness of fit, with values closer to 1 indicating stronger explanatory power. The formulas for each metric are as follows:

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{({\mathrm{y}}_{\mathrm{i}}-{\widehat{\mathrm{y}}}_{\mathrm{i}})}^2}$$

(14)

$$\mathrm{M}\mathrm{A}\mathrm{E}={\displaystyle \frac{1}{\mathrm{N}}}{\sum }_{\mathrm{i}=1}^{\mathrm{N}}\left|{\mathrm{y}}_{\mathrm{i}}-{\widehat{\mathrm{y}}}_{\mathrm{i}}\right|$$

(15)

$${\mathrm{R}}^2=1-{\displaystyle \frac{{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{({\mathrm{y}}_{\mathrm{i}}-{\widehat{\mathrm{y}}}_{\mathrm{i}})}^2}{{\sum }_{\mathrm{i}=1}^{\mathrm{N}}{({\mathrm{y}}_{\mathrm{i}}-\stackrel{\mathrm{-}}{\mathrm{y}})}^2}}$$

(16)

where ${\mathrm{y}}_{\mathrm{i}}$ is the actual value of the i-th sample, ${\widehat{\mathrm{y}}}_{\mathrm{i}}$ is the corresponding predicted value, $\stackrel{\mathrm{-}}{\mathrm{y}}$ is the average of the values, and N is the total number of samples.

3. Results and Discussion

The performance metrics of the ensemble model are shown in

Table 1. Evaluation metrics of the ensemble model.

Evaluation Metrics	RMSE	MAE	R2
Value	0.0868	0.0708	0.9602

. The model’s RMSE is 0.0868, and MAE is 0.0708, indicating a small average deviation between predicted and actual observed values, and thus high prediction accuracy. The close values of these two error metrics suggest a relatively uniform error distribution without significant abnormal deviations. Furthermore, R² is 0.9602, indicating that the model explains 96.02% of the variance in the dependent variable, demonstrating significant fitting properties.

Figure 3 shows the scatter distribution between the predicted and actual values of the integrated model, with the diagonal representing ideal predictions. Most points cluster tightly along this line, demonstrating high predictive accuracy. Despite minor outliers, the strong linear trend reflects excellent prediction consistency, confirming the model’s capability to capture underlying data patterns and its robust generalization performance.

Figure 4 presents the error distribution of the ensemble model, which approximates a normal distribution. The mean of the error distribution is −0.002625, and the median is −0.001481. Both values are close to zero and relatively close, indicating that the overall prediction bias of the model is small and there is no obvious systematic bias, further verifying the accuracy and stability of the model. The concentration of errors around zero further validates how the XGBoost + LightGBM integration enhances overall predictive performance through complementary advantages.

Figure 5 shows the predicted permeability indexes of XGBoost, LightGBM, and the ensemble model against the true values over the time series. The prediction trajectories of all three models align with the trend of the true values. Among them, the predictions of the ensemble model most closely fit the true values, especially in regions where the true values change significantly, where the ensemble model demonstrates more stable predictive capability.

4. Conclusions

Based on 25 initial parameters from a domestic blast furnace (January to June 2024), 20 key parameters affecting the permeability index through Spearman correlation analysis were identified, including operational parameters (e.g., charging rate, blast volume/pressure), thermal indicators (e.g., combustion temperature, cooling stave differential), and gas flow indices. The proposed XGBoost + LightGBM ensemble model achieves outstanding permeability index prediction performance, with an RMSE of 0.0868, an MAE of 0.0708, and an R² of 0.9602. These results demonstrate high accuracy, robust stability, and effective trend-tracking capability, establishing a reliable technical foundation for intelligent blast furnace control.