An Online Blast Furnace Condition Recognition Method Based on Spatiotemporal Texture Feature Coupling and Diffusion Networks

Abstract

Real-time and accurate identification of the blast furnace (BF) condition is essential for maintaining stability and improving energy efficiency in steelmaking. However, the harsh environment inside the BF makes direct acquisition of the BF condition extremely difficult. To address this challenge, this study proposes an online BF condition recognition method based on spatiotemporal texture feature coupling and diffusion networks (STFC-DN). The method employs a multi-domain Swin-Transformer module (MDSTM) combined with wavelet decomposition and channel attention to extract the gas flow region. A temporal feature pyramid network module (T-FPNM) is then used to capture both the global and local spatiotemporal characteristics of this region. Heuristic clustering and an idempotent generative network (IGN) are introduced to obtain standardized BF condition features, enabling intelligent classification through multi-metric similarity analysis. Experimental results show that the proposed STFC-DN achieves an average accuracy exceeding 98% when identifying four BF conditions: normal, hanging, oblique stockline, and collapsing, with an inference speed of approximately 28 FPS. This approach demonstrates both high accuracy and real-time capability, showing strong potential for advancing the intelligent and sustainable development of the steel industry.

1. Introduction

The blast furnace (BF) is the core facility in the steelmaking process; its operation consumes substantial energy and directly determines steel quality [1,2]. BF conditions serve as key indicators of BF operational stability and smelting efficiency [3,4]. Therefore, precise and real-time monitoring of BF conditions is critically important [5,6]. However, the high-temperature, high-pressure, sealed, and dusty environment inside the BF renders direct acquisition of BF condition information highly challenging [7]. Although the high-temperature industrial endoscope developed by Chen, Yi et al. [8,9,10], and colleagues enables real-time acquisition of BF videos, the absence of auxiliary light sources often causes BF burden surface images to exhibit blurred textures and uneven illumination, severely limiting subsequent BF condition analysis [11]. Therefore, there is an urgent need to develop efficient BF condition recognition methods to achieve intelligent control of the BF ironmaking process [12].

Existing studies on BF condition recognition can be broadly categorized into two approaches: mechanism-based modeling and data-driven, multimodal fusion with intelligent prediction [13].

(1) Mechanism-based modeling and data-driven: BF condition recognition is commonly performed under the constraints of mechanistic models, where operational data are integrated and deep learning is employed for modeling. Xu et al. [14] combined industrial endoscopy with a virtual multi-camera array to achieve real-time 3D measurement of the BF burden surface under extreme environments, demonstrating reliable performance for BF condition monitoring and control. Zhou et al. [15] established quantitative criteria for abnormal condition recognition by detecting coke particle size and temperature distribution in the tuyere region. Duan et al. [16] developed a novel deep neural network, ES-SFRNet, which achieves intelligent recognition of BF tuyere conditions by fusing tuyere images with temporal data. Zhao et al. [17] introduced the GT-MSPC model, which leverages multi-objective optimization for abnormal condition detection and early warning. These methods can achieve recognition accuracy above 95% under specific operating conditions. However, their performance relies heavily on sensor precision and the accurate measurement of internal furnace parameters. When the operating conditions become complex, the prediction error of the models exceeds 10%, which limits their applicability in industrial environments.

(2) Multimodal fusion with intelligent prediction: To overcome the limitations of single data sources, some studies have attempted to integrate radar point clouds, visible/infrared images, and temperature data to construct multi-source information-driven discriminative models. Guo et al. [18] proposed a BF condition classification model based on multi-source fusion and Dynamic Process Adaptive Kernel Fisher Discrimination (DPAKF), which achieved high accuracy. Tian et al. [13] employed RGB-D cameras and digital elevation models to measure burden surface roughness and perform state discrimination, enabling comprehensive BF condition recognition. Lee et al. [19] combined hybrid quantum machine learning with pulverized coal injection control to improve the stability of BF temperature prediction. This type of method achieves an average classification accuracy of 98.2% for BF conditions, representing a 25% improvement in prediction accuracy compared with traditional models. However, issues such as high computational complexity, difficulty in cross-modal data alignment, and limited interpretability still remain.

To address these issues, this paper proposes an online BF condition recognition method based on spatiotemporal texture feature coupling and diffusion networks (STFC-DN). The main contributions are as follows:

(1) To overcome the problem of BF blurred burden surface images that obscure critical gas flow regions, a multi-domain Swin-Transformer module (MDSTM) is proposed to enable precise localization and segmentation of these regions.

(2) To handle the challenge that gas flow regions change dynamically over time and are difficult to comprehensively represent using conventional methods, a temporal feature pyramid network module (T-FPNM) is proposed, which integrates spatial features with temporal dynamic features to achieve multi-scale representation of BF conditions.

(3) To reduce the strong dependence on BF condition labels, alleviate the difficulty of manual annotation, and improve classification accuracy, an intelligent recognition method based on feature generation is introduced. This approach effectively enhances recognition accuracy and robustness under complex BF conditions.

The remainder of this paper is organized as follows: Section 2 introduces BF condition classification and BF burden surface image preprocessing. Section 3 elaborates on the proposed recognition method. Section 4 presents the experimental design and analysis of results. Section 5 concludes the paper and outlines future research directions.

2. Classification of BF Conditions and Color Rendering of BF Burden Surface Images

2.1. Classification and Feature Description of BF Conditions

BF conditions are generally classified into two categories: normal conditions and abnormal conditions. Abnormal conditions primarily include typical types such as hanging, oblique stockline, and collapsing. Table 1 summarizes the characteristic features associated with different BF conditions, and Figure 1 illustrates representative image examples.

Table 1. Classification and Characteristics of BF Conditions.

BF Condition	Characteristics
Normal condition	Stable combustion inside the BF, uniform gas flow distribution, consistent brightness, clear gas flow morphology, and sharp image display.
Hanging	The burden surface cannot descend steadily, permeability decreases, gas flow is obstructed, and central gas flow brightness intensifies.
Oblique stockline	Uneven burden descent, unreasonable gas flow distribution, asymmetric gas flow pattern, and low brightness.
Collapsing	Sudden massive collapse of burden, chaotic and irregular image appearance, and low brightness.

Figure 1. Illustrations of BF conditions: (a) normal condition, (b) hanging, (c) oblique stockline, and (d) collapsing.

2.2. Image Enhancement and Color Rendering of BF Burden Surface

The BF burden surface images captured by a high-temperature industrial endoscope, as illustrated in Figure 2, are of poor quality. To address this limitation, STFC-DN incorporates image clarification and color rendering into the preprocessing stage.

Figure 2. BF burden surface image acquired by high-temperature industrial endoscope: (a–c) BF burden surface images obtained from different batches.

During image clarification, the BF burden surface image is first decomposed into an illumination map and a reflectance map using an encoder–decoder network. Subsequently, gamma correction is applied to the illumination map to improve brightness uniformity, while a bilateral texture filtering-based detail enhancement method [20] is applied to the reflectance map to emphasize the edges and texture features of the gas flow region, as shown in Equation (1).

$$\left\{\begin{array}{c}\mathrm{BS}=\mathrm{Illumination}\otimes \mathrm{Rflectance}\hfill \\ I_{\mathrm{out}}=I_{\mathrm{in}}^{\gamma }\hfill \\ R_{\mathrm{out}}=BTF\left(R_{\mathrm{in}}\right)+{\sum }_{i=1}^{L-1}\mathrm{Enhance}\left(R_{\mathrm{in}}-BTF\left(R_{\mathrm{in}},k_i\right),k_i\right)\hfill \end{array}\right.$$

(1)

where BS denotes the BF burden surface image; Illumination and Rflectance represent the decomposed illumination and reflectance components; $\gamma$ is the gamma value; $I_{\mathrm{in}}^{\gamma }$ and $I_{\mathrm{out}}$ denote the input and enhanced illumination information; $\mathrm{BTF}\left(R_{\mathrm{in}},k_i\right)$ refers to the bilateral texture filtering operation; $k_i$ indicates the filter size of the i-th layer; $\mathrm{Enhance}\left(d_i,k_i\right)$ denotes the enhancement function applied to the texture detail image $d_i$ ($d_i=R_{\mathrm{in}}-\mathrm{BTF}\left(R_{\mathrm{in}},k_i\right)$); and $R_{\mathrm{out}}$ represents the enhanced reflectance information.

During color rendering, regions with varying temperature and brightness distributions are mapped into a pseudo-color space to enhance the distinction of gas flow patterns. The experimental results are presented in Figure 3.

Figure 3. BF burden surface image after enhancement and pseudo-color rendering: (a,c) clarified BF burden surface images and (b,d) color-rendered BF burden surface images.

3. Methods

STFC-DN first utilizes the MDSTM to precisely extract the gas flow region. Subsequently, the multi-scale texture features and spatial structural features of this region are computed separately and then fused to construct a BF condition representation vector. Finally, an intelligent recognition method based on feature generation is introduced to enable precise identification of the BF condition. The overall framework of the proposed method is illustrated in Figure 4.

Figure 4. Overall framework of the proposed STFC-DN method.

3.1. Gas Flow Region Extraction Based on Multi-Domain Swin-Transformer Module (MDSTM)

The MDSTM incorporates multi-scale wavelet decomposition and a channel attention mechanism into the Swin-Transformer [21] to enhance detail capture and global structural perception in gas flow regions. The structure of the MDSTM is illustrated in Figure 5.

Figure 5. Structure of the proposed MDSTM.

First, during the model training stage, a total of 2069 representative blast furnace (BF) burden surface images were selected, with each frame corresponding to one second of the internal furnace state. The dataset included 758 frames of normal BF condition, 547 frames of hanging, 435 frames of oblique stockline, and 329 frames of collapsing. The high-temperature gas flow region in each frame was manually annotated based on three criteria: the regional brightness exceeded the background by more than 25%, the region exhibited plume-like or radially diffused bright textures, and its edge morphology remained stable while changing continuously across adjacent frames. These annotations were used to generate binary supervision labels denoted as $M_{gt}$. In addition, multi-scale features $G_f$ obtained through two-dimensional wavelet transform were introduced as guiding information. Subsequently, the channel attention weights $A_c$ were calculated according to the correlation within the gas flow region, and the extracted features were weighted as follows:

$$G_{f_{\mathrm{weighted}}}=A_c\odot G_f$$

(2)

The feature set $G_{f_{\mathrm{weighted}}}$ was then fed into multiple layers of the multi-domain Swin-Transformer module (MDSTM). The window-based multi-head self-attention mechanism (W-MSA) was employed to capture local feature dependencies, while the shifted window strategy (SW-MSA) was used to achieve information interaction across different windows:

$$G_{\mathrm{output}}=\mathrm{SwinBlock}\left(\mathrm{Shift}(\mathrm{Multi}-\mathrm{Attention}\left(G_{f_{\mathrm{weight}}}\right))\right)$$

(3)

Finally, the output features $G_{\mathrm{output}}$ are mapped to the probability space of the gas flow regions through a fully connected layer (FC), yielding the segmentation results, calculated as follows:

$$P_{\mathrm{flame}}=\mathrm{FC}\left(G_{\mathrm{output}}\right)$$

(4)

The model was trained in a supervised manner, where the input images and their corresponding labels $M_{gt}$ were used for feature learning. The loss function was defined as a weighted combination of binary cross-entropy loss and dice loss, written as follows:

$$\mathcal{L}={\lambda }_1{\mathcal{L}}_{BCE}(M_{pred},M_{gt})+{\lambda }_2{\mathcal{L}}_{Dice}(M_{pred},M_{gt})$$

(5)

where, ${\lambda }_1$ and ${\lambda }_2$ represent adjustable weighting coefficients.

3.2. Feature Computation of Gas Flow Regions

After the MDSTM precisely extracts the gas flow region from the BF burden surface image, the texture features and spatiotemporal structural characteristics of this region need to be further quantified.

3.2.1. Multi-Scale Texture Features

The multi-scale texture features comprise the perimeter P, area A, brightness B, and sharpness S of the gas flow region, which are combined through weighted fusion, as formulated in Equation (6), written as follows:

$$F_{\mathrm{fusion}}={\omega }_{1}P+{\omega }_{2}A+{\omega }_{3}B+{\omega }_{4}S$$

(6)

where ${\omega }_1$∼${\omega }_4$ denotes a learnable weight, which is automatically updated through a Multi-Layer Perceptron (MLP) followed by Softmax normalization:

$$[{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4]=\mathrm{Softmax}\left(\mathrm{MLP}(P,A,B,S)\right)$$

(7)

The fused feature preserves geometric shape information while integrating edge characteristics, thereby providing input that combines both global and local information for BF condition recognition.

3.2.2. Spatiotemporal Structural Feature Extraction Based on Temporal FPN Module (T-FPNM)

The structure of the T-FPNM is illustrated in Figure 6. First, a multi-scale convolutional network is employed to extract the spatial features of the gas flow region frame ${\left\{R_t\right\}}_{t=1}^T$ and to generate multi-level feature maps $\{F_t^1,F_t^2,\dots ,F_t^L\}$. Subsequently, feature maps at the same level are stacked into a sequence $F_{\mathrm{stack}}^l$ and fed into a convolutional long short-term memory (ConvLSTM) network to produce feature maps that incorporate temporal information [22]:

$$F_{\mathrm{temporal}}^l=\mathrm{ConvLSTM}\left(F_{\mathrm{stack}}^l\right)$$

(8)

where l denotes the the layer index. Finally, a top-down strategy is employed to iteratively fuse features across layers. The feature map $F_{\mathrm{temporal}}^{l+1}$ is upsampled to match the resolution of $F_{\mathrm{temporal}}^l$ and then fused with it via element-wise addition:

$${\widehat{F}}^l=\mathrm{Upsample}\left(F_{\mathrm{temporal}}^{l+1}\right)+F_{\mathrm{temporal}}^l$$

(9)

Figure 6. Structure of the proposed T-FPNM.

By propagating down to the bottom layer, the complete multi-scale spatiotemporal feature set $\{{\widehat{F}}^1,{\widehat{F}}^2,\dots ,{\widehat{F}}^L\}$ is obtained. This set is then combined with the multi-scale texture features $F_{\mathrm{fusion}}$ extracted by the MDSTM to construct the BF condition representation vector $\{F_{\mathrm{fusion}},{\widehat{F}}^1,{\widehat{F}}^2,\dots ,{\widehat{F}}^L\}$.

3.3. Intelligent BF Condition Recognition Method Based on Feature Generation

The method first obtains BF condition labels through heuristic fuzzy clustering (HFC), then employs an idempotent generative network (IGN) [23] to generate standardized feature representations of BF conditions, and finally combines multi-metric similarity measurements to achieve BF condition classification. The workflow is shown in Figure 7.

Figure 7. Workflow of the proposed method.

First, the HFC uses the multi-scale texture features of the gas flow region as clustering centers to derive labels $Z_i$ for four categories of BF conditions, which are subsequently fed into the IGN network:

$$G_i=\mathrm{IGN}(Z_{\mathrm{i-embed}},{\eta }_i)$$

(10)

where ${\eta }_i$ represents noise, $G_i$ denote the generated BF condition features. Subsequently, the BF condition representation vector $F_{\infty }=\{F_{\mathrm{fusion}},{\widehat{F}}^1,{\widehat{F}}^2,\dots ,{\widehat{F}}^L\}$ is compared with the generated features $G_i$:

$$\left\{\begin{array}{c}{S}_{cos}(F_{\infty },G_i)={\displaystyle \frac{F_{\infty }·G_i}{\left|\right|F_{\infty }|{|}_2\left|\right|G_i|{|}_2+ϵ}}\hfill \\ D_{\mathrm{maha}}(F_{\infty },G_i)=\sqrt{{(F_{\infty }-G_i)}^T{\sum }^{-1}(F_{\infty }-G_i)}\hfill \\ D_{\mathrm{euc}}(F_{\infty },G_i)={\displaystyle \frac{{∥F_{\infty }-G_i∥}_2}{\sqrt{m}}}\hfill \end{array}\right.$$

(11)

where $S_{cos}$, $D_{\mathrm{maha}}$, and $D_{\mathrm{euc}}$ denote the modified cosine similarity, the normalized Mahalanobis distance, and the normalized Euclidean distance, respectively. m represents the feature dimension. The results of the three metrics are integrated using an adaptive weight learning network. Finally, the BF condition category is determined by the maximum discrimination score:

$$Z_{\mathrm{pred}}=arg\underset{i}{max}{M}_i$$

(12)

where $M_i$ denotes the integrated discrimination score for BF conditions.

4. Experiments

The experimental data were obtained from independent BF condition videos captured by a high-temperature industrial endoscope in a steel plant. The dataset includes four typical BF conditions: normal, hanging, oblique stockline, and collapsing. A total of 162 video clips were selected, each with a duration of approximately 10 s. Sampling at 1 fps resulted in 2956 image frames, including 1083 frames of normal, 782 of hanging, 621 of oblique stockline, and 470 of collapsing. All data were divided into training, validation, and test sets in a ratio of 7:2:1. Representative samples are shown in Figure 8. All experiments were implemented in Python 3.10.8 on the PyCharm Professional 2023.2 platform (JetBrains s.r.o., Prague, Czech Republic), using an NVIDIA GeForce RTX 3090 GPU (NVIDIA Corporation, Santa Clara, CA, USA) with 24 GB of memory.

Figure 8. BF Burden surface images captured by a high-temperature industrial endoscope: (a–d) images from different batches.

4.1. Generation of BF Condition Clustering Labels

To reduce reliance on manual annotation, the HFC is employed to automatically generate BF condition labels. The clustering results are illustrated in Figure 9, showing that different BF conditions can be effectively distinguished: normal conditions correspond to three clusters, hanging conditions to four clusters, oblique stockline conditions to five clusters, and collapsing conditions to six clusters. These results validate that the proposed method can automatically uncover potential BF condition structures from image features.

Figure 9. Clustering results of BF condition labels obtained by HFC. (a) normal condition, (b) hanging, (c) oblique stockline, and (d) collapsing.

4.2. Accuracy and Efficiency in BF Condition Recognition

To evaluate the overall performance of the STFC-DN, a comparative analysis was conducted against several representative classification networks, including ResNet-50 [24], EfficientNet-B3 [25], HRNet-W32 [26], and the Vision Transformer [27], as shown in Table 2. The evaluation metrics included Precision, Recall, F1-score, and average inference speed, as defined in Equation (13). Precision, Recall, and F1-score were computed from the confusion matrix, which was generated by frame-by-frame comparison between model predictions and manual annotations.

Table 2. Recognition accuracy and inference speed of different models under various BF conditions.

Model	BF Condition	Precision (%)	Recall (%)	F1-Score (%)	Inference Speed (FPS)
ResNet-50	Normal condition	96.22	95.88	96.00	20.48
	Hanging	95.81	95.25	95.52	19.57
	Oblique stockline	95.18	94.54	94.85	19.67
	Collapsing	98.95	99.00	98.97	25.49
EfficientNet-B3	Normal condition	96.98	96.41	96.67	13.78
	Hanging	96.36	95.93	96.11	12.64
	Oblique stockline	95.74	95.00	95.43	12.49
	Collapsing	99.12	98.90	99.01	14.95
HRNet-W32	Normal condition	95.48	94.76	95.00	24.69
	Hanging	95.27	94.37	94.88	24.13
	Oblique stockline	94.86	93.85	94.29	23.49
	Collapsing	98.74	98.42	98.55	25.41
Vision Transformer	Normal condition	96.51	95.98	96.26	26.42
	Hanging	96.14	95.45	95.84	25.73
	Oblique stockline	95.95	94.98	95.48	25.98
	Collapsing	99.00	98.85	98.97	26.91
MDSTM	Normal condition	98.24	97.72	98.42	28.16
	Hanging	97.86	97.33	97.56	27.91
	Oblique stockline	97.58	96.92	97.28	26.73
	Collapsing	99.47	99.34	99.36	28.37

Note: The bold values indicate the best results among all models under each BF condition.

The results indicate that STFC-DN achieved average Precision, Recall, and F1-score values of 98.29%, 97.83%, and 98.16%, respectively, across the four BF conditions, with an average inference speed of 27.79 FPS. These results outperform those of the comparison models and demonstrate that STFC-DN can fully meet the real-time recognition requirements of blast furnace operations.

$$\left\{\begin{array}{c}Precision={\displaystyle \frac{TP}{TP+FP}}\hfill \\ Recall={\displaystyle \frac{TP}{TP+FN}}\hfill \\ F1={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}\hfill \end{array}\right.$$

(13)

where $TP$, $FP$, and $FN$ represent the numbers of true positive, false positive, and false negative samples, respectively, and N denotes the number of BF condition categories.

4.3. Feature Analysis and Ablation Experiments of High-Temperature Gas Flow Regions

4.3.1. Multi-Scale Feature Computation of High-Temperature Gas Flow

The multi-scale texture features and spatiotemporal structural features of the gas flow region were computed to quantify the dynamic differences among various BF conditions. Table 3 presents the feature distributions under four representative BF conditions. Overall, in the abnormal conditions, the gas flow region exhibits significant deviations in area, brightness, and spatiotemporal characteristics compared with the normal condition. These deviations indicate stronger local instability and more pronounced dynamic fluctuations within the gas flow region.

Table 3. Comparison of Gas Flow Region Features under Different BF Conditions.

BF Condition	Area (/px2)	Perimeter (/px)	Brightness	Sharpness	Spatiotemporal Feature
Normal condition	$3.28\times {10}^4$	$2.27\times {10}^4$	0.988	0.053	0.045
Hanging	$9.80\times {10}^3$	$1.24\times {10}^4$	1.000	0.043	0.065
Oblique stockline	$1.71\times {10}^4$	$2.58\times {10}^4$	0.999	0.041	0.061
Collapsing	$3.51\times {10}^4$	$1.75\times {10}^3$	0.996	0.001	0.094

4.3.2. Ablation Experiments on the Extraction of High-Temperature Gas Flow Regions

To assess the effectiveness of multi-domain guidance information in the MDSTM, the wavelet transform module was removed, and the model was subsequently retrained. Figure 10 compares the extraction results of the high-temperature gas flow region with and without the wavelet transform module, while Table 4 reports the corresponding BF condition recognition accuracy.

Figure 10. Extraction results of the high-temperature gas flow region: (a) original image; (b) extraction with multi-domain guidance information; (c) extraction without multi-domain guidance information.

Table 4. Comparison of BF condition recognition with and without multi-domain guidance.

Model	Condition
	Normal	Hanging	Oblique Stockline	Collapsing
With multi-domain guidance	98.24%	97.86%	97.58%	99.47%
Without multi-domain guidance	93.22%	96.81%	97.36%	98.64%

Note: The bold values indicate the best recognition accuracy under each BF condition.

It is evident that the recognition accuracy decreases substantially after removing the multi-domain guidance information, with the decline being particularly pronounced under normal conditions. This finding indicates that frequency-domain details are essential for the precise extraction of gas flow regions and further validates the necessity of the multi-domain feature fusion design.

5. Conclusions

This study addresses the challenges of the harsh internal environment of the BF and the difficulty of obtaining accurate BF condition information by proposing the STFC-DN method. The method employs the MDSTM to achieve precise extraction of the gas flow region, integrates the T-FPNM to characterize its dynamic spatiotemporal evolution, and utilizes feature generation together with a multi-metric discrimination mechanism to accomplish intelligent classification and online recognition of BF conditions. The proposed model can be integrated with the BF process control system to dynamically adjust key operational parameters such as burden distribution eccentricity, tuyere air volume, and pulverized coal injection rate based on recognition results. This enables early warning and prevention of abnormal BF conditions, thereby avoiding interruptions in furnace operation.

However, the performance of STFC-DN remains sensitive to the quality of BF burden surface images and illumination conditions during data acquisition. Its inference speed is limited on devices with low computational capability, and its generalization performance across different furnace types and operating conditions still requires further validation. Future work will focus on model lightweighting, robustness enhancement, and multimodal data fusion to improve the system’s long-term stability and adaptability in complex industrial environments.