Application of Efficient Channel Attention Residual Mechanism in Blast Furnace Tuyere Image Anomaly Detection

Abstract

In the steelmaking industry, the state of the blast furnace tuyere is an important basis for obtaining the internal information of the blast furnace. Traditional detections mainly rely on manual experience judgment, which is a time-consuming and tiring procedure for a human. In order to improve the efficiency of the detection, this paper is devoted to applying artificial intelligence methods to blast furnace anomaly detection. However, because of the low imaging degree of the abnormal state monitoring of the furnace mouth, the difference in the abnormal category is inconspicuous, and it is difficulty to extract the features with the existing intelligent models. To solve these problems, a novel and stable method is proposed in this paper to classify the image recognition of the abnormal state of the tuyere into one category; this is a new architecture that combines multiple technologies. For the fine-grained image classification task, an improved abnormal state recognition algorithm of the blast furnace tuyere based on the channel attention residual mechanism is proposed. In the model, the dataset is augmented by rotating it at random angles to balance the amount of data in each category; then, the residual module is used to integrate high- and low-order feature information and optimize the network; then, the multi-layer channel attention module is added based on the channel attention residual mechanism, and it obtains the optimal parameter combination of the model through k-fold cross-validation. Moreover, the number of channels was reduced by half after channel fusion, which could effectively reduce the model parameters and model complexity. It is shown in our experiments that the proposed method has an accuracy rate of 97.10% in identifying the abnormal state of the tuyere in our collection of blast furnace tuyere datasets. In order to test the performance of the proposed method, some existing models, such as SERNet, ResNeXt, and repVGG, are involved for comparison, and the proposed method has a better classification effect in comparison to them.

1. Introduction

Blast furnace steelmaking is a key task in the iron and steel industry and accounts for more than 95% of global iron production. However, blast furnace steelmaking is a dangerous industry [1]. The blast furnace is sealed vessel with a high temperature and a high pressure; thus, improper operation or untimely abnormal state detection will lead to explosion accidents. For example, in 2021, a blast furnace steelmaking accident occurred in Xingtai, Hebei province, resulting in six deaths and six injuries; in 2019, a blast furnace pipe explosion at Fangda Special Steel caused one death and nine injuries. So, it is important to detect the state of the blast furnace. The detection of the working state of the blast furnace is mainly carried out through the tuyere, and the tuyere of the blast furnace has become an important “window” [2,3]. In the traditional blast furnace detection, the operators observe the raceway from the peephole and judge the combustion condition according to experience, but it is difficult to make timely and accurate judgements of the combustion state in the blast furnace. We study the intelligent detection of blast furnace steelmaking, which does not require staff to inspect the blast furnace tuyere in real time; so, it is of great significance to the development of the blast furnace steelmaking industry and the safety of blast furnace workers.

In recent years, the intelligent detection of steelmaking has mainly focused on detecting the blowing state of the steelmaking, and many flame detection applications have been developed based on flame detection image technology. Li Pengju et al. [4] used the hue saturation value (HSV) to extract flame features. After Gaussian normalization, the Canberra distance was used to judge the flame combustion stage of the blast furnace, but this method could only extract low-latitude features with low efficiency. Jiang Fan et al. [5] used the maximum inter-category variance method combined with hue saturation intensity (HIS for short) for image segmentation and took the flame image as the center for image interception. The convolutional neural network model based on the LeNet-5 network was built to predict the early, middle, and late stages of blast furnace smelting, but the abnormal state of the smelting process could not be detected. Liu Hui et al. [6] segmented the image and then detected the flame edge, using the general regression neural network (GRNN for short) to complete the classification prediction. This method was prone to incomplete feature detection, especially in image segmentation; the features were easy to ignore. The accuracy of the subsequent detection was affected. To sum up, many researchers were devoted to the blast furnace smelting in the former period at the end of the testing, but the research on the unusual situation of blast furnace steelmaking recognition is rare. Zhang Tianfang et al. [7] based their research on a spatial attention mechanism combined with the network VGG16 model for the blast furnace tuyere image intelligence early warning, but the large field background changes affected the recognition accuracy, and there were no detailed model implementation details in the text. In image classification, in addition to common influencing factors, such as light, motion, and line of sight, the similarity between different classes of fine-grained datasets will be greater, and the difference between classes will be smaller. Therefore, fine-grained images have higher requirements for classification models and need better recognition of subtle differences. Deng Xuran et al. [8] showed that fine-grained image classification was very important in image processing, and the fine-grained image classification had a better classification effect in the classification of subtle features that were difficult to distinguish. In the processing of images that were difficult to distinguish, such as the image of the blast furnace tuyere, it was necessary to combine multiple methods to achieve the effect of accurate discrimination. Yu Hongsheng et al. [9] calculated the edge complexity, weighted the sub-interval of the histogram, simplified the weighted histogram, and used the simplified histogram to distinguish the features of the image, which was significantly better than the gray histogram. Image enhancement is a common method of image processing and plays an important role in image processing. Chen Si et al. [10] added smoothing to fuzzy enhancement to improve the image fuzzy enhancement system. Li Jun et al. [11] proposed a membership function to extract pixels near the object boundary and adopted the threshold selection algorithm to determine the threshold value so that the gray value of the enhanced image could be more balanced.

Although researchers have made great efforts in developing the detection technology of the blast furnace steelmaking image, the automatic detection of blast furnace combustion is still a preliminary application, and there is still much room for improvement in the detection effect. Because of the special structure and complexity of the combustion process, there are many problems to be overcome. First, a burner has only one fixed detection “window”, the distance between the raceway and the camera is long, resulting in a small field of view of the raceway flame, and the raceway can only be viewed from this one direction. Secondly, not only is the combustion temperature very high, but there is also the interference of the pulverized coal and coke in the raceway; the information obtained from the detection system is largely limited by these factors. In recent years, deep learning has become a research hotspot. It has been widely applied and has achieved remarkable results in image processing, speech recognition, and other fields, especially in image classification and image recognition [12,13,14,15]. In our work, an abnormal detection method of the blast furnace tuyere based on an improved channel attention mechanism and a residual network is proposed to learn the actual abnormal condition of the blast furnace raceway. A monitoring camera is used as a sensor to obtain the image of the blast furnace tuyere. First, the frame is read to obtain the image, and the region of interest of the furnace mouth is intercepted to eliminate the influence of the noise and background. Moreover, the channel attention mechanism is improved; combined with the residual network, the abnormal detection of the blast furnace tuyere is studied.

The aim of our methodology is to replace manual detection while greatly improving efficiency and accuracy so as to achieve a set of automatic, efficient, and very accurate intelligent recognition data. There were also some detection errors in the previous manual detection, but now, there are a variety of sensor alarms so that errors can also be dealt with at an early stage, and our method has a very high accuracy rate; even if there is a very small probability of error, it will not have a big impact. Additionally, the model will be gradually applied in the case of semi-manual intervention, and only when there are no major problems will it fully replace manual labor, and a complete early warning system will be developed before the model is applied.

The Section 2 of the article concerns the materials and methods and introduces the related materials and the specific structure of the model. The Section 3 is on the results; this section introduces the procedure of the experiment and the discussion of the results. The Section 4 summarizes the method we proposed and the results we obtained.

2. Materials and Methods

In this section, we will introduce the related materials and the specific method we propose.

2.1. Related Materials

In the video image of the blast furnace tuyere, the parts to be detected include the blast furnace tube wall area, the inner bright area, the spray gun area, and the coal lump area. The blast furnace tuyere structure is shown in Figure 1. In the actual blast furnace working environment, the placement position and the angle of the monitor are different, and the obtained video images are greatly different. Furthermore, the shape of the spray gun shot will be different; the same state of different videos will show different forms. Therefore, it is not easy to detect different working states, and the judgment of the same working states also has an impact; that is, it is not easy to distinguish intraclass differences from interclass differences. A large amount of the image classification is mainly based on deep learning to distinguish images with obvious differences in features [16,17]. For example, ImageNet’s 1000 categories have great differences and features that are easy to distinguish. However, the problem of the image classification of the blast furnace tuyere is mainly caused by different states at the same furnace mouth, and the same state will also occur at different furnaces. Therefore, conventional deep learning networks cannot accurately detect the specific categories of air outlet anomalies.

Jie Hu et al. [18] focused on the dependence between the channels and proposed “squeeze and excitation” (SE) blocks, which adaptively selected channel features and formed a SENet structure through layer upon layer of SE blocks. The imaging degree of the blast furnace tuyere images obtained in this paper is low, and the differences between the classes are small. Therefore, abnormal condition detection is classified as fine-grained image classification to solve the problem of the low imaging degree and the difficulty in distinguishing feature differences. In this paper, the abnormal state recognition algorithm of the blast furnace tuyere based on the attention residual mechanism is improved on the basis of the SE block, and combined with the residual network, a residual convolutional neural network based on channel attention is constructed, named ESERNet (Efficient Squeeze-and-Excitation Residual Network). This network model is applied to the detection and classification of the different situations in the blast furnace tuyere pictures.

2.2. Proposed Methods

At present, the existing methods are rarely used in this field, and the accuracy is not high enough. In order to solve this problem, we propose a new model. The channel attention residual convolutional neural network model consists of the squeeze-and-excitation block (SEB) and the residual convolutional block (RCB). The structure is shown in Figure 2. The model network consists of one convolution layer, four stacked modules, one full connection layer, and the last softmax layer. The stack of four-layer modules is composed of SEB and RCB, which constructs a deep network structure. The interconnection of the two modules is used to extract multidimensional features so as to realize the classification detection of images under the different conditions of the blast furnace tuyere.

2.2.1. Reconstruct the Channel Attention Module

The channel attention mechanism is to focus on the relationship between the channels and to use their dependence relationship to strengthen the ability of the feature screening and the extraction of the network. By grasping the global information, the network can screen out useful information to strengthen the extraction of this part of the feature and suppress useless features. The channel attention mechanism consists of squeeze and excitation. First, the global space characteristics of each channel are compressed. The information is compressed into a descriptor to represent the information for each channel. Then, the dependence of the channels is learned, the weights of the different channels are obtained, and the feature graphs are adjusted according to the weight information.

In this paper, based on the channel attention mechanism, the SEB attention convolution block is constructed to fuse global information and capture rich context associations to solve the problem of image classification. This module is added to deepen the connection between the channels on the basis of the original channel attention mechanism; so, a 1 × 1 convolution is first used for information fusion for each channel when obtaining the global information. The operational differences between the original channel attention module and the channel attention module in this paper are shown in Figure 3. Figure 3a represents the operation of the SE block. Firstly, the global spatial information is compressed into a channel by generating statistics through global average pooling. Then, two fully connected (FC) layers are added before and after the non-linear to control the change of dimension; that is, the first fully connected layer is a dimension reduction layer, and then, Relu is added. Then, another dimension is connected to add a layer back to the channel dimension of the transform output. Finally, the sigmoid activation function is used to learn the non-linear interactions between the channels, ensuring that multiple channels are allowed to be emphasized. The final output is to rescale the feature by multiplying the output activated by the previous layer. Before global pooling, a 1 × 1 convolution operation is added to the SEB convolution block in Figure 3b for channel fusion, and then, the number of channels is halved to reduce the computation of the model.

2.2.2. Residual Convolutional Block

In order to avoid the degradation of the learning and training ability in the process of the model training, RCB residual blocks are constructed on the basis of the residual network in combination with the literature [19,20,21] to ensure that network performance is improved while the number of layers of the network model is continuously deepened. Figure 4 shows the comparison between the general convolutional neural network and the network with the residual mechanism, where Figure 4a represents the operation of the common convolutional network. After obtaining the input of the model, simple convolution, batch normalization, and superposition of the non-linear activation are carried out. In Figure 4b, the introduction of the network residual mechanism increased the jump connection and strengthened the characteristic information of the weakening. For the two steps of the convolution operation of the input information, that on the left side is the same as the ordinary convolution network, and the right enters information for the shallow convolution; then, the convolution information can be obtained by superposition of the two parts. Finally, the Relu activation function can be that of an RCB output module.

3. Results

In this section, we will describe the procedure of the experiment and the discussion of the results.

3.1. Experiments

In this part, we conducted experiments on the collected blast furnace tuyere video dataset and compared the ESERNet model with the SERNet proposed in [22], the repVGG proposed in [23], and the ResNeXt model proposed in [24] in the training set and validation set for comparison. This paper compared the models through two aspects: one was to use the number of parameters for comparison to compare the complexity of the model; the other was to compare the average running time of the different models; the third was to compare the classification results with the average accuracy rate, which was defined as shown in Formula (1),

$$\overline{A}=\frac{1}{n_c}{{\displaystyle \sum }}_{i=1}^{n_c}\frac{n_{ii}}{n_i},$$

(1)

where $n_c$ represents the total number of sample categories, and we set it as six in our work; i is the category label, which is selected from 1 to 6. $n_i$ represents the total number of samples of category i; and $n_{ii}$ represents the total number of accurate predictions in class i.

3.1.1. Dataset and Preprocessing

In the early stage of the abnormal detection of the blast furnace tuyere, the videos were shot with a tuyere camera purchased from Beijing Shenwang Technology Co., Ltd. (Beijing, China) The camera device has a built-in CCD camera, which is placed outside the furnace mouth to take pictures of the situation inside the furnace. The blast furnace tuyere camera device is shown in Figure 5. The 167 videos, with a total length of about 26.5 h, of the blast furnace tuyere were collected. The videos were cut into pictures by frame, and the pictures were added with classification labels according to different abnormal states. The status of the blast furnace tuyere at work is mainly divided into six categories [7].

• Large chunks falling: large areas of darkening appear at the air outlet, and the large chunks are melted and disappear by the blast furnace, and the air outlet returns to brightness;
• Material block: gray coke in the blast furnace at the wind outlet for winding movement;
• Normal: the edge of the tuyere is smooth; there are no impurities in the tuyere and no dark area, and the dark area of the coal gun and coal injection is clearly visible;
• Coal breaking: the tuyere boundary is smooth, the tuyere is clear and bright, and there is no dark area produced by the coal injection;
• Slag hanging: there is uneven residue at the edge of the tuyere, and there is a small dark area at the slag hanging, and the gray level is large;
• Wind off: the coke rotation speed is slow, the carbon block gradually accumulates, and the wind is not bright and gradually darkens.

The categories of the blast furnace tuyere images are shown in Figure 6.

The uneven amount of data obtained by each category will affect the training effect of the model. In order to obtain a better training effect, the following operations should be performed on the data before the model input [25].

• Expand the data to make the model have a strong generalization ability and to avoid over-fitting. The images are randomly rotated and flipped, and slight changes are added to enrich the dataset;
• In order to reduce the impact of image noise, the region of interest (ROI) of the image is intercepted;
• The image input size is 256 × 256, which reduces the calculation amount of the model and speeds up the calculation speed.

Therefore, the image dataset of the blast furnace tuyere after screening and expansion contains 21,120 pictures, including 3520 pictures of the falling block, material block, normal, broken coal, hanging slag, and rest wind, which are used for training and testing in the experiments.

3.1.2. Experimental Parameter Setting

In order to avoid memory overflow, the ESERNet model with SERNet, repVGG, and ResNeXtare were involved for performance comparison, in which the same training set and verification set were used. The model is built by pytorch and the experiment was carried out on pycharm. The number of iteration rounds was set to 100, and the loss function was CrossEntropyLoss. Stochastic gradient descent (SGD) was selected as the optimization algorithm. The setting of the batch size and learning rate was carried out by the grouping and combination method. Among them, the batch sizes were set to 16, 32, and 64, and the learning rates were set to 0.1, 0.01, and 0.001, which formed a total of nine combinations. At the same time, the results of the dataset training under different partitioning methods fluctuated greatly, and the average results obtained from the repeated experiments could obtain a good approximation to the generalization results. Therefore, this paper adopts k-fold cross-validation to select the parameters, improve the accuracy of the model evaluation, and finally determine the optimal parameter combination. The model structure and parameters are used to classify the abnormal state of the blast furnace tuyere images.

In this paper, k is set to five, which means that there are five folds in the cross-validation. The dataset is equally divided into five parts, four parts are used as the training set and one part is used as the validation set. The experiment is run five times as a whole, and the average value of the five validation results is taken as the validation error of this model. The k-fold cross-validation results are shown in Figure 7. The results suggest that in the ESERNet model, when the batch size is 16 and the learning rate is 0.1, the average accuracy reaches 96.73%, which yields a best performance; when the learning rates are 0.01 and 0.001, the average accuracy decreases significantly. When the batch size was set to 32 or 64, the average accuracy of the predictions was the highest when the learning rate was 0.1, while when the learning rates were 0.01 and 0.001, the average accuracy decreased significantly and finally reached about 50%. The SERNet and repVGG models are consistent with the ESERNet model. In the ResNeXt model, when the batch size is 16 and the learning rate is 0.01, the average accuracy of the model is the highest, while when the learning rates are 0.1 and 0.001, the average accuracy decreases significantly. When the batch sizes are 32 and 64, the average accuracy rate is the highest when the learning rate is 0.1, while the average accuracy rate decreases significantly when the learning rates are 0.01 and 0.001. The optimal parameter combination of each model is shown in

Table 1. The Optimal Parameter Combination of Different Models.

Model	Batch Size	Learning Rate	Average Accuracy
ESERNet	32	0.1	96.73%
SERNet [22]	16	0.1	97.01%
repVGG [23]	32	0.1	96.74%
ResNeXt [24]	32	0.1	96.69%

. The batch size of the ESERNet model is 32, the learning rate is 0.1, and the average accuracy rate is 96.73%; the batch size of the SERNet model is 16, the learning rate is 0.1, and the average accuracy rate is 97.01%; the batch size of the repVGG model is 32, the learning rate is 0.1, and the average accuracy rate is 96.74%; the batch size of the ResNeXt model is 32, the learning rate is 0.1, and the average accuracy rate is 96.69%. In contrast, the SERNet model has the best effect. Due to the introduction of the squeeze-and-excitation and residual modules, the model can better learn the hidden information in different data so as to obtain better results; the ESERNet introduces the squeeze-and-excitation and residual modules, but due to the reduction in the number of channels, the complexity of the model is reduced, and the ability of the model to extract features is also slightly reduced.

3.2. Discussion of Results

In this part, we analyze the results of the experiment and discuss the accuracy, the number of parameters, the average running time of the models, and the classification of the various anomalies.

3.2.1. Comparison of Different Indicators of the Models

Different models adopt their own optimal parameter combination training models. In Figure 8, the average classification accuracy of the datasets on the different models and the number of model parameters are illustrated. The results show that the ESERNet model proposed in this paper has little difference in classification accuracy compared with the other models in our dataset of the images of blast furnaces, but the number of parameters is reduced by 6.23~56.71%. This is because channel fusion is carried out in the SCB module of the ESERNet model, and the number of channels is halved, leading to a significant reduction in the number of model parameters. The accuracy of the ESERNet model is only 0.28% lower than that of the SERNet model, and the number of parameters is reduced by 26.62%, which not only achieves higher classification accuracy, but also greatly reduces the model complexity. Compared with the repVGG model, although the average accuracy of the ESERNet model decreases by 0.01%, the number of parameters decreases by 56.71%. Compared with the ResNeXt model, the ESERNet model improves the classification accuracy by 0.04% and reduces the number of parameters by 6.23%. The average running time per image of the different models is listed in

Table 2. Average Running Time of Different Models.

Model	Average Running Time (ms)
ESERNet	2.94
SERNet	3.32
repVGG	4.13
ResNeXt	3.63

, and the results show that our proposed model has the lowest average running time. Because of the reduction in the model complexity, the running time of the model is reduced, and the performance of the model is improved. Overall, ESERNet provides a significant improvement in model performance as well as a competitive classification accuracy. For the classification of each state of the blast furnace tuyere, the ESERNet model has the best results in the categories of hanging slag and broken coal, as shown in

Table 3. Classification Accuracy of Blast Furnace Tuyere Images by Different Models.

Image Type	ESERNet	SERNet	repVGG	ResNeXt
Wind off	99.09%	100.00%	100.00%	98.01%
Slag hanging	100.00%	100.00%	100.00%	100.00%
Coal breaking	100.00%	100.00%	100.00%	100.00%
Large chunks falling	96.96%	96.88%	98.07%	97.36%
Material block	93.52%	93.41%	93.41%	93.41%
Normal	97.53%	96.90%	96.90%	97.51%

. The data in the box in the table represent the best accuracy of the corresponding image types. They can obtain the best classification performance in all the other categories except for the rest wind and falling bulk. The accuracy of slag hanging in Table 3 is entirely 100% because the image features of this class are better captured, and the accuracy of each model is better in this class; the three similar images are material block, large chunks falling, and normal for the model; it may be difficult to extract the features of these three types of images, but an accuracy rate of more than 90% can still be achieved.

3.2.2. The Predicted Results of the Model

This article lists the confusion matrices for the ESERNet, SERNet, repVGG, and ResNeXt models on the test set to show the specific classification of each model. As shown in Figure 9, each column of the confusion matrix represents the prediction category, and the cumulative quantity of each column represents the number of predicted categories. Each line represents the real category of data, and the cumulative quantity of each line represents the total quantity of this category. The value at the intersection of the columns and rows represents the number of predicted classes, and the diagonal represents the number predicted correctly. The results in Figure 9 show that the image of the material block and the large chunks falling is not easily distinguished, and the material block is as easy to identify as normal or large chunks falling, and large chunks falling is as easy to identify as the material block. This is because, for the steelmaking process, it is not easy to determine the boundary of the block size; it is not easy to judge the early stage of block falling, but for the middle stage of large chunks falling and material block, the classification accuracy of the four models is high. The ResNeXt model achieves the best accuracy in the classification of coal breaking, but the prediction effect on the wind off is not obvious. The ESERNet model can achieve the best accuracy in coal breaking, slag hanging, and wind off, and other categories can also achieve a high recognition rate, indicating that the ESERNet model can provide a reference for the anomaly detection and classification of the blast furnace tuyere images.

4. Conclusions

In this paper, a recognition algorithm for the abnormal state of the blast furnace tuyere based on the attention residual mechanism is proposed. It is improved on the basis of channel attention, combined with the residual mechanism; the ESERNet model is constructed to detect and classify the abnormal state of the blast furnace tuyere images and is compared with the SERNet, repVGG and ResNeXt models. In our dataset of images of blast furnaces, the results show:

• The ESERNet model greatly reduces the number of parameters, which are 6.23–56.71% less than the other three models, and the average running time of ESERNet is the least; ESERNet greatly reduces the model complexity;
• On the basis of reducing the complexity of the model, the model has high classification performance, and the recognition accuracy reaches 97.10%. The best accuracy of the model is achieved in the classification of coal breaking, wind off, and slag hanging. In the classification of coal breaking, slag hanging, material block and normal, the classification accuracy is the highest compared with the other models.

At present, the model has solved the problem that the differences of the tuyere anomaly category are too small and difficult to classify and has realized the intelligent identification of the tuyere state with improved model performance. The limitation of the method is that we only apply it well to our own collected datasets, with high accuracy, but when migrating to a new dataset, the model may need to be retrained because the resolution of the sensors is different and to prevent different locations. All we need to do is retrain the model with the new dataset. Although retraining is required, the training time is not long and does not necessarily affect the applicability of the method. In addition, the method we proposed slightly reduces the accuracy, and we will further improve the accuracy of the model.