Research on Improving Denoising Performance of ROI Computer Vision Method for Transmission Tower Displacement Identification

Abstract

The health monitoring technology of transmission towers based on vibration data had become a research hotspot. At present, vibration data mainly relied on sensors installed on the tower, which was time-consuming and laborious. Nevertheless, the ROI computer vision method could achieve long-distance, multi-point, and non-contact monitoring, which offers a new possibility for the structure-safety identification of power transmission towers. However, transmission towers are generally located in the field environment, and the background is complicated, resulting in the ROI key point method for vibration data acquisition encountering various types of noise. Thus, the key point in practice was clearing the noise and reducing the impact of noise on identification accuracy. The subpixel corner method was used to detect a minor error with the research object of pixel sets. The dilation + erosion method could reduce image noise. Under white noise with a variance of 0.05, the dilation + erosion could reduce average error (E_mae) and mean square error (E_mse) by 27% and 23% and increase percentages of data with absolute error less than 5 mm and 10 mm in the total number of data (σ₅ and σ₁₀) by 8% and 4.3%, respectively, which was compared to median filter + sharpen. The histogram equalization method was used to balance background lighting conditions and reduce identification errors from non-uniform illumination. E_mae and E_mse were reduced by 92% and 99%, and σ₅ and σ₁₀ were increased by 5 and 3 times, respectively, and the identification time was cut by 62% with the histogram equalization method. Under white noise with a variance of 0.15 or lower, the three methods combined increased the numerical stability of E_mae, E_mse, σ₅, and σ₁₀, which indicated that the combination of the three methods could improve the anti-noise performance, robustness, and identification accuracy of the ROI computer vision method for transmission tower displacement identification.

1. Introduction

According to the statistics in the National Electricity Reliability Report 2021 released by the National Energy Administration, the length of transmission line circuits of 220 kV and above nationwide was 814,000 km as of 2021, with an average annual growth of 35,000 km in the past three years. According to the calculation of 2.3 transmission towers per kilometer of transmission lines by 2021, the total number of 220 kV and above towers in service nationwide was about 1.87 million. The transmission towers built in the early stage are already in end-of-life or overdue service. Assessing the health of in-service transmission towers had become a research hotspot. The traditional transmission tower health monitoring method was to collect vibration data through sensors installed on transmission towers, which were detected by a data acquisition and analysis system on the ground [1,2,3]. The traditional method had many problems in data collection, such as damage to the transmission tower during the installation of sensors and the risk of falling and electrocution for the installer. Meanwhile, this method was not convenient for operation and maintenance, and the sensors need to be replaced regularly, which was time-consuming and labor-intensive. Therefore, it is necessary to urgently address the selection of a non-destructive and rapid tower health inspection method that could evaluate the existing towers, which could save monitoring costs based on the operational safety of transmission lines in the meantime.

The method of structural displacement detection based on computer vision had the advantage of long-distance, multi-point, and contactless monitoring [4,5]. In the field of crack identification and monitoring [6,7,8], the features of concrete, geological, and other surface cracks were identified by monocular or binocular cameras, and data analysis was performed using compressive sensing, deep learning, finite element, and other methods to carry out structural health inspection research, which had excellent benefits. The ROI (region of interest) method is based on computer vision monitoring, which could achieve long-distance transmission tower structural displacement monitoring without targets. This method took the inner contour of the transmission tower structure as the ROI, combined the SUZUKI contour tracking algorithm [9] and the Harris corner point detection algorithm [10] to extract and track the key points, and then obtained the displacement changes of the key points of the transmission tower. However, the actual transmission tower was located in a harsh environment with complex backgrounds such as clouds, trees, houses, and non-uniform illumination caused by different sunlight irradiation angles that would affect the recognition accuracy of the method. Thus, it was the key to reducing the impact of noise on recognition accuracy and whether the method could be successfully applied in practice.

In the field of image noise removal, the frequently used methods were the subpixel corner point method [11], median filtering [12,13], Gaussian filtering [14], sharpening [15], expansion and erosion [16], deep learning [17], histogram equalization [18,19,20,21], etc. Among these methods, the median filtering method was the most commonly used, including single-threshold, double-threshold, adaptive thresholding, and other algorithms, which could achieve better noise-removal effects in Gaussian noise and pretzel noise cooperating with the image sharpening method. The sharpening method was mainly based on component replacement, multi-resolution analysis, variational optimization, and deep learning, as well as other methods, which were fast and simple to implement but prone to aberration, distortion, and other problems. The deep learning method was an emerging noise-removal technology with better results. However, it was based on neural network training, which required more computational power and took a longer time, so it could not be widely used. Obviously, various noise-removal methods had shortcomings and could not be well combined with ROI target-free computer vision monitoring methods. Therefore, finding a noise-removal method that could effectively improve the robustness, computational speed, and generality was very valuable, and the method could achieve long-distance transmission tower structure displacement monitoring.

Image noise-removal methods were mostly carried out using a single method, which can achieve better results under specific noise conditions. However, the noise environment was often complicated in reality, because of which using a single method could not meet the detection accuracy requirements. The subpixel corner point method could reduce the pseudo corner point recognition error. The expansion and erosion method was suitable for many image processing fields, such as noise removal and edge detection, which was based on convolution operation. The histogram equalization method was more applied in the fields of fog removal and noise reduction under low-light conditions; this method was effective for grayscale image processing. This study analyzed the advantages of a subpixel corner point, expansion + erosion, and histogram equalization methods and discussed the effect of the combined use of multiple noise-reduction methods on the noise immunity performance of ROI computer vision monitoring methods under comprehensive noise conditions, which aimed to improve the recognition accuracy of transmission tower structure displacement monitoring in noisy environments.

2. ROI Key Point Method

The ROI key point method was a scale-free computer vision recognition method, based on the internal contour characteristics of transmission tower structure for displacement monitoring. The main steps of the ROI key point method include ROI extraction, key point detection, coordinate conversion, etc.

2.1. Extraction of the Region of Interest

Firstly, image pre-processing was carried out, which included grayscale and binarization. The grayscale conversion method [22] is shown in Equation (1), and the binarization used the OTUS algorithm, whose expression is shown in Equation (2).

$$GRAY=B\times 0.114+G\times 0.587+R\times 0.299$$

(1)

where B, G, and R denote the color intensity of the blue, green, and red pixels.

$${\sigma }^2=\frac{{(mG\times p_1-m)}^2}{p_1\times (1-p_1)}$$

(2)

where p₁ denotes the probability that the gray level is greater than k, mG denotes the global mean, and m denotes the cumulative mean.

Identify the contours of binary images to calculate the center of gravity position, perimeter, area, and tightness of all contours, which used the SUZUKI identical contour tracking algorithm (Figure 1).

The N-nearest neighbor minimum energy search method was adopted to extract the contour region of interest, which was used to calculate the difference with the previous frame as the energy E_i by taking the position of the gravity center of the inner contour P_i, area A_i, and tightness T_i as geometric features, as shown in Equation (3). The contour with the minimum energy was the ROI contour.

$$E_i=\alpha \Delta (P_i^{t_0},P_i^{t_0-1})+\beta \Delta (A_i^{t_0},A_i^{t_0-1})+\gamma \Delta (T_i^{t_0},T_i^{t_0-1})$$

(3)

where α, β, γ denote the weights of the contour’s position, the weights of size, and the weights of tightness.

2.2. Corner Detection

The Harris corner detection algorithm was used for corner detection, and the corner detection results are shown in Figure 2.

2.3. Coordinate Conversion

The pixel difference d was calculated frame by frame according to the difference in the position of the key corner pixels. The coordinate conversion was performed by calculating the scale factor R between the pixels and the actual length based on the known structure size D, as shown in Equation (4).

$$R=\frac{d}{D}(\mathrm{pixels}/\mathrm{mm})$$

(4)

where R is the scale factor, d is the pixel difference, and D is the actual distance.

3. Research on Image Denoising Method

3.1. Experimental Design

A model of a transmission tower pedestal was constructed with a laser sensor, model HL-G112-A-C5, mounted on the side of the model (Figure 3). The vibration displacement of the model was monitored at a frequency of 1000 Hz, which would be used as a comparison group. Meanwhile, a camera was placed in front of the model to capture the vibration video, with a recording frequency of 60 fps and a resolution of 4096 × 2160. Adopt the ROI key point method for displacement monitoring, and the selected experimental samples were HS-33 (normal) and HS-42 (non-uniform illumination). The artificial noise was white noise with a mean value μ of 0, and the variance σ was selected according to the noise magnitude.

3.2. Main Sources of Errors

(1) Pseudo corner point error

When applying the ROI key point method displacement to monitor the sample HS-33, results were obtained as shown in Figure 4; furthermore, the tiny bumps that appeared at certain locations in the figure were analyzed. Intercepting the corner point detection results of this location and adjacent frames, the following problems were found.

① Noise appears near the key corner points at this location, as shown in Figure 5a, which can lead to detection errors.

② As shown in Figure 5b–d, the key corner point is near a collection of pixels, and there is a pixel difference in different images. Therefore, taking the position of the rightmost pixel as the recognition corner point will generate 1- to 2-pixel errors.

(2) Non-Uniform Illumination Error

In the real environment, transmission towers were often subjected to non-uniform light intensity. To better simulate the real environment and study the influence of lighting conditions on the displacement recognition accuracy of the ROI key point method, this study selected the asymmetric lighting experimental sample HS-42 as the research object for contour recognition and applies the white noise of σ = 0.05 to intervene it.

As shown in Figure 6b,c, under the non-uniform illumination conditions, the contour recognition results were irregular, mismatched with the actual contour edges, and some contours were lost. When the white noise of σ = 0.05 is applied, the contour recognition results were even more unsatisfactory. It could be seen that the influence of non-uniform illumination on the accuracy of contour recognition cannot be ignored.

As shown in Figure 7, under the condition of the applied white noise of σ = 0.05, the displacement analysis curve and the laser measurement curve showed a large difference. The average error reaches 46 mm, and the mean square error reaches 1786 mm. Thus, the recognition error increased substantially in the presence of both noise and non-uniform illumination.

3.3. Subpixel Corner Point Detection

In order to solve the pseudo corner point error problem, the neighborhood concatenation and subpixel corner point detection [8] methods were chosen. Neighborhood concatenation was to link the pixels adjacent to the corner points of the previous frame to form a pixel set. Subpixel corner point detection calculates the subpixel point that minimizes the squared sum of the distances of all pixels in the pixel set based on the least squares theory, which was used for the displacement calculation of corner points.

$$E={\displaystyle \sum _{i=1}^n{k}_i{s}_i}$$

(5)

where E denotes the sum of the squares of the distances between pixels and valid corner points in the pixel set, k_i denotes the weights, and s_i denotes the square of the distances between pixels and valid corner points.

Applying the subpixel corner point detection method to the pixel set avoided the selection of pseudo corner points as key points, which effectively reduced the pixel error and improved the measurement accuracy.

3.4. Histogram Equalization

The histogram equalization method was used to balance the background lighting conditions, which could reduce the recognition errors caused by non-uniform lighting conditions. The histogram was a function of the image’s gray level, which was defined as shown below.

$$P(r_k)=\frac{n_k}{n},k=0,1,\cdots ,L-1$$

(6)

where r_k is the k-level gray value, n_k is the number of pixels with a gray value of r_k, and n is the total number of pixels.

In the case of non-uniform illumination, the grayscale values of different edges of the image exhibited large differences. Then, because the binarization method was based on grayscale values, edge information would be lost, and the result of contour recognition would be inaccurate.

Histogram equalization was to correct the image histogram to a uniform function by transforming the function T, as shown in Equation (7), which converts pixels with the gray level of r_k in the original image to pixels with a gray level of s_k in the new image by the cumulative distribution function. Histogram equalization equalizes the gray value of the whole image while retaining the local gray difference, which strengthened the image edges and enhanced the contour recognition accuracy.

$$s_k=T(r_k)={\displaystyle \sum _{k=0}^{L-1}P(r_k)}={\displaystyle \sum _{k=0}^{L-1}\frac{n_k}{n}}$$

(7)

The comparison of the grayscale image before and after histogram equalization was shown in Figure 8, and the comparison of the binary image is shown in Figure 9. It could be seen that the overall uniformity of the grayscale image was better and that the edge details were more obvious after the histogram equalization process. The binary image before processing was non-uniform to the left and right, and there were a large number of missing edges. After histogram equalization, the edges of the binary image were obvious and prominent, which was beneficial to contour recognition in the later stage.

To quantitatively analyze the influence of the noise-removal method on the displacement recognition results, four error indicators were selected for analysis, which was the average error (E_mae), the mean square error (E_mse), and the percentage of data with absolute error less than 5 mm and 10 mm compared to the total number of data (σ₅ and σ₁₀). The error indicators were calculated as shown below.

$$E_{m\mathrm{ae}}=\frac{1}{n}{\displaystyle \sum _{i=1}^n(\left|u_v(i)\right|-\left|u_l(i\right|)}$$

(8)

$$E_{mse}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{(\left|u_v(i)\right|-\left|u_l(i\right|)}^2}$$

(9)

where u_v denotes the displacement calculated by the ROI key point method, and u_l denotes the displacement by laser ranging.

The results of the error analysis are shown in

Table 1. Error indicators (histogram equalization).

Noise	Classification	${\mathit{E}}_{\mathit{m}\mathit{a}\mathit{e}} \left(\mathbf{mm}\right)$	${\mathit{E}}_{\mathit{m}\mathit{s}\mathit{e}} \left(\mathbf{mm}\right)$	${\mathit{\sigma }}_5$	${\mathit{\sigma }}_{10}$
No noise	No treatment	3.11	5.62	0.86	0.99
	Histogram equalization	2.79	4.85	0.85	0.99
$\sigma =0.05$	No treatment	45.46	1786.30	0.12	0.23
	Histogram equalization	3.62	8.06	0.74	0.97

. Under the no-noise condition, the histogram equalization method reduced E_mae and E_mse by 10% and 13%, respectively, but the difference between σ₅ and σ₁₀ was not significant. Under the noise ofthe σ = 0.05 condition, E_mae and E_mse were reduced by 92% and 99%, respectively, and σ₅ and σ₁₀ were improved by five times and three times, respectively. Consequently, the use of the histogram equalization method could largely reduce the error, eliminate the influence of noise, and improve the accuracy of displacement monitoring.

Under the no-noise condition, the recognition errors were 0.14 s/frame and 0.16 s/frame before and after using the histogram equalization method, and the difference between them was not significant. Under the white noise condition, with σ = 0.05 imposed, the recognition time was reduced from 0.84 s/frame to 0.32 s/frame by using the histogram equalization method, which was 62% lower. Obviously, the histogram equalization method could effectively improve the calculation speed.

Generally, due to non-uniform illumination, noise, and other factors, the process of transmission tower displacement monitoring would generate a large number of pseudo contours and pseudo corner points, resulting in slowing down the calculation speed. Nevertheless, the histogram equalization method effectively avoided or reduced the generation of pseudo contours and pseudo corner points and improved the calculation efficiency.

3.5. Expansion + Erosion

We compared the applicability of two methods, expansion + erosion, and median filtering + sharpening, in the application of the ROI key point method for displacement monitoring, using the original video and the video with the addition of white noise as the research object. Expansion and erosion were a pair of operations that were usually used together. The principle was the addition and subtraction of vectors, and the operations are shown in Equations (10) and (11).

$$X\oplus B=\{p\in {\epsilon }^2,p=x+b,x\in X,b\in B\}$$

(10)

$$X\Theta B=\{p\in {\epsilon }^2,p+b\in X,\forall b\in B\}$$

(11)

The results of the Harris corner point detection of the transmission tower were subjected to expansion + corrosion operation. Under the original video conditions, the transmission tower model background was a wall, which is relatively single and less noisy. The effects before and after the expansion + corrosion treatment are shown in Figure 10a,b. Compared with the median filtering + sharpening method, the noise was significantly reduced by applying the expansion + erosion method, and the contour corner points in the region of interest were more obvious, as shown in Figure 11a.

Under the condition of white noise with mean μ = 0 and variance σ = 0.05 applied to the image, the corner points, processed by the expansion + erosion method, were more clearly as shown in Figure 10c,d. The corner points processed by the median filtering + sharpening method had the situation that the contour corner points in the region of interest were mistakenly deleted, and the error was larger, as shown in Figure 11b.

Four error indicators E_mae, E_mse, σ₅, and σ₁₀, were selected for the quantitative analysis of the identification before and after treatment with the expansion + corrosion method.

As shown in

Table 2. Error indicators (dilation and erosion).

Noise	Classification	Emae (mm)	Emse (mm)	σ5	σ10
no noise	expansion + corrosion	2.90	5.25	0.86	0.99
	median filtering + sharpening	2.95	6.29	0.82	0.98
σ = 0.05	expansion + corrosion	3.09	6.62	0.81	0.96
	median filtering + sharpening	4.26	8.62	0.75	0.92

, the expansion + corrosion method had certain advantages over the median filtering + sharpening method in both the conditions of no noise and applied white noise of σ = 0.05. Especially under the increased-noise condition, the error indices E_mae and E_mse were reduced by 27% and 23%, and σ₅ and σ₁₀ were improved by 8% and 4.3%, respectively. It could be seen that the expansion + erosion method was more effective in noise removal when using the ROI key point method for transmission tower structure displacement monitoring.

4. Combined Application of Noise-Resistance Methods

Combined subpixel corner point, expansion + corrosion, and histogram equalization methods to increase the recognition accuracy of the ROI computer vision transmission tower structure displacement monitoring method. To verify its performance, the experimental sample HS-42 with asymmetric illumination was studied, and white noise of σ = 0.05, σ = 0.1, σ = 0.15, and σ = 0.2 were applied sequentially, and the video recognition results were compared and analyzed.

A deep convolutional neural network approach (CNN) was used for noise immunity comparison. The training device parameters were i7-12700H (2.30 GHz), 32 GB RAM, and GeForce RTX 3060 (6 GB) graphics card. The network model was a VGG residual model with 28 network layers and 64 convolutional kernels, and the optimizer was the stochastic gradient descent method (SGD), and the activation function is Leaky ReLU. A total of 1000 images were selected for each noise; the image resolution was 3840 × 2160; the images were segmented by 128 × 128 pixels, and the edge part was filled with 0, resulting in a total of 1,224,000 samples, of which 80% were taken as the training set and 20% as the test set for network parameter training, 300 rounds of training, with a training time of about 108 h and an accuracy of 94.2%.

As shown in Figure 12, when the condition of asymmetric illumination and white noise less than σ = 0.15 was applied, the displacement analysis curve of the combined noise-resistance method fit well with the laser measurement curve, which could better reflect the actual displacement situation and had better noise immunity and robustness. Under the condition of white noise of σ = 0.2, the displacement analysis curve of the combined noise-resistance method had a large difference from the laser measurement curve.

The results of the quantitative analysis are shown in

Table 3. Error indicators (performance).

Noise	Emae (mm)	Emse (mm)	σ5	σ10	t (s/Frame)
no noise	2.79	4.85	0.85	0.99	0.16
σ = 0.05	3.62	8.06	0.74	0.97	0.32
σ = 0.10	4.26	9.13	0.72	0.92	0.60
σ = 0.15	6.26	11.08	0.69	0.89	0.71
σ = 0.20	52.10	1914.50	0.12	0.20	0.72
σ = 0.05 (CNN)	4.69	9.04	0.71	0.93	\
σ = 0.10 (CNN)	6.21	10.58	0.67	0.87	\
σ = 0.15 (CNN)	7.52	12.36	0.65	0.81	\
σ = 0.20 (CNN)	12.35	18.25	0.42	0.51	\

. With the increase in noise intensity, E_mae and E_mse of the combined noise-resistance method gradually increased, and σ₅ and σ₁₀ gradually decreased. Under the condition of white noise with a variance less than 0.15, the variation of error indicators was small, and the accuracy of displacement monitoring results could be guaranteed; compared with the CNN method, the error metrics were not significantly different. Still, the CNN method required a longer time for network training using dedicated equipment, which must remove the noise from the whole sample before performing displacement recognition. Thus, the CNN method could not display the results promptly, and the time parameters are not included in Table 3.

However, the error increased substantially, and the monitoring accuracy decreased when the white noise variance reached 0.2, which was due to the large amount of interference noise superimposed near the corner points, and the real corner points would be mistakenly deleted when the combined noise-resistance method was used. However, the use of the CNN method could avoid such problems.

5. Conclusions

In this study, a computer vision monitoring method based on the region of interest (ROI) was implemented for target-free and long-range transmission tower structure displacement monitoring. Meanwhile, the noise-removal characteristics of a subpixel corner point, expansion + corrosion, and histogram equalization methods were analyzed, and a combined noise-resistance method was proposed, which had the following advantages through experiments and results analysis.

(1) The subpixel corner point method could reduce the recognition error caused by pseudo corner points. The expansion + erosion method could improve the accuracy of displacement recognition. The histogram equalization method could effectively reduce the displacement recognition error caused by uneven illumination and improve recognition accuracy.

(2) The combined noise-resistance method had better recognition accuracy, better noise immunity, and robustness in the complex noise environments of asymmetric illumination superimposed on white noise, which also had higher applicability in the use of the ROI key point method for transmission towers based on computer vision.