Detection and Imaging of Corrosion Defects in Steel Structures Based on Ultrasonic Digital Image Processing

Abstract

Corrosion is one of the critical factors leading to the failure of steel structures. Ultrasonic C-scans are widely used to identify corrosion damage. Limited by the range of C-scans, multiple C-scans are usually required to cover the whole component. Thus, stitching multiple C-scans into a panoramic image of the area under detection is necessary for interpreting non-destructive testing (NDT) data. In this paper, an image mosaic method for ultrasonic C-scan based on scale invariant feature transform (SIFT) is proposed. Firstly, to improve the success rate of registration, the difference in the probe starting position in two scans is used to filter the matching pairs of feature points obtained by SIFT. Secondly, dynamic programming methods are used to search for the optimal seam path. Finally, the pixels in the overlapping area are fused by fade-in and fade-out fusion along the seam line. The improved method has a higher success rate of registration and lower image distortion than the conventional method in the mosaic of ultrasonic C-scan images. Experimental results show that the proposed method can stitch multiple C-scan images of a testing block containing artificial defects into a panorama image effectively.

1. Introduction

Corrosion is a common kind of defect in steel structures and is one of the main direct or indirect causes of structural failure [1,2]. A common case is the corrosion of buried water pipelines. Water pipeline corrosion directly harms the pipe itself, contributing to longitudinal stress, metal loss, pipe wall perforation, and reductions in service life, as well as cathodic or coating disbanding and hydrogen embrittlement [3]. Another common case is corrosion in structural steel due to exposure to moisture and oxygen. The annual cost due to the corrosion of steel structures, especially at coastal sites, can be tremendous [4]. There are different types of corrosion in steel, such as general, galvanic, bimetallic, embedded, atmospheric, and cut-edge corrosion. Correctly determining the spatial distribution of steel corrosion is critical for estimating the remaining service life of deteriorating steel structures.

Currently, non-destructive techniques such as ultrasonic inspection [5,6], radiography [7], thermography [8], optical inspection [9], etc. have been employed to monitor and identify the corrosion damage in the steel structures. In the detection of general corrosion that causes large area thickness loss in metal, phased array ultrasonic testing (PAUT) has many advantages, including increased flaw detection ability, increased inspection speed, and reliability of results. PAUT has been widely used for the in-service detection and characterization of corrosion in pipes, tanks, vessels, and other critical assets [10,11,12]. Due to their larger footprint, PAUT probes can cover a larger surface at higher speeds, leading to a significant time reduction and enhanced resolution.

The observation of corrosion defects by PAUT is divided into two parts: the outline and the depth. The outline of corrosion can be observed by the color change in the C-scan image, and the depth of corrosion can be determined by the arrival time of the ultrasonic signal. Ultrasonic C-scan uses a projection of the ultrasonic data onto a plan view of the component being tested to create an image, which is the most used view in phased array corrosion mapping. However, in the detection of large-area general corrosion by PAUT, limited by the range of a single C-scan, multiple C-scans are usually required to cover the whole component. Stitching multiple partially overlapped C-scan images into a panorama image brings great convenience to interpreting NDT data.

Image mosaicing is an effective means of constructing a single seamless image by aligning multiple partially overlapped images [13]. Image mosaic is divided into two steps: image registration and image fusion. Image registration is a process that combines different images obtained from different times, different views, or different sensors altogether [14]. After image registration, the original images are placed in the same coordinate system. In the stage of image fusion, the images are merged to form a panorama image. At present, point-feature-based registration is the most widely used method in the mosaic of NDT images, in which the representative scale invariant feature transform (SIFT) algorithm was proposed by David G. Lowe in 1999 [15] and upgraded in 2004 [16]. The features extracted by SIFT are invariant to image scale and rotation and are shown to provide robust matching across a substantial range of affine distortion, change in 3D viewpoint, addition of noise, and change in illumination.

The SIFT algorithm has been used in the registration and mosaic of optical images [17], synthetic aperture radar (SAR) images [18], microscopic images [19], remote sensing images [20,21], and other fields. And SIFT is also commonly used to stitch NDT images. Huang et al. [22] proposed an improved SIFT image mosaic algorithm to stitch the optical images of printed circuit board bare plates. The accuracy and efficiency of image mosaics were effectively improved by designing a block strategy and using the similarity criterion. Zhao et al. [23] used the improved SIFT algorithm to stitch multiple videoscope images of the inner wall of the pipeline. The videoscope image of the inner surface of the pipeline was completely reconstructed by this method.

However, because of the low image resolution, random defect distribution, and image error caused by probe jitter and coupling conditions in ultrasonic NDT images, it is difficult for conventional image mosaic algorithms to stitch ultrasonic NDT images. Consequently, it is necessary to introduce other methods to improve the robustness of the SIFT algorithm. Qiu et al. [24] combined features from the accelerated segment test (FAST) operator and SIFT descriptor to detect and describe image features. The fast library for approximate nearest neighbors (FLANN) algorithm with adaptive threshold and the improved random sample consensus (RANSAC) algorithm were used to remove the mismatching points. The speed and accuracy of feature matching were obviously improved. Teng et al. [25] proposed a SIFT-based technique that was modality-invariant and still retained the strengths of local techniques. Histogram weighting strategies were introduced to improve the accuracy of descriptor matching. The results showed that the method could not only improve multimodal registration accuracy but also had the potential to improve the performance of all SIFT-based applications. Hossein-Nejad et al. [26] developed SIFT using adaptive RANSAC. The threshold value of RANSAC exploited the variance of the distances in addition to the mean value. Simulation results confirmed the superiority of the method in terms of the correct matching rate. Hossein-Nejad et al. [27] also proposed a method called clustered redundant key point elimination (CRKEM) to remove the redundancy of key points extracted by SIFT. The experimental results confirmed the superiority of the proposed method in image mosaicing as well as in image registration and matching.

In this paper, a novel image mosaic method based on SIFT is proposed to stitch the ultrasonic C-scan images acquired by PAUT. The matching of feature points is first filtered by the difference in probe starting position between two scans and then further filtered by RANSAC to improve the success rate of image registration. Also, the dynamic programming method and the fade-in and fade-out image fusion methods are introduced to reduce the distortion in the fused image. Multiple C-scan images of a testing block containing artificial defects are stitched into a high-quality panorama image by this method.

2. Principles of SIFT Image Registration

SIFT features are the local image features that remain unchanged during rotation, zooming, and lighting and keep somewhat stable during view changing, affine transformation, and noise emission. The SIFT algorithm is implemented in two phases. The first phase is the generation of SIFT features, namely extracting the feature points irrelevant to scale zooming and rotation from the images to be matched. The second phase is the matching of SIFT feature points.

The SIFT algorithm is implemented in the following four steps: check the extremum of scale space, refine the position of feature points, calculate the descriptive information of feature points, and generate local feature descriptors. Use the Gaussian filter first to filter the original images several consecutive times and establish the first scale group. Then, halve the image size and implement the same Gaussian filtering to form the second scale group. Repeat this process until the image size is smaller than a given threshold. Next, differentiate the Gaussian images in every scale group to obtain the Gaussian difference scale group (DOG) images. The generation of DOG images is shown in Figure 1. Then, extract the local extremums of these Gaussian difference images and obtain the feature points of the images in the scale space domain.

The principal direction assignment of key points is based on the local gradient direction of the image. Using the gradient characteristics and direction distribution of pixels in the neighborhood of key points, the gradient modulus and direction can be obtained as follows:

$$m\left(x, y\right)=\left(I\left(x+1, y\right)-I\left(x-1, y\right)\right)+\left(I\left(x, y+1\right)-I\left(x, y-1\right)\right)$$

(1)

$$\theta \left(x,y\right)=\frac{arctan(I\left(x,y+1\right)-I\left(x,y-1\right) }{I\left(x+1, y\right)-I\left(x-1, y\right)}$$

(2)

A key point descriptor is created by first computing the gradient magnitude and orientation at each image sample point in a region around the key point location. An 4 × 4 arrays of 8 bin histogram is created and shown in Figure 2. The 16 × 16 pixels in the neighborhood are divided into 16 sub-regions with 4 × 4 pixels, and the 128D feature descriptor of SIFT can be obtained by calculating the eight gradient directions of each region. Then, the Best Bin First algorithm is used to search for the nearest neighbor and the next nearest neighbor of a key point. If the searched feature point is E, and P and P′ are the two feature points with the shortest and the second shortest Euclidean distance, the distance ratio Z of EP and EP′ is calculated. If the Z value is smaller than a certain threshold, the feature point pair is matched. The RANSAC algorithm is used to extract the final matching pairs to calculate the transformation matrix. The images will be registered by the transformation matrix.

3. Implementation of Image Mosaic Based on the SIFT Algorithm

The error rate of feature matching in the registration of an ultrasonic C-scan image is high for the conventional SIFT algorithm. Moreover, there is an obvious deviation in the fused image, which brings inconvenience to interpreting NDT data. In this paper, the difference in probe starting position between two scans is used to filter the matching pairs of feature points obtained by SIFT. And then the matching pairs are further filtered by the RANSAC algorithm to obtain the correct transformation matrix. The dynamic programming method is used to search for an optimal seam line, and the pixels of two images are fused by fade-in and fade-out fusion along the seam line. The flow chart of the image mosaic method for ultrasonic C-scan images is shown in Figure 3.

3.1. Acquisition of C-Scan Images

The instrument used in this paper is shown in Figure 4. The test instrument is a Multi 2000 Pocket ultrasonic phased array produced by M2M in France, connected to an E6A2-CW3E rotary encoder produced by OMRON Corporation in Japan to acquire C-scan images. According to the sound velocity, attenuation, and thickness of the testing block, the following parameters are chosen: The central frequency of the probe is 5 MHz. Simultaneously excite 16 of the 64 elements for detection. The focusing law is 0-degree beam steering, and the system gain is 57 dB. The encoder step value is 0.5 mm. The mechanical scanning range is 105 mm. Select the highest signal peak value exceeding the threshold of the gate for C-scan imaging. The width of the C-scan image is 24.8 mm under the above condition. Experimental results show that using the above-mentioned parameters, images of higher quality can be obtained.

In order to imitate the irregular shape of an actual corrosion defect, a testing block containing an artificial defect is made. Metal loss with different depths and contours is made using a milling machine and a milling cutter of 15 mm in diameter. The depths of the irregular stepped area are 1 mm, 2 mm, and 5 mm, respectively. The outline of the defect is randomly selected in the milling process. The testing block containing artificial defects is shown in Figure 5.

The testing block is divided into four parts to be scanned, and the C-scan images for each scan are shown in Figure 6, respectively. The C-scan image generated by PAUT takes the direction of encoder stepping as a horizontal axis and the direction of probe electronic scanning as a vertical axis. In the C-scan image, a pixel region of fixed length and width represents the signal peak of a single A-scan. Hence, the proportion of C-scan image output directly is not consistent with the actual proportion. If the image is compressed according to the actual proportion, the compression process will affect the matching of the feature point. Therefore, C-scan images used to be stitched are directly output. After the image mosaic is all completed, the panorama image will be compressed according to the actual proportion.

3.2. Image Registration

Because the resolution of the ultrasonic C-scan image is low and the texture features are obscure, the matching pairs of features calculated by the conventional SIFT algorithm include a large number of mismatching pairs. To solve this problem, the difference in probe starting position between two C-scans is used to screen the matching pairs. The matching pairs are further screened by RANSAC, and the final matching pairs are used to calculate the transformation matrix. The principle of matching pair filtering is shown in Figure 7.

Set the starting points of the probe in the first and second C-scans to $(x_a,y_a)$ and $(x_b,y_b)$, respectively. There are a total of m matching pairs calculated by the SIFT algorithm, and the corresponding feature point coordinate sets are as follows:

$$\left\{\begin{array}{c}{loc}_a=\left(x_{a1},y_{a1}\right),\left(x_{a2},y_{a2}\right),\cdots \left(x_{ai},y_{ai}\right)\cdots \left(x_{am},y_{am}\right)\\ {loc}_b=\left(x_{b1},y_{b1}\right),\left(x_{b2},y_{b2}\right),\cdots \left(x_{bi},y_{bi}\right)\cdots \left(x_{bm},y_{bm}\right)\end{array}\right.$$

(3)

The formula for matching pair filtering is as follows:

$$\left\{\begin{array}{c}1-k\le \frac{x_{bi}-x_{ai}}{x_a-x_b}\le 1+k\\ 1-k\le \frac{y_{bi}-y_{ai}}{y_a-y_b}\le 1+k\end{array}\right.$$

(4)

where k is the error coefficient allowed in the screening of matching pairs.

Taking the stitching of the first and second C-scan images as an example of image mosaic, the feature point matching pairs of the conventional SIFT algorithm and the improved registration method are respectively shown in Figure 8. The same numbers in the images represent the feature point matching pairs between the two original images. In Figure 8a, only 3 of the 16 pairs are matched correctly by the conventional SIFT algorithm. In Figure 8b, 11 of the 16 pairs are matched correctly by the improved image registration method.

The transformation matrices calculated by the matching pairs shown in Figure 8a,b are respectively used to stitch the original images directly. The fused images are shown in Figure 9. By the conventional SIFT algorithm, the transformation matrix is miscalculated due to the excessive error rate of matching pairs, leading to the failure of the image mosaic shown in Figure 9a. The transformation matrix is calculated correctly by the improved registration method, and the fused image is shown in Figure 9b. However, the fused image shown in Figure 9b has obvious differences on both sides of the seam line, and the pixel transition in the overlapping area of two images is not smooth.

3.3. Image Fusion

If two C-scan images are stitched directly, the difference on both sides of the seam line is obvious in the fused image. For this reason, this paper introduces an optimal seam line searching algorithm and fade-in and fade-out fusion to eliminate the difference in the fused image.

3.3.1. Improved Optimal Seam Searching

To make the seam line ideal for segmenting the overlapping area in the fused image, the difference in intensity and the difference in geometrical structure around the seam line should be minimal. For this reason, the reference [28] puts forward the criterion value for searching for the optimal seam line as follows:

$$E\left(x,y\right)={E^2}_{color}\left(x,y\right)+E_{geometry}\left(x,y\right)$$

(5)

where $E_{color}(x,y)$ represents the value difference between overlapping pixels in two images, and $E_{geometry}(x,y)$ represents the structural difference between overlapping pixels in two images. To emphasize the difference between the four edge pixels in the evaluation criterion of structure similarity, the gradient operator templates shown below are used to calculate $E_{geometry}(x,y)$ as follows:

$$S_x=\left[\begin{array}{ccc}-2& 0& 2\\ -1& 0& 1\\ -2& 0& 2\end{array}\right] \mathrm{and} S_y=\left[\begin{array}{ccc}-2& -1& -2\\ 0& 0& 0\\ 2& 1& 2\end{array}\right]$$

Set the two original images to $f_1$ and $f_2$, and $E_{geometry}(x,y)$ can be obtained from the following formula:

$$E_{geometry}=Diff\left(f_1\left(x,y\right),f_2\left(x,y\right)\right)$$

(6)

$Diff$ is obtained by calculating the product of the gradient differences between two images in the x and y directions.

The optimal seam line searching algorithm is divided into three steps:

• Initialization. Each pixel in the first column is regarded as the starting point of a seam line. The intensity value of the seam line is initialized to the criterion value of the pixel in the first column.
• Expansion. The column that has calculated the intensity value extends down to the last column. For each seam line, the criterion values sum to the current pixel, and one of the three pixels in the next column is calculated. The direction of the minimum criterion value is chosen as the expansion direction.
• Selecting the optimal seam line. The seam line with the minimum criterion value is selected as the optimal seam line.

The schematic diagram of searching for an optimal seam line is shown in Figure 10.

According to the above method, the path of the optimal seam line and the fused image are shown in Figure 11.

3.3.2. Fade-in and Fade-out Fusion

After two C-scan images are stitched according to the optimal seam line, the difference on both sides of the seam line has improved, but the transition of pixel value is still not smooth. To improve the quality of the fused image, fade-in and fade-out image fusion is employed along the optimal seam line. Fade-in and fade-out image fusion is shown in Figure 12.

Fade-in and fade-out image fusion assigns different weights to the pixels in the overlapping area of two images, which can transform the pixels smoothly and preserve the information of the original images as much as possible. Fade-in and fade-out image fusion can be expressed by the following formula:

$$F\left(x,y\right)=\left\{\begin{array}{l}{f}_1\left(x,y\right) \left(x,y\right)\in f_1 \\ {\omega }_1\left(x,y\right)f_1\left(x,y\right)+{\omega }_2\left(x,y\right)f_2\left(x,y\right) \left(x,y\right)\in f_1\cap f_2\\ f_2\left(x,y\right) \left(x,y\right)\in f_2 \end{array}\right.$$

(7)

In the formula, $f_1\left(x,y\right)$ and $f_2\left(x,y\right)$ are the upper and lower images to be fused, respectively; ${\omega }_1\left(x,y\right)$ and ${\omega }_2\left(x,y\right)$ are the distribution weights of pixel values of the upper and lower images, respectively.

The distribution weights should be selected according to the image’s characteristics. To avoid excessive smoothing of the fusion region, new distribution weights are adopted, as shown in Figure 13b. Set the width of the overlapping area of two images to R, and select the area with the width of R/3 along the seam line as the fusion region. The sum of ${\omega }_1\left(x,y\right)$ and ${\omega }_2\left(x,y\right)$ is constantly equal to 1, as ${\omega }_1\left(x,y\right)$ decreases from 1 to 0 and ${\omega }_2\left(x,y\right)$ increases from 0 to 1 in the form of a cubic function.

The fused images obtained by different fusion methods are shown in Figure 14. The fused image, which is stitched directly, is shown in Figure 14a, and the fused image stitched by the improved fusion method is shown in Figure 14b. The results show that pixel values in the fusion region are transformed smoothly by the improved fusion method.

According to the above method, four C-scan images are stitched into a panorama image. Because the proportion of the C-scan images used for stitching is different from the actual proportion, the panorama image is compressed according to the actual proportion after stitching, as shown in Figure 15a. It is hard to quantitatively evaluate the fused image through visual observation. Therefore, we flip the photo of the defect horizontally and overlap it with the fused image, which is shown in Figure 15b. By calculating the area deviation of each region between the fused image and photo, we confirm that the fused image corresponds well to the photo of the testing block.

4. Evaluation of Image Mosaic Results

According to the above method, the four C-scan images are stitched three times to obtain a panorama image. The result of image registration can be evaluated by the number and accuracy of feature point matching pairs. The evaluation criteria for image fusion are divided into subjective evaluation and objective evaluation [29]. In this paper, peak signal-to-noise ratio (PSNR) [30] and structure similarity (SSIM) [31] in the objective evaluation index are used to evaluate the results of image fusion.

4.1. Evaluation of the Image Registration Results

Image registration is always needed for the three stitches. The number and accuracy of matching pairs directly affect the success rate of image registration. In the processes of three stitching, the number and accuracy of matching pairs of the conventional SIFT algorithm and the improved method are shown in

Table 1. Evaluation of the results of image registration.

	Conventional SIFT		Improved Algorithm
	Number of Matching Pairs	Matching Accuracy	Number of Matching Pairs	Matching Accuracy
The first stitching	16	18.75%	16	68.75%
The second stitching	16	31.25%	16	62.50%
The third stitching	16	12.50%	11	72.73%

. After screening the matching pairs according to the probe starting position, the final number of matching pairs may be less, but the accuracy is greatly improved. Because there is only the relationship of rotation and translation between the C-scan images, the transformation matrix can be calculated by a rigid-body transformation model. At least two sets of feature point matching pairs are needed to calculate the transformation matrix. Therefore, the accuracy of matching pairs has a greater impact on the success rate of image registration.

4.2. Evaluation of Image Fusion Results

PSNR is the most widely used objective image evaluation index. It represents the ratio of the energy of the peak signal to the average energy of the noise in the image. PSNR can be obtained by using the following formula:

$$PSNR=10{log}_{10}\left(\frac{{{(2}^n-1)}^2}{MSE}\right)$$

(8)

In the formula, n is the number of bits per pixel, which is generally taken as 8; MSE is the mean square error of the current image $I_1$ and the reference image $I_2$. The MSE formula for the image of “$H\times W$” size is as follows:

$$MSE=\frac{1}{H\times W}\sum _{x=1}^H\sum _{y=1}^W{\left(I_1\left(x,\mathrm{y}\right)-I_2\left(x,y\right)\right)}^2$$

(9)

The value of PSNR is inversely proportional to the distortion of the fused image. The higher the PSNR value, the less the image distortion.

Considering that PSNR is an error-sensitive evaluation method, the results of PSNR and human vision are different. To avoid the obvious difference between objective evaluation results and visual evaluation results, this paper also introduces SSIM to evaluate image fusion.

SSIM is an index to measure the similarity of two images. It mainly considers three key features: brightness, contrast, and structure. SSIM is more consistent with the intuitive perception of human eyes. The SSIM index can be calculated using the following formula:

$$SSIM\left(x,y\right)=L\left(x,y\right)\times C\left(x,y\right)\times S\left(x,y\right)$$

(10)

In the formula, $L\left(x,y\right)$ represents the brightness difference in two images, $C\left(x,y\right)$ represents the contrast difference in two images, and $S\left(x,y\right)$ represents the structural similarity difference in two images. The formulas for calculating $L\left(x,y\right)$, $C\left(x,y\right)$, and $S\left(x,y\right)$ are as follows:

$$L\left(x,y\right)=\frac{2{\mu }_x{\mu }_y+C_1}{{{\mu }_x}^2+{{\mu }_y}^2+C_1}$$

(11)

$$C\left(x,y\right)=\frac{2{\sigma }_x{\sigma }_y+C_2}{{{\sigma }_x}^2+{{\sigma }_y}^2+C_2}$$

(12)

$$S\left(x,y\right)=\frac{2{\sigma }_{xy}+C_3}{{\sigma }_x{\sigma }_y+C_3}$$

(13)

In the formula, ${\mu }_x$ and ${\mu }_y$ refer to the average brightness of two images; ${\sigma }_x$ and ${\sigma }_y$ refer to the standard difference of two images; ${\sigma }_{xy}$ is the covariance of two images; and $C_1,C_2,$ and $C_3$ are constants. $C_1$ is often taken as ${{(K}_1\mathrm{L})}^2$; $C_2$ is often taken as ${{(K}_2\mathrm{L})}^2$; and $C_3$ is often taken as $0.5C_2$. The higher the SSIM value, the less the image distortion.

Both PSNR and SSIM are full reference evaluation indexes, so it is necessary to select an ideal image as the reference image. This paper uses the C-scan image covering the fusion region obtained with the same parameters as the reference image. The fusion region in the fused image is compared with the same region in the reference image to calculate the PSNR and SSIM. The PSNR and SSIM of the images fused directly and fused by the improved method are shown in Figure 16. The results show that the image distortion caused by the improved fusion method is lower.

5. Conclusions

In this paper, an image mosaic method for ultrasonic C-scan based on the SIFT algorithm is proposed. Four C-scan images of the testing block containing artificial defects are stitched into a qualified panorama image by this method. From the current study, the following conclusions can be drawn:

• To improve the success rate of image registration, the feature point matching pairs obtained by the SIFT algorithm are screened by the difference in probe starting position in two C-scans. And then the matching pairs are further screened by RANSAC to obtain the final matching pairs used to calculate the transformation matrix. The success rate of feature point matching has greatly improved.
• After registration, a dynamic programming method is used to search for the optimal seam line. The pixels on both sides of the seam line are fused by the fade-in and fade-out image fusion. In this way, the image error on both sides of the seam line is eliminated, and the pixels in the fusion region are transformed smoothly.
• The panorama image is obtained by stitching four C-scan images three times. The experimental results show that the improved image mosaic method has higher accuracy of feature point matching and smaller image distortion, which is significantly better than the conventional SIFT algorithm.

The image mosaic method is suitable for the complete imaging of corrosion defects in plate-like structures. It is important to note that the use of the image mosaic method may have some technical limitations. One case where the method may not be applicable is when testing on a curved surface. In this case, the process of ultrasonic imaging may change the local features of C-scan images, which brings difficulty to image registration. The above-mentioned limitation will be addressed in future work by the authors.