Computer Vision and Image Processing Approaches for Corrosion Detection

Abstract

Corrosion is an undesirable phenomenon resulting in material deterioration and degradation through electrochemical or chemical reactions with the surrounding environment. Additionally, corrosion presents considerable threats in both the short and long term because of its ability to create failures, leakages, and damage to materials, equipment, and environment. Despite swift technological developments, it remains difficult to determine the degrees of corrosion due to the different textures and the edgeless boundary of corrosion surfaces. Hence, there is a need to investigate the robust corrosion detection algorithms that are suitable for all degrees of corrosion. Recently, many computer vision and image processing algorithms have been developed for corrosion prediction, assessment, and detection, such as filtering, texture, color, pixelation, image enhancement, wavelet transformation, segmentation, classification, and clustering approaches. As a result, this paper reviews and discusses the state-of-the-art computer vision and image processing methods that have been developed for corrosion detection in various applications, industries, and academic research. The challenges for corrosion detection using computer vision and image processing algorithms are also explored. Finally, recommendations for future research are also detailed.

1. Introduction

Corrosion or rust is an undesirable phenomenon resulting in material deterioration and degradation through electrochemical or chemical reactions with the surrounding environment [1]. Additionally, corrosion presents considerable threats in both the short and long term; this costs billions of dollars because of its ability to create failures, leakages, and damage to materials, equipment, and environment. The drawbacks of corrosion, such as expensive overdesign, costly maintenance, reduced efficiency, product contamination, plant shutdown, production loss, nuclear hazards, and heat transfer reduction, have burdened a lot of industries [2]. It also can threaten human health by damaging the respiratory tract, digestive tract, skin, and eyes [3]. Human tissues can be destroyed and burned on contact, and corrosion can cause permanent lung injury, blindness, scarring, and even death [4].

Computer vision involves developing algorithms and techniques that enable machines to simulate or replicate human vision using machine learning. The machine or computer has the ability to perceive and analyze patterns using visual inputs. It imitates the intricacy of the human visual system and analyzes inputs based on attributes like color, size, texture, and other factors. Computer vision is used for corrosion detection since it can locate and classify the corroded area in images along with the other machine learning techniques. Other examples of computer vision applications include face detection, car detection, and iris classification. Moreover, image processing refers to a technique that involves adjusting various attributes and parameters of images in order to improve their quality, enhance their appearance, and modify them in accordance with specific needs. There are numerous techniques available to transform images. These digital images contain specific values and positional elements, including visual elements, picture elements and pixels. Therefore, image processing has practical applications in detecting corrosion because its techniques are capable of segmenting, filtering, sharpening, and enhancing corroded areas in images. Examples of image processing applications include illumination correction, noise reduction, contrast enhancement, and image rescaling. The detection of corrosion through computer vision and image processing involves five primary processes: image acquisition, image pre-processing, image segmentation, feature extraction, and image classification.

Despite the fact that chemical and physical tests are highly efficient for corrosion detection, it remains difficult to implement these methods in confined spaces, narrow pipelines, small spaces, dangerous areas, and large surface areas. Consequently, computer vision and image processing approaches have been adopted to detect, assess, and predict corrosion progression, deterioration, and degradation on equipment and materials. Both approaches are inexpensive, reasonably accurate, fast, reliable, and can be implemented on large and wide areas such as bridges, ships, buildings, etc. However, it is hard to detect and predict corrosion in images captured in different environments, with different levels of quality, and different parameters, using a single technique. One computer vision or image processing technique is not enough to manage all types of corrosion images without considering the harsh environment and other external factors. Hence, there is a need to investigate various computer vision and image processing techniques such as color analysis, texture detection, denoising, registration, edge and line detection, filtering, pixelation, clustering, classification, segmentation, wavelet transformation, pattern recognition, morphological operation, and image enhancement that can be integrated or combined for corrosion detection, as well as the assessment and prediction of different corrosion conditions. Hence, this paper reviews and investigates state-of-the-art computer vision and image processing solutions in terms of prediction modelling and detection approaches that have been developed for various corrosion applications, industries, and academic research. The challenges and future recommendations for both approaches are also presented.

This paper is organized as follows. Section 2 presents the background of this study. Section 3 discusses the corrosion prediction model for computer vision and image processing. Section 4 reviews the corrosion detection approaches in terms of computer vision and image processing. Section 5 presents and discusses the challenges in corrosion detection. Section 6 presents the discussion and future recommendations. Finally, Section 7 details the conclusion of this study.

2. Background

This section will explain the background of the corrosion, computer vision, image processing, and corrosion detection processes of this study.

2.1. Corrosion

The strength of a metallic material, its operation, physical appearance, and mechanical properties can be affected by corrosion. However, the degradation of metallic materials is inevitable and they are susceptible to the surrounding environment [5]. The adjacent environment can be classified into wet corrosion and dry corrosion categories. Wet corrosion happens because of the formation of reactive electrochemical cells in the presence of liquid. Examples of wet corrosion include galvanic corrosion, erosion corrosion, and crevice corrosion. Moreover, dry corrosion refers to the reaction of metallic material with oxygen in the air, without the presence of liquid. Examples of dry corrosion include hydrogen attacks, molten-salt, and oxidation. The most corrosive environments are coastal areas and marine environments. The combination of sodium chloride (salt), oxygen, and moisture can rapidly increase the corrosive level in both environments. Moreover, seawater is more corrosive because of its higher penetrating power and conductivity through the metallic surface films. Furthermore, acid, polluted air, hot air, and hot water are more corrosive than the normal environment. Recently, the high toxicity level in the wastewater of domestic areas, urban sewage plants, and industrial plants has become a major concern as it can lead to a corrosive environment due to the presence of dissolved and suspended compounds, organic and inorganic chemicals, pathogenic and non-pathogenic compounds, etc. [6].

Other than environmental exposure, such as wet corrosion and dry corrosion, corrosion can also be categorized in accordance with attack morphology. There are eight categories of corrosion morphology, including general attack, erosion corrosion, intergranular corrosion, stress-corrosion cracking, galvanic corrosion, pitting, crevice corrosion, and selective leaching [7]. The uniform electrochemical reaction that occurs on the surface of a material is known as a general attack. It can be contained and is predictable. Erosion corrosion occurs due to the accelerated corrosion process resulting from the high velocity of corrosive fluid on the metal surface. Abrasion and mechanical wears are also involved. Intergranular corrosion is a specific form of corrosion that targets the boundaries between grains in a material. It creates a small, corroded area. Corrosive mediums and tensile stress can result in stress-corrosion cracking. A crack is produced from brass season cracking and steel caustic embrittlement. Galvanic corrosion occurs because of the potential difference between two submerged metals in a corrosive solution [8]. The metal with resistance to corrosion acts as a cathode whereas the other metal acts as an anode. The anode metal will corrode more than the cathode metal. Pitting is a type of corrosion that is highly destructive and causes localized damage, resulting in the formation of holes in the metal surface. Due to the accumulation of corrosion products, it can be challenging to identify these areas of damage. Crevice corrosion is a localized corrosion with small amounts of stagnant solution. It always attacks the holes, gaskets, and lap joints. Selective leaching is a type of corrosion that occurs when a particular element from an alloy is extracted [9]. This happens to aluminum, zinc, cobalt, chromium, and iron.

In relation to ship structures, the weakening caused by corrosion and fatigue can decrease the durability of vessels and ships, leading to potential structural failures [10]. The primary factors causing corrosion in ship structures include the single metal element (such as the element at the ship’s hull), two different metals (such as those used for the metallic coatings and rivets), and the concentration element (such as the presence of electrolytes and metal). Moreover, ship structures can be categorized into atmospheric zones and immersed zones of stiffeners and plates. The factors that can influence corrosion in the atmospheric zone include pressure, humidity, temperature, ventilation, steel type, oxygen, carbon dioxide, sulfur dioxide, sulphuric acid, and chloride concentrations. Additionally, the main influencing factors for the immersed zone include salinity, temperature, pH value, seawater velocity, seawater pressure, hydrogen sulfide concentrations, sulphuric acid, and carbon dioxide. Therefore, it is essential to implement an effective maintenance strategy that includes corrosion detection and maintenance activities. Corrosions on ship structures are depicted in Figure 1.

2.2. Computer Vision and Image Processing

Computer vision involves developing algorithms and techniques that enable machines to simulate or replicate human vision using machine learning. The machine or computer has the ability to perceive and analyze patterns using visual inputs. Images and videos are the most common visual inputs for computer vision. Moreover, object detection and object identification are the outputs of computer vision. Image processing is one of many techniques used in computer vision. Other techniques, such as instance segmentation, semantic segmentation, object racking, object detection, and image classification, are also important and widely used in computer vision. Corrosion detection benefits from the use of computer vision as it is capable of identifying and categorizing areas of corrosion within images, along with other machine learning techniques such as the use of the convolutional neural network (CNN) and the genetic algorithm. It is a non-destructive test and can be operated from a distance. Other examples of computer vision applications include face detection, car detection, and iris classification.

Image processing involves adjusting the various parameters and features of an image to enhance and transform it for improved clarity and quality. The images can be transformed using many techniques such as stretching, smoothing, sharpening, deblurring, filtering, inpainting, restoration, compression, and segmentation. These digital images comprise specific values and locations, which are represented by pixels, visual components, and other picture elements. Each of the images consists of rows and columns of pixels. The two-dimensional signal is processed in accordance with the given or specific task. The inputs for image processing are images and videos, which are similar to computer vision. Moreover, the outputs are the transformed input images and videos. Image processing is considered to be a subset of computer vision since it is a technique used in computer vision. Hence, corrosion detection utilizes image processing techniques since it can be used to segment, filter, sharpen, and enhance the corroded area in images. Hence, corrosion can be detected in the images using the aforementioned processes. Other examples of image processing applications include illumination correction, noise reduction, contrast enhancement, and image rescaling.

The main difference between computer vision and image processing can be understood as follows. Image processing involves processing and manipulating raw images into new images using different features. Moreover, computer vision involves extracting meaningful information or data from the images. Computer vision will not enhance features in images, but it will gain a high level of understanding from them. Machine learning techniques allow machine vision to interpret images in the same way humans do. Furthermore, image processing focuses on processing, whereas computer vision focuses on pattern recognition and data understanding. Moreover, computer vision requires training with a significant amount of data. Nevertheless, image processing does not require training and can be implemented straight away.

2.3. Corrosion Detection Process

The computer vision and image processing processes for corrosion detection can be divided into five main processes or elements: image acquisition, image pre-processing, image segmentation, feature extraction, and image classification. The corrosion detection process is illustrated in Figure 2.

2.3.1. Image Acquisition

Image acquisition is a process that captures images from digital cameras, thermal cameras, videos, computed tomography, etc. Images can be taken directly from cameras. Moreover, to obtain the desired images from a video, they need to be extracted frame by frame. A tomogram is an image produced by a tomograph which uses penetrating wave and reconstruction algorithms to produce a cross-sectional image. Images can be captured either automatically or manually [11].

2.3.2. Image Pre-Processing

The second process is image pre-processing. Pre-processing is used to enhance the quality of raw images. Generally, raw images contains a great deal of unwanted background noise. As a result, several pre-processing techniques can be applied, such as histogram equalization, morphological operations, and median filters. These techniques can reduce and eliminate noise, distortion, and interference in the raw images.

2.3.3. Image Segmentation

The third process is image segmentation, which plays a prominent role in corrosion detection. The corroded regions in an image are detected and isolated in accordance with similarities in discontinuity and criteria. Examples of segmentation techniques include region-based, edge detection, threshold-based, clustering, pattern matching, and artificial neural network (ANN) segmentation.

• Region-based: This segmentation technique locates connected pixels of similar colors and intensities in an image. Normally, similar pixel regions are classified using some predefined rules between adjacent pixels. The segmented regions are obtained from the neighboring pixels and they are connected to the seed pixels. This technique works better with high contrast images.
• Edge detection: This technique detects object boundaries by locating and identifying points of brightness discontinuities or sharp changes in an image. These points are also called the image edges. Widely used edge detection methods include Canny edge, Prewitt edge, Sobel edge, and Laplacian edge methods. The aim of edge detection methods is to reduce the probability of detecting wrong edges in an image by optimizing edge detection algorithms.
• Threshold-based: This segmentation method converts color or grayscale images into binary images based on a threshold value. The threshold value is obtained from the original image’s pixel intensity. The pixels in the original images are converted for easier image analyzation. This method separates the desired object from the background pixels.
• Clustering: This technique partitions an image into clusters or groups by identifying the relationship between adjacent pixels in an image. Then, groups of similar patterns are clustered to explore during data analysis. The most used clustering methods are the fuzzy c-means, k-means, improved k-means, and hierarchical methods.
• Pattern matching: Pattern matching, or template matching, is a technique that allows template localization in images. The patterns in images are matched to each other in terms of their special features by restricting the searching region. The corroded region can be segmented by matching a predefined template with the small parts of an image. Examples of pattern matching include Hamming distance and Harris corners.
• ANN segmentation: This method uses an encode–decoder structure for the segmentation of 3D images [12]. It works with the height, width, and channel number. The first two dimensions represent the image resolution. Moreover, the third dimension represents the red, green, and blue channels. This segmentation technique uses machine learning in its segmentation process.

2.3.4. Feature Extraction

Feature extraction is the fourth process, and it involves extracting corrosion characteristics from the image that has been segmented [13,14]. Corrosion features are retrieved to obtain the most significant information from an image. Examples of feature extraction techniques include the Gabor filter, histogram of oriented gradients (HoG), and Laplacian of Gaussian (LoG).

2.3.5. Image Classification

The final process is image classification, wherein the groups of pixels are labelled and categorized in accordance with specific rules. This process will label an image based on its visual content. The groups of pixels can be categorized with textural or spectral characteristics. The three types of image classification used were supervised, unsupervised, and object-based analysis.

3. Corrosion Prediction Model

This section will review the state-of-the-art corrosion prediction models used in computer vision and image processing approaches. The corrosion prediction models are divided into the knowledge-based model, probabilistic model, statistical model, and deterministic model.

3.1. Knowledge-Based Model

The knowledge-based model involves utilizing artificial intelligence techniques to anticipate the advancement of corrosion based on images and videos. The learning process is adopted by the algorithms used for the corrosion images. The knowledge of the algorithm is continuously updated throughout the learning process. The learning process can be categorized into supervised learning, unsupervised learning, and reinforcement learning. Supervised learning is a learning process used by labeled corrosion images. These images are labeled manually before being trained by the algorithm. Unsupervised learning is a learning process used by unlabeled corrosion images. The algorithm will train itself to explore the unlabeled images and predict the corroded area. Reinforcement learning is a learning process that uses trial and error in an interactive corrosion environment. The algorithm will create a sequence of decisions, based on the learning process, by providing penalties or rewards for each solution concerning corrosion detection. The feedback of each action will enable the algorithm to provide optimal results. Generally, the knowledge-based model will complement the other computer vision and image processing corrosion detection approaches due to its high efficiency, low level of human error, and improved workflows.

Several studies [15,16] have used the knowledge-based model for automated sewer inspections. The images and videos were captured with closed-circuit television (CCTV). The CCTV was installed in sewers, and corrosion assessments were conducted from a remote place. The data from the CCTV were analyzed in order to detect the corroded area. However, not all images or videos were clear enough for corrosion detection. Consequently, pre-processing techniques were applied, such as the support vector machine (SVM) [17] for the pixel classification of corroded pipes. CNN [18,19] can also be used for corrosion detection. This method pre-processed 47,000 sewer pipeline images and managed to classify six types of defects. This method achieved 96.33% accuracy for defective sewer pipelines. However, the classification of imbalanced CCTV data can occur in long-distance sewer pipelines. Defective classifications in a 24.7 km sewer pipeline were investigated as only a 64.8% accuracy was obtained from the CCTV with regard to corrosion detection. However, this method did not use pre-processing for imaging input enhancement. This proved that pre-processing is an important step in computer vision and image processing-based corrosion detection. Similar knowledge-based approaches [20,21,22] also used a combination of CCTV and machine learning in their methodology for corrosion detection in pipelines.

3.2. Probabilistic Model

The probabilistic model is utilized for corrosion detection in scenarios where there is inadequate historical data and real-time data are available. Corrosion is a time-dependent and space-dependent process, thus, the probabilistic model must be able to ascertain the uncertain nature of materials. This model requires expertise and an in-depth knowledge of physics and mathematics to develop the sophisticated mathematical model. Consequently, this model is quite difficult to develop. However, the probabilistic model can be replaced by the fuzzy logic model if the available data are ambiguous and vague. The fuzzy logic model also requires professional experience and expert judgments in order to determine the rule set for corrosion detection.

Gamma distribution [23] was used to investigate the cumulative deterioration of materials. The reliability of the corrosion progression was analyzed using a decreasing monotonic pattern. The number of counts was used to reconstruct a 2D image of the materials. The corrosion pattern can be analyzed using the reconstructed 2D image. Moreover, the gamma process and copulas [24] of the spatio-temporal aspects of the image were used to predict and detect the corrosion of the buried pipelines. The correlation length model calibrated the temporal and spatial correlations of the probabilistic model. The failure reliability can be estimated, and thus, corrosion progression can be detected. The probabilistic model and finite element [25] were used to study the effect of fly ash, the rebar diameter, and current density on the spatial variability of a concrete beam based on X-ray images. Gumbel distribution parameters were obtained from the spatial probabilistic corrosion model. The results showed that the uniform corrosions in images can be detected with a high current density. Moreover, a low current density can detect non-uniform corrosion patterns such as large cracks and pits. The probabilistic prediction model managed to derive the accurate Gumbel distribution parameters from the steel weight loss. The Monte Carlo finite element [26] was used with the probabilistic framework to estimate the corrosion-induced cracks in images. The corrosion was predicted and assessed with an efficient computational approach and two machine learning models. The deterioration prediction of the structural capacity was demonstrated with this method. The accurate corrosion classification of the probabilistic model was further improved with linear population size reduction and image processing [27]. The automated pitting corrosion was detected with multilevel thresholding for the metal surface extraction of corrosion images. This method also managed to extract the corrosion features of multilevel color spaces, such as grayscale and color channels.

3.3. Statistical Model

The statistical model is a prediction method that uses statistical analysis to detect and predict corrosion progression using historical data. Information can be obtained from CCTV systems installed in various places such as gas pipelines, sewers, and others. A wide range of statistical methods can be used to develop a prediction model, such as the Markov chain, Bayesian inference, distribution fitting, polynomial regression, linear regression, etc. The accuracy of the statistical model depends on the data quality and sufficient input.

The corrosion mechanism for marine material is difficult to understand because of the diversity involved with corrosion morphology. As a result, the Markov chain with the gray level co-occurrence method [28] were introduced in the seawater to detect corrosion damages on Q420 steel. The extraction of the corrosion morphology from images was achieved, and it was consistent with the electrochemical results. However, there were discrepancies with regard to pit depths, interconnected corrosion holes, and irregular corrosion morphologies. Despite these issues, they can be overcome with Bayesian optimization (see the image-based corrosion rating in [29]). Bayesian optimization was used to fine-tune the corrosion classifiers. The corroded regions in the images were characterized with the local texture features and high discriminatory power. Consequently, the cross-validation error was minimized after the hyperparameters were fine-tuned. The other statistical models, such as polynomial regression [30] and linear regression [31,32], can also be used to develop the statistical prediction model for corrosion detection using corrosion images.

3.4. Deterministic Model

The deterministic model involves the examination of the relationships between the parameters or variables of corroded materials through field experiments conducted with the aid of images and videos. This prediction model is easier to develop compared with the other models. Vast expertise or knowledge in corrosion is not crucial for the development of the deterministic model. Nevertheless, this model cannot extract the realistic nature of corrosion in the environment. Thus, the developed prediction model could be inaccurate in terms of the extrapolated results [33].

Examples of deterministic models for corrosion detection using computer vision and image processing approaches can be found in [34,35]. The structural health monitoring (SHM) and digital twin models were used in [34] to increase the durability of infrastructural assets. Corrosion progression can be regularly revalidated in terms of the structure’s reliability. The results showed that this method can improve the behavioral prediction of metallic structures. Moreover, the texture descriptors with cellular automata [35] were used when the training data were not sufficient. This method counterbalanced the formulation with controlled deterministic chaos. According to the results, the proposed method outperformed the other approaches during the real-world detection process. Other deterministic models in the literature can be explored in [36,37,38]. The comparisons of different corrosion prediction models are illustrated in

Table 1. Comparisons of different corrosion prediction models.

Prediction Model	Description	Application/Method
Knowledge-Based	AI is used to predict corrosion progression from images and videos.	Automated sewer inspection [15,16]. Combination of machine learning and CCTV [20,21,22].
Probabilistic	This model is used when historical and real-time data are insufficient. Requires expertise and an in-depth knowledge of physics and mathematics to develop sophisticated mathematical models. Quite difficult to develop.	Gamma distribution [23]. Gamma process and copulas of spatio-temporal aspects [24]. Probabilistic model and finite elements [25]. Monte Carlo finite elements [26].
Statistical	Statistical analysis is used to predict and detect the progression of corrosion using historical data. Gathers data from CCTV systems installed in various locations such as gas pipelines, sewers, etc.	Markov chain with gray level co-occurrence method [28]. Polynomial regression [30] and linear regression [31,32].
Deterministic	Field experiments involving images and videos are used to examine the connections between the parameters/variables of corroded materials. Easier to develop. Extrapolation accuracy.	Structural health monitoring and digital twin models [34]. Texture descriptors with cellular automata [35].

.

4. Corrosion Detection Approaches

This section will describe the state-of-the-art computer vision and image processing corrosion detection approaches. There is no single corrosion detection technique that is suited to all types of environments. The different corrosion detection techniques must be used for different environmental conditions and other external factors.

4.1. Ground Penetrating Radar

Ground penetrating radar (GPR) uses electromagnetic reflection to penetrate the surface of materials used for corrosion detection. A short, microscopic electromagnetic energy pulse is transmitted to the surface in the frequency range of 500 MHz to 2.5 GHz. The amplitude-time signal is recorded using the reflected waves of GPR antennae. The advantages of GPR include fast scanning, being sensitive to embedded material in the structure, good quality images of the structure, and greater depth penetration. However, GPR data are quite difficult to interpret, therefore, they require expert knowledge and skilled operators. The deep post-processing of captured images is also required. Moreover, this method must be applied periodically, and it is not capable of monitoring corrosion progression online. These shortcomings are caused by data analysis instead of equipment. Thus, computer vision and image processing approaches can be used to accurately analyze complicated GPR data.

Researchers in [39] employed a combination of GPR data, ANN, and image processing to identify the degree of corrosion in reinforced steel concrete structures. Edge detection and k-means clustering image processing techniques were used to investigate differences between corrosion features. ANN was used to interpret and extract complicated features data from GPR images. The images were taken in a short period of time to reduce the blurry effect on GPR images. The results showed that k-means clustering managed to extract and partition a corrosion image into multiple segments. The obtained semantic information classified the area into no corrosion, low corrosion, middle corrosion, and high corrosion categories. The accurate edges of multiple segments areas were detected with a Sobel edge detector. Moreover, GPR was used with Fourier transform and a Gaussian filter [40] with 1 GHz frequency to monitor and predict the degree of corrosion. Reinforced steel structures, such as floors, roofs, and concrete walls, were scanned and the obtained images were pre-processed to detect noise and anomalies. The results were validated using simulation and field data tests. This approach yielded the most accurate prediction with regard to the degree of rebar corrosion, making it a superior technique for detecting and identifying corrosion. Moreover, GPR can also be combined with other imaging techniques to further improve corrosion detection accuracy. Acoustic scanning and GPR were deployed in [41] for corrosion detection in straight and curved concrete bridge decks. A frequency range of 0.5 to 5 KHz was used to reconstruct 2D images from the energy map. The results showed that by combining both deterioration and acoustic scanning maps, comprehensive corrosion detection can be provided in the bridge decks.

4.2. Thermography

Recent advancements in solid-state technology have allowed an improved infrared detector to be developed. This new infrared detector contributes to better resolutions and higher accuracies being obtained during corrosion monitoring, detection, and prediction. The cost of online monitoring with this type of sensor also has been considerably reduced. However, thermal images obtained from the infrared sensor can be affected by low signal strengths and noise. Consequently, image processing techniques and signal analysis are proposed to overcome these problems, especially to enhance the thermal images’ quality.

Infrared thermography (found in [42,43,44]) is a common computer vision and image processing technique for corrosion detection. This technique records the electromagnetic energy emitted by metallic materials. Then, corrosion patterns can be located from the thermal images. Other widely used thermography-based techniques include optical [45], laser [46], induction [47], and microwave [48] techniques. Each of these techniques have their own advantages and limitations in terms of energy, surface inspection, practicalities, source, etc. Recently, a new thermography technique was introduced to improve corrosion classification and detection. The photovoltaic (PV) electroluminescence module [49] was developed to detect cracks and corrosion on pipelines. The electroluminescence (EL) images were parsed from PV cells to machine learning algorithms such as CNN, random forest, and SVM. EL images were pre-processed with regression fitting, convex hull, thresholding, and filtering to create the planar index module. The results showed that CNN on EL images outperformed the other machine learning algorithms for pipeline corrosion degradation. Nevertheless, these methods required a proper sample for training, evaluation, and testing in order to increase the accuracy of corrosion prediction.

4.3. Computed Tomography

Computed tomography is a computerized imaging technique which generates cross-sectional images from radiation rays that are aimed and rotated around the object. The cross-sectional images or slices are also called tomographic images. Geometry processing generates 2D and 3D images of an object which contain important information such as shape and dimension. Several slices are stacked together to reconstruct a 3D image for corrosion detection. Computed tomography is a non-invasive technique in computer vision and image processing.

The thermal spraying method [50] was developed in order to assess the corrosion mechanism and coatings. X-rays and Raman spectroscopy were used to protect the steel of anodic metals such as aluminum and zinc. This method was compared with electrochemical experiments such as open circuit potential and electrochemical impedance spectroscopy (EIS). The results showed that the corrosion resistance property was improved with X-ray computed tomography. The corner steel bar in concrete can also be detected with X-rays [51]. Moreover, novel tomographic acoustic microimaging (TAMI) [52] presented an approach to assess the depth of corrosion and pitted areas in images obtained from scanning acoustic microscopy (SAM). These images were processed with binarization under the C-mode to identify the corrosion pits under the rust. The results of the corrosion pit morphology were validated with optical microscopy and the 2D and 3D surface methodology, which achieved a 40% accuracy with regard to pit regions. Synchrotron radiation computed tomography (SRCT) [53] was tested using electrochemical reaction visualizations, composite failure analysis, and corrosion rate measurements. This tomography technique used a resolution of less than 1 µm to create the 3D material microstructures. This technique was further improved by adding a fourth temporal dimension to analyze the under-load material response. The reconstructed 4D images managed to enhance the textural details of corrosion detection. On the other hand, neutron tomography [54] was mobilized to provide the details of the internal structure of heritage objects. Nevertheless, this technique can also be used for detecting the degree of degradation of heritage objects. The internal defects and spatial distribution information were obtained using modern mathematical algorithms and neutron attenuation coefficients. The 3D structural volume data and images were obtained, which provided opportunities for the future implementation and improvement of corrosion detection and prediction. Magnetic Resonance Imaging (MRI) can also be implemented for corrosion analysis, as presented in [55,56,57].

4.4. Color Space Detection

Color space is the color representation described by mathematical models in an image. The image pixels are defined using the characteristics of color spaces. The color space segmentation is a technique wherein the meaningful object in an image corresponds with the clusters of color features in image pixels. These image clusters share similar properties with regard to color. However, there is no single-color space that fits with all kinds of images [58]. Consequently, many experiments have been conducted to select the best color space that suits specific conditions and environments. The well-known color spaces are RGB (red, green, and blue), YUV (luminance and chroma), CMY (cyan, magenta, and yellow), HSV (hue, saturation, and value), and XYZ (x, y, and z coordinates). An example of corrosion detection using a RGB color space is illustrated in Figure 3.

The HSV color space [59] can be used to locate corroded and non-corroded regions based on the color and roughness of materials. In this work, HSV was used for the color analysis of a corrosion-representative histogram. Moreover, the grayscale level was used for roughness analysis. Then, the photographic images of components and structures were applied using this image processing technique. This color space was combined with artificial intelligence algorithms to improve the detection accuracy of the corrosion damage assessment. HSI (hue, saturation, and intensity) [60] can also be applied for corrosion detection. An image enhancement technique was used to improve the corrosion areas with histogram equalization to obtain better contrast and brightness. The HSI was converted using the RGB images. Nevertheless, robust corrosion detection can be developed by extracting CbCr (blue chroma and red chroma) elements from the corrosion images [61]. The textures of corrosion can be distinguished, which can improve the computation and measurement of ununiform corrosion. A novel color space based on the color shade index (CSI) and rusty area ratio (RAR) was introduced in [62] to ensure electric energy transmission by detecting the damper rust defect in a timely manner. The complex damper images were acquired in grayscale and enhanced with Gaussian kernel, edge strength mapping, and local difference processing. Then, the images were classified into severe rust, medium rust, slight rust, and rust-free categories. The results showed effective segmentation and rusted dampers were identified with 93% accuracy.

4.5. Wavelet Domain

Wavelet domain is a zero-averaging fast decaying technique using both frequency and spatial domains. It can be used to measure the corroded regions in an image. The minimization of entropy is used for illumination component correction in the corrosion image. Then, the wavelet transform is applied to the RGB channels to measure the entropy and energy values. The feature vectors are extracted, thus allowing the accurate detection of corroded regions. It is possible to recover noisy corrosion images without muddling or blurring the details of the corrosion in the image.

The wavelet image coefficient [63] on the grid metal frame was studied to determine the atmospheric corrosion characteristics. This metal frame was used for power transmission in Chongqing. The structural materials were investigated in terms of corrosion product, quantity, and morphology using the wavelet transform algorithm. The images of the detection results were evaluated using the on-site samples which were consistent with the actual corrosion progression. Moreover, wavelet analysis [64] was used to determine the effect of nitrogen for pitting corrosion. The standard deviations of the partial signal and energy distribution were used to measure the energy of the low frequency signal. The hybrid wavelet packet transform [65] was combined with the support vector classifier for carbon-steel pipeline corrosion detection. The corrosion grades were predicted and determined using fast Fourier transform. This technique was evaluated with the linear polarization resistance (LPR) laboratory test to determine its prediction accuracy. The results showed that the proposed technique was effective for the detection and assessment of corrosion. The similar 2D-wavelet filtering technique can be used for damage detection in structures [66].

4.6. Classification with SVM

SVM is an image classification technique wherein two separate zones can be partitioned from a vector space. It is a powerful machine learning technique, especially for image segmentation and pattern recognition. SVM is a supervised learning technique used for regression, classification, and outlier detections. This technique is highly effective in dimensional spaces. SVM transforms the corrosion image using a kernel trick, and then, it locates the optimal corrosion boundaries between all possible outputs. Both colors of the corroded region, and the degree of corrosion, can be analyzed using SVM.

The applications of SVM for corrosion detection can be observed in water pipelines [67], underwater pipelines [68], steel bars [69], bridge cables [70], equipment [71], aircraft structures [72], wind turbine blades [73], and many more. SVM can also be improved using other methods to obtain a better accuracy for corrosion detection and assessment, such as HOG [74], scale-invariant feature transform (SIFT) [75], and speeded up robust features (SURF) [76]. The corroded and non-corroded regions can be distinguished from colors [77] and textures [78] using SVM.

4.7. Damage Analysis with NDE and SOM

NDE (non-destructive evaluation) is a method that is used to detect the texture changes in a corrosion image. This method is also known as non-destructive testing (NDT) and non-destructive inspection (NDI). NDE enables the assessment of the characteristics of a component, material, or structure without damaging the subject itself. Moreover, the SOM (self-organizing map) is used to classify the corroded regions in the segmented NDE images. SOM is an unsupervised learning technique for the classification of similar patterns and features within the same class. The local parameters are modified using a group of neurons instead of the sample training of class membership. The texture variations of corrosion images can be classified using NDE and SOM.

NDE and SOM were used in [79] to investigate deteriorations caused by rebar corrosion and corrosion-induced cracks. The electromagnetic wave radar of the NDE assessed the cover thickness of the specimens, which were 30 mm and 60 mm. The internal damage and corrosion rate were judged during the maintenance inspection. The electrolytic corrosion experiment was conducted to evaluate the developed NDE and SOM techniques. According to the results, this technique managed to detect the internal crack region and rebar corrosion of concrete structures. Moreover, the SOM-based neural network [80] was used with ant colony optimization to identify the prestressed steel mechanism and corrosion evolution analysis. Additionally, the acoustic emission of NDE was used to characterize the signals from the prestressed steel. Four acoustic emission sources were used to identify four damaged areas on the prestressed steel strands. This showed that this technique was another alternative for corrosion analysis and detection.

4.8. Texture Analysis

Texture analysis (as depicted in Figure 4) is one of many image processing techniques that can be used for corrosion detection. Computer vision also uses this technique for the object classification of computer image analysis. This technique calculates the edge boundaries of various textured corroded areas in the images. The enhancement of classification accuracy is possible via texture analysis, which effectively reduces errors when detecting isolated data. A fundamental aspect of this technique is that the corroded materials will increase the surface roughness over time. It can be measured using the surface pit depths, which provide shadow lengths that are produced from a single incident light source. Then, reconstructing a pseudo 3D material exterior allows for the average corrosion rate to be obtained. The corroded area is segmented from the edge pixels in the image. These edge pixels lie on a corroded region boundary with different grayscale values. The difference between the edge pixels and neighboring pixels is calculated to determine an accurate corroded boundary. The intensity characteristic is used to measure discontinuities between grayscale values in the image. Examples of the texture analysis technique in literature can be observed in [81,82,83,84]. The texture analysis technique can accurately detect, recognize, and classify corroded regions in images. The identification of both corroded and non-corroded areas is possible.

Furthermore, the texture analysis approach can be a combination of any existing segmentation methods, including region-based, thresholding-based, edge detection, clustering, etc. Hence, we propose a combination of edge detection and thresholding-based methods by using the active contour segmentation method. Active contour has been proven to be accurate for segmenting objects with uneven, blurry, and weak boundaries [85,86,87,88]. Thus, this method will be modified to accommodate weak corrosion boundaries by formulating an initial contour, curve restrainer, and stopping function. Then, the thresholding of pixel property method will be used to lead the modified active contour to segment the accurate corrosion regions. The preliminary results of the proposed method are illustrated in Figure 5. From this figure, it can be observed that the active contour can be used to segment corrosion boundaries on ship structure images. Nevertheless, more analysis in terms of recognition accuracy and segmentation accuracy needs to be undertaken in order to analyze the accuracy and efficiency of the proposed method. Moreover,

Table 2. Summary of corrosion detection approaches in computer vision and image processing.

Detection Approach	Description	Limitation
Rossouw and Doorsamy [89]	Historical data and machine learning to predict downstream test post on ship. Survival, regression, and classification analysis for evaluation. Detect and estimate maintenance time of corrosion progress. Effective corrosion control via mechanism for external corrosion prevention.	Various AI techniques can augment the intricacy of the model.
Vu and Dong [90]	Assess corrosion effect on bulk carrier. Identify and mitigate the effects of corrosion on a ship’s hull.	Sensitivity of a model used in maintenance planning can be adversely affected by the absence of historical data. Unexpected corrosion behavior can still happen.
Canca and Kokkulunk [91]	Explore the current corrosion maintenance employed in maritime sector. Monitor very low sulphur fuel oil (VLSFO) characteristics to lower critical risk. Provide detection and corrective action for corrosion caused by sulphur emissions.	Only for Sulphur 2020 amendment.
Bouzaffour et al. [92]	Detect corrosion of steel in a marine environment. Sensor detects corrosion region by calculating the difference between concrete moisture degree and steel layer. Supervise and manage the steel’s mass loss by detecting corrosion status.	Inaccurate sensor network for physic-based model.
Yarveisy et al. [93]	Pit depth prediction that requires immediate maintenance. Degradation process from real-time data is used, considering peaks over threshold. Assess and detect corrosion failures with high performance.	Multiple sections are not implemented in the same workflow. Requires parameterization and fine-tuning for assessing corrosion failure of multiple sections.
Kim et al. [94]	Predict corrosion damages on ship structures. Determine RUL and corrosion damage over time.	Requires fine-tuning for each onshore, offshore, and nearshore ship structure.
Cheliotis et al. [95]	Detect particular fault indicators, like air temperature and pressure, that could prove beneficial in the formulation of a corrosion model.	Occurrence failures exhibit reliability, safety, and energy efficiency. The occurrence of corrosion and fouling within the turbo-charger and nozzle ring of the vessel is assessed.
Makridis et al. [96]	Machine learning and time-series anomaly detection. Detect and predict corrosion on specific parts of a ship engine.	Accuracy may suffer as sensors are placed in certain areas of the vessel. Need fine-tuning and parameterization for different parts of vessel.
Kim et al. [97]	Historical and real-time data for employment of generalized extreme value distribution. Adapt to defect depth distribution and high reliability.	Even with the assistance of historical data, the estimation of parameters using real-time data is highly delicate. Unavailability of historical data can reduce model’s accuracy.
Anyfantis [98]	Specific location can be determined.	Not feasible to closely monitor areas of a ship’s hull that are prone to corrosion.
Jimenez et al. [99]	Initiate solution for hull corrosion. Breakdowns and failures can be reduced with predictive maintenance.	The complexity involved in implementing it could cause significant disruptions within the shipping industry.

represents the other corrosion detection approaches in the computer vision and image processing literature.

5. Challenges in Corrosion Detection

The implementation of machine vision and image processing approaches for corrosion detection, assessment, and prediction is fraught with challenges. In terms of sewage pipes, it is critical to address the challenges of assessing and maintaining corrosion in situ. The sewage pipes contain many bacteria and other microorganisms that settle inside the pipes which can pose health threats to the corrosion inspectors because of practical reasons. Machine vision and image processing approaches could be considered for corrosion inspection in sewage pipes because of their NDT and remote inspection. The roles of corrosion inspectors can be further reduced, with minimal exposure and physical contact with the health hazards.

The other concern for computer vision and image processing approaches for corrosion detection is the availability or readiness of the imaging equipment technology. The information of the structure’s condition cannot be provided using a camera or video camera. This equipment and its technology are not able to penetrate the material itself, thus the degree of corrosion cannot be identified. The information and condition of the outer material can be analyzed, although the inner side would be totally ignored. Only a general understanding of the corroded area can be provided instead of individual flaws. This approach also has an invasive installation process, as the equipment is not water-resistant and is not durable enough to be operated during the underwater and inner pipe corrosion inspections. Dewatering is required, thus, the operation can be interrupted. Operation interruptions will affect production and runtime, which can reduce productivity. Moreover, there is a significant problem in terms of replacing the infrared detectors and sensors in small spaces and confined spaces such as in the ship engine room and wastewater pipeline. The detectors and sensors might stop working and break down due to longevity and durability issues.

In terms of ship structures and pipelines, the main concern is how to reach these structures in order to capture images documenting their physical condition [100]. The most affected areas on the ship, such as the engine room and ship hull, are dangerous and difficult to reach [101]. Moreover, the oil and gas pipelines are isolated and located underwater. The wastewater pipelines are also too small for humans to reach. Consequently, robotic vehicles and drones can be used to reach those places. However, the cost of this equipment is not cheap, and it might require a significant number of robotic vehicles and cameras to be installed on those structures.

Moreover, the ultrasound and GPR can be employed for concrete corrosion detection. Both types of equipment can be considered as being part of the machine vision and image processing approaches. The boundary layer depth and concrete flaws can be detected microbiologically. However, the sensors used in both methods can be constrained by sound coupling, inaccurate predictive models, and moisture level. Hence, there is a need to address these problems as the aggregate location of corrosion cannot be accurately detected, and thus, it can produce a false corrosion segmentation.

The combination of digital CCTV with infrared scanning, sonar, and radar can also be used for corrosion inspection and detection. These imaging techniques can be enhanced using multiple-angles, fisheye lenses, and higher-resolution cameras. These methods can be employed in small spaces, confined spaces, and out-of-reach areas. Thus, the presence of a corrosion inspector is minimized. Nevertheless, these methods are error-prone as they need to be mounted on a robot crawler which is reliant on the operator’s expertise. On the other hand, advances in robotic technology, machine learning, and LiDAR (light detection and ranging) sensors can help operators to minimize these issues.

LiDAR with point cloud can be used for corrosion detection on the ship hull. This imaging method can estimate the corroded areas by using an integrated 2D camera and RGB values on the ship structures. However, this technique relies heavily on the reflected data of the material surface. Consequently, some corrosion progresses on the material surface cannot be detected, especially on blurry areas, weak boundaries, and disoriented surfaces. This method is best suited for use with geometrical deviation surfaces.

Computed tomography with nuclear sources or gamma rays is another method for corrosion detection. This technique uses a radioactive source, such as cobalt and cesium, to develop and reconstruct a 3D image profile of an object or material. Radioactive sources can penetrate thick materials or insulation for corrosion detection. Nevertheless, radioactive particles are very dangerous and must be handled and prepared by a competent person with adequate certifications. This competent person must also have a fundamental knowledge of computer vision, image processing, and artificial intelligence. Moreover, specialized training is required before the person can be employed, which can increase upskilling costs and training costs.

Data complexity remains a critical issue in the implementation of computer vision and image processing approaches in corrosion studies. The low and medium quality of images and videos can produce false corrosion detection results. Noise, distortion, and attenuation also pose critical challenges for object detection and classification. Consequently, some data from images must be reconstructed and converted into frequency domains or time domains in order to obtain the relevant information. Then, the relevant and significant features can be extracted for corrosion detection, assessment, prediction, and analysis. Big data and advanced data processing can improve the accuracy and efficiency of corrosion detection, and thus, it can further reduce the incidence of unexpected detection failures.

6. Discussion and Future Recommendation

Computer vision and image processing techniques can be considered as a major topic, not only in corrosion detection and maintenance, but also in the manufacturing, aeronautical, power and energy, food and beverage, wastewater management, and construction industries. Most of the papers that were published concerned the topics of the corrosion prediction model, corrosion detection, and corrosion maintenance, and were considered to be computer vision and image processing studies. Complex methodologies, such as in prediction modelling, is one of the most pressing issues and is a frequent topic of research in computer vision and image processing.

Computer vision and image processing approaches can provide good insight into the condition of materials. These approaches are convenient to industry, researchers, and academia, as the corrosion can be detected, assessed, analyzed, and maintained from a distance with remote monitoring. The exposed components, materials, and equipment are not taken out of operation, thus, productivity is not affected. Moreover, the lining deterioration, deformation characteristics, and visible internal damage can be located with these approaches. The use of high-resolution images can provide a comprehensive assessment of the corroded area. Automated analysis is conducted on the nodes or edges, and thus, it can reduce the data transmission cost and increase the corrosion detection speed. Digital measurements of a corroded area can be obtained much faster than conventional methods. Immediate results can also be obtained as zooming in on images is possible, and thus, more details for the corrosion analysis can be obtained.

IR4.0 (Industrial Revolution 4.0) helps the industry, researchers, and academia to use more advanced technologies for corrosion detection, such as the advanced sensor, data analytics, signal processing, machine learning, big data, and the internet of things (IoT). The computer vision approach certainly needs imaging equipment to monitor or capture the suspected corroded area. Advanced sensors with a 5G network can complement the computer vision approach for more accurate and efficient corrosion detection. Advanced and intelligent sensors, such as gas sensors [102,103], temperature and humidity sensors [104,105], and pH sensors [106,107], have been developed, and they can intelligently capture and analyze the important corrosion parameters at the site. These sensors are well equipped with machine learning, which can improve the accuracy and efficiency of their sensing and analyzing capabilities. Machine learning, particularly deep learning, has provided a great opportunity for corrosion prediction, assessment, and detection in terms of the applicability of processing images and video data. The use and integration of the current equipment with IR4.0 technology are fully recommended for corrosion detection [108]. The future integration between LiDAR and infrared technology in remote sensing can provide more details and information for corrosion detection, assessment, and maintenance. Remote sensing imagery contains spatial, temporal, spectral, geometric, and radiometric resolutions which can be used to extract more information from the respective corrosion images. Hence, considering recent advances in IR4.0, a high level of computer vision, image processing, and artificial intelligence can be achieved for corrosion detection, assessment, and prediction.

In addition, the combination of computer vision, micro-indentation mapping, pH profiling, scanning electron microscopy, and neutron tomography can be used as a novel methodology for microbiological corrosion detection. So far, there is no standard evaluation approach for microbiological corrosion [109]. Microbiological corrosion is common in sewage pipes where acids are formed from chemical compounds and acidophilic bacterial colonies. The computer vision and image processing techniques can complement chemical approaches for microbiological corrosion assessment, detection, and maintenance [110].

Furthermore, crucial elements to develop the computer vision and image processing prediction models include the level of knowledge or expertise, availability of real-time data and historical data, computational power, and prediction or detection accuracy. In the case of insufficient data from images and videos, corrosion prediction and detection can be implemented using the probabilistic model. This prediction model is suitable for predicting the failure states of materials with higher accuracy. Nevertheless, the probabilistic model requires a high level of modelling expertise and computational expense. In cases of data abundance, the knowledge-based and statistical models can perform better in terms of computational power, accuracy, and expertise level. It is also recommended to use fuzzy logic for prediction modelling if the data are not available, ambiguous, and/or vague. Fuzzy logic can utilize engineering judgments and expert opinions when developing the corrosion prediction model.

The poor quality of corrosion images, because of the presence of noise, interference, distortion, illumination, and image stabilization, is the main concern for computer vision and image processing techniques. Automated image acquisition is highly recommended when pre-processing techniques are all embedded in the imaging equipment, thus, it can produce a high-quality image. On the other hand, most of the developed computer vision and image processing techniques were implemented on simple curves and flat surfaces. It is recommended to include other complex structures and geometry, as observed on heavy machines, ship hulls, bridges, etc. Furthermore, the trend analysis can be implemented to predict corrosion progression or growths on materials. The prediction model can be used to predict maintenance in order to invent an automated corrosion maintenance system.

7. Conclusions

Corrosion continues to be the biggest concern faced by researchers, academics, and industries in this field, affecting ship builders, wastewater pipelines, and oil and gas pipelines. In this paper, state-of-the-art computer vision and image processing approaches have been assessed and summarized in relation to corrosion prediction models and corrosion detection approaches. The fundamental advantages and limitations of these methods and approaches have also been discussed. Most of the examined techniques have provided considerable quantitative and qualitative data regarding the detection and advancement of corrosion from images and videos. As a result, it is possible to evaluate the current state of corrosion and forecast its future condition. Moreover, it is impossible to obtain all necessary information from a single computer vision or image processing technique. As a result, multiple approaches or techniques are suggested to be used when testing the different environmental conditions or parameters for corrosion detection, assessment, and prediction. Furthermore, this paper has presented challenges and future recommendations for computer vision and image processing approaches in corrosion detection, assessment, and prediction, which could contribute toward a clearer direction for more pragmatic research. On-time monitoring with computer vision and image processing approaches are crucial from safety, economic, and health perspectives. These approaches can also be implemented with IoT, big data, wireless networks, laser scanning, ultrasonic scanning, and advanced sensor networks. The findings of this review have the potential to introduce new and supplementary insights into computer vision and image processing approaches for corrosion prediction, detection, and assessment. Further studies are necessary to capitalize on and maximize the potential of the recent advancements in IR4.0 technology, including advanced sensors and artificial intelligence, to improve future corrosion prediction, detection, and assessment.