1. Introduction
As a quality renewable energy generator, the safety operation of hydropower plants is crucial to social and economic development. The inlet pressure steel pipe is an important functional part of a hydropower generator unit. It is a large structural component that utilizes the pressure difference generated by the height to drive the water flow into the hydraulic turbine. The pressure steel pipe inspection mainly refers to the detection of anti-corrosion layer damage on the inner wall, surface, and internal cracks on the welding seam. Its specific process includes high-pressure water cleaning, surface visual inspection, and internal flaw detection. Traditional manual inspection has the disadvantages of low efficiency and high operational risk. Considering the magnetic conductivity of the pressure steel pipe, a novel non-contact magnetic climbing robot is proposed to complete the inspection tasks in this work.
Wall-climbing robots are specialized robots which are capable of moving on steeply inclined or even vertical surfaces, by utilizing adhesion and movement mechanisms [
1]. If the stable adhesion and flexible movement can be guaranteed, the unrestricted environmental adaptability of wall-climbing robots would be beneficial for inspection, maintenance, and manufacturing tasks in engineering fields [
2], such as aircraft skin inspections [
3], offshore wind turbine inspections [
4], glass façade cleaning [
5], and cable-stayed bridge maintaining [
6].
Magnetic adhesion, one of the most mature methods, can be used to achieve stable adhesion on the pressure steel pipe by employing electromagnets or permanent magnets. The control of magnetic force is easy by adjusting current intensity in electromagnets. But the heating effect and the force disappearance caused by power loss are unacceptable flaws in climbing applications. Thus, it is much more common that permanent magnets are chosen as the magnetic adhesion units in climbing robots. A permanent magnet adhesion wall-climbing robot [
7] was developed for inspection and maintenance on ships. The magnetic disks were embedded into the tracks, rolling with the tracks and sequentially attaching or detaching from the surface. Park [
8] presented a magnetic adhesion wall-climbing robot with each sub-robot connected via hinges. By coordinating the movement states of each sub-robot, safe transitions between different planes could be achieved. Nguyen [
9,
10] utilized a linear reciprocating mechanism, hinge mechanism, and magnetic tracks to enable the robot to adapt to surfaces of different configurations. A serial structure with magnetic wheels enables the climbing robot to traverse most complex geometric structures on steel structures and move efficiently. Adinehvand [
11] installed magnetic suction cups at the end of a robotic arm to develop a compact, multifunctional wall-climbing manipulator. Eto [
12] innovatively designed magnetic units, developing spherical magnetic wheels with two rotational degrees of freedom, aligning the magnetic force with the normal of the adhered surface.
To realize the flexible movement on the pressure steel pipe, wheeled mode and tracked mode are more efficient than leg mode in climbing robots. Compared to wheeled mode, tracked mode has superior obstacle-crossing capabilities due to the significantly larger contact area between the tracks and the surface. This enables climbing robots to easily traverse surface obstacles such as weld seams and rusted protrusions. Kermorgant [
13] developed a tracked wall-climbing robot for welding on the outer surfaces of ships. By integrating the adhesion modules into the tracks, the loading capacity was over 100 kg. Seo [
14] connected two tracked wall-climbing robots using flexible joints to achieve compliant motion, enabling the robot to smoothly navigate extreme positions such as inner and outer corners of steel frames.
The primary objectives of pressure pipe inspection typically involve detecting corrosion on the welded surfaces of steel plates and cracks within the weld seams. This necessitates that the climbing robot be capable of identifying weld seams under non-structured light conditions, thereby guiding the robot to move along the weld seams and complete the inspection tasks. Visual image processing systems have been applied as a means of weld recognition in industries [
15], such as chemicals and shipbuilding. Traditional image processing methods involve denoising and enhancing the image, followed by feature extraction methods to extract information related to the weld [
16,
17], such as edge detection, region growing, and template matching. Weld recognition methods based on semantic segmentation have attracted increasing attention from researchers in recent years due to their high efficiency and accuracy. Semantic segmentation based on convolutional neural networks (CNN) is designed to extract features and perform classification, then employ upsampling or deconvolution techniques to convert the results back to the original image size. Convolutional semantic segmentation networks with an encoding–decoding architecture are common neural network architectures used for image semantic segmentation tasks. Some classic convolutional semantic segmentation networks that use the encoder–decoder architecture include U-Net [
18], SegNet [
19], and FCN [
20]. Among these, FCN uses fully convolutional layers that skip connections to connect feature maps from the encoder to those in the decoder, helping to fuse information from different resolutions and improve segmentation accuracy. DeepLab is a series of semantic segmentation networks developed by Google [
21], including DeepLabv1, DeepLabv2, DeepLabv3, and DeepLabv3+. It uses atrous convolutions to increase the receptive field and conditional random fields (CRF) to improve the spatial consistency of segmentation. Pyramid scene parsing network (PSPNet) is a semantic segmentation method proposed by Zhao [
22], based on a pyramid pooling strategy. The core innovation of PSPNet is the introduction of a pyramid pooling module. This module divides the input feature map into multiple regions of different scales, performs pooling operations in each region, and then concatenates these pooled features. This enables the model to consider contextual information at different scales simultaneously, thereby improving segmentation performance.
In this work, a non-contact magnetic, tracked climbing robot for pressure steel pipe inspection is designed. Based on the magnetic adsorption analysis, the payload capacity is over 120 kg, which is sufficient for carrying several equipment for inspection. To enable climbing robots to follow the welding seam during the inspections, the improved Deeplabv3+ semantic segmentation method is proposed for welding seam recognition. The climbing robot was field-tested in the real pressure steel pipe, and the results show that the climbing robot can achieve reliable adsorption and flexible movement on the internal face of the inlet pressure steel pipe. The climbing robot can meet the requirements of safety and efficiency for pressure steel pipe inspection processes.
This paper is organized as follows:
Section 2 presents the mechanical structure and control framework of the climbing robot.
Section 3 focuses on the analysis of the slippage and overturning situations and presents the design of the magnetic adsorption unit.
Section 4 shows the frameworks of welding seam recognition.
Section 5 presents the experimental results and concludes with a discussion on the findings and future work.
4. Welding Seam Recognition
Since the inspections of pressure steel pipe primarily focus on the welded areas, the rapid and accurate weld recognition is critical in guiding the climbing robot to complete the inspections. As shown in
Figure 12, the weld recognition frame is designed based on traditional image processing technology and computer vision technology. Firstly, the collected weld images are preprocessed by filtering and enhancement which removes high-frequency noise and balances image brightness to overcome the impact of uneven ambient lighting. Secondly, the preprocessed weld images are used to build a dataset for training the semantic segmentation model for weld recognition. Then the trained model is applied process actual weld seam images for the climbing robot’s trajectory guidance.
Based on our preliminary work [
25], it shows that the quality of the weld image can be improved by filtering techniques, such as mean filtering, Gaussian filtering, and edge-preserving filtering, as well as enhancement techniques, such as contrast-limited adaptive histogram equalization. However, although the brightness and contrast of the enhanced weld images are balanced, the weld features are not prominent enough. Therefore, computer vision technology is needed to improve the accuracy of weld recognition.
DeepLab is a series of semantic segmentation networks developed by Google. These networks utilize atrous convolutions to increase the receptive field and employ conditional random fields to improve the spatial consistency of segmentation. DeepLabv3 introduces a multiscale feature fusion module, which combines feature maps from different convolutional layers to achieve better segmentation performance. Additionally, DeepLabv3 employs deformable convolutions to further enhance the receptive field. DeepLabv3+ adds a decoder module to DeepLabv3 to restore the spatial resolution of segmentation results. This helps reduce jagged boundary effects in segmentation results and improves segmentation quality.
The algorithmic model of DeepLabv3+ is shown in
Figure 13, which adopts the encoder–decoder structure. The encoder containing the dynamic convolution neural network component serves as the backbone network for semantic segmentation. To adapt to the processing ability of the perception controller and enhance recognition efficiency, this work replaces the classic Xception with MobileNetv2 [
26] as the backbone network.
MobileNetv2 is a lightweight convolutional neural network composed of two types of blocks, which are distinguished based on stride. It employs a series of lightweight design strategies, including separable convolutions, linear bottlenecks, and inverted residual structures. Feature extraction is divided into two parts: high-level semantic extraction and low-level semantic extraction. Firstly, 1 × 1 convolution kernels are used to perform sampling on the channels, serving a role similar to fully connected layers. Next, feature extraction is performed using three dilated convolution kernels. Dilated convolution increases the receptive field without adding parameters, helping to capture broader contextual information. Finally, pooling operations are performed to reduce the size of the feature maps, and the extracted feature maps are concatenated using the concatenate operation. The 1 × 1 convolution is then applied again to alter the number of channels, thereby obtaining high-level semantic information. For the low-level feature maps output by the backbone network, a 1 × 1 convolution is performed to transform the channels. The high-level feature information obtained through the above combination and channel transformation is then sampled by a factor of 4 to match the size of the low-level feature maps. The results are combined again using the concatenate operation, and the combined result is finally sampled by a factor of 4 to restore it to the input size, yielding the final output.
5. Experiments
The prototype of climbing robot designed in this paper is shown in
Figure 14, and its technical specifications are shown in
Table 1. To verify the adsorption reliability, weld seam recognition, and ability to complete pressure steel pipe inspections, experiments were conducted inside an actual pressure steel pipe at the Three Gorges Power Plant. The size of the chosen pressure steel pipe is huge; its dimensions are as follows: 120 m of axial length, 12 m of diameter, 30 m of radius of curvature in the turning section, 54 degrees of inclination, 40–60 mm of thickness, which is also the depth of the welding seam, and 25–35 mm of welding seam width.
5.1. Adsorption Reliability and Motion Test
Adsorption reliability is the most important prerequisite for inspections. According to task requirements, the climbing robot is driven to perform circular movements along the circumferential welds on the inner side of the steel pipe as shown in
Figure 15. According to the analysis in
Section 3.2, the climbing robot is prone to slipping, longitudinal overturning, and detachment hazards at 2.32 rad, 2.48 rad, and 3.12 rad, respectively. With a load of 100 kg and without any external safety measures, the experimental results showed that the climbing robot successfully completed the circular motion without any hazardous situations occurring.
The designed weld height for pressure steel pipes does not exceed 5 mm, and the protrusions caused by damage to the corrosion-resistant coating on the weld surface also do not exceed this value. Additionally, to compensate for deformation caused by thermal expansion and contraction, pressure steel pipes are equipped with expansion joints internally. The maximum distance from the protrusions and sink points to the surface of the pipe is 8 mm. Therefore, the climbing robot must be capable of traversing expansion joints under conditions where the tracks are subjected to pressure-induced deformation. Furthermore, since there is standing water and water flow inside the pressure steel pipe, the climbing robot must have water-crossing capability to ensure a continuous inspection process. As shown in
Figure 16, the climbing robot can pass through expansion joints without the chassis scraping. Additionally, the climbing robot can safely traverse water flows with a depth of 10 mm, enabling flexible movement along the inner surfaces of the pressure steel pipe.
5.2. Weld Seam Recognition
In this work, weld seam images are captured by the vision sensor. From the captured images, 300 images are selected from each weld seam to construct the dataset, with a sample ratio of 9:1 between the training set and the test set. The weld seam positions are manually annotated. Data augmentation is performed on the annotated sample set using techniques such as rotation and exposure adjustment, expanding the sample size to 1500 images. The training set is input into a semantic segmentation model for training, and the model’s weight parameters are iteratively optimized.
The weld seam recognition model runs on the perception controller with Ubuntu 18.04. During model training, a loss function is defined to quantify the degree of difference between the output of semantic segmentation model and the true values. The model updates its weight parameters through backpropagation to reduce the difference between the output and the true values, making the predictions more accurate. The loss function selected in this work is binary cross-entropy [
27], which effectively avoids gradient vanishing. The optimizer uses the adaptive moment estimation [
28]. It calculates the first-order moment estimate and second-order moment estimate of the gradient to design independent adaptive learning rates for the parameters in the model. In this paper, the cosine annealing algorithm [
29] is used to dynamically update the learning rate of the optimizer. The decline process of the loss function during the training of the semantic segmentation model is shown in
Figure 17. A total of 50 training iterations were performed, with the loss function value rapidly decreasing within the first 20 iterations and converging by the approximately 40th iteration.
In this test, images were captured from five weld segments at different locations within the same pressure steel pipe, and the model trained in the previous section was used for weld recognition. The recognition results are shown in
Figure 18. Given the wide variety of weld types, the semantic segmentation model needs to have a certain degree of generalization ability. Weld I in this test consists of images of different locations of welds in the training set, while welds II to V did not appear in the training set. The recognition results clearly show that the model has a certain degree of generalization ability and can still complete the recognition task with new welds. The mean intersection over union (
mIoU) is used for evaluating the performance of semantic segmentation model. That means the ratio of the intersection to the union of the model’s predicted results for each class and their true values, summed and averaged. The formula is as follows:
where
IoUi means the intersection-over-union (
IoU) ratio for the
i-th class label. It can be expressed as
where true positive (
TP) represents correctly classified positive samples, false positive (
FP) represents incorrectly classified positive samples, and false negative (
FN) represents incorrectly classified negative samples. The
IoU for the weld seam is also presented in
Table 2 as a quantitative reference for model performance.
The IoU can be understood as the overlap between the predicted value and the true value, reflecting the accuracy of weld recognition. The actual requirement for the degree of overlap of weld recognition results should be at least 65%. The quantitative data of the segmentation model shows that the average intersection ratio of the recognition results outside the training set reached 0.8325, meaning that the degree of overlap between the recognition results and the actual image position of the welds reached more than 80%.
5.3. Pressure Steel Pipe Inspections
As shown in
Figure 19, under the conditions of reliable adhesion, flexible movement, and efficient weld seam recognition, the inspection functions of the climbing robot are tested on an actual pressure steel pipe by carrying different inspection equipment (
Supplementary Materials).
The pump is used to supply high-pressure water at 240 bar to clean the weld seam surface by removing surface-adhering mud and damaged anti-corrosion coatings. The on-site water flow is chosen as the working medium. By controlling the rotation angle and swing speed of the joint motor connected to the water jet and by coordinating with the climbing robot’s speed, the cleaning effect and efficiency are ensured, with a cleaning speed of 4 m2/s.
Surface anti-corrosion layer detection can be performed after the cleaning. The climbing robot is controlled to move along the weld seam. The network camera transmits images of the area near the weld seam back to the motion controller, where staff members assess the damage condition. For areas with severe surface damage, image processing is used to calculate the damaged area and record it for future reference.
Internal flaw detection is performed using a PAUT detector. The detector is mounted on the body frame of the climbing robot via a connection mechanism that securely attaches the probe. The motion controller accesses the PAUT detector via Ethernet. In accordance with the process requirements for internal flaw detection, the inspection speed is set to 1 m/s.
For the three inspection tasks, internal flaw detection requires high precision in the relative pose control between the climbing robot and the weld seam, which is also the purpose of weld seam recognition. The detection data is primarily collected and recorded by the PAUT detector. With technological advancements, the use of detectors with online identification capabilities may significantly enhance detection efficiency. In the surface anti-corrosion layer detection, the location and area of corrosion near the weld seam vary significantly, and the view field of the vision sensor cannot meet the requirements, so a network camera is used for image acquisition.
As shown in
Table 3, the climbing robot for pressure steel pipe inspections possesses sufficient adhesive force to meet the high load requirements imposed by inspection equipment and long-distance composite cables. Compared with related works, the climbing robot offers higher mobility and operational capabilities, enabling efficient completion of tasks involved in inspections. It can be found that the climbing robot exhibits a lower adhesion-to-weight ratio, as contact-based adhesion generates greater magnetic force than non-contact mode. However, in tracked contact-type adhesion scenarios, non-operational magnets exist during track movement, reducing magnet utilization efficiency. Additionally, contact magnets may cause abrasion on anti-corrosion coatings during attachment and detachment. Through the motion test, the climbing robot has demonstrated the ability to achieve reliable adhesion without damaging anti-corrosion coatings.
There are still some issues in pressure pipe inspections that we will focus on in our subsequent research, including the positioning of the climbing robot inside pressure steel pipes, which is key to recording excessive defects and subsequent traceability; further improvement in weld recognition effectiveness and efficiency; and the design of offset strategies during internal flaw detection, mainly addressing the issue of mismatches between the climbing robot’s posture and the probe adjustment requirements. Increasing the degree of freedom of the probe position can be considered in future works to improve the probe adjustment effect.
6. Conclusions
This paper designs a multifunctional climbing robot for performing cleaning with high-pressure water, surface anti-corrosion layer detection, and internal flaw detection involved in pressure steel pipe inspections. To achieve reliable adhesion, two types of non-contact magnetic adhesion units were designed based on sliding and overturning analysis of the climbing robot. By utilizing Halbach magnetization, a high magnetic force-to-weight ratio can be achieved. To enhance weld seam recognition efficiency, an improved Deeplabv3+ semantic segmentation method was applied for weld seam recognition. The climbing robot prototype described in this paper was tested within an actual pressure steel pipe at the Three Gorges Power Plant. Experimental results demonstrate that the climbing robot possesses the required mobility and weld seam recognition capabilities, enabling it to complete pressure steel pipe inspection tasks.
At the same time, it can be found that the performance limitations still exist. Firstly, the current state of weld recognition needs to be improved. The weld seams inside pressure steel pipes are multi-layer, multi-pass welds constructed manually. This results in non-structural characteristics in weld surface width, bead distribution, and weld bead height. Combined with varying degrees of anti-corrosion coating damage caused by water erosion, the current weld recognition cannot achieve fully intelligent, unmanned motion navigation. In future work, we will focus on optimizing the visual unit’s lighting, vision sensor, and image processing methods to enhance image recognition accuracy. Additionally, integrating line laser ranging sensors to measure weld bead height for improved recognition accuracy is under consideration.
Secondly, the current climbing robot requires mounting water pipes during cleaning and uses composite cables for power and communication. The maximum equivalent load reaches 30 kg, increasing demands on the adhesion force. This necessitates larger dimensions and heavier weight, consequently reducing mobility. Given the substantial weld lengths within pressure pipes, single-operation completion is impractical. Future designs will explore swarm robotics approaches, deploying distinct robotic systems for tasks requiring water supply and composite cables versus the other tasks. For instance, surface anti-corrosion layer detection could utilize battery-powered systems to reduce weight and enhance mobility. Parallel operations can also be adopted throughout the pipeline to enhance overall inspection efficiency.
Simultaneously, to address existing defects, the current climbing robot could be further optimized by integrating articulated mechanisms and grinding/welding/coating equipment to repair these defects. This will ultimately achieve full-task operation for pressure steel pipe inspection and maintenance.