Application of EfficientNet and YOLOv5 Model in Submarine Pipeline Inspection and a New Decision-Making System

Abstract

Submarine pipelines are the main means of transporting oil and gas produced offshore. The present work proposed a deep learning technology to identify damage caused by characteristic events and abnormal events using pipeline images collected by remotely operated vehicles (ROVs). The EfficientNet and You Only Look Once (YOLO) models were used in this study to classify images and detect events. The results show that the EfficentNet model achieved the highest classification accuracy at 93.57 percent, along with a recall rate of 88.57 percent. The combining of the EfficentNet and YOLOv5 models achieved a higher accuracy of detecting submarine pipeline events and outperformed any other methods. A new decision-making system that integrates the operation and maintenance of the model is proposed and a convenient operation is realized, which provides a new construction method for the rapid inspection of submarine pipelines. Overall, the results of this study show that images acquired via ROVs can be applied to deep learning models to examine submarine pipeline events. The deep learning model is at the core of establishing an effective decision support system for submarine pipeline inspection and the overall application framework lays the foundation for practical application.

1. Introduction

Numerous industries are turning their attention to artificial intelligence and robotics [1,2,3] and how to best combine their features in the future [4,5]. At present, new technology has been widely applied to pipeline inspection under or over water because the safe use of water resources is closely linked to the pipeline network. For example, transient test-based techniques (TTBTs) are used for the fault monitoring of water transmission networks (TM) [6]. This technology is different from traditional technology, showing excellent performance. In recent years, seawater resources have received widespread attention. Seawater is also an important source of drinking water for humans, and desalinated water is able to be consumed by people [7]. The natural cycle of water is closely related to the sea. Guarding seawater resources is of great significance to human safety and health. Submarine pipelines are one of the most important ways to transport marine oil and gas [8], and their safety is of great concern. Submarine pipelines are mostly located in seabed environments which are complex and changeable geomorphologies. Erosion, bottom trawling for fish and other hazards can all cause pipeline accidents serious enough to produce extensive water pollution and ecological damage and result in significant impact to the marine industry [9,10,11]. It is possible to prevent and eliminate the frequency of submarine pipeline accidents by accurately detecting the state and position of the pipelines. However, because of the increasing length and operation time of submarine pipelines, the traditional inspection and maintenance methods appear to be unable to meet the requirements of marine safety management [12]. In order to ensure the normal operation of pipelines, it will be necessary to upgrade inspection methods with innovative technologies to ensure their long-term safe operation. Research on these new methods has received broad attention.

Some conventional methods have been used. For example, Xing et al. [13] proposed an IPSO-GRU neural network for quantitatively identifying submarine pipeline defects, achieving an average accuracy of 96 percent. Luo [14] detected the leakage point of submarine pipelines based on a deep learning method that determined the location of leakage points using binocular vision. It achieved good real-time and accuracy results, but was only based on a small amount of data. Ma et al. [15] designed an autonomous inspection system using light vision technology that comprehensively evaluated pipeline anomalies with a You Only Look Once (YOLO) model. It had a recognition accuracy of 96 percent, but the system has still not been studied in deep-sea environments. Recently, the University of Tokyo in Japan developed an underwater robot named Twin-Burger2 to track underwater pipelines, but it cannot be applied in real time in complex offshore environments [16]. Developing new methods of submarine pipeline inspection under complex environmental conditions remains a challenge.

Remotely operated vehicles (ROVs) have become an important tool for underwater pipeline inspection worldwide [17,18,19]. They can be equipped with various sonar sensors and high-definition cameras that can transmit real-time monitoring data. ROVs can enhance detection in many ways. For example, Chen et al. [20] placed forward-looking sonar equipment on an ROV to inspect a pipeline in turbid water through edge detection and feature extraction. Sravya et al. [21] used a support vector machine and a random forest classifier with five features to inspect a pipeline, and saw the random forest classifier with directed gradient histogram achieve a 98.81 percent detection accuracy. Schyen et al. [22] studied a fast, accurate and automated method that used deep learning of multi-beam echo sounder data, a customized deep residual network (ResNet) and a YOLO deep learning model. They achieved real-time and accurate detection superior to baseline maximum likelihood methods. This new approach provides a convenient method for the intelligent detection of submarine pipeline defects, and provides evidence that further research on using ROVs to enhance submarine pipeline inspection would be beneficial.

While applying machine learning to the use of ROVs for the inspection of submarine pipelines (as mentioned above) has been reported throughout the world, this study provides a new method. Using submarine pipeline images collected by an ROV, we studied a deep learning inspection technology to establish a system to identify characteristic and abnormal events of submarine pipelines. Our research focused on creating effective learning methods and inspection systems to ensure the feasibility of detection. We compared the performance of many deep learning models, hoping to obtain a better model. These studies enhanced the application of ROVs. It would provide great benefits for the timely detection of underwater pipeline defects to prevent water pollution caused by accidents. In addition, it would reduce false alarms in visual inspections and improve the overall efficiency of underwater pipeline inspection. Finally, we proposed an effective overall framework for controlling the inspection of submarine pipelines. This study would be important for the inspection of any underwater pipeline.

2. Data Collection and Models

2.1. Data Collection

In this study, we collected images of subsea pipeline using Fugro’s FCV3000 ROV instrument, The Netherlands. They were labeled in two categories, with 1 indicating that there was an event in the image and 0 indicating that there was no event in the image. The labeled images were used as the training set and verification set for model learning, and a certain number of unlabeled images were selected as a test set. There were 8961 images selected to train the ROV image binary classification model (corresponding to 0 and 1). The ratio of the training set to the verification set was 17:3 (5200 samples for training set and 916 samples for validation), and the source of the test set (2845 images) was real-time images of submarine pipeline inspection. Each evaluation index value of the experiment was finally averaged by 5-fold cross validation. All the experiments were performed on an Amax NVIDIA Titan V server with a 12-gigabyte GPU.

The images captured by the ROV were labeled with event detection, the location and size of the event were marked with a rectangular frame, and the event detection category was marked. The detected events included subsea pipe field joint, anode, debris (e.g., metal, plastic, wood and fishing net), sleeper, free span, gravel and sandbag. The labeled images were used as the training set and validation set for the learning of the detection model, and a certain number of unlabeled images were selected as the test set. The dataset is shown in

Table 1. Event detection dataset.

Event Name	Quantity	Training Dataset	Validation Dataset	Test Dataset
Fieldjoint	2872	1995	351	526
Anode	1662	1155	203	304
Debris	2206	1531	270	405
Sleeper	901	685	120	96
Freespan	238	180	32	25
Gravel	216	164	29	23
Sandbag	841	640	112	89

. The 8936 images were selected to train the ROV image event detection model, in which the ratio of the training set to the verification set was 17:3, and the source of the test set was real-time images from ROV inspection. Each evaluation index of the model was finally averaged by 5-fold cross validation. All the experiments were performed on an Amax NVIDIA Titan V server with a 12-gigabyte GPU.

2.2. Classification and Detection of Images by Deep Learning Models

Subsea pipeline inspection results include the output event category and the location of the event on the map. With the powerful fitting ability of deep neural networks, deep learning detection technology can input images end to end as it outputs inspection results, all while avoiding a large number of manual design features of traditional machine learning and achieving better detection results. The factors to be identified during patrol inspections included characteristic events such as flanges, cement briquettes, cement sleepers, pipe head connection points and anodes. There were also abnormal events such as pipeline suspension, garbage, anode corrosion and damage. Because it presented the diversity of events and seafloor images, we classified events using the convolutional neural network, EfficientNet, and the specific categories and locations of events were detected by YOLO v5 (hereafter, YOLO refers to the model). Overall, the subsea pipeline inspection methods tested in this study were divided into object classification and object detection.

2.2.1. EfficientNet Model Classification

The YOLO event detection algorithm was only used to detect the type of event and its location in the image. This not only improved the overall efficiency of submarine pipeline event detection, but also helped to find and record unknown events. Before the detection, we needed to know the event classification, expressed by images in 0 and 1.

2.2.2. EfficientNet Model Learning

The ocean’s complex environments can lead to the serious degradation of underwater images obtained by optical vision systems, so this study used deep learning technology to detect submarine pipeline targets. Convolutional neural networks have achieved significant success in image processing. Alternately, stacked convolutional and pooling layers can effectively capture image-rich features and use them to make fully connected layers that show the output of individual classes or class probabilities. EfficientNet is a hybrid model-scale deep convolutional neural network that balances the depth, width and input image size by setting certain parameter values to achieve the best performance. In this section, we built an image binary classification model based on EfficientNet, and the component structure that was tested in this study is shown in Figure 1. The network used the preprocessed image (denoted as X) as the input and the manually labeled class probability (denoted as Y) as the label to classify the image (0/1). In this study, images were used as the data source to construct the binary classification model and the detection model. The image resolution was 720 × 576. The image classification can preliminarily detect whether the submarine pipeline event occurs or not.

The model we established used three basic modules. Each child module block was connected to the previous block creating a skip connection. Each square in a module represented an operation consisting of Conv2d, batch normalization, rescaling, activation and global average pooling. We used the Swish activation function of $f\left(x\right)=x·sigmoid\left(\beta x\right)$ where $\beta$ is a parameter that can be obtained through AI learning, and x is the input value and using it is considered better than the ReLU activation function in training the CNN network model [23,24,25]. In this study, we used the Conv2D which is a depthwise separable convolution [26]. Finally, the corresponding feature maps were added.

EfficientNet produces eight different versions by gradually increasing the depth, width and input image size. The figure shows the B0 version (hereafter, EfficientNet refers to the model). This version has the least parameters and the highest efficiency. For details of other versions, please refer to the specific paper [27]. The binary classification model was constructed based on EfficientNet, where the input of the model was the enhanced inspection image, and the output was 0 or 1, where 0 represented no event in the image and 1 represented an event. The loss function for training the binary classification model is CrossEntropyLoss [28], which was used to determine the similarity between the actual output and the predicted output in the classification of submarine pipeline images. The true class is calculated in the form of one-hot coding. The loss function is shown in Equation (1).

$$\begin{array}{cc}\hfill CrossEntropyLoss\left(y,class\right)& =-\mathit{log}\left(\frac{\mathit{exp}\left(y\left[class\right]\right)}{{\sum }_{j}exp\left(y\right[j])}\right)\hfill \\ & =-y\left[class\right]+\mathit{log}\left({\sum }_j\exp \left(y\left[j\right]\right)\right)\hfill \end{array}$$

(1)

where y is a vector $y=(y_1,y_2,\dots ,y_n)$, where each value y_i represents the probability that the predicted sample is of class i, class is the true class of the sample, and j represents the total number of classes in the dataset. Cross entropy is used to determine how similar the actual output is to the predicted output when performing sea tube image classification.

Due to the limitation of image classification efficiency in the process of submarine pipeline inspection, we selected classification methods such as residual network (ResNet50) [29], DenseNet [30] and PyramidNet [31] to compare with EfficientNet. ResNet50 quickly learns the characteristics of events in tube images by building deeper networks through residual connections. The input of each layer of dense convolutional network (DenseNet) includes the output of the previous layer and the input of all layers before the most recent one. Through dense connection, the gradient disappearance is alleviated, the feature multiplexing and propagation are enhanced, and the number of parameters is reduced, which improves classification. PyramidNet improves on ResNet by using an additive pyramid to incrementally increase the dimensionality. It also uses a zero-padded directly connected identity map, which is wider, more accurate and more generalizable than DenseNet.

2.2.3. YOLO Model Detection

After the second classification of the subsea pipeline images, this section carried out specific event target detection for images with events, determining the type of event and its position and size in the image. The timely discovery of conventional and abnormal events is very important for the normal operation and maintenance of the subsea pipelines despite challenges such as image blurring and events that are too large or too small. Target detection is divided into an anchor-based method and anchor-free method [32]. The anchor-based method views target detection as a problem of classification and regression of candidate region, and it is divided into a one-stage method and two-stage method. The one-stage method has a faster detection speed and the two-stage method has better detection accuracy. Anchor-free technology abandons anchor and completes the detection by determining the key points. This greatly reduces the number of network hyper parameters, but causes an unstable detection effect. In order to balance the detection speed and detection effect, this study selected the one-stage YOLO end-to-end training target detection model. Manual judgment and identification of events such as sundries, suspended pipelines, anodes and nodes with ROV video images consumes a lot of manpower and material resources, and also reduces the efficiency of the operation. In addition, deep-sea images have low definition, and there are many kinds of subsea pipeline events with different sizes. Both manual detection and traditional detection methods have low efficiency and low accuracy. Therefore, building an efficient and accurate detection system based on artificial intelligence is very important. The YOLO algorithm uses a separate CNN model to achieve end-to-end target detection. The detailed parameters and operation processes tested in this study are shown in Figure 2.

The backbone portion of this system is mainly used for the feature extraction of submarine pipeline images, in which Focus is used for downsampling to reduce the amount of calculation and speed up the network. ResUnit is used to learn more abundant features, and spatial pyramid pooling (SPP) is used to realize the fusion of local features and global features at the feature map level. The subsequent structure of the backbone network fuses the extracted features and transfers them to the prediction layer. Finally, the head portion outputs the detection result. The YOLO model first crops the input image to a uniform size, then sends it to the backbone network, and finally processes the prediction results to get the detected target. The YOLO model uses Mosaic data enhancement to account for limited data when required, and also uses the non-maximum suppression algorithm (NMS) to deal with multiple target detections, which can achieve classification while detecting. The event detection model was constructed based on the YOLO model, and the loss function of the detection training model consisted of three parts: confidence loss (L_obj), class loss (L_class) and location loss (L_box) [33]. The L_obj and L_class used cross-entropy loss function corresponding to BCE with logits loss and focal loss, respectively. The focal loss is a strategy that considers the serious imbalance of positive and negative samples in target detection, reduces the influence of sampling, which is easy to classify under the loss function, and focuses on the training of samples which are difficult to classify. The L_obj and L_class can be expressed using the following form (see Equation (2)):

$$L_{obj}/L_{class}=-\frac{1}{n}\sum _{i=1}^n[y_i\times \log \left(\sigma \right({y^{\prime }}_i))+(1-y_i)\times \log (1-\sigma \left({y^{\prime }}_i\right))]$$

(2)

where ${y^{\prime }}_i$ represents the confidence of the predicted output, $y_i$ is the true probability of 0 or 1 in L_obj. ${y^{\prime }}_i$ represents the confidence of the predicted output and $y_i$ is the true probability of 0 or 1 in L_obj. $\sigma$ is calculated as follows (see Equation (3)):

$$\sigma \left({y^{\prime }}_i\right)=\frac{1}{1+\mathrm{e}\mathrm{x}\mathrm{p}(-{y^{\prime }}_i)}$$

(3)

L_box employs GIOU loss as shown in Equation (4) where GIOU means taking two arbitrary boxes A and B, and finding a minimal closed shape C so that C can contain A and B. Calculate the ratio of the area of C that does not cover A and B to the total area of C, and then subtract this ratio by the intersection and concatenation ratio (IOU) of A and B. Finally, subtracting GIOU by 1 transforms the optimization problem into a minimum.

$$\begin{array}{cc}\hfill L_{box}& =1-\mathrm{G}\mathrm{I}\mathrm{O}\mathrm{U}\hfill \\ & =1-\mathrm{I}\mathrm{O}\mathrm{U}+\frac{|\mathrm{C}\backslash (\mathrm{A}\cup \mathrm{B}\left)\right|}{\left|\mathrm{C}\right|}\hfill \end{array}$$

(4)

Finally, we selected methods widely used in the field of image detection for comparison with the method tested in this study, such as Single Shot MultiBox Detector (SSD) [34] and Scalable and Efficient Object Detection (EfficientDet) [35].

2.3. Model Evaluation Index

Accuracy, precision, recall and F1 score are effective parameters to evaluate the performance of neural network models [36]. In this study, they were used as classification evaluation indexes for measuring the performance of subsea pipeline image classification by EfficientNet, as shown in Equations (5)–(8). Accuracy referred to the proportion of correctly classified samples to the total number of samples and was also called precision. Recall referred to the proportion of positive samples with correct classification to the number of real positive samples. The lower the recall was, the more images of missed events there were. F1 score is the harmonic average of precision and recall, which comprehensively evaluates the effectiveness of the model.

$$Accuracy=(TP+TN)/(TP+FP+TN+FN)$$

(5)

$$Precision=\left(TP\right)/(TP+FP)$$

(6)

$$Recall=\left(TP\right)/(TP+FN)$$

(7)

$$F1 score=2\ast (Recall\ast Precision)/(Recall+Precision)$$

(8)

where TP represents the number of samples in which the positive sample was judged to be a positive example, TN represents the number of samples in which the negative sample was judged to be a negative example, FP represents the number for which the negative sample was judged as a positive example and FN represents the numbers of samples for which the positive samples were judged to be negative examples.

The frame rate (FPS) and mean average precision (mAP) were used to evaluate the detection of subsea pipeline events. FPS was used to evaluate the speed of object detection (number of images processed per second, or image refresh rate). The accuracy for an image for events of category C is shown in Equation (9), which represents the number of events detected correctly and the number of events not detected correctly (denoted as false detection or missed detection).

$$P_C=\frac{(T{P)}_C}{(T{P)}_C+(F{P)}_C}$$

(9)

The accuracy over all images for events of category C is shown in Equation (10) where N was the total number of images.

$$A{P}_C=\frac{1}{N}\sum P_C$$

(10)

The accuracy map of events of all categories on all images is shown in Equation (11), which represents the number of categories (N_c).

$$mAP=\frac{1}{N_c}\sum {AP}_C$$

(11)

It was always calculated on a fixed dataset, and it looked at the average accuracy of each class while analyzing the model results, which avoids the preference for some easy-to-detect classes and was suitable for comparing different target detection methods.

3. Results and Discussion

3.1. Characterization of Sample

The data in this study came from image frames of video collected by ROV inspection of a submarine pipeline. Once the left, middle and right image frames were obtained, the middle image was preprocessed, then all the extracted images were labeled by two classifications and event detection. The images are shown in Figure 3a–c. In the middle of the images, light absorption led to color deviation, and scattering led to low contrast and blurred details. When deep learning technology is applied to submarine pipeline images, image degradation will not only affect the processed results, but, in serious cases, also invalidate deep neural network models. So, image enhancement has become a necessary link before detection. Using multi-operation joint enhancement technology can obtain a stable and clear image. In this scheme, images are enhanced using operations such as optimized contrast enhancement [37], adaptive histogram equalization with limited contrast (CLAHE) and color-level adjustments [38,39].

Optimized contrast enhancement constructs a cost function consisting of a contrast term and an information loss term, enhances the contrast by minimizing the cost function and optimally preserves the information to achieve image deblurring. CLAHE determines a mapping curve through the gray distribution histogram of the image, which is used to transform the gray level and improve the contrast of the image. The color gradation adjustment obtains the adjustment coefficient by calculating the deviation between the black field and the white field, and adjusts the R, G and B channels of the image to achieve color cast correction. The image enhancement effect is shown in Figure 3b. Different images, showing events such as field joint, anode, sleeper, debris, gravel, free span and sandbag, are shown in Figure 4.

3.2. Classification Results

The performance of various classification methods used to train the binary datasets of subsea pipeline images is shown in Figure 5. The figure shows the accuracy, precision, recall and F1 score values. The results show that for underwater images with color deviation and blurred details, ResNet50 overcame network depth gradient vanishing by residual connection, and achieved 80.74 percent accuracy and 94.82 percent recall, but the classification efficiency of ResNet50 took 100 milliseconds for one sheet, which is not very efficient. DenseNet achieved 80.74 percent accuracy and 94.82 percent recall, but its repeated splicing led to considerable computer memory consumption and the efficiency was not able to meet the demand of real-time tasks. PyramidNet utilized the golden tower instead of downsampling, obtained more image features and achieved 80.74 percent accuracy and 94.82 percent recall, but it was also inefficient due to the wider network. EfficentNet searched the network architecture for network depth, width and image size, and constructed a better model relative to other network architectures, achieving 93.57 percent accuracy and 88.57 percent recall, while classifying an image in only 0.5 s, which meets the real-time requirements of subsea pipeline inspection. EfficentNet also showed good results for precision and F1 score with the highest marks for those evaluation indexes. For detection speed, the DenseNet model had the fastest response time as indicated by FPS (see Figure 5b), followed by EfficentNet. However, DenseNet’s other indexes values were not as good as EfficentNet’s values. Recently, another study also gave positive results. For example, 2D convolutional neural networks achieved an F1-Score of 90% [40]. Xie et al. [41] used acoustic signals to train a time-frequency distribution map-based convolutional neural network model for underwater pipeline leakage detection, showing 95.5% of accuracy. All of the results showed that our model performance is not worse or better than the results of other models.

3.3. Detection Results

The YOLO method’s detection results are shown in Figure 6. When compared with the other methods, it achieved an mAP value of 0.9242 and reached 25 FPS, which were the best results, showing that YOLO can be the optimum choice for the detection of subsea pipeline events. For several events such as nodes, anodes, sundries, supports and hanging submarine pipeline, YOLO gave good results, especially for the detection of field joint, anode, debris, sleeper and free span, where its detection accuracy was over 0.95. However, Yolo only had detection accuracies of 0.7453 for gravel and 0.7883 for sandbag. However, overall, the YOLO method showed significant potential for determining the status of pipelines in the complex environment of the seabed.

3.4. Influence of Complex Environments on Detection

In the above study, we built deep learning detection models for the effective recognition of objects, and we examined the accuracy of deep learning in recognizing different objects. In this study, we also found that even under complex environmental conditions, these models do a good job detecting events. This can be seen in Figure 7, which shows the accuracy of different models for recognizing objects. There are combinations of multiple objects in these complex environmental conditions, and the accuracy of recognition for some clutter is not high, but many are still detected. This indicates that the deep learning model is a highly feasible method to be applied to objects under submarine pipelines. However, because of the complexity of the seabed, it is important to improve the recognition ability of deep learning accuracy under these conditions. At the same time, the quantization problem due to the existence of seabed objects deserves attention. When objects are detected, how much influence these objects have on the submarine pipeline has not been explored in depth, and quantitative analysis appears to require more research.

3.5. Discussion of Submarine Pipeline Inspection Management Systems for Industry Applications

Deep learning models are at the heart of subsea pipeline inspection. Few studies have addressed an overall operational framework for better monitoring in rapid applications. However, while subsea pipeline monitoring is important, it also needs to be supported by visualization systems that connect to ROVs. How to build an effective organizational framework is an important aspect of current submarine pipeline inspection. In this study, a framework was developed based on the trained deep leaning model. It provides functions for two types of users. The first is the system administrator, who manages the basic data, including event type. The second is the patrol inspector, who creates the inspection task and executes it. This includes connecting the ROV equipment, playing the ROV the inspection video, turning on the event detection switch in the process of playing the video, recording the detection results of conventional events and abnormal events, querying the event record results and exporting the submarine pipeline patrol inspection report according to the template. The whole operation process is shown in Figure 8.

A well-functioning system facilitates the monitoring and operation of the system. The interaction of hardware, software and operators is critical. The subsea pipeline patrol monitoring process includes the equipment access layer, the patrol inspection operation client and the video recognition algorithm layer. The service objects of the system include the system management unit, the patrol inspectors, the department heads and the enterprise leaders. The overall architecture is shown in Figure 9. The device access layer manages the relevant equipment involved in patrol inspection, such as ROV devices, video devices and positioning devices. The device access layer saves the network address and the service port and communication protocol information of the devices. These devices can access the submarine pipeline inspection system through the network, and the information of the acquisition equipment can be used as the parameter input of the subsequent algorithm layer. The client is the main entry for the daily work functions of the patrol operation users. After logging into the system, users can perform various business operations, such as parameter setting, creating patrol tasks, executing patrol tasks, connecting devices, playing ROV videos, enabling event detection, recording abnormal events and exporting patrol reports.

The algorithm layer of the submarine pipeline inspection system integrates a traditional vision algorithm and machine learning vision algorithm to process video images. The algorithm includes image enhancement, image classification, image segmentation, target detection and anomaly detection. It is a comprehensive algorithm processing service platform, providing algorithm support for target event recognition. The submarine pipeline inspection system functions include system management, parameter setting, inspection tasks, equipment management, offline video, inspection operations, event management and other modules.

3.6. Submarine Pipeline Inspection and Management Framework

In this study, we proposed a method for constructing a submarine pipeline management system that includes an operable process and maintainable function. The basic components consist of an inspection operation unit and a system management unit. Submarine pipeline management maintains inspection information, and user management maintains the addition, modification, deletion and enabling and disabling of basic information. Only the basic information of inspectors added in the user management can be entered into the submarine pipeline inspection system. Event type provides the classification function of common events in the inspection process, which should also include the maintenance function of event information, such as event code and event level. The overall system is shown in Figure 10.

During the patrol inspection process, a task must first be created by selecting the submarine pipeline to be inspected, then filling in the name of the operating vessel and the ROV equipment number, and finally, generating a “pending” patrol inspection. After the task is executed, the ROV device can be connected, and the client software will play video in real time. The ROV device can also be disconnected in device management. The client software supports the selection of local video files for loading and playing through the offline video function. During video playback in the client software, event detection can be started or stopped. When the task is started, the patrol task status is not modified to “Patrol”, the video captured by the camera is transmitted to the algorithm platform and the video is analyzed and processed in real time to detect whether there is a routine event or an abnormal event. “Suspend Patrol” indicates that the patrol task is suspended, and the video will not be transmitted to the algorithm platform for analysis and processing. At the same time, no routine events or abnormal events will be detected. We built a Graphical User Interface (GUI) system (see Figure 11), which is able to perform the above steps well and conveniently.

4. Conclusions

Due to the complex and changeable seafloor environment, submarine pipelines are inevitably subject to natural erosion and abrasion as well as man-made hazards that can cause damage that needs to be identified quickly. Because the manual detection of events is inefficient, and false detection and leakage rates remain high, this study proposed a submarine pipeline auxiliary detection method based on deep learning technology. It automatically identifies and records various events by learning the characteristics of the events present in the pipeline. The presence or absence of events in an image is recognized by a classification method that facilitates the identification of previously unrecorded events. Then for images where events exist, the exact location and category of the events are determined by the target detection method. The recognized events are also recorded by the designed detection system for subsequent analysis and query. We conducted subsea tests of the proposed method and determined that it can greatly improve the efficiency and accuracy of submarine pipeline inspection compared with manual inspection. Overall, this study proposed an effective subsea pipeline inspection model and decision-making system. However, since this model also faces the challenges of complex environments, we suggest further improvement of the submarine pipeline inspection system by identifying event types and quantitatively analyzing events such as suspension, tearing and damage.