A Deep Learning-Based Three-Stage Method for Spillway Blockage Detection in Reservoirs

Abstract

Spillway blockage detection is crucial for flood prevention and disaster reduction in reservoirs. To address the challenge of detecting the spillway blockages under complex environmental conditions such as rain and fog, this study proposed a three-stage spillway blockage detection method based on deep learning. This method involved the removal of the rain and fog interference, the segmentation of the spillway boundary region, and blockage detection. First, a rain and fog interference removal algorithm based on the dark channel prior theory was developed. Next, an improved lightweight DeepLabv3+ semantic segmentation algorithm was adopted to segment the spillway region from the images. Finally, the improved YOLOv7 object detection algorithm was utilized to identify the blockage debris within a segmented spillway area. The experimental results indicated that the proposed method achieved an average precision of 80.32% under normal conditions and 77.77% under complex conditions, representing improvements of 9.93% and 6.65%, respectively, compared to traditional methods. This method significantly enhanced the detection and identification of blockages in complex environments and could provide effective support for intelligent reservoir flood control and disaster reduction.

1. Introduction

Reservoir projects have demonstrated substantial benefits in flood control, irrigation, water supply, power generation, and ecology, with the flood discharge capacity being critical to the safety of both reservoir and flood control efforts. The spillway that is a primary water discharge structure is vital for effective flood discharge [1]. However, in actual operation, safety hazards such as mountain floods or geological disasters can cause debris such as silt and falling rocks to obstruct the spillway, significantly compromising its flood discharge capacity and the overall safety of the reservoir [2]. In China, where reservoirs are numerous, the routine safety inspections of spillways typically rely on manual inspections combined with video surveillance [3]. This method is often inhibited by high labor costs and issues with timeliness and accuracy [4], particularly under adverse weather conditions, such as rain and fog. Therefore, the application of video surveillance images for intelligent inspection is essential for promptly detecting and addressing blockages or other abnormalities within the spillway. This approach not only enhances the real-time defense and early warning capabilities for reservoir safety and flood disaster prevention, but also reduces the operational management costs, improves the reservoir safety monitoring, and supports the national water security and the green development of water conservancy projects.

In recent years, the development of image processing and computer vision technologies has significantly increased the efficiency and value of applications in various fields. In the field of flood control and management, researchers have integrated machine vision technology to enhance disaster detection and provide timely warnings. For instance, Lohumi and Roy [5] combined deep neural networks with the gated recurrent units to propose a CNN-GRU model that effectively predicted the floods, enabling timely rescue operations. Vallimeena et al. [6] developed a face recognition model using convolutional neural networks, employing the face as a reference object and calculating a person’s height based on the golden ratio to analyze the flood depth and spread. Feng et al. [7] designed a CDDS network structure for crack image segmentation, improving the accuracy and intuitiveness of dam health assessments by detecting cracks more effectively. Yokoya et al. [8] constructed a flood and debris flow simulator based on the attention U-Net network and LinkNet architecture regression model. The proposed framework breaks the limitations of traditional remote sensing and can quickly estimate flood depth and terrain changes, providing important information for rescue. Basnyat et al. [9] implemented the U-Net semantic segmentation model to detect river water levels by segmenting the shape of the river, triggering an alarm when the water level reached a critical threshold. Yang et al. [10] employed drones to capture urban building images, which were analyzed using the YOLOv3 target detection model to determine flood coverage and assess the extent of flooding. Lin et al. [11] adopted the YOLOX network model to detect garbage blockages on manhole covers, aiding sanitation workers in managing waste accumulation, and preventing road waterlogging by ensuring timely sewage discharge. Kazemi Garajeh et al. [12] used a time series collection of remote sensing images to investigate the impact of shallow flood spread on vegetation density. After experimental comparison, they found that deep learning convolutional neural networks were superior to traditional models in detecting and mapping flood-affected areas, improving effectiveness in the accurate identification of flood-affected areas. Li et al. [13] proposed a TRCAM-Unet image segmentation model to monitor water levels in harsh environments. Based on the Unet network, the Transformer module and residual attention mechanism were integrated to enhance the association between the regional features and the expression of valuable information. Qiu [14] combined various image processing algorithms to effectively extract the parameters related to underwater dam cracks, enabling timely safety measures. Wu et al. [15] proposed a classification framework SRNet for various types of sedimentation in water conveyance tunnels, which enables the rapid and accurate identification of sedimentation in water conveyance pipelines and provides important technical support for disaster prevention. Furthermore, some researchers have conducted research on the automated detection of spillways, but most of their studies focus on crack detection and area recognition of spillways. In 2021, Feng et al. [16] proposed a lightweight STDD network that combines separable convolution and asymmetric convolution to improve the efficiency of defect detection in spillway tunnels, achieving the accurate real-time detection of spillway tunnels. In 2023, Zhang J et al. [17] proposed a deep learning-based algorithm for detecting small defects on spillway surfaces by introducing a spatial attention mechanism and an SE module, effectively solving the problem of the difficult detection of small targets. The algorithm has good generalization ability. In 2024, Wan et al. [18] developed a recognition model based on Mask RCNN to address the low efficiency of traditional methods for identifying soil and rock dam structures. They used ROIAlign to solve the quantization error problem in ROI Pooling and improve the recognition accuracy of spillway areas. It can be seen that deep learning technology and digital image processing technology can effectively achieve flood control and disaster reduction. However, the detection of foreign object blockage in spillways is still in a blank stage. Scholars have also conducted relevant research on foreign object blockage detection. In 2022, Iqbal et al. [19] employed the YOLOv4 network to create a culvert blockage model, helping to prevent flood disasters by ensuring timely water drainage. However, the detection of foreign objects blocking culverts remains limited, and no advanced methods have been proposed for specialized detection targets, such as blockages. In 2024, Vandale et al. [20] designed an unsupervised garbage screen blockage detection model based on the EfficientNet-B4 network to address the issue of clogging in garbage screens. This model made the detection process more efficient, but it still has limitations in dealing with complex environmental changes and has poor adaptability.

Current research in flood management mostly emphasizes fracture detection, flood area monitoring, water level assessment, vegetation density analysis, and the identification of foreign item obstructions in culverts. Nonetheless, research on identifying foreign object obstructions in reservoirs during inclement weather, such as rain and fog, remains insufficient. The spillway’s surrounding environment is intricate, and the identification of foreign item obstructions is significantly affected by influence from external objects, leading to substantial false positives. This research offers a three-stage method for detecting foreign item blockages in reservoir spillways, utilizing deep learning, to address the aforementioned issues and enhance flood warning management for reservoirs. The objectives of this investigation are as follows:

• Due to the complex and variable weather conditions in a reservoir, the rain and fog generated around the spillway will pose certain difficulties in detecting foreign object blockages. Therefore, a rain and fog interference removal algorithm is proposed to denoise the image and improve its quality.
• For the recognition scenario of a reservoir spillway environment, an improved DeepLabv3+ spillway boundary segmentation algorithm is proposed to avoid interference from objects in nonspillway areas. This algorithm enhances the recognition ability of spillway boundary areas via a lightweight backbone network and by introducing a CFF module, providing more accurate feature input for subsequent foreign object detection.
• In order to enhance the feature extraction and capture capabilities of the network, an improved YOLOv7 foreign object blockage target detection algorithm is proposed. By reconstructing the efficient backbone network and introducing the SPPFCSPC-M module, the interference of non-target features is effectively suppressed, and the accuracy of blockage recognition is improved.
• We implemented the rain and fog interference removal algorithm, spillway boundary area segmentation algorithm, and foreign object blockage target detection algorithm, and developed a three-stage detection method to enhance image clarity and eliminate extraneous interference factors outside the spillway area, thereby achieving precise recognition of blockage targets in intricate environments.

2. Three-Stage Reservoir Spillway Blockage Detection Method Based on Deep Learning

The proposed detection method consisted of three stages (Figure 1). First, the captured image of the reservoir spillway was preprocessed. If the image was affected by rain and fog, a rain and fog interference removal algorithm based on the dark channel prior theory could be applied to obtain a clear image. Subsequently, the improved DeepLabv3+ semantic segmentation model was implemented to isolate the spillway boundary from the complex environment, excluding the external interference factors. Finally, the segmented image was analyzed using the improved YOLOv7 target detection model to detect the blockages. This study aimed to design a blockage detection model with strong environmental adaptability and the high detection accuracy, thereby promoting the intelligent monitoring and early warning capabilities of reservoir flood control.

To further explain the implementation of the proposed method, the following details the three-stage algorithm.

2.1. Rain and Fog Interference Removal Algorithm

In summer, the reservoir surface exposed to the atmosphere often reaches high temperatures. During the rain or flood discharge periods, heat causes water molecules to evaporate, generating large amounts of water vapor near the spillway, which can produce fog and obscures images, hindering blockage detection. Therefore, after acquiring images from the monitoring equipment, it is essential to apply the digital image processing techniques to remove the rain and fog interference around the spillway for clearer images. This study introduced a rain and fog interference removal algorithm specifically designed for this purpose, comprising two main components: the dark channel prior algorithm and the global histogram equalization algorithm. The detailed process of the algorithm is illustrated in Figure 2.

2.1.1. Dark Channel Prior Algorithm

Under rainy and foggy conditions, water vapor alters the light path, leading to the low contrast and reduced clarity of the images captured by surveillance cameras, which can inhibit the spillway blockage detection. The dark channel prior algorithm effectively enhances the image clarity and color restoration. Therefore, this algorithm was employed to remove most rain and fog from the images. The specific steps of the algorithm are as follows.

The atmospheric scattering model [21] was applied to address the issue of the reduced visibility in the spillway images under severe weather conditions. This model is presented in Equation (1).

$$I\left(x\right)=J\left(x\right)t\left(x\right)+A(1-t(x\left)\right),$$

(1)

where $I\left(x\right)$ represents the image with rain and fog, $J\left(x\right)$ represents the processed image, $t\left(x\right)$ represents the degree of attenuation of light in rain and fog environments, with smaller values indicating higher levels of rain and fog; $A$ represents the global atmospheric light intensity coefficient, which can dynamically obtain the top 0.1% pixel area with the lowest transmittance in the dark channel image based on the initial image brightness and contrast, in order to determine the value of $A$.

In a spillway image devoid of a localized sky area, the reflective properties of the object’s surface and the light absorption effect result in one color channel across the RGB spectrum consistently approaching a value of 0 [22]. The dark channel prior expression is provided in Equation (2).

$$J^{dark}=\underset{y\in \Omega \left(x\right)}{\min }\left(\underset{c\in (r,g,b)}{\min }{J}^c\left(y\right)\right)\to 0,$$

(2)

where $J^{dark}$ represents the dark channel image; $J^c\left(y\right)$ represents one of the three-channel color images; and $\Omega \left(x\right)$ denotes the filter window centered on pixel $x$.

By normalizing Equation (1) and substituting it into Equation (2), the atmospheric transmittance equation $t\left(x\right)$ is derived, as shown in Equation (3):

$$t\widetilde{(}x)=1-\underset{y\in \Omega \left(x\right)}{\min }(\underset{c}{\min }{\displaystyle \frac{I^c\left(y\right)}{A}}),$$

(3)

By combining Equations (1) and (3) and setting the minimum threshold of $t\left(x\right)$ to $t_0$, a clear image can be obtained after removing the particles. The formula for image restoration is given by Equation (4).

$$J\left(x\right)={\displaystyle \frac{I\left(x\right)-A}{\mathrm{m}\mathrm{a}\mathrm{x}[t\left(x\right),t_0]}}+A,$$

(4)

The image obtained may exhibit varying processing effects due to the different concentrations of rain and fog. If the parameters in the dark channel prior algorithm were not adjusted promptly, the processed image could become dark, adversely affecting the subsequent detection. To address this issue, this study employed a global histogram equalization algorithm to adjust the image brightness.

2.1.2. Global Histogram Equalization Algorithm

The probability $P\left(k\right)$ of a pixel with a gray level relative to the total number of pixels in the entire image is given by Equation (5):

$$P\left(k\right)={\displaystyle \frac{n_k}{n}},$$

(5)

where $k$ represents the gray level; $n$ represents the total number of pixels in the image; and $n_k$ denotes the number of pixels contained when the gray level is $k$.

The grayscale cumulative histogram represents the number of pixels in an image with grayscale levels below a specified threshold. The formula for the cumulative distribution function is given by Equation (6):

$$s(k)={\sum }_{i=1}^k{p}_i,(k=\mathrm{0,1},2\dots ,L-1),$$

(6)

where $L$ is the number of gray levels of the entire image, and $p_i$ represents the number of pixels when the gray level is $i$.

The updated grayscale value determined by the grayscale transformation function is $j$, as expressed in Equation (7):

$$j=INT(\left(L-1\right)\times s\left(k\right)+0.5),$$

(7)

where $s\left(k\right)$ represents the cumulative distribution of pixels when the gray level is $k$.

After processing with the dark channel prior algorithm and subsequent global histogram equalization, the grayscale pixels for each level were relatively evenly distributed in the histogram following the mapping transformation. This adjustment stretched the dynamic range of the original image, resulting in an enhanced image with the brightness being generally centered near the median of the entire grayscale range [23]. Figure 3 and Figure 4 illustrate the spillway blockage images before and after global histogram equalization, respectively.

2.2. Spillway Boundary Region Segmentation Algorithm Based on Improved DeepLabv3+

The research environment around the reservoir was complex, with the monitoring image including not only the spillway but also objects such as soil, branches, and the ground on either side of the spillway. Effectively separating the spillway from this complex background enhanced the blockage detection accuracy. To address this, this study proposed an improved DeepLabv3+ segmentation algorithm for the spillway boundary region, featuring a lightweight backbone network and deep fusion of different feature scales.

2.2.1. Improved DeepLabv3+ Algorithm Network Architecture

To address the issue of a DeepLabv3+ network with a large number of parameters and inadequate interaction between different feature layers in the spillway boundary area segmentation, this study developed an enhanced DeepLabv3+ segmentation algorithm. The improved algorithm was trained on a reservoir spillway dataset to develop a spillway regional segmentation model. The enhancements included (1) replacing the original Xception backbone network with the MobileNetv2 network to create a lightweight backbone, and (2) integrating the CFF module to refine the feature fusion network and improve the segmentation accuracy. The updated DeepLabv3+ network structure is shown in Figure 5.

2.2.2. Lightweight Backbone Network

The Xception backbone network adopted in the DeepLabv3+ algorithm can provide better training results with a larger number of segmentation categories, whereas it is complex and has a slow processing speed [24]. For the spillway boundary area segmentation where the number of categories is limited and a fast inference speed is crucial, the MobileNetv2 network [25] can be more appropriate. It serves as a lightweight backbone network, significantly reducing the number of model parameters and facilitating their deployment.

2.2.3. Improvement of Feature Fusion Network by Combining CFF Module

In the DeepLabv3+ semantic segmentation backbone network, various feature layers of different scales were generated with the gradual extraction of the features. The shallow features captured the local and detailed information, whereas the deep features provided the global and generalized information. During decoding, the DeepLabv3+ network only combined the deep features with the shallow features of the 128 × 128 resolution for simple feature fusion that could lose the detailed spillway boundary information. To improve the semantic segmentation capability of the network model, the CFF module was integrated to enhance the feature fusion across different layers of the DeepLabv3+ backbone network.

The CFF module [26], or the cascade feature fusion unit, facilitates the mutual fusion of feature maps with varying resolutions. The structure of the CFF module is illustrated in Figure 6.

The CFF module was integrated with the lightweight MobileNetv2 backbone network to improve feature fusion. Initially, the first CFF module combined the 128 × 128 resolution feature map from the fourth layer of MobileNetv2 with the 64 × 64 resolution feature map from the seventh layer, resulting in an output feature map with dimensions of (128 × 128 × 80). Subsequently, the second CFF module fused the 32 × 32 resolution feature map from the 11th layer with the feature map produced by the first CFF module, yielding a shallow feature map incorporating multiple feature scales. Finally, the feature map from the deep network was fused with the shallow feature map obtained from the multi-scale feature fusion. This enhanced network structure effectively maintained the detailed spillway boundary information and extracted the overall information of the spillway image based on the context, contributing to improved segmentation accuracy for the spillway area boundary.

2.3. Blocked Target Detection Algorithm Based on Improved YOLOv7

Certain traditional models, such as R-CNN [27], SSD [28], and RetinaNet [29], exhibit the high algorithm complexities, low detection accuracies, and suboptimal recognition for blockages in spillways with irregular shapes and sizes. To address these issues, this study introduced an improved YOLOv7 target detection algorithm. This enhanced algorithm not only efficiently reconstructed the backbone network, but also increased the inference speed of the detection model.

2.3.1. Improved YOLOv7 Algorithm Network Architecture

The overall parameter count of the YOLOv7 model can be large and may not effectively avoid the non-obstructing target information. To address the detection needs for reservoir spillway blockages, this study implemented an improved YOLOv7 blockage detection network model. First, the efficient attention module ECA was introduced to utilize the MP1-ECA module for reconstructing an efficient backbone network that improved the feature extraction of the obstructing objects without significantly increasing the parameter count. Second, an advanced spatial pyramid pooling cross-local-stage convolution module, SPPFCSPC-M, was proposed to expand the receptive field for feature extraction while enhancing the inference speed and reducing the number of parameters. The structure of the improved YOLOv7 algorithm is illustrated in Figure 7.

2.3.2. Efficient Backbone Network Construction

The spillway area can be influenced by strong light, water flow, bubbles, and other factors, which can obscure the feature information of blocking foreign objects and negatively impact recognition accuracy. To address the interference of factors such as strong light, water flow, and bubbles on the feature extraction of blocking foreign objects, this study introduced the ECA efficient channel attention module into the YOLOv7 backbone feature extraction network. Additionally, an improved MP1-ECA transition module was proposed to enhance the accuracy of the feature extraction within the backbone network of the model.

The ECA attention module [30] employs a partial cross-channel interaction strategy without dimensionality reduction. This approach can effectively reduce the model complexity while maintaining the performance. The structure of the ECA module is illustrated in Figure 8.

The proposed MP1-ECA module enhanced the detection performance by refining the MP transition module in the backbone network. This improvement involved two branches, where one branch performed two convolution operations to extract the detailed feature information, and the other adopted a maximum pooling layer to adjust the dimensions and extract the most significant features. Subsequently, a convolution layer was introduced along with the ECA efficient channel attention module, which facilitated the cross-channel information interaction. This approach prevented the learning of the non-target redundant features in the spillway images without significantly increasing the memory overhead, thereby improving the detection of obstructing foreign objects. Finally, the feature maps from both branches were spliced and fused. The structure of the MP1-ECA module is illustrated in Figure 9.

2.3.3. Improved Spatial Pyramid Pooling Cross-Local Stage Convolution Module

The SPPCSPC module can enhance the global receptive field, reduce the dependence on spatial information [31], and aid the model in distinguishing the obstructions from the background in complex environments while capturing the obstruction information across various scales. However, owing to the varying camera angles and obstruction sizes in actual environments, the maximum pooling layer with three different pooling kernel sizes in the original module is inadequate. Hence, this study employed an improved SPPCSPC-M module to effectively address these limitations.

The SPPFCSPC-M module proposed in this study was based on a fast spatial pyramid pooling structure (SPPF). It replaced the original three maximum pooling modules in the SPPCSPC structure with seven maximum pooling modules, each with a 3 × 3 pooling kernel. These seven layers were connected in series, increasing the receptive field after each pooling operation. Four outputs were generated after the first, third, fifth, and seventh pooling layers, corresponding to maximum pooling sizes of 3 × 3, 7 × 7, 11 × 11, and 15 × 15, respectively. These outputs were then spliced and fused to produce the final result. This improved SPPFCSPC-M module enhanced the fusion of the features across different resolution sizes of obstructions and increased the inference speed of the model. The structure of the SPPCSPC-M module is illustrated in Figure 10.

3. Experimental Results and Analysis

3.1. Experimental Preprocessing and Training Parameter Setting

This study utilized real data collected from a reservoir spillway and simulated blockage data consisting of 714 real scenes and 207 simulated blockage scenes. A data enhancement algorithm was employed to expand the dataset, thereby improving the robustness and generalization of the network. The dataset was divided into training and validation sets in a 9:1 ratio, and the training set was further split into training and validation subsets in the same ratio. A total of 1080 training images, 121 validation images, and 134 test images were obtained. For the region segmentation, the LabelMe tool was utilized to generate the image segmentation labels for preparing a spillway boundary region segmentation dataset. LabelMe was used to create the blockage detection labels by retaining the color of the segmented area and masking the background in black. The boundary area images of the spillway were then utilized as the target detection dataset to identify the foreign object blockages.

The experimental platform is equipped with an Intel Core i9-12900K CPU (manufactured by NVIDIA Corporation in Santa Clara, California, USA), 64 GB RAM (manufactured by Kingston Technologies Corporation in Fangquan Valley, California, USA), and an NVIDIA GeForce RTX 4090 GPU with 24 GB of video memory (manufactured by Colorful Technology Development Co., Ltd. in Shenzhen, Guangdong Province, China). It ran Windows 10 and utilized the CUDA 11.7 parallel computing architecture. The PyTorch 1.13.1 deep learning framework with Python 3.9 was employed for the model construction and enhancement. In the phase of identifying foreign object blocking targets, transfer learning was employed to enhance the model’s convergence speed. The model training batch size was configured to 8, with a maximum of 200 iterations. During the boundary area segmentation phase, the model was trained from the ground up using the spillway’s boundary region segmentation dataset. The training batch size was 8, the maximum number of iterations was 200, and the learning rate was 0.007. The stochastic gradient descent (SGD) optimizer was employed to enhance the algorithm, while the cosine annealing method was utilized to adjust the learning rate.

3.2. Experimental Results and Comparative Analysis

This study proposed a three-stage method for detecting reservoir spillway blockages in complex environments using deep learning, which involved two training steps. The first step entailed training the DeepLabv3+ semantic segmentation network to develop a spillway boundary area model, and the second step focused on training the YOLOv7 object detection network to create a spillway blockage model. The models were refined based on the characteristics of the spillway boundary area and blockages. To validate the efficiency and reliability of the networks, a series of ablation experiments were conducted and compared for the improved DeepLabv3+ and YOLOv7 networks.

The reservoir environment presented significant complexity, with surrounding objects potentially causing substantial interference in the blockage detection. To assess the effectiveness of segmenting the spillway boundary as an image preprocessing step, the performance of the single-stage model was compared with that of the cascade model. Additionally, the variable weather conditions, such as rain and fog, could blur the detection line of sight. To address this, a rain and fog interference removal algorithm was employed to enhance image clarity, thereby improving the accuracy of the subsequent cascade network. The feasibility of this algorithm was then validated by comparing the effects and data before and after applying the rain and fog interference removal processes.

3.2.1. Analysis of Spillway Boundary Area Segmentation Performance

To evaluate the performance of the spillway area segmentation network and the effectiveness of the improvements, the comparative experiments were conducted using the Unet, PSPnet, DeepLabv3+, and improved DeepLabv3+ networks. The results depicted in Figure 11 revealed that the UNet and PSPnet networks (Figure 11b,c) were susceptible to interference from surrounding objects, leading to missed and false detections. Their segmentation performance was poor for the accurate extraction of the spillway boundary area. In contrast, the improved DeepLabv3+ network, illustrated in Figure 11d,e, effectively preserved the regional information of the spillway while avoiding background interference. This network achieved both a lightweight backbone and a high accuracy in spillway area segmentation.

To evaluate the boundary area segmentation performance of the spillway more effectively, the experiment employed several evaluation criteria: the mean intersection over union (MIoU), the mean pixel accuracy (MPA), the parameter quantity, and the inference speed. The calculation formulas for MIoU and MPA are as follows:

$$MIoU={\displaystyle \frac{1}{k+1}}{\sum }_{i=0}^k{\displaystyle \frac{p_{ii}}{{\sum }_{j=0}^k{p}_{ij}+{\sum }_{j=0}^k{p}_{ji}-p_{ii}}},$$

(8)

$$MPA={\displaystyle \frac{1}{k+1}}{\sum }_{j=0}^k{\displaystyle \frac{p_{ii}}{{\sum }_{j=0}^k{p}_{ij}}},$$

(9)

where $MIoU$ is used to calculate the ratio of the intersection and union between the true value and the predicted value set; $MPA$ represents the average probability of correct classification of pixels in each category; $k+1$ represents the total number of categories and backgrounds; and $p_{ij}$ denotes the probability that a pixel with the true category label $i$ is predicted to have label $j$. The experimental data presented in

Table 1. Experimental results of spillway area segmentation.

Models	mPA/%	MIoU/%	Param/M	FPS/Hz
Unet	93.16	86.99	24.892	51.97
PSPnet	93.77	88.44	46.716	102.67
DeepLabv3+	97.51	95.13	54.709	67.97
DeepLabv3+-MobileNetV2	96.28	92.60	5.813	141.12
DeepLabv3+-MobileNetV2-CFF	96.63	93.27	5.916	105.71

included the results from the ablation study of the improved DeepLabv3+ network. The data indicated that the MIoU of the enhanced DeepLabv3+ segmentation model reached 93.27%, with both the mPA and MIoU values surpassing those of the other network models. This demonstrated the superior segmentation performance of the spillway.

3.2.2. Single-Stage Model Spillway Blockage Target Detection Performance and Comparative Analysis

The single-stage model refers to the detection of spillway blockage using only the target detection network model without integration with other algorithms. To assess the performance of the proposed improved YOLOv7 single-stage model, this study used accuracy (P), recall (R), mean average precision (mAP), parameter quantity (param), and inference speed (FPS) as the evaluation indicators. P measured the correctness of model predictions (Equation (10)), R assessed the ability of the model to detect the target data (Equation (11)), and mAP evaluated the effectiveness of the algorithm in predicting the target location and category.

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(10)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(11)

where $TP$ is the number of correctly identified blockages, $FP$ is the number of false detections where other objects are identified as blockages, and $FN$ is the number of missed detections where blockages are not correctly identified.

In the same environment, the SSD, YOLOv5, and original YOLOv7 networks were trained using the spillway obstruction dataset and compared with the improved YOLOv7 network. The experimental results (

Table 2. Blockage detection experiment results.

Models	P/%	R/%	mAP/%	Param/M	FPS/Hz
SSD	69.24	52.82	56.90	24.146	135.73
YOLOv5	78.05	55.50	68.29	46.653	90.11
YOLOv7	72.99	67.67	70.39	37.216	69.49
YOLOv7 + ECA efficient backbone network	75.64	68.62	73.35	37.216	64.51
YOLOv7 + ECA efficient backbone network +SPPFCSPC-M	74.30	71.56	73.93	37.478	73.87

) indicated that the improved YOLOv7 network achieved a 3.54% increase in mAP compared to the original YOLOv7 network, while also demonstrating a faster inference speed.

3.2.3. Cascade Model Spillway Blockage Detection Performance and Comparative Analysis

The single-stage detection method treats the blockage issue in the spillway as a target detection problem, allowing for the direct application of the target detection model for blockage detection. However, this study adopted a cascade approach combining the semantic segmentation and the target detection models to effectively identify the background object interference and enhance the accuracy of the reservoir spillway blockage detection.

To assess the performance of the proposed cascaded DeepLabv3+ and YOLOv7 networks for the spillway blockage detection, a comparison was made with the single-stage YOLOv7 network. The results (Figure 12 and Figure 13) indicated that the single-stage YOLOv7 network struggled with accurate detection, owing to the interference from the surrounding objects. In contrast, the cascaded DeepLabv3+ and YOLOv7 methods effectively filtered the background environment prior to the target detection, resulting in no false or missed detections.

To better evaluate the detection performance of the cascade model for blockages, this study implemented a spillway blockage detection model using the YOLOv7 single-stage detection method. We compared the blockage detection performance of the DeepLabv3+ and YOLOv7 networks before cascade improvement with that of the improved DeepLabv3+ and YOLOv7 cascade models. The experimental data in

Table 3. Experimental comparison results of cascade model and single-stage model.

Models	P/%	R/%	mAP/%	FPS/Hz
YOLOv7 spillway blockage detection model before single-stage improvement	72.99	67.67	70.39	69.49
DeepLabv3+ and improved YOLOv7 cascade spillway blockage detection model before improvement	82.18	74.22	79.76	66.77
Improved DeepLabv3+ and improved YOLOv7 cascade spillway blockage detection model	83.43	77.49	80.32	67.96

revealed that the average precision of the single-stage target detection method was relatively low. In contrast, the cascaded model with the improved DeepLabv3+ and YOLOv7 networks achieved an mAP of 80.32%, which was approximately 9.93 percentage points higher than that of the single-stage YOLOv7 target detection model, while there was a minor reduction in inference speed.

3.2.4. Performance and Comparative Analysis of Blockage Recognition in Complex Rain and Fog Environment

The performance comparison analysis in Section 2.2.3 indicated that the average accuracy of the proposed improved DeepLabv3+ and improved YOLOv7 cascade spillway blockage detection model reached 80.32%, demonstrating a significant improvement over the single-stage model in identifying blockages. However, the complex and variable weather conditions at the reservoir, such as rain and fog, could substantially reduce the detection accuracy by blurring the field of vision. To further assess the efficacy of the proposed three-stage method, Figure 14 presents a comparative analysis of simulation scenarios under rain and fog interference conditions. The scenarios include no interference removal, preprocessing with Contrast Limited Adaptive Histogram Equalization (CLAHE), preprocessing with the dark channel prior (DCP) algorithm, and preprocessing with the proposed rain and fog interference removal algorithm. In Figure 14a, the interference from rain and fog blurs the spillway boundaries and obstructed areas, resulting in substantial missed detections and false positives. Although the CLAHE algorithm achieves certain optimization effects on the image, as shown in Figure 14b, the segmentation quality of the spillway area remains poor, with extraneous interference outside the target area still being detected, leading to relatively low overall accuracy. Figure 14c shows the inference image preprocessed with the DCP algorithm, where false positives are significantly reduced; however, due to the overall low brightness of the preprocessed image, missed detections still occur. Figure 14d displays the detection results after using the proposed rain and fog interference removal algorithm. The comparison shows that the proposed algorithm achieves high precision and recall rates, which are advantageous for further cascade network detection.

According to the actual scenario, this study applied an optical model to overlay a specified amount of rain and fog noise onto the original image. The ratio of the images with rain and fog interference to those without such interference in the test set was approximately 1:1. Following the introduction of the improved cascade network, the experimental results without using the rain and fog interference removal algorithm were compared with the experimental results preprocessed using CLAHE, DCP, and rain and fog interference removal algorithms. The comparison results are shown in

Table 4. Comparative experimental results of blockage recognition in complex rain and fog environments.

Methods	P/%	R/%	mAP/%
Failure to eliminate rain and fog interference	73.51	68.31	71.12
CLAHE	82.02	69.25	74.75
DCP	84.13	71.62	76.51
Rain and fog interference removal algorithm	83.08	73.55	77.77

.

The experimental results presented in Table 4 revealed that in the rainy and foggy environments of the reservoir, the preprocessed approach, utilizing rain and fog interference removal, exhibits superior average accuracy and recall compared to other preprocessing methods, resulting in enhanced detection efficacy. Compared with image preprocessing alone, the average accuracy value improved by nearly 6.65%.

This work presents a three-stage detection system that effectively identifies foreign item blockages in the spillway, following the elimination of rain and fog interference and the segmentation of the spillway’s perimeter region.

Table 5. Experimental comparison results of different stage combination methods under rain and fog interference conditions.

Methods	P/%	R/%	mAP/%	FPS/Hz
YOLOv7 with single-stage improvement	80.92	68.66	73.61	73.87
Two stages (improved YOLOv7 and improved DeepLabv3+)	73.51	68.31	71.12	67.96
Two stages (rain and fog interference removal algorithm and improved YOLOv7)	79.88	71.63	75.31	67.66
Three stages (rain and fog interference removal algorithm, improved YOLOv7, and improved DeepLabv3+)	83.08	73.55	77.77	66.46

presents the detection outcomes of several stage combinations in situations affected by rain and fog interference. The data indicate that in the single-stage processing mode utilizing solely the object detection model, the inference speed is the highest, yet the detection accuracy is comparatively low. Conversely, employing the enhanced YOLOv7 and DeepLabv3+ two-stage network for image inference, the presence of rain and fog obscured the spillway boundary, leading to inadequate segmentation of the spillway boundary and adversely impacting the subsequent detection of foreign object obstruction. The application of the rain and fog interference removal algorithm, combined with the enhanced YOLOv7 two-stage prediction method, resulted in a substantial increase in average accuracy, demonstrating the algorithm’s efficacy in eliminating rain and fog noise from images, thereby enhancing the precision of subsequent detection. Conversely, the implementation of the three-stage strategy markedly enhanced the model’s accuracy, recall, and average precision, yielding an average precision increase of 4.16%, greatly improving its focus on the target area.

The experiment demonstrates that the rain and fog removal algorithm, along with the boundary area segmentation of the spillway, contributes to denoising and area constraint in the three-stage method. This allows the final object detection model to concentrate on the foreign object blockage area, thereby minimizing false detections of the surrounding environment and validating the efficacy of the three-stage method in intricate rain and fog conditions.

4. Conclusions and Discussion

According to the weather and environmental conditions of the reservoir spillway, this study proposed a three-stage reservoir spillway blockage detection method based on deep learning for complex environments. The method initially applied a rain and fog removal algorithm to address the challenges posed by the complex rain and fog environments. Subsequently, it enhanced the DeepLabv3+ network model by improving and lightweighting its backbone network, enabling the model to filter environmental interference and segment the boundary area of the spillway while reducing the number of model parameters. Finally, for the obstruction detection, an efficient backbone network, was developed based on the original YOLOv7 network to inhibit the non-target features and improve the overall accuracy. Additionally, the spatial pyramid pooling cross-stage module was refined to expand the receptive field in the feature extraction network and accelerate the reasoning speed of the network. The experimental comparison results of the spillway scene in a simulated rain and fog environment demonstrated that the proposed three-stage reservoir spillway blockage detection method achieved a detection accuracy of 77.77% in the test set of rain and fog interference images. Under clear conditions, the cascaded detection network significantly outperformed the single-stage network. The enhanced DeepLabv3+ network excelled in MIoU and MPA indicators, providing complete and clear segmentation of the boundary area and minimizing interference from nonspillway regions. Additionally, the improved YOLOv7 network offered more accurate blockage detection, effectively enhancing the identification of reservoir spillway blockage risks. This advancement could support theoretical and technical foundations for the safety of reservoir flood control areas and the intelligent management of reservoirs. Nevertheless, when the concentration of rain and fog interference is excessively high or in more intricate spillway situations, the model has certain obstacles regarding its generalization performance. Elevated levels of rain and fog may hinder the efficacy of algorithms designed to eliminate interference, thereby impacting the segmentation of spillway regions and the identification of obstructions. Furthermore, overflowing scenarios in intricate ecosystems will provide additional non-target information, hence impacting the model’s robustness and accuracy. Consequently, subsequent research may investigate the enhancement of algorithms’ adaptive capabilities across varying environmental conditions to augment their usability and stability in real-world applications.