Intelligent Segmentation and Change Detection of Dams Based on UAV Remote Sensing Images

Abstract

Guilin is situated in the southern part of China with abundant rainfall. There are 137 reservoirs, which are widely used for irrigation, flood control, water supply and power generation. However, there has been a lack of systematic and full-coverage remote sensing monitoring of reservoir dams for a long time. According to the latest public literature, high-resolution unmanned aerial vehicle (UAV) remote sensing has not been used to detect changes on the reservoir dams of Guilin. In this paper, an intelligent segmentation change detection method is proposed to complete the detection of dam change based on multitemporal high-resolution UAV remote sensing data. Firstly, an enhanced GrabCut that fuses the linear spectral clustering (LSC) superpixel mapping matrix and the Sobel edge operator is proposed to extract the features of reservoir dams. The edge operator is introduced into GrabCut to redefine the new energy function’s smooth item, which makes the segmentation results of enhanced GrabCut more robust and accurate. Then, through image registration, the multitemporal dam extraction results are unified to the same coordinate system to complete the difference operation, and finally the dam change results are obtained. The experimental results of two representative reservoir dams in Guilin show that the proposed method can achieve a very high accuracy of change detection, which is an important reference for related research.

1. Introduction

Guilin has a tropical monsoon climate. According to the average annual statistics, the number of precipitation days is about 160 days, the rainfall is 1887.6 mm, and the rain collecting area is 19,288 square kilometers. According to the statistical document issued by the Guilin Municipal Bureau of Statistics, the land used for water conservancy facilities in Guilin has reached a high proportion of 60% of the urban land. At present, there are 137 reservoirs in Guilin, which are used for irrigation, flood control, water supply and power generation.

For a long time, there has been a lack of systematic and full-coverage remote sensing safety monitoring for all reservoir dams in Guilin. It is a new and effective technical method to realize the change detection of reservoir dams based on high-resolution unmanned aerial vehicle (UAV) remote sensing. UAV remote sensing has the advantages of a good maneuverability, a high resolution, and a high timeliness.

In recent years, some scholars have conducted research on UAV remote sensing change detection, such as Wei et al. [1], Dong et al. [2], Lu et al. [3] and Zhong et al. [4]. Change detection refers to the extraction of change information of objects in different time phases and in the same geographical area [5]. Change detection is commonly used in urban planning [6,7], land use [8,9], vegetation cover [10,11], disaster detection [12,13,14], etc. However, as of now, there are few studies using UAV images to detect changes in dams, especially lacking a well-developed technical process.

This paper proposes an intelligent segmentation change detection method for reservoir dams based on UAV remote sensing, which realizes the fusion of linear spectral clustering (LSC) and enhanced GrabCut, completes the UAV high-resolution remote sensing flight experiments of two representative reservoir dams, and finally, conducts a multitemporal and multiscale model evaluation and a precision analysis.

This research includes the following parts:

(1)

The acquisition and image preprocessing of high-resolution UAV remote sensing data. Through the actual survey of multiple reservoirs in Guilin, the remote sensing data of two typical reservoir dams were collected by using high-resolution UAV. The data are unique and diverse.

(2)

An enhanced GrabCut that fuse with the LSC superpixel mapping matrix and the Sobel edge operator is proposed to extract the features of reservoir dams. The edge operator is introduced into GrabCut to redefine the new energy function’s smooth item, which makes the segmentation results of enhanced GrabCut more robust and accurate. Then, through image registration, the multitemporal dam extraction results are unified to the same coordinate system to complete the difference operation, and finally, the dam remote sensing image change results are obtained.

(3)

A precision analysis and evaluation of the experimental results.

2. Related Work

Many scholars have done a lot of research on the detection of feature changes based on remote sensing images. For example, Kim studied the change detection of floods using Sentinel-1 synthetic aperture radar (SAR) satellite images [15]; Adediji used satellite remote sensing images to detect changes in major dams in the United States region [16]; Al-Obaidi used satellite images and GIS technology to detect changes on an Iraqi dam lake in the northern part of the country [17]; etc. These studies are mainly divided into conventional image detection methods and deep-learning-based detection methods [18,19]. Further, we list some representative change detection methods, including change detection method based on principal component analysis (PCA) and K-means [20,21], a deep learning model based on reconstruction loss (DLMRL) [22,23], the OTSU image difference method [24], the robust change vector analysis (RCVA) detection method [25,26,27] and differential principal component analysis (DPCA) [28].

Superpixel segmentation is an image segmentation method that allows the image to be split into large blocks of pixels, each of which has approximately the same texture inside, while still maintaining the edges of similar objects very well. In this paper, the use of superpixels is not only beneficial in achieving a fast processing of images but also in maintaining the edge contours of dams, which is especially beneficial in segmenting precise dam targets. The generation of superpixels includes graph theory methods and gradient descent methods. Graph theory methods can usually maintain better segmentation edges, but it is difficult to achieve real-time processing due to the high computational complexity. Superpixel segmentation methods based on graph theory generally include normalized cut algorithm [29], graph-based algorithm [30] and LSC [31]. Gradient descent methods can improve computational efficiency but may lead to blurred boundaries or oversegmentation. Gradient descent algorithms generally include Turbopixel [32], simple linear iterative cluster (SLIC) [33], superpixels extracted via energy-driven sampling (SEEDS) [34], etc.

GrabCut is a very effective energy optimization algorithm proposed by Carsten Rother, Vladimir Kolmogorov and Andrew Blake at the Microsoft Research Center in the UK [35,36]. By combining hard constraints (energy function) and local constraints (flow minimization, local penalties, etc.), iterative graph cuts are used for interactive foreground extraction, which has excellent performance and convenient operability.

3. Methods

In order to realize the intelligent extraction of the reservoir dams proposed in this paper, firstly, UAV remote sensing was used to collect multitemporal and high-resolution image data of the reservoir dams, and then the image calibration preprocessing was completed, including image distortion correction, orthorectification, rotation cropping and color equalization.

Secondly, superpixels were used instead of pixels as the smallest unit of image processing in GrabCut to reduce the amount of computation. By analyzing the defects and deficiencies in GrabCut, edge operators were introduced to redefine the energy function of GrabCut to make the segmentation results more accurate and segmented edges smoother and more robust.

Finally, image registration was used to unify the extracted mask results of multiple time-phase data under the same coordinates and obtain the change results of reservoir dams with a differential operation.

The overall technical route of this paper is shown in Figure 1. Through route planning, airborne camera and scene parameter settings, the UAV could realize a fully automated flight, which greatly saved the cost of the experiment, and allowed us to complete the multitemporal remote sensing change detection of the Guilin reservoir dam.

3.1. Image Preprocessing

Firstly, the camera distortion model with the internal and external parameters was applied to restore the original features of the UAV remote sensing images; secondly, orthorectification was used to complete the tilt and projection correction in order to acquire the orthophoto with the characteristics of the mapping and remote sensing image; thirdly, rotating the image to the horizontal direction made the quality of the subsequently generated superpixel blocks higher, the blocks less in number, and better in visual perception; finally, histogram equalization and normalization were used to unify the color of the multitemporal UAV remote sensing image, so as to reduce the influence of changes in color, brightness, contrast and other color factors in different temporal phases.

The processing of distortion correction, orthorectification and color equalization was very conventional. The handling details of the rotation and interception is shown in Figure 2.

3.2. The Fusion of Sobel Edge Operator and Enhanced GrabCut

As an interactive segmentation algorithm, GrabCut can effectively separate the target area from the background and can continue to segment the target multiple times by supplementing the background or foreground labels. However, this algorithm uses the region information and gray value features in the image and does not take into account the edge information that may exist between adjacent pixels. Especially when the target in the selection area is continuously segmented, if there is too much color difference, the segmented target may be removed, resulting in a hole inside the segmented target. At the same time, the nonsegmented objects in the selection area are retained due to the obvious regional characteristics, resulting in noise outside the nonsegmented objects.

In view of the above problems, this paper introduced the edge operator into GrabCut. By redefining the new energy function’s smooth term, the enhanced GrabCut segmentation results are more robust and accurate. In this paper, the Sobel operator based on the weighted difference of adjacent pixels in the image was a discrete differential operator that combined Gaussian smoothing and differential derivation. It had the advantages of a strong anti-noise ability, high operating efficiency, and fast speed. The specific process of the 3 × 3 Sobel operator acting on the image was as follows:

$$\left\{\begin{array}{c}{G}_x=I\left(x,y\right)⨂\left[\begin{array}{ccc}-1& 0 & 1\\ -2& 0 & 2\\ -1& 0 & 1\end{array}\right]\\ G_y=I\left(x,y\right)⨂\left[\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right]\end{array}\right.,$$

(1)

In Formula (1), ⨂ is the convolution operation. It indicates that the Sobel operator convolution operation is performed on the image $I$ in the $x$ and $y$ directions, respectively.

$$\left|G\right|\approx \sqrt{{G_y}^2+{G_x}^2},$$

(2)

where $G_x$ represents the gradient in the $x$ direction, $G_y$ represents the gradient in the $y$ direction, and $G$ is the approximate value of the gradient. $I\left(x,y\right)$ is an image pixel, and the convolution kernel is generally an odd number. Typically, when the gradient value changes abruptly in both directions, it is considered as an edge. Thus, when the gradient $G$ is greater than a certain threshold, the point is considered as an edge point.

We rewrote the Gibbs energy function of the entire image in GrabCut.

$$E\left(L\right)=E_{data}\left(L\right)+\lambda E_{smooth}\left(L\right),$$

(3)

where $L \in \{0, 1\}$ represents the data label. $E\left(L\right)$ represents the energy function. $E_{data}\left(L\right)$ is the energy function of the data item, and $E_{smooth}\left(L\right)$ is the energy function of the smooth item. $\lambda$ is a balance factor, which is used to balance the weight of data items and smooth items, and generally it is 0.5.

$$E_{data}\left(L\right)=\sum _{p\in n}D\left(l_p,\theta \right)$$

(4)

where $D\left(l_p,\theta \right)$ is a data item, which is used to constrain the consistency between the label $L$ and the observed data $\theta$.

$$E_{smooth}\left(L\right)=\sum _{\left\{p,q\right\}\in N}{S}_{<p,q>}\times \delta \left(l_p,l_q\right),$$

(5)

The calculation formulas for the two terms on the right side of the equation are as follows:

$$S_{<p,q>}=\exp \left(-\sigma {‖F_p-F_q‖}_2^2\right),$$

(6)

$$\delta \left(l_p,l_q\right)=\left\{\begin{array}{c}0 if l_p=l_q \\ 1 {if l}_p\ne l_q\end{array}\right.,$$

(7)

$F_p$ and $F_q$ are the grayscale value vectors of the BGR color space, and the L2 norm is used to measure the similarity of the color features of two pixels. When the two pixels are less similar, that is, the greater the difference, the more likely they are assigned to different labels. $\sigma$ is used to adjust the influence of the contrast of adjacent pixels when the image contrast is different.

In this paper, the Sobel operator was added to the edge gradients. Among them, the gradient in the $x$ direction was $G_x$, and the gradient in the $y$ direction was $G_y$. $G(p,q)$ was defined as the possibility value of an edge between pixels $p$ and $q$, which was replaced by a gradient approximation.

$$G\left(p,q\right)=\left\{\begin{array}{c}{G}_x\left(p,q\right) if p=\left(x,y\right),q=\left(x+1,y\right)\\ G_y\left(p,q\right) if p=\left(x,y\right),q=\left(x,y+1\right)\end{array}\right.,$$

(8)

We redefined the smooth term of the new energy function:

$$S_{\left(p,q\right)}=\mathsf{\gamma exp}\left(-\sigma {‖F_p-F_q‖}_2^2\right)+\left(1-\gamma \right)\exp \left(-\sigma G\left(p,q\right)\right),$$

(9)

where $\mathsf{\gamma }$($\mathsf{\gamma }\in \left[\mathrm{0,1}\right]$) is the weighted reconciliation factor of the gray intensity and gradient intensity in the smooth item. When $\mathsf{\gamma }$ is set to 1, it is the original GrabCut. It is generally set to 0.5.

The data items in GrabCut were defined as follows:

$$D\left({\alpha }_n\right)=-ln{P}_r\left(z_n|{\alpha }_n,\theta \right),$$

(10)

According to the gaussian mixture model (GMM) to estimate the probability of belonging to the background or foreground, the specific definition is as follows:

$$P_r\left(z_n|{\alpha }_n,\theta \right)=\sum _{k=1}^k{\pi }_k{\ast P}_r\left(z_n|{\alpha }_n,{\theta }_{\pi }\right),$$

(11)

$$P_r\left(z_n|{\alpha }_n,{\theta }_{\pi }\right)=\frac{1}{\sqrt{2\pi \left|\sum k\right|}}\ast e^{\left[-\frac{1}{2}{\left({\alpha }_n-{\mu }_k\right)}^T{\sum }_k^{-1}({\alpha }_n-{\mu }_k)\right]},$$

(12)

In Formulas (11) and (12), $P_r(\ast )$ is the probability that the current pixel belongs to the foreground or background, $z_n$ is the grayscale value of the pixel in the BGR color channel, ${\alpha }_n\in \{0:\mathrm{b}\mathrm{a}\mathrm{c}\mathrm{k}\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d},1:\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{r}\mathrm{o}\mathrm{u}\mathrm{n}\mathrm{d}\}$; $\theta$ is the parameter of the GMM, $\mu$ is the mean value, $\sum k$ is the covariance matrix of the $k$th GMM, and ${\pi }_k$ is the weight of the $k$th GMM.

Finally, we reconstructed the energy function as:

$$E\left({\alpha }_n,l_p,l_q\right)=D\left({\alpha }_n\right)+\lambda S_{\left(p,q\right)},$$

(13)

The steps of the LSC superpixel fused with enhanced GrabCut are as follows:

• Input the image for processing;
• Perform a superpixel segmentation;
• Label each pixel block in the superpixel segmentation result, mark the subpixels it contains and generate a new mapping matrix $I_{new}$ by arranging the pixels according to the arrangement rules in the superpixel segmentation result;
• Input the image $I_{new}$ into enhanced GrabCut for the interactive segmentation;
• Obtain the segmentation result, then refill the image based on the subpixels indexed in the superpixel blocks of the superpixel result to restore the resolution;
• Output the result.

3.3. Image Registration and Change Detection

Image registration is to match the images of the same ground scene taken by different perspectives, different time phases and different equipment, so that they can be unified in the same coordinate system. In this research, the images for the dam area contained obvious feature points, corner points, edges and other information to complete the detection of dam changes.

The scale-invariant feature transform (SIFT) is stable and accurate and is very suitable for high-precision image processing tasks. In this paper, the SIFT was used for feature extraction. After the SIFT feature points were detected, the SIFT feature points of the two images were matched, and the good matching points were screened out by the random sample consensus (RANSAC) method and the ratio method. After feature matching, the transformation matrix was estimated by the least-squares method for image registration so that the images of different phases were unified into the same coordinate system, and then the difference operation was performed on the images to obtain the mask map of the dam change result. The specific process is shown in Figure 3.

4. Experiments

4.1. Dataset

The drone used was the DJI Phantom 4 Pro and the camera model was FC6310S. The parameters of the visible-light camera carried on the UAV in this experiment were as follows: COMS 1 inch, 12.8 mm × 9.6 mm and a focal length of 8.8 mm. The resolution of the collected images was 4864 × 3648, and the relative heights were 50 m, 100 m and 200 m, respectively. The overlap rate in the heading direction was 60%, and the overlap rate in the sideways direction is 70%.

4.1.1. Data of Dalingtou Reservoir Dam

Dalingtou Reservoir is a medium-sized reservoir with an area of 30 square kilometers. It is mainly used for irrigation, combined with water supply and fish feeding. The main area is located at latitude 25.383696°N~25.391237°N and longitude 110.489544°E~110.498664°E. The satellite location of Dalingtou Reservoir is shown in Figure 4. The data of time phase I of the Dalingtou Reservoir were taken in November 2022, and the data of time phase II were taken in April 2023.

4.1.2. Data of Qingshitan Reservoir

Qingshitan Reservoir is the largest reservoir in the north of Guilin, covering an area of 141.7 square kilometers. It is mainly used for irrigation combined with water supply, power generation, flood control, shipping, fish feeding, tourism, etc. The main area of Qingshitan Reservoir is located at latitude 25.525228°N–25.530553°N and longitude 110.223555°E–110.232127°E. The satellite location of Qingshitan Reservoir is shown in Figure 5. The data of time phase I of Qingshitan Reservoir were taken in November 2022, and the data of time phase II were taken in April 2023.

4.2. Implementation Details

This experimentation and software development was carried out in a computer hardware environment with a windows 10 operating system, an [email protected] CPU and 32 GB of RAM. The development platform was visual studio C++ 2017.

4.2.1. Comparison of Superpixel Segmentation Effects

The experimental results of the superpixel segmentation are shown in Figure 6. Figure 6a is the original image of the Qingshitan Reservoir dam used in this superpixel segmentation experiment. Figure 6b–d are the segmentation results when the superpixel segmentation algorithm SLIC, simple linear iterative cluster zero (SLICO) and LSC were applied, separately, whose superpixel size was set to 10. Figure 6e–g are the segmentation results when the superpixel size was set to 30. Figure 6h is the segmentation result when the superpixel algorithm SEEDS was adopted, whose number of superpixels was set to 1500.

SLIC worked better in similar-texture regions, where the generated superpixels were more regular, while in complex-texture regions, regardless of the size of the superpixels, the generated blocks of superpixels were more scattered and had a poor edge retention. The superpixels of SLICO in the whole image were relatively regular, the superpixel blocks generated by the small-sized superpixels were better, and the edge preservation was also good. However, when the superpixels were large in size, the generated superpixel blocks were poorly preserved at the edge; SEEDS could not directly specify the superpixel size, but the number could be set, and the edge retention was better in areas where the color difference was large. However, the edges in regions with similar color and texture remained unstable, and the generated superpixels were very uneven and irregular. Regardless of the superpixel size, LSC generated relatively regular superpixel blocks. At the same time, it also presented an excellent edge fit in complex texture areas, which were more pronounced at larger superpixel sizes. SLC had the best overall results.

4.2.2. Results of LSC Fused with Enhanced GrabCut Segmentation

This experiment was based on the orthophoto from the UAV remote sensing image of the Dalingtou dam area. The superpixel size was set to 10. The extraction experiment was realized by using the LSC fused with the enhanced GrabCut, as shown in Figure 7. The processing sequence is shown in Figure 7a–f.

The orthophoto of the Dalingtou dam is shown in Figure 7a; the result of the LSC superpixel segmentation is shown in Figure 7b; the new superpixel mapping image is shown in Figure 7c. The operation of the newly mapped image was as follows: firstly, we randomly selected 70% pixels from each superpixel block obtained by LSC, then we took the weighted mean value of each color channel of the selected pixels and remapped them into points, and finally, we rearranged them into a three-channel mapping matrix.

It can be concluded that the orthophoto of the Dalingtou dam with an original resolution of 11,844 × 7896 was only 1184 × 789 after mapping, and then the mapped image was passed into the enhanced GrabCut for interactive segmentation, as shown in Figure 7d. In order to improve the segmentation effect, some target pixels were selected as seed points with red marks during segmentation, and the result after segmentation by GrabCut is shown in Figure 7e. Finally, the segmentation result was restored to the original resolution, as shown in Figure 7f. The final segmentation worked well, with precise segmentation edges.

4.2.3. Detection Results of Dam Changes

The results of the change detection of the two reservoir dams when the superpixel size was set to 10 are shown in Figure 8 and Figure 9, respectively.

Figure 8 shows the experimental results of the Dalingtou Reservoir dam. Figure 8a is the data of time phase I. Figure 8b is the data of time phase II. Figure 8c,d are the extraction results of the dam. Figure 8e is the registration result for the reservoir dam of time phase II. Figure 8f is the extraction result of the reservoir dam by the mask operation. Figure 8g is the difference calculation result by the reservoir dam’s masks of two different time phases. Figure 8h is the final detected change result of the Dalingtou Reservoir dam. Similarly, Figure 9 shows the experimental results of the Qingshitan Reservoir dam.

It can be seen from the experimental results that the two reservoir dams went through relatively obvious changes over a period of a half year. For the Dalingtou dam, part of the image below the dam was lost, while a part was added to the right. For the Qingshitan dam, a part was added below the dam in the image. After analysis, it was determined that: (1) the subtle changes around the reservoir dam were caused by the fact that, as the change in seasons, vegetation grew on the border of the dam body, so the edge of the dam body was covered; (2) when the rainy season occurred from March to May, the continuous rainfall made part of the dam body wash away due to the sand and stones.

5. Discussion

Superpixel segmentation is a key factor affecting the boundary extraction of reservoir dams. Based on the above experimental results, this section discusses and analyzes the influence of different superpixel segmentation methods on the accuracy of reservoir dam boundary extraction and the influence of the superpixel size on the detection accuracy of the reservoir dam’s changes. In addition, comparisons with other mainstream methods are presented.

5.1. Comparison and Analysis of Different Superpixel Segmentation Methods

From the three indicators in Figure 10, Figure 11 and Figure 12, it can be seen that the LSC superpixel segmentation has the highest boundary recall (BR) value and the lowest under-segmentation error (UE) value. It shows that the segmentation result of LSC has the most accurate edges and fits the best boundary. However, there are cases where the CO value is relatively low, indicating that the superpixel blocks generated by LSC are not very regular, although it does not affect the accurate extraction of edge features.

This application research requires completeness and precise boundaries of the reservoir dam body when superpixel presegmentation is applied, and due to the unique color attributes of the reservoir dam body, the segmentation boundary is significantly different from the color texture of other objects, so LSC is more suitable than other superpixel segmentation methods.

5.2. Analysis of the Influence of Superpixel Size Setting on the Detection Accuracy of Dam Changes

When the superpixel size was set to 10, the detection results of the dam changes in Qingshitan and Dalingtou Reservoirs are shown in Figure 13.

The masks in Figure 13a,c are the true labels and the masks in Figure 13b,d are the experimental extraction results. The red color is the missing label of the reservoir dam by the proposed extraction algorithm. It can be seen from the comparison of the above figures that the change detection results of the two reservoir dams are accurate. However, both reservoirs are with partial occlusion (as shown with red color), resulting in the failure to extract the dam body of the obscured part in every group of two-time-phase data. In actual situations, the relative height of the flight is relative to the take-off point, while the plane of the reservoir is often lower than the actual take-off point, and due to the two-dimensional image projection, only the projected area of change of the reservoir dam can be calculated. The three-dimensional change calculation requires combining more information.

The pixel size of the image sensor was (12.8 × 10³/4864) ≈ 2.6315µm. When the relative height of the UAV above Dalingtou Reservoir is 100 m, the ground spatial resolution can be calculated as 2.99 cm/pixel through the camera parameters and flight height of the UAV: 2.6315 × 10⁻⁴ × 10⁴/(8.8 × 10⁻¹) ≈ 2.99 (cm/pixel). Moreover, the number of pixels in the changed area is 965,524. It can be calculated that the plane area of the dam change of Dalingtou Reservoir is 965,524 × 2.992 × 10⁵ ≈ 863.19 m². When the relative height of the UAV above Qingshitan Reservoir is 55 m, the ground spatial resolution is 2.6315 × 10⁻⁴ × 10⁴/(8.8 × 10⁻¹) ≈ 1.64 (cm/pixel). The number of pixels in the changed area is 1,950,977 according to the statistics. It can be calculated that the planar area of Qingshitan Reservoir dam change is about 1,950,977 × 1.642 × 10⁻⁵ ≈ 524.73 m².

The calculated results of the above parameters are shown in

Table 1. The calculated parameters of the UAV experiment.

Parameter	Flight Height\m	Ground Spatial Resolution\(m/pixel)	Number of Changed Pixels	Change Area\m2
Dalingtou Reservoir	100	0.0299	965,524	863.192
Qingshitan Reservoir	55	0.0164	1,950,977	524.73

.

From the segmentation indicators in Figure 14 and Figure 15, it can be seen that when detecting reservoir changes from UAV remote sensing images, the smaller the superpixel size, the more accurate the results of the dam change detection.

The evaluation indices of the change results when the superpixel size of the two reservoir dams was set to 10 are shown in

Table 2. The evaluation metrics for the dam change results in Dalingtou and Qingshitan Reservoirs.

Method	ACC	Recall
Dalingtou reservoir	0.9674	0.9623
Qingshitan reservoir	0.9812	0.9723

. From the above four evaluation indicators, it can be concluded that the evaluation indicators of the final change results of the two reservoir dams were both better than 95%.

The detection results of the dam changes in Dalingtou Reservoir and Qingshitan Reservoir were as follows: the F1-score indices were 0.9648 and 0.9767 compared with the label value, and the Kappa indices were 0.9625 and 0.9674, respectively. At the same time, it was also verified that as the relative flight height of the Qingshitan test (55 m) was lower than the relative flight height of the Dalingtou test (100 m), the pixel representation in the image of the Qingshitan dam was finer, so that the accuracy of the change detection results of the Qingshitan dam was better than that of Dalingtou dam.

5.3. Comparison with Other Mainstream Methods

In this section, we describe in detail the specific models and experimental results of several comparative methods and make a comparative analysis with the LSC fused with enhance GrabCut proposed in this paper. Through these comparative experiments, we comprehensively assess the performance of various methods for remote sensing change detection in reservoirs. It is worth noting that the change detection images used in the comparison test were fully consistent with the UAV remote sensor images in previous studies to ensure the comparability and accuracy of the experimental results.

In our study methodology, outlined in

Table 3. Comparisons with different methods.

Method	Research Area	ACC	Recall	F1-Score	Kappa
Ours	Dalingtou Reservoir	0.9674	0.9623	0.9648	0.9525
	Qingshitan Reservoir	0.9812	0.9723	0.9767	0.9674
PCA + K-means	Dalingtou Reservoir	0.9922	0.8402	0.3226	0.3202
	Qingshitan Reservoir	0.9454	0.4227	0.5122	0.4846
DLMRL	Dalingtou Reservoir	0.9045	0.1458	0.1738	0.1249
	Qingshitan Reservoir	0.9739	0.0748	0.1019	0.0905
OTSU image difference	Dalingtou Reservoir	0.9413	0.0619	0.1119	0.1015
	Qingshitan Reservoir	0.9277	0.2433	0.2697	0.2321
RCVA	Dalingtou Reservoir	0.9942	0.7121	0.2916	0.2898
	Qingshitan Reservoir	0.9491	0.4471	0.5292	0.5033
DPCA	Dalingtou Reservoir	0.9201	0.0711	0.1324	0.1219
	Qingshitan Reservoir	0.8018	0.1283	0.2115	0.1497

, we evaluated the performance metrics for identifying changes in two reservoir dams by using a superpixel size of 10. For the above four evaluation metrics, it was clear that the final performance of the change detection for both reservoirs exceeded 95%. Notably, it yielded F1-score metrics of 0.9648 and 0.9767 for the Dalingtou and Qingshitan Reservoir dams’ change detections, respectively. Additionally, their Kappa metrics were 0.9625 and 0.9674, respectively, demonstrating a reliable correspondence with ground truth labels. The test results demonstrated that this model exhibited a strong change detection capability and accuracy, effectively capturing the changing information in the reservoir images. The enhanced model fused with the superpixel segmentation proposed in this study demonstrated outstanding performance in reservoir remote sensing change detection tasks.

(1)

Change detection method based on PCA and K-means [20,21]: The proposed method involves the division of difference images into nonoverlapping blocks. It utilizes PCA to extract orthogonal feature vectors and create a vector space from these features. Each pixel is projected onto this space utilizing S-dimensional feature vectors. Subsequently, the application of K-means clustering aims to partition the feature vector space into two clusters. Based on the minimum Euclidean distance, pixels are then assigned to one of these clusters. This process achieves unsupervised detection of both changed and unchanged regions. On the Dalingtou Reservoir image, this method yielded results similar to our research approach. However, its recall was still lower than our method, resulting in more false positives and false negatives. In the case of the Qingshitan Reservoir image, there were a significant number of false negatives, indicating limited model performance.

(2)

DLMRL [22,23]: The deep learning model based on reconstruction loss is trained using unlabeled, single-time, single-image data and attempts to perform change detection during inference by reconstructing one of the input images. The reconstruction-loss-based deep learning model achieved a lower accuracy on the Dalingtou Reservoir image, but the recall, F1-score and Kappa coefficient were higher. However, the recall, F1-score and Kappa coefficient were worse, and the accuracy was higher on the Qingshitan Reservoir image. The results of different images were more biased, as there were more missed areas. The robustness of their models was not strong, and the results were all poorer compared to the methods in this study.

(3)

OTSU image difference method [24]: This method is based on the OTSU thresholding technique and performs change detection by computing the global threshold for an image’s gray levels and then subtracting one time phase of remote sensing images from another. It is a classic threshold-based image change detection method. Experimental results indicated that compared to our method in this paper, it performs poorly in the context of change detection in reservoir images, which is a complex scene. The recognition results between the two images showed significant discrepancies, indicating that the model lacked robustness. It was not sensitive to subtle changes in the area and tended to have a higher number of missed detection areas.

(4)

Robust change vector analysis detection method [25,26,27]: This method is used to extract change vectors and perform change detection. The RCVA demonstrated moderate performance. Compared to our research method, it had a higher query accuracy, but the other three indicators such as the F1 score were lower. In the context of reservoir change detection in our study, this method did not provide a comprehensive coverage of change areas and had a relatively high rate of missed detections. Its effectiveness was limited in this specific scenario.

(5)

Differential principal component analysis [28]: This method first calculates the difference between multiband images from two different time periods, resulting in difference images. Subsequently, it performs a principal component analysis on these difference images to extract the top principal components, which represent the primary differential information in the images and are considered important features representing change information. In comparison to our research method, this approach exhibited higher rates of false positives and false negatives, and its overall evaluation metrics were lower. It did not perform well in detecting changes in reservoir images.

All in all, when comparing with other models in comparative experiments, some models showed subpar performance. For example, the supervised transfer learning method based on U-Net exhibited a poor transferability to reservoir images, lacking generalizability, and failing to accurately detect changes in reservoirs. The change detection methods based on PCA + K-means, deep learning models based on reconstruction loss, RCVA, DPCA and OTSU image difference method all had limitations in reservoir change detection tasks, including issues with false positives, false negatives and other challenges.

Finally, it should be pointed out that when the lighting difference between two different phase images is large, we first need to process the image for color consistency, which is conducive to the later image alignment; in addition, if the selected weather contains a large haze, the image will become unclear, which should first be de-fogged, and then processed for change detection.

6. Conclusions

The method proposed in this paper combined LSC superpixel segmentation with enhanced GrabCut, fusing Sobel edge operator and GrabCut, and could effectively perform intelligent segmentation and dam extraction from high-resolution UAV remote sensing images. The superpixels were mapped to points and rearranged into a mapping matrix, which was introduced into the segmentation algorithm, and then the Sobel edge operator was introduced into GrabCut to improve the accuracy and smoothness of the segmentation edge. Finally, the image registration method was used to obtain the change information of reservoir dams. In the change detection experiment on a reservoir dam, the smaller the superpixel size, the stronger the edge points of the mapping matrix, and the higher the change detection accuracy.

The innovation and effectiveness of the model proposed in this paper were verified by the UAV high-resolution remote sensing change detection from two typical reservoir dams in Guilin with two different time phases. This paper adopted UAV high-resolution remote sensing for the first time to detect changes in reservoir dams in Guilin and provided a technical feasibility verification for the subsequent realization of systematic and full-coverage remote sensing monitoring of 137 reservoir dams in Guilin.