Water Stream Extraction via Feature-Fused Encoder-Decoder Network Based on SAR Images

Abstract

The extraction of water stream based on synthetic aperture radar (SAR) is of great significance in surface water monitoring, flood monitoring, and the management of water resources. However, in recent years, the research mainly uses the backscattering feature (BF) to extract water bodies. In this paper, a feature-fused encoder–decoder network was proposed for delineating the water stream more completely and precisely using both the BF and polarimetric feature (PF) from SAR images. Firstly, the standard BFs were extracted and PFs were obtained using model-based decomposition. Specifically, the newly model-based decomposition, more suitable for dual-pol SAR images, was selected to acquire three different PFs of surface water stream for the first time. Five groups of candidate feature combinations were formed with two BFs and three PFs. Then, a new feature-fused encoder–decoder network (FFEDN) was developed for mining and fusing both BFs and PFs. Finally, several typical areas were selected to evaluate the performance of different combinations for water stream extraction. To further verify the effectiveness of the proposed method, two machine learning methods and four state-of-the-art deep learning algorithms were utilized for comparison. The experimental results showed that the proposed method using the optimal feature combination achieved the highest accuracy, with a precision of 95.21%, recall of 91.79%, intersection over union (IoU) score of 87.73%, overall accuracy (OA) of 93.35%, and average accuracy (AA) of 93.41%. The results showed that the performance was higher when BF and PF were combined. In short, in this study, the effectiveness of PFs for water stream extraction was verified and the proposed FFEDN can further improve the accuracy of water stream extraction.

1. Introduction

Surface water, including rivers, lakes, reservoirs, and ponds, generally has certain connectivity and forms a water stream [1]. Extracting the water stream accurately and completely can provide powerful data for surface water resources management and planning as well as drought and flood disaster prevention and control [2], which contributes to socio-economic development and ecological protection [3].

Nowadays, remote sensing technology is regarded as an efficient means for the investigation of surface water resources and has a high accuracy for the extraction of the water stream in large-scale images because it can make use of various features [4]. At present, optical remote sensing [5,6,7,8] and digital elevation models (DEMs) [9,10,11,12] obtained from different satellites have been widely used with a range of supervised and unsupervised methods [13]. However, the optical data can be prone to being affected by clouds, rain, and fogs, so it may not be possible to obtain high-quality images at the specific time [14]. Meanwhile, the methods using DEMs also have some problems such as low accuracy in extracting water bodies [15]. As a result, it is difficult to extract water body by using these data in specific area or time. Fortunately, synthetic aperture radar (SAR), as an active earth observation system, can detect targets at all times which are not affected by weather and meteorological factors and can detect targets at all times [16,17]. Water bodies have a lower scattering value and present as dark area in SAR images compared with other ground objects, which make it easy to extract surface water.

Based on these characteristics, scholars have put forward many different water extraction methods, including threshold segmentation [18,19], mathematical statistical algorithms [20], machine learning [21,22,23,24], and deep learning [25,26,27]. The threshold method determines whether the pixels are water bodies through a suitable preset threshold. This method is easy to implement, but the setting of suitable threshold is difficult to find [28]. The Wishart classifier utilizes polarimetric information to extract water bodies and has more accurate extraction results in small areas, but the spatial continuity is weak [29]. Machine learning methods, such as random forest (RF) [30], support vector machine (SVM) [31], and Markov random field (MRF) [32], require higher computing power and are always affected by speckle noise in SAR images, which lead to inaccurate extraction results. The above methods attempt to establish a mapping relationship to classify each pixel, which is difficult due to the influence of speckle noise [33]. In recent years, deep learning methods have overcome these problems because of their powerful feature extraction capabilities. Convolutional neural networks (CNN), such as U-Net, DeepLab, and others, have achieved remarkable results [34,35,36,37]. Nemni et al. used an improved U-Net to rapidly extract water bodies with VV polarization backscattering images of Sentinel-1 [38]. Based on the DeepLab model, Guo et al. established a new model to quickly extract flooded areas by using both VV and VH polarization backscattering images [39]. Xue et al., proposed dense-coordinate-feature-concatenate network (DCFNet) and combines the features of HH and HV polarization backscattering images [40]. Chen et al., proposed a model named MSF-MLSAN (multi-scale spatial feature-multi-level selective attention network) to extract a water body in backscattering images of millimeter-wave SAR data. [41]. Zhang et al., proposed a U-Net-based model, WENET, to extract the waterline from Sentinel-1 SAR images [42]. Baumhoer et al., used a U-Net model to extract the Antarctic coastline for tracking glacier and ice shelf front movement using Sentinel-1 data [43]. Combining the holistically-nested edge detection (HED) and U-Net, Heidler et al. recently successfully extracted the Antarctic coast’s waterline and completed the sea–land classification simultaneously for Sentinel-1 SAR imagery [44]. However, these deep learning methods were mainly focused on the backscattering features (BFs) and select one or multiple BFs as the input of the model; the polarimetric features (PFs) have not been utilized adequately.

This paper proposed a deep learning method, which fully integrated BFs and PFs, and whether the fused features can extract water stream more accurately from Sentinel-1A SAR images was discussed. First, the BFs of VV and VH were obtained and the PFs were gained using the model-based decomposition method. Then, a feature-fused encoder–decoder Network based on U-Net [45] was proposed for extracting and fusing valid water features from all of the above features to extract the water stream more precisely and connectively. The main contributions of this paper are:

• The water features from BFs and PFs are fully integrated by using feature-fused block and the influence of different combinations on water stream extraction is explored. Especially, the influence of PFs obtained by the newly model-based decomposition adapted to dual-pol SAR images was first discussed in the task of water stream extraction.
• An effective water stream extraction model FFEDN is proposed. It has an outstanding capability of feature learning and feature fusing, which improves the extraction accuracy.

The rest of this paper is organized as follows. Section 2 describes the study area and datasets for data pre-processing, model training, and the accuracy assessment. Section 3 shows the results of extraction and Section 4 discusses the results as remaining limitations of our method. Finally, Section 5 summarizes the findings of this paper.

2. Materials and Methods

2.1. Study Area

As shown in Figure 1, Henan Province (110°21′E–116°39′E, 31°23′N–36°22′N) is located in the center of China, which has an area of 167,000 km². Henan Province is the only one that spans the Yangtze River Basin, the Yellow River Basin, the Huaihe River Basin, and the Haihe River Basin, with numerous tributaries crisscrossing the territory. However, the total amount of water resources is at the middle level of the whole country, and the per capita water resources are less than 1/6 of the whole country. Meanwhile, there has been a serious shortage and uneven distribution in space and annual water due to the influence of latitude and climatic conditions. Therefore, high precision monitoring of the complete water stream in the study area is of great practical significance for the planning and utilization of water resources.

2.2. Data and Pre-Processing

2.2.1. Sentinel-1A Data and Pre-Processing

Sentinel-1A SAR images acquired by the European Space Agency (ESA) have a short revisiting period of 12 days and a high spatial resolution of 5 × 20 m [46]. The parameters of the data are shown in

Table 1. Parameters of Sentinel-1A SAR Images.

ID	Time (M/D/Y)	Range Spacing (m)	Azimuth Spacing (m)	Orbit Direction	Processing Level
1	14 May 2021	2.33	13.94	Ascending	L1-SLC (IW)
2	14 May 2021	2.33	13.95	Ascending	L1-SLC (IW)
3	14 May 2021	2.33	13.95	Ascending	L1-SLC (IW)
4	14 May 2021	2.33	13.94	Ascending	L1-SLC (IW)
5	21 May 2021	2.33	13.94	Ascending	L1-SLC (IW)
6	4 June 2021	2.33	13.94	Ascending	L1-SLC (IW)
7	4 June 2021	2.33	13.93	Ascending	L1-SLC (IW)
8	20 July 2021	2.33	13.95	Ascending	L1-SLC (IW)
9	20 July 2021	2.33	13.94	Ascending	L1-SLC (IW)
10	22 July 2021	2.33	13.94	Ascending	L1-SLC (IW)
11	27 July 2021	2.33	13.95	Ascending	L1-SLC (IW)
12	27 July 2021	2.33	13.94	Ascending	L1-SLC (IW)
13	27 July 2021	2.33	13.95	Ascending	L1-SLC (IW)
14	27 July 2021	2.33	13.93	Ascending	L1-SLC (IW)
15	27 July 2021	2.33	13.94	Ascending	L1-SLC (IW)

. Single Look Complex (SLC) image, which includes amplitude and phase information, was chosen because it can provide abundant polarimetric information. As the scale of Sentinel-1A images can not cover the whole study area, a total of 14 scene SAR images from 14 May 2021 to 27 July 2021 were selected which can observe the entire study area. Among them, 13 scene SAR images were selected for model training and 1 scene SAR images for validation. Finally, 14 scene SAR images were spliced to obtain the result of water stream extraction in Henan Province.

The pre-processing steps for obtaining the BF and the polarimetric matrix from Sentinel-1A images were shown in Figure 2. For the same steps, the orbit file application, thermal noise removal, deburst, multi-looking, refined Lee filtering, and range-Doppler terrain correction were used. It should be noted that the number of range looks and azimuth looks is 4 and 1 in the step of multi-looking. In addition, the type of filter is refined Lee and window size is 7 × 7 pixels. Additionally, in the range-Doppler terrain correction, the shuttle radar topography mission (SRTM) with a resolution of 30 m was used to acquire the precise geographic information. Each burst was merged which has the effective signal parts by the operation of deburst. The impact of coherent speckle noise can be reduced by the multi-looking and filtering operation. Among the steps, the difference lied in that the output of calibration was σ0 band to obtain the BF, while the output of calibration was a complex band to obtain the polarimetric matrix. In addition, the coherency matrix was generated after the deburst to obtain the polarimetric matrix.

2.2.2. Ground Truth Data and Pre-Processing

In order to obtain a comprehensive and accurate view of the water bodies, the labeling process of samples can be divided into three steps. The first step was using the traditional threshold method for preliminary mapping. In the second step, the preliminary mapping of water bodies was corrected by visual interpretation to obtain more accurate labeling. Images of the same area and similar dates obtained by Google Earth were used as references. By comparison, the mistakenly extracted road and hill shade could be removed. Finally, the binary map was created by setting the pixel values of water and non-water to 0 and 1, respectively.

The sample set was divided into 1024 × 1024 with a 50% overlap area to make the sample representative and the model stable. In the training process, a total of 1521 sliced pictures were obtained by the 13 scene SAR images. 80% of the sliced pictures were randomly selected as training sets and 20% as test sets. Eventually, 1217 pictures were used as training samples and 304 pictures were used as test samples. By obtaining the binary map of water or non-water, the parameters of the samples were illustrated in

Table 2. Ground truth samples in one scene SAR image in the study area.

Label	Type	Total Number of Samples	Number of Training Samples	Number of Test Samples
0	Water	239,232,614	191,386,092	47,846,522
1	Non-Water	1,355,651,482	1,084,521,186	271,130,296

. Figure 3 shows the BF of VV and VH polarization and corresponding samples of two regions.

2.3. Water Stream Extraction

The flow chart of the proposed method was presented in Figure 4. After the data pre-processing, the water stream extraction method was mainly divided into 3 steps. In step 1, standard BFs were extracted and the PFs were obtained based on the model-based decomposition of dual-pol SAR data [47]. Additionally, the above features were used to form five groups of candidate feature combinations. In step 2, FFEDN was proposed based on U-Net, including feature-fused block, encoder, pyramid pooling module, and decoder in order to improve the ability to mine and fuse both BFs and PFs, thus, better representing the characteristics of water bodies. In step 3, the extraction results of the water stream with different combinations were explored based on FFEDN by quantitative and qualitative evaluation. Through the analysis of precision and efficiency, the optimal combination was selected.

2.3.1. Feature Acquisition and Combination

The BFs of Sentinel-1 with two polarization modes, namely VV and VH, can be obtained through the pre-processing. The BFs can be calculated as follows:

$${\sigma }_0^{VV}=10lo{g}_{10}\left(\frac{\left|D{N}_{VV}\right|}{A_{VV}}\right)$$

(1)

$${\sigma }_0^{VH}=10lo{g}_{10}\frac{\left|D{N}_{VH}\right|}{A_{VH}}$$

(2)

where $DN$ is the original value of SAR image and $A$ depends on the lookup table of auxiliary data.

The Sentinel-1 datasets used in this study were dual-polarimetric images with $VV$ and $VH$ polarization, for which the scattering matrix is defined as:

$$S=\left[\begin{array}{cc}0& 0\\ S_{VH}& S_{VV}\end{array}\right]$$

(3)

The corresponding target scattering vector $k$ can be expressed as:

$$k={\left(\sqrt{2}{S}_{VH},S_{VV}\right)}^T$$

(4)

The Sentinel-1 dual-polarimetric SAR data can be represented by the following polarimetric covariance matrix:

$$C_{2X2}=k{k}^{\gamma }=\left[\begin{array}{cc}{c}_{11}& c_{12}\\ c_{21}& c_{22}\end{array}\right]=\left[\begin{array}{cc}\langle 2{\left|S_{VH}\right|}^2\rangle & \langle \sqrt{2}{S}_{VH}{S_{VV}}^{\ast }\rangle \\ \langle \sqrt{2}{S}_{VV}{S_{VH}}^{\ast }\rangle & \langle {\left|S_{VV}\right|}^2\rangle \end{array}\right]$$

(5)

where $\gamma$ represents the conjugate transpose and $\ast$ is the conjugate operator.

To make full use of the polarimetric information of Sentinel-1 images in water stream extraction, the polarimetric decomposition parameters (volume scattering $m_v$, remaining scattering $m_s$, and the difference between them $m_{rat}$) were obtained from Sentinel-1 images by adopting the newly model-based decomposition adapted for dual-polarimetric SAR images [46].

Transforming $C_{2X2}$ into Stokes vector:

$$\underset{\_}{S}=\left[\begin{array}{c}{s}_1\\ s_2\\ s_3\\ s_4\end{array}\right]=\left[\begin{array}{c}{c}_{11}+c_{22}\\ c_{11}-c_{22}\\ 2Re\left(c_{12}\right)\\ 2Im\left(c_{12}\right)\end{array}\right]$$

(6)

where $Re\left(c_{12}\right)$ and $Im\left(c_{12}\right)$ represent the real part and the imaginary part of $c_{12}$, respectively.

Then, transforming $\underset{\_}{S}$ into the sum of three polarimetric components:

$$\underset{\_}{S}=m_v{\underset{\_}{s}}_v+m_s{\underset{\_}{s}}_p+n{\underset{\_}{s}}_n$$

(7)

where ${\underset{\_}{s}}_v$ represents an arbitrary volume model, ${\underset{\_}{s}}_p$ represents the polarized Stokes vectors, $n$ is noise term, ${\underset{\_}{s}}_n$ is a randomly polarized Stokes vector that can be ignored using noise subtraction techniques [48], $m_v$ is the volume scattering component, and $m_s$ is the remaining scattering component.

Therefore, two items should be considered as follows:

$$\underset{\_}{S}=m_v\left[\begin{array}{c}1\\ \pm 0.5\\ 0\\ 0\end{array}\right]+m_s\left[\begin{array}{c}1\\ cos2\alpha \\ sin2\alpha cos\delta \\ sin2\alpha sin\delta \end{array}\right]$$

(8)

$$\left\{\begin{array}{c}\alpha =\frac{1}{2}co{s}^{-1}\left(\frac{s_2}{s_1}\right)\\ \delta =\arg \left(s_3+i{s}_4\right)\end{array}\right.$$

(9)

where $\alpha$ and $\delta$ are the ratio parameters, symbol + is for H transmit, and − for V.

According to the above formulas, $m_v$ and $m_s$ can be calculated as follows.

$$m_v=\pm \frac{2Re\left(c_{12}\right)}{\arcsin \left(co{s}^{-1}\left(\frac{c_{11}-c_{22}}{c_{11}+c_{22}}\right)\right)\arccos \left(\arg \left(2Re\left(c_{12}\right)+2Im\left(c_{12}\right)\right)\right)}$$

(10)

$$m_s=c_{11}+c_{22}-m_v$$

(11)

The $m_{rat}$ can be calculated as follows:

$$m_{rat}=m_v-m_s$$

(12)

Subsequently, the three extracted PFs were obtained by decomposition.

After BF and PF were obtained by pre-processing, five combinations shown in

Table 3. Five Candidate Feature Combinations.

	BF		PF
	${\mathit{\sigma }}_{\mathbf{0}}^{\mathit{V}\mathit{V}}$	${\mathit{\sigma }}_{\mathbf{0}}^{\mathit{V}\mathit{H}}$	${\mathit{m}}_{\mathit{v}}$	${\mathit{m}}_{\mathit{s}}$	${\mathit{m}}_{\mathit{r}\mathit{a}\mathit{t}}$
Combination A	√	√
Combination B	√	√	√
Combination C	√	√		√
Combination D	√	√			√
Combination E	√	√	√	√	√

were formed for the extraction of the water stream. In previous studies, water extraction was based on BF. In order to effectively verify the performance of the model after adding PF, combination A was BFs. Different PFs, which represent different characteristics of the water body, were added to form different combinations.

2.3.2. FEEDN Model

FFEDN was built to obtain a more accurate result of the water stream by fusing and using the BFs and PFs. The structure of FFEDN, shown in Figure 5, consisted of four main components: feature-fused block, encoder block, pyramid pooling module, and decoder block. Feature fusion [49] and the encoder–decoder framework [50,51,52,53] were widely used in deep learning, which had proved to be effective for feature mining and fusing with different images and resolutions. The encoder–decoder framework was adapted as the backbone network and the skip connection was used between the encoder and decoder to obtain features with different resolution scales. The pyramid pooling module lay between the encoder and decoder part. The attention block is mainly inspired by the attention gate model [54], which is used to identify the irrelevant part of the model and learn the characteristics related to the task. The FFEDN attempted to mine water features of BFs and PFs and combine them together effectively, and the features of different scales were fused through encoder–decoder to improve the ability to extract the water stream.

In the feature-fused block, convolutional filters were set up to further mine the deep and correlation information from BFs and PFs. After the convolution layer, the batch normalization (BN) layers and ReLU activation function followed, respectively, which can improve network generalization capabilities and effectively avoid the problems of gradient explosion and gradient disappearance [55]. The fused feature maps were fed into the encoder and compressed into a potential space representation.

As shown in Figure 6, the pyramid pooling module (PPM) [56,57] fuses features of four different scales (1 × 1, 2 × 2, 3 × 3, and 6 × 6). The outputs of different levels contain the feature maps with different sizes. To maintain the weight of the global feature, the 1 × 1 convolution after each pyramid level was used to reduce the dimensions to $1/N$ of the original features, where $N$ is the size of the pyramid level. Then, the bilinear interpolation upsampling was used to restore the channel dimension of the low-dimensional feature map to the original feature map. Finally, different levels of features were concatenated as the final global feature of pyramid pooling. The pyramid pooling module collected multi-level features of multi-scale water bodies, such as lakes or small rivers with different characteristics, and combined them with the original feature map extracted for the encoder.

Then, the decoder aimed to reconstruct the spatial representation, which mapped the feature vectors back to the original input image size. The skip connection between the encoder and decoder is developed to provide rich fundamental information. At last, a water–non-water binary map was generated by convolution operation with the kernel size of 1 × 1. Especially, as shown in Figure 7, the attention block (ABL) [58,59] was used in the decoder to make the network focus on water stream extraction by identifying the irrelevant part and learning the characteristics related to the task. The $g$ represents the gating signal output from the downsampling layer and $X^1$ represents the feature map of the upsampling layer passed by the skip connection. DConv2d represents the mean pool after using the dilated convolution kernel. Then, the gate signal and feature map are connected and the ReLU activation function, dimension reduction, and Sigmod activation are used. The results are the dot multiplied with $X^1$ to obtain the features concerned.

It should be noted that the proportions of water and background pixels were imbalanced, thus, using Dice loss which was designed to address the extremely imbalanced samples [60] to solve the problem. In the task of water bodies extraction, the Dice loss function was as follows:

$$L_{Dice}=1-Dice$$

(13)

where $L_{Dice}$ represents the Dice loss and $Dice$ represents Dice coefficient. Using $L_{Dice} ϵ \left[0, 1\right]$ to balance the importance of water and non-water pixels which can avoid non-water pixels dominating the gradient. Larger $Dice$ puts more weight on water pixels.

The calculation equation of the Dice coefficient was as follows:

$$Dice=\frac{2\left(A\cap B\right)}{\left|A\right|+\left|B\right|}$$

(14)

where $A\cap B$ specifies the intersection of the generated results and the ground truth value and $\left|A\right|$ and $\left|B\right|$ represent the pixel number of the extraction results and ground truth, respectively. The extraction performance was better when the Dice coefficient was higher.

In summary, the FFEDN model contained four blocks: feature-fused block, encoder block, pyramid pooling module, and decoder block. The feature-fused block was used to extract and fuse the water features from different kinds of features. The encoder block, which extracts features using convolution, increased the receptive field by the pool layer. The pyramid pooling module could realize multi-scale feature extraction from the feature map of both small rivers or large water bodies with different characteristics. In the decoder, through deconvolution and the attention block, the features were reproduced, and the upsampling was restored to the original size of the input. The process used supervised learning to discover internal correlations between the BF and PF and extract as much useful information as possible. The detailed parameters of FFEDN are shown in

Table 4. Parameters of FFEDN.

BFs (1024,1024,2)		PFs (1024,1024,3)
Conv2D Filters = 64, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(1024,1024,64)	Conv2D Filters = 64, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(1024,1024,64)
Conv2D Filters = 64, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(1024,1024,64)	Conv2D Filters = 64, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(1024,1024,64)
Layer		Parameters	Output shape
Concat			(1024,1024,128)
MaxPooling2D		Kernel_size = 2	(512,512,128)
Conv2D		Filters = 256, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(512,512,256)
MaxPooling2D		Kernel_size = 2	(256,256,256)
Conv2D		Filters = 512, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(256,256,512)
Up-Sampling		Kernel_size = 2	(512,512,512)
Pyramid Pooling Module		Kernel_size = 1,2,3,6	(512,512,512)
Attention Block			(512,512,512)
Conv2D		Filters = 256, kernel_size = 3, activation = ‘ReLU’, BatchNormalization	(512,512,256)
Up-Sampling		Kernel_size = 2	(1024,1024,256)
Conv2D		Filters = 128, kernel_size = 3, activation = ’ReLU’, BatchNormalization	(1024,1024,128)
Conv		kernel_size = 1, activation = ’Sigmod’,	(1024,1024,1)

.

2.3.3. Optimal Combination Selection

In order to select the optimal combination, confusion matrix was used to evaluate the extraction results. As shown in

Table 5. Confusion matrix.

		Generated Label
		Water	Non-Water
Ground truth	Water	True Positive (TP)	False Negative (FN)
	Non-Water	False Positive (FP)	True Negative (TN)

, the definitions of the four indicators were given in the confusion matrix.

Precision, recall, intersection over union (IoU), overall accuracy (OA), and average accuracy (AA) were chosen as evaluation indicators to carry out quantitative analysis. The calculations of them were illustrated in the Equations (15)–(19), respectively. High precision means that the extraction results are more accurate. A high value of recall means that the model can find more water bodies in the image. IoU represents the overlap ratio of the model extraction results.

Precision:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(15)

Recall:

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(16)

IoU:

$$\mathrm{IoU}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}+\mathrm{FN}}$$

(17)

OA:

$$\mathrm{OA}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{FN}+\mathrm{FP}+\mathrm{TN}}$$

(18)

AA:

$$\mathrm{AA}=\frac{1}{2}\left(\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}+\frac{\mathrm{TN}}{\mathrm{FP}+\mathrm{TN}}\right)$$

(19)

The McNemar test [61] was used to access the statistical significance of differences in the extraction accuracy. McNemar’s test principle is based on the evaluation of a contingency table with a 2 × 2 dimension considering only correct and incorrect points for two different methods. (

Table 6. Data layout for McNemar test between two extraction results.

		Extraction Result 2
		Correct	Incorrect	Total
Extraction Result 1	Correct	$f_{11}$	$f_{12}$	$f_{11}+f_{12}$
	Incorrect	$f_{21}$	$f_{22}$	$f_{21}+f_{22}$
	Total	$f_{11}+f_{21}$	$f_{12}+f_{22}$	$f_{11}+f_{12}+f_{21}+f_{22}$

).

In the relative comparison between different combinations in our manuscript, the chi-square (${\chi }^2$) has one degree of freedom and uses exclusively discordant samples as shown in Equation (20):

$${\chi }^2=\frac{{(f_{12}-f_{21})}^2}{f_{12}+f_{21}}$$

(20)

3. Results

3.1. Implementation Details

All of the experiments were completed in the TensorFlow environment with GeForce RTX 3080Ti and 64G RAM. The FFEDN was trained by an Adam optimizer [57,58] which has computational efficiency and low memory requirements. The weight decay was set to 0.004 and the learning rate was set to 0.0001. The model was trained on 100 epochs to obtain the final results and the batch size was 32. The hardware and software configuration of the network was presented in

Table 7. Hardware and software configurations of the experiments.

Configuration	Version
GPU	GeForce RTX 3080Ti
Memory	64 G
Language	Python 3.8.3
Frame	Tensorflow 1.14.0

.

The FLOPs of FFEDN were 62,049,671 and the model size was 359 MB. On average, it took around 4–5 h to train the model. To ensure a fair comparison, all parameters were set as the same in all experiments. Figure 8 showed the loss and accuracy plot for FFEDN obtained on training and validation images of the SAR sub-image dataset.

3.2. Evaluation of Different Combinations with FFEDN

There were 14 images in the experiment, 13 of which were used to train the FFEDN model. After training the model, the remaining SAR image not included in the training process was used for validation. In the task of water stream extraction, the fine water bodies were easily omitting, the shadows were lightly classified as water bodies, and the small rivers were easily extracted incompletely. All the above phenomena will lead to inaccurate results of water stream. Thus, three different areas were selected, namely, rural (Region A), hilly (Region B), and urban (Region C) which includes water bodies with different characteristics, as shown in Figure 9a–c, respectively. The selected area was 1024 × 1024, as shown in the orange rectangle in Figure 9. The relatively complete water stream can be constructed if the water bodies can be extracted in the different areas.

3.2.1. Qualitative Evaluation

The five combinations shown in Table 3 were used as the inputs of FFEDN and the optimal one was explored through qualitative evaluation and quantitative evaluation, respectively. The extraction results of Region A, B, and C using different combinations were shown in Figure 10.

Whether in Region A, B or C, when the Combination C was used as input, the extracted results were more refined and complete. In Region A, as shown in Figure 9c, both the large water bodies and fine water bodies were extracted more completely and accurately. In Region B, as shown in Figure 10(c2), especially in the orange rectangle, the erroneous extraction of hilly shadows as water was effectively reduced. In Region C, marked with blue rectangle in Figure 10(c3), the small rivers flowing through the urban area could be effectively extracted.

For Combination A, there were errors of false and missing information in extracting fine water bodies, as shown in Figure 10(a1). The shadow of hills was extracted mistakenly as a water body in the Region B marked with a yellow rectangle, as shown in Figure 10(a2). The river in Region C was not extracted connectively, as shown in the blue rectangle in Figure 10(a3). For Combination B, the fine water stream is extracted accurately to a certain degree compared with Combination A, as shown in Figure 10(b1). However, the shadows were still mistakenly classified as a water body in Figure 10(b2). The results of river extraction in the Region C were more accurate which also had a missing extraction in the blue rectangle in Figure 10(b3). For Combination D, the extraction result in Figure 10(d1) was complete. However, the hilly shadow was still wrongly classified as a water body in Figure 10(d2) and the river flowing through the Region C in Figure 10(d3) was not correctly identified to some extent. For Combination E, the fine water body, hilly shadows, and small rivers were not correctly extracted.

3.2.2. Quantitative Evaluation

The validation samples were fed into the FFEDN, then the outputs of models were compared with the labels to generate confusion matrices, which were shown in Figure 11. According to the confusion matrices, the different combinations were quantitatively evaluated.

The results were listed in

Table 8. Results of quantitative evaluation by different combination.

	Precision	Recall	IoU	OA	AA
Combination A	81.63%	79.50%	67.43%	80.29%	80.31%
Combination B	83.52%	87.73%	74.79%	85.92%	86.01%
Combination C	95.21%	91.79%	87.73%	93.35%	93.41%
Combination D	89.73%	81.66%	74.68%	84.79%	85.13%
Combination E	88.14%	87.02%	77.90%	87.50%	87.50%

. Among them, the indicator values were lowest when using Combination A as input. The precision, recall, OA, and AA were around 80% and the IoU was lower than 70%. Combination C achieved the best accuracy which precision scored 95.21%, recall scored 91.79%, IoU scored 87.73%, OA scored 93.35%, and AA scored 93.41%. This was because the scattering type of water body in the SAR image is surface scattering which can be represented by ${\mathit{m}}_{\mathit{s}}$. Therefore, the indicator value was highest when using Combination C as input. Precision of Combination B was 83.52% because ${\mathit{m}}_{\mathit{v}}$ is different from the characteristics of water body in the SAR image. Precision, recall, IoU, OA, and AA of Combination D were lower than Combination C 5.48%, 10.13%, 7.96%, 13.05%, 8.56%, and 8.82%, respectively. This was because the features of ${\mathit{m}}_{\mathit{v}}$ and ${\mathit{m}}_{\mathit{r}\mathit{a}\mathit{t}}$ have a negative influence on water extraction, resulting in low indicator values. Precision, recall, OA, and AA of Combination E were all above 85%, but the IoU was still low which was due to the negative effects of ${\mathit{m}}_{\mathit{v}}$ and ${\mathit{m}}_{\mathit{r}\mathit{a}\mathit{t}}$.

McNemar test results highlight that the extraction results between different Combinations. The chi-square value between Combination A and other Combinations (B, C, and D) were given in

Table 9. Result of McNemar test.

	${\mathit{\chi }}^{\mathbf{2}}\mathbf{\text{-}}\mathbf{Value}$
Combination A and B	52.67
Combination A and C	107.92
Combination A and D	61.75
Combination A and E	56.01

. The ${\chi }^2$ values were larger than 3.84 and the results confirm the better performance after fusing BFs and PFs.

3.3. Ablation Study of FFEDN with Feature Combination C

In order to verify each improvement measures’ effectiveness in FFEDN, the experiments were conducted. The ablation study contains two parts: pyramid pooling module and attention block. The baseline was FFEDN with the feature Combination C as input.

Table 10. Different strategies for the ablation study.

	Method	PPM	ABL
Strategy A	FFEDN
Strategy B	(w/o) PPM	-
Strategy C	(w/o) ABL		-
Strategy D	(w/o) PPM and ABL	-	-

showed different strategies for the ablation study.

3.3.1. Qualitative Evaluation

The extraction results of the different strategies were shown in Figure 12. After removing the PPM from FFEDN, the details such as small rivers tributaries in the middle of the rivers were missed, as shown in Figure 12(b3). When ABL was removed, the extraction result in Region B was not accurate with classifying shadows as water bodies, as shown in Figure 12(c2). When both PPM and ABL were removed, the model only can produce low confidence predictions and blurred boundary.

3.3.2. Quantitative Evaluation

As can be seen from

Table 11. Quantitative evaluation of different strategies for the ablation study.

	Precision	Recall	IoU	OA	AA
Strategy A	95.21%	91.79%	87.73%	93.35%	93.41%
Strategy B	89.17%	85.51%	75.01%	85.20%	81.92%
Strategy C	87.36%	86.69%	76.57%	86.64%	86.01%
Strategy D	83.15%	81.39%	68.85%	81.19%	79.62%

, after removing the PPM, the precision, recall, IoU, OA, and AA of the water extraction task were decreased by 6.04%, 6.25%, 12.72%, 8.15%, and 11.49%, respectively. After removing the ABL, the indicators were decreased as well. When the model was trained without PPM and ABL, the indicators were lowest. Both PPM and ABL improved the performance of FFEDN.

3.4. Evaluation of Different Methods with Feature Combination C

3.4.1. Compared Methods

In order to verify the effectiveness of the methods proposed, several machine learning methods and deep learning methods were selected for comparison. The classical machine learning methods (RF, SVM) and some CNN-based methods (U-Net, U-Net-Resnet [62], MSF-MLSAN [41] and WENET [42]) were selected. SVM is a supervised classification method for binary cases. It uses a subset of training samples (called support vectors) close to the decision boundary to calculate the linear decision hyperplane. RF is a classifier that consists of many decision and regression trees to train and predict the classification results. The U-Net and U-Net-Resnet model were widely used for feature extraction and water extraction. MSF-MLSAN [41] was proposed to extract water bodies in mountain regions by using millimeter wave SAR images and had achieved potential performance. WENET [42] integrates the features of SAR images at different scales for water extraction. The hyperparameters of the SVM and RF were listed in

Table 12. Hyperparameters of the SVM and RF.

Classifier	Parameters	Description	Value
SVM	C Kernel	Penalty coefficient Kernel function	2 Rbf
RF	N_estimators	Number of decision trees	550

.

3.4.2. Qualitative Evaluation

The extraction results of the seven methods selected above were shown in Figure 13. All methods extracted water accurately in a large area and the proposed method had the best effect, especially mapping the water bodies in Region B and drawing small rivers flowing through the Region C thoroughly, as shown in Figure 13(a2,a3). For machine learning methods, as shown in Figure 13(b1,c1), RF and SVM can also extract small water bodies. However, the two methods mistakenly extracted hilly shadows as water, as shown in Figure 13(b2,c2). Meanwhile, as shown in Figure 13(b3,c3), the results presented a missing extraction phenomenon of relatively small rivers as shown in the blue rectangle. For deep learning methods, the water body in Region A was extracted accurately as shown in Figure 13(d1–g1). As shown in Figure 13(f2,g2), MSF-MLSAN and WENET mapped the water body in Region B accurately without extracting shade mistakenly. Meanwhile, the other deep learning methods also extracted hill shade mistakenly. As shown in Figure 13(d3–g3), the four deep learning methods presented a missing extraction phenomenon of relatively small rivers as shown in the blue rectangle, which was due to the influence of urban buildings or trees.

3.4.3. Quantitative Evaluation

The validation images were fed into the models, then the outputs of models were compared with the labels to generate confusion matrices, which are shown in Figure 14. According to the confusion matrices, the methods were quantitatively evaluated. The results were listed in

Table 13. Quantitative evaluation of different models.

Methods	Precision	Recall	IoU	OA	AA
FFEDN	95.21%	91.79%	87.73%	93.35%	93.41%
RF	78.99%	73.56%	61.38%	75.15%	75.30%
SVM	76.29%	73.49%	59.83%	74.39%	74.43%
U-Net	87.85%	78.94%	71.17%	82.21%	82.62%
U-Net-ResNet	84.99%	83.07%	72.44%	83.84%	83.85%
MSF-MLSAN	90.14%	85.55%	78.23%	87.46%	87.56%
WENET	89.21%	84.43%	76.61%	86.38%	86.50%

.

Among them, the proposed FFEDN achieved the best accuracy. Precision scored 95.21%, recall scored 91.79%, IoU scored 87.73%, OA scored 93.35%, and AA scored 93.41%. The machine learning methods cannot effectively establish a mapping relationship to classify each pixel, so values of indicators were lowest. Especially in the Region B and Region C, complex scenarios made them more difficult to extract water accurately. For deep learning methods, the indicator values improved compared with machine learning methods. The precision of U-Net and U-Net-ResNet was 87.85% and 84.99%, respectively, which indicates that the encoder–decoder network can fuse abstract high-level information and detailed low-level information. MSF-MLSAN and WENET performed well and the indicators were higher. However, compared with the proposed method, the other deep learning methods can not map a complete water stream because the water bodies in complex scenes were difficult to extract.

To objectively assess the model ability of each method, a visual comparison of the accuracy evaluation indicators precision, recall, IoU, OA, and AA were presented in Figure 15. The OA can be ranked in the following order: FFEDN > MSF-MLSAN > WENET > U-Net-ResNet > U-Net > RF > SVM. It demonstrated that the deep learning methods outperformed the machine learning methods on the task of water stream extraction. Furthermore, the proposed network model outperformed other models.

3.5. Extraction Result of Study Area

As shown in Figure 16, using the proposed model, FFEDN, the final water stream of the study area was extracted from the stitched image of the 14 SAR images. It can be shown that rivers, lakes, reservoirs, and ponds were all extracted. The regional distribution of water resources in Henan Province was uneven. The plain areas of eastern and northern Henan were densely populated, which were the main grain producing areas. However, the serious shortage of water resources in this area directly restricted the sustainable development of the local national economy. Accurate extraction of the water stream can provide powerful data for surface water resources management and planning as well as drought and flood disaster prevention, and it can also contribute to social and economic development and ecological protection. The Landsat-8 optical image on 18 May 2021 with cloud cover of 0.13% was used for qualitative evaluation. The band combinations of the Landsat-8 image can extract specific information to help better understand the ground objects. The image of bands B4 (Red), B3 (Green), and B2 (Blue) combination was defined as a pseudo-color image, as shown in Figure 16b. As shown in Figure 16b–d, compared with the optical image and the ground truth, the tributaries were extracted accurately, thus, constructing a more connective water stream. The code of the implemented methods is available at https://github.com/yuandadada/SARwater (accessed on 15 November 2022).

4. Discussion

In this work, the water stream extraction method using BF and PF was carried out. First, the BFs were extracted and PFs were obtained through model-based decomposition. Then, a new model FFEDN was developed for the water stream extraction task, which mined and combined water features from both BFs and PFs. Lastly, by comparing the results of the water stream extraction with different combinations, the optimal one was chosen. The results showed that the extraction results were improved indicating that PF can effectively manifest the physical scattering mechanism of water bodies in SAR images. To further verify the effectiveness of the proposed method, two machine learning methods and the other four deep learning methods were chosen to compare with FFEDN and the results demonstrated the validity of FFEDN in the extraction of the water stream. The water stream of the study area was extracted by the method proposed in this paper.

First of all, the model-based decomposition of dual-pol SAR data was used to extract the polarimetric features from Sentinel-1A SAR data. Other decomposition methods, such as Freeman and Yamaguchi, have been widely used in many applications. However, most of these methods were designed to be used with quad-pol data and always lead to the loss of polarimetric information when decomposing the dual-pol SAR data. Fortunately, the model-based decomposition used in this paper can extract important underlying physical information from Sentinel-1A images, providing the data for the correct interpretation of parameters of different polarimetric components.

Moreover, this paper demonstrated that the FFEDN outperformed the machine learning methods SVM and RF, and deep learning methods U-Net, etc. Based on the optimal feature, the most representative machine learning methods and deep learning methods, RF, SVM, U-Net, U-Net-Resnet, MSF-MLSAN, and WENET, were chosen for comparison. As can be seen in Figure 13, the two machine learning methods did not achieve good results in both simple and complex environments. The deep learning methods achieved good results in the water extraction when the scene was not complicated, but the extraction accuracy in the hilly area affected by shadows and buildings was lower, so these deep learning methods found it impossible to form a connected water stream. In addition, the comparative deep learning methods did not input the PFs when establishing the model. In particular, the proposed FFEDN showed good extraction performance in every scene, exceeding 95%, 91%, 87%, 93%, and 93% of precision, recall, IoU, OA, and AA, respectively.

However, the local rivers may have had seasonal cut offs at the time of data acquisition and the extracted water stream may not be accurate. This problem should be considered and overcome by using multi-temporal data in the same area. Meanwhile, because of the small shape and narrow width of lakes and rivers are similar to the spatial resolution of the SAR images. Another work direction is to apply the proposed method with higher resolution SAR data to extract the water stream more connectively.

5. Conclusions

In this study, a deep learning method was proposed for water stream extraction named FFEDN using Sentinel-1A SLC images. The method first obtained the BFs and the PFs from Sentinel-1A images using model-based decomposition. Then, based on U-Net, FFEDN was constructed. Next, BF and PF, which formed different combinations, were used as inputs to FFEDN to select the optimal features. Finally, through qualitative analysis, the accuracy, integrity, and connectivity of the extracted water body were analyzed and the optimal feature combination was selected. Through a comparison with two machine learning methods and four other deep learning methods, the application potential of the proposed method in water stream extraction was verified. The proposed method achieved the best results in both qualitative and quantitative analysis. Therefore, FFEDNN was used to extract the water stream of Henan Province and the results were analyzed accordingly.

Furthermore, a few interesting things were found in this paper. For example, the combination of BF and PF can obtain higher extraction accuracy.

This study demonstrated the significance of PFs from Sentinel-1 data. Meanwhile, deep learning methods had great potential for water stream extraction. In the future, with the continuous optimization of decomposition methods and deep learning models, water streams will be extracted more accurately.