Sub-Pixel Surface Water Mapping for Heterogeneous Areas from Sentinel-2 Images: A Case Study in the Jinshui Basin, China

Abstract

Mapping high-spatial-resolution surface water bodies in urban and suburban areas is crucial in understanding the spatial distribution of surface water. Although Sentinel-2 images are popular in mapping water bodies, they are impacted by the mixed-pixel problem. Sub-pixel mapping can predict finer-spatial-resolution maps from the input remote sensing image and reduce the mixed-pixel problem to a great extent. This study proposes a sub-pixel surface water mapping method based on morphological dilation and erosion operations and the Markov random field (DE_MRF) to predict a 2 m resolution surface water map for heterogeneous regions from Sentinel-2 imagery. DE_MRF first segments the normalized difference water index image to extract water pixels and then detects the mixed pixels by using combined morphological dilation and erosion operations. For the mixed pixels, DE_MRF considers the intra-pixel spectral variability by extracting multiple water endmembers and multiple land endmembers within a local window to generate the water fraction images through spectral unmixing. DE_MRF was evaluated in the Jinshui Basin, China. The results suggested that DE_MRF generated a lower commission error rate for water pixels compared to the comparison methods. Because DE_MRF considers the intra-class spectral variabilities in the unmixing, it is better in mapping sub-pixel water distribution in heterogeneous regions where different water bodies with distinct spectral reflectance are present.

1. Introduction

Urban and suburban surface water bodies, including lakes, rivers, small reservoirs, fishponds, on-farm reservoirs, and paddy fields, are crucial natural resources. As such, it is important for water resource assessment and for agricultural and irrigation management to accurately monitor and map these surface water bodies to understand their spatial extent and temporal dynamics [1]. Monitoring large-scale water bodies has been greatly facilitated by the development of optical and synthetic aperture radar satellite remote sensing. Compared with synthetic aperture radar images, which are often noisy, optical remote sensing images, in which water and land are distinct in spectral reflectance, have been extensively utilized in mapping surface water at different scales [2]. For example, high-spatial-resolution images, such as from QuickBird and PlanetScope, have greatly facilitated the monitoring of small water bodies and rivers at resolutions finer than 10 m [2,3,4]. However, the high-spatial-resolution images have not been planned to cover the entire surface of the world in a regular time frame, and the commercial images are costly and have a narrow image span, which typically limits their use in mapping large-scale surface water bodies. Compared with high-spatial-resolution images, low-spatial-resolution images such as those from the moderate-resolution imaging spectroradiometer (MODIS), which has a spatial resolution of 250 m to 1 km, enable the mapping of large lakes at the global or continental scale at a daily frequency [5,6] but are limited to monitoring the spatial details of many small ponds and river nets. The medium-spatial-resolution Landsat series, with a history of approximately 50 years, has mapped global and regional water bodies, including rivers and reservoirs, at relatively medium spatial (typically 30 m) and temporal (approximately 16-day repetition rate) resolutions [7,8]. The medium-spatial-resolution Sentinel-2 satellite, launched in 2015, provides a multispectral high-spatial-resolution image at a typically 10 m spatial resolution, which enables the mapping of small water bodies, such as ponds and on-farm reservoirs, in heterogeneous regions [3,9,10,11].

Sentinel-2 image, which has 13 spectral bands, has been extensively used in many surface water mapping studies [9,12,13,14,15,16,17]. With the rich spectral information from Sentinel-2, studies have reported mapping water bodies on the basis of state-of-the-art machine learning methods, such as random forest [18,19] and the naïve Bayes classifier [20,21]. Spatial and spectral information from Sentinel-2 has been combined in an object-based analysis to map surface water bodies [22,23]. Moreover, with the development of deep learning techniques, the convolutional neural network has been utilized to extract deep semantic information about water bodies for Sentinel-2 surface water mapping [15,24,25,26]. The machine learning and deep learning methods require supervision and prior information about the water bodies [15,24,25,26,27]. The water index thresholding method is unsupervised and fully automatic, and has been extensively used because of its robustness and simplicity without inputting any prior information about the study area [14,28,29,30,31]. In particular, the water index method first extracts water indices, such as the normalized difference water index (NDWI) [32], modified NDWI (MNDWI) [33], automated water extraction index (AWEI) [34], 2015 Water Index (WI2015) [35], and Sentinel-2 water index (SWI) [16], and then utilizes thresholding algorithms to the water index image to produce the binary surface water map.

Although the aforementioned surface water mapping methods have mapped various water bodies from Sentinel-2 imagery, they all assign a Sentinel-2 pixel to either the water or land label and produce a binary water–land map. Therefore, these methods suffer from the mixed-pixel problem when both water and land classes contribute to the observed spectral response of the pixel. The mixed-pixel problem results in a large uncertainty in the accurate extraction of the water–land boundary [36,37,38,39]. To reduce the mixed-pixel problem in surface water mapping, spectral unmixing has been utilized to estimate surface water fractions (or area proportions) within each pixel [11,40]. Although the surface water fraction images contain sub-pixel water areal information relative to the pixel-based binary classifications, the spatial distributions of the surface water within the pixels are still unknown.

Sub-pixel mapping (SPM) is a method that predicts the land cover spatial distribution within mixed pixels [41,42]. SPM is a post-processing method applied to class fraction images and provides more information than spectral unmixing in land cover mapping. Various SPM methods have been proposed, including Markov random field (MRF) [43,44], Hopfield neural networks [45,46,47], pixel swapping algorithm (PSA) [42], spatial regularization [48], soft-then-hard model [49,50], attraction model [51,52], maximum a posteriori model [53], multi-objective optimization [44], spatial allocation [54], evolution algorithm [55], radial basis function interpolation [56], and deep learning [57,58]. Furthermore, SPM has been utilized in many fields, including mapping sub-pixel scale forests [59,60], trees [61], impervious surfaces [62], and surface water. For example, SPM has been used to map the spatial extent [36,38,63] and spatial–temporal dynamics of lakes [37]. The sub-pixel waterlines mapped from SPM have satisfied the mapping at 1:5000 standards using 20 m resolution imagery [64,65]. SPM has also been extensively used in mapping floods in floodplain regions [66], wetland regions [67,68], urban regions [69,70], and river basins [71,72,73]. In SPM for surface water mapping, the main focus is mapping sub-pixel surface water distribution within the water–land mixed pixels. Most SPM methods first detect the water–land mixed pixels from the image, which are then spectrally unmixed to water fractions. Finally, an SPM algorithm is utilized as a means to post-process the water fractions and map sub-pixel surface water within the mixed pixels [36,70]. Although SPM can greatly reduce the mixed-pixel problem in surface water mapping, several challenges still exist for SPM in heterogeneous regions.

First, the accuracy of SPM is directly dependent on the detection of the mixed pixels. If the mixed-pixel number is underestimated, the mixed pixel is incorrectly labeled as pure water or land pixels. Recent studies have used different thresholding segmentation methods to detect pure pixels in SPM. For example, Li et al. [36] applied a thresholding method to the histogram of the 250 m MODIS band to find pure water pixels. Liu et al. [70] applied OTSU thresholding segmentation to the NDWI image to extract pure water pixels. To the best of our knowledge, all of these methods apply only the morphological dilation operation to the water pixels in the binary water map to detect the mixed pixels. Dilation operation adds pixels to the boundaries of water pixels, which are assumed to be the water–land mixed pixels in the SPM model. However, the dilation operation fails to consider the existence of the potential mixed pixels within the water pixels that are segmented from the thresholding method. Moreover, considering that the detection of pure water pixels is difficult and has uncertainties and that the detected pure water pixels may be mixed in real scenarios in heterogeneous regions, it may be inappropriate to only use the morphological dilation operation in detecting mixed pixels for SPM.

Furthermore, SPM utilizes spectral unmixing for the mixed pixels, and the accuracy of the surface water fraction images is greatly dependent on the selected endmembers. Currently, sub-pixel surface water mapping has considered the variability of endmembers in unmixing the mixed pixels and uses local water (or land) endmembers within a local window around each target mixed pixel for spectral unmixing [36,70,74,75]. Although spectral unmixing is locally adaptive, recent studies often averaged the water spectrum and the land spectrum within the local window. The methods used in these studies are suitable for mapping large lakes where the water or land reflectance is homogeneous within the local window, but are unsuitable in heterogeneous regions where various water reflectances are encountered owing to factors such as different water contaminations and turbidities. The previous SPM methods, which averaged different water spectra and averaged different land spectra to produce local water and land endmembers in a local window, failed to consider the intra-class variability for water class and land class in spectral unmixing, and may have generated inaccurate water fraction images for SPM as a result.

Finally, various SPM algorithms can be applied to the surface water fraction images to produce the sub-pixel surface water map. The accuracy of the resulting surface water map is dependent on the algorithm used. Most studies often adopt SPM algorithms such as PSA, the attraction model, or the soft-then-hard model, which do not change the water fractions between the water fraction image (as the SPM input) and the sub-pixel surface water map (as the SPM output). In other words, the errors in the spectral unmixing are propagated into the resulting sub-pixel surface water map, resulting in lower accuracy and jagged land cover boundaries in the resulting map [76,77]. Therefore, the use of other SPM algorithms that can deal with spectral unmixing errors should be explored in Sentinel-2 surface water mapping.

This study proposes a new sub-pixel surface water mapping method using Sentinel-2 images that addresses the aforementioned challenges. In particular, the proposed method combines morphological dilation with morphological erosion, which removes pixels on water boundary pixels, to decrease the uncertainty in detecting the mixed water–land pixels, which are then unmixed on the basis of multiple water endmembers and land endmembers. In addition, the proposed method uses the MRF-based SPM algorithm to reduce the impact of water fraction image errors on the resulting surface water map. Unlike other sub-pixel surface water mapping methods that are applied to Landsat images, the proposed sub-pixel surface water mapping using dilation and erosion operations and the MRF (DE_MRF) is applied to a 10 m Sentinel-2 image to produce a 2 m surface water map. The proposed method was applied to the Jinshui Basin, China, which has a heterogeneous landscape of various water bodies, including lakes, rivers, paddy fields, and small reservoirs, near Wuhan, China. The proposed method was then assessed and compared with the state-of-the-art machine learning and deep learning water mapping methods [14,78,79] and several SPM algorithms [36,42].

2. Methods

DE_MRF predicts a 2 m resolution surface water map using Sentinel-2 multispectral images. First, the 20 m Sentinel-2 bands, including B5 (visible and near-infrared (VNIR) band), B6 (VNIR band), B7 (VNIR band), B8a (VNIR band), B11 (short-wave infrared (SWIR) band), and B12 (SWIR band), are downscaled to 10 m on the basis of the area-to-point regression kriging method, which has the advantage of preserving the spatial details when downscaling the low-spatial-resolution band to a high-spatial-resolution scale [80]. Then, the NDWI image is generated, and the edge-guided OTSU method is utilized to produce an initial 10 m binary surface water map. Both the morphological dilation and erosion operations are applied to the water pixels to generate the initial water pixels, land pixels, and mixed pixels. The water pixels and land pixels are then divided into water or land pixels of different sub-classes on the basis of k-means clustering. The mixed pixels are unmixed based on local water and land endmembers from multiple sub-classes (clusters) within a moving window. With the 10 m water fractions in the mixed pixels, an MRF-based SPM is used for producing the final 2 m surface water map. Figure 1 presents a flowchart of DE_MRF.

2.1. Sentinel-2 Mixed-Pixel Detection Using Morphological Operations

The DE_MRF produces an initial 10 m resolution binary surface water map from the Sentinel-2 image and then detects the mixed water–land pixels used for spectral unmixing and SPM. In this study, the water index thresholding method, which is fully automatic without any prior information about the study area, is used to produce the binary surface water map. The NDWI image calculated from the multispectral Sentinel-2 images is selected as the water index used for mapping surface water [81]. The main reason for selecting NDWI is that it is computed from the 10 m resolution green and VNIR bands from Sentinel-2, whereas most of the other water indices, such as MNDWI [33], AWEI [34], WI2015 [35], and SWI [16], require the Sentinel-2 SWIR bands at 20 m spatial resolution in which some spatial details of the water bodies may be distorted [30]. Although pansharpening algorithms can downscale the 20 m SWIR bands to 10 m in the computation of the water index, great uncertainty exists in the selection of the pansharpening algorithm and the selection of a pan-like band in pansharpening [14,70]. In addition, previous studies have suggested the effectiveness of using only finer-spatial-resolution bands instead of combining all of the spectral bands in surface water spectral unmixing and SPM. For example, the spectral unmixing [74] and SPM [36] studies adopted only the 250 m MODIS bands and discarded the 500 m MODIS bands in water–land segmentation to maintain spatial details of water bodies. Finally, the NDWI image has indicated its effectiveness in the SPM for water bodies from Landsat imagery [70]. Therefore, the NDWI image generated from the 10 m Sentinel-2 bands is adopted and segmented into an initial binary water map using the edge-guided OTSU method [82]. Compared with the traditional OTSU segmentation method that is applied to entire image pixels to search for the optimal threshold for distinguishing water/land, which may be inaccurate when the water pixels only account for a small proportion in the image, the edge-guided OTSU selects pixels from edge pixels, so the number of potential water and land pixels used for determining the optimal threshold is similar. With the optimal NDWI threshold determined by the edge-guided OTSU segmentation, the initial 10 m binary water body map is produced.

DE_MRF applies morphological operations to the binary water body map to determine the mixed pixels in the image. In the binary surface water map, the mixed pixels are assumed to be located near the border between the water pixels and land pixels. In contrast to previous methods that only use the dilation operation to determine the mixed pixels [36,70], the proposed method considers that the mixed pixel can be located in both the initial water pixels and land pixels at the water–land pixel borders and uses both dilation and erosion operations to determine the mixed pixels. Assume A is the binary water map and B is the structuring element (e.g., the 3 × 3 structuring element highlighted with gray squares in Figure 2). The morphological dilation operation, i.e., $A\oplus B$, is used for expanding the binary water map (A) using the structuring element (B), and the morphological erosion operation, i.e., A$\ominus B$, is used for shrinking the binary water map (A) using the structuring element (B). Therefore, dilation adds pixels to the boundaries of water objects, whereas erosion removes pixels on water object boundaries. Figure 2 presents a sketch map illustrating the morphological dilation and erosion operations using the 3 × 3 binary structuring element.

2.2. Mixed-Pixel Spectral Unmixing Using Local Multiple Endmembers

The multispectral Sentinel-2 image is spectrally unmixed using endmembers automatically estimated from the Sentinel-2 image. Considering that the water and land endmembers vary greatly in the image and that using global water and land endmembers in the image would result in a loss of local endmember variability information, the proposed method uses local endmembers for spectral unmixing of the mixed pixels. Thus, for a detected mixed pixel in the Sentinel-2 image, the local endmembers are selected using a local square window that uses the target mixed pixel as the window center. Specifically, for a detected mixed pixel, the local endmembers are selected on the basis of a 5 × 5–sized square local window with the target mixed pixel as the window center. The 5 × 5 local window may be relatively small and does not contain the refined pure water or pure land pixels within the window. In this case, the window size is enlarged iteratively (from 5 × 5 size to 7 × 7, 9 × 9, …, and the target mixed pixel has remained as the local square window center) until the window contains at least one refined pure water pixel and one refined pure land pixel. Then, the refined pure water and pure land pixels within this square window are used to produce the local water and local land endmembers for the target mixed pixel. The local window is applied to each mixed pixel to select the corresponding local endmembers for that mixed pixel.

Considering the heterogenic landscapes in urban and suburban regions, the local window may contain water pixels of various water bodies, such as different ponds, rivers, lakes, and reservoirs, and different land objects, such as impervious surfaces, soil, and vegetation, which have different spectral values (Figure 3). Previous SPM methods average the spectra from all pure water (or land) pixels within the local window to produce the local water (or land) endmember, whereas the proposed method considers the intra-class spectral variability within each local window. In particular, the spectral values of the refined pure water pixels are clustered into n_water clusters on the basis of the k-means algorithm, so the pure water pixels belonging to the same water cluster have similar spectral values (Figure 1). Similarly, the spectral values of the refined pure land pixels are clustered into n_land clusters on the basis of the k-means algorithm (Figure 1). Then, for each water or land cluster, the local endmembers are calculated by averaging the spectra from pixels belonging to this cluster within the local window. If the local window does not contain the pixels of the cth cluster, then the endmember of the cth cluster is not considered in unmixing this target mixed pixel within the local window.

A sketch map of selecting local pure water pixels from different ponds in a paddy field region is portrayed in Figure 3. The target mixed pixel, which can be detected by applying the morphological operations to the binary surface water map as presented in Figure 2, is highlighted in the red rectangle, and the blue rectangles depict the pure water pixels within the 5 × 5–sized local window with the target mixed pixel as the window center. The target mixed pixel is composed of water from three different ponds in which the water has different spectral reflectances and is thus represented with different colors. The 5 × 5–sized local window contains eight pure water pixels, i.e., P_1,1, P_1,2, P_2,1, P_2,2, P_3,4, P_4,4, P_5,4, and P_5,2 (the subscript number indicates the row and column number in the local window₎, highlighted with blue rectangles in Figure 3c. Pixels P_1,1, P_1,2, P_2,1, and P_2,2 belong to pond 1; pixels P_3,4, P_4,4, and P_5,4 belong to pond 2; and P_5,2 belongs to pond 3 in Figure 3a. The pixels in the same pond are similar in optical presence, and the pixels in different ponds are dissimilar. Traditional SPM methods average the pure water spectra of the pure water pixels from all three ponds, i.e., {P_1,1 ∪ P_1,2 ∪ P_2,1 ∪ P_2,2 ∪ P_3,4 ∪ P_4,4 ∪ P_5,4 ∪ P_5,2}, as one water endmember in the unmixing without considering the difference in water pixels (Figure 3d). By contrast, the proposed DE_MRF could consider the water spectra from different ponds to incorporate intra-class spectral variability of water and may group different water pixels into different subsets in spectral unmixing (Figure 3d). Therefore, the proposed SPM that uses different subsets of local pixels in endmember estimation and spectral unmixing is expected to generate a more accurate surface water fraction for the mixed pixel.

With the selected local water and local land endmembers, DE_MRF applies the fully constrained least squares linear spectral mixture (FCLS) to the target mixed Sentinel-2 pixel to generate water fractions within this pixel. With the water fractions predicted by FCLS in the mixed pixels and by assigning the water fractions of 100% for the pure water pixels and 0% for the pure land pixels, the 10 m surface water fraction image can be generated (Figure 1).

2.3. Mixed-Pixel Sub-Pixel Mapping

The MRF-based SPM of surface water is utilized to map the water bodies at the sub-pixel scale within mixed pixels. The MRF-SPM is formulated by utilizing the maximum a posteriori rule in the Bayesian framework. The optimal surface water map at the sub-pixel scale is solved by maximizing the posterior probability of the sub-pixel map given the surface water fractions, and it selects the most likely sub-pixel map among all the possible maps given the surface water fraction image. Assuming the resulting sub-pixel map has the Markov random field property, the model assumes that neighboring pixels more possibly belong to the same land cover class than different classes. The MRF-SPM was first proposed by Kasetkasem et al. [44] and has been extended in many fields [36,41,62]. DE_MRF focuses on surface water mapping and adopts the water fraction images as the input, whereas the classic MRF-SPM in [44] focuses on multiple land covers and adopts the multispectral remote sensing images as the input. In particular, the MRF-SPM framework can be demonstrated in Equation (1):

$$X=\mathrm{argmax}\left[Pr\left(X|Y\right)\right]=\mathrm{argmax}\left[Pr\left(Y|X\right)Pr\left(X\right)\right]$$

(1)

where X is the high-spatial-resolution surface water map, Y is the surface water fraction image, $Pr\left(X|Y\right)$ is the posterior probability of X given Y, $Pr\left(Y|X\right)$ is the probability of Y given X, and $Pr\left(X\right)$ is the marginal probability density function of X. The details of the MRF-SPM model and parameter setting are introduced in the Supplementary Materials.

Before SPM, the 10 m surface water fraction image is downscaled to a 2 m resolution surface water map with a scale factor s = 5. In this step, if the surface water fraction is f% in a target Sentinel-2 pixel, a total of s × s × f% pixels at 2 m spatial resolution are assigned to water, and s × s × (100% − f%) pixels at 2 m spatial resolution are assigned to land. Then, the surface water pixels are randomly allocated within this Sentinel-2 pixel. Sub-pixels in the pure water pixels and pure land pixels are directly labeled as water and land, respectively. As this study focuses on the sub-pixel mapping of water–land mixed pixels, MRF-SPM was applied only to the detected mixed pixels. Moreover, the simulated annealing (SA) mode is used to update the sub-pixel labels within the range defined by the mixed pixels in the initial 2 m surface water map to produce the final surface water map. The SA ends if less than 0.1% of pixels change between two iterations. Note that the proposed method is only applied to the mixed 10 m Sentinel-2 pixels, and the labels of water and land pixels in the pure water pixels and pure land pixels are unchanged in each iteration.

3. Experiments

3.1. Study Area and Data

Jinshui Basin, located at the border of Wuhan and Xianning City, China, was selected as the study area. This region covers an area of approximately 2616 km² and has an elevation of approximately 17–25 m (Figure 4). This basin has several lakes, including the three main lakes of Futouhu Lake, Xiliang Lake, and Luhu Lake, and is connected to the Yangtze River. The basin also contains several wetlands and plays an important role in fishing, irrigation, rainwater storage, and shipping.

A Sentinel-2 image acquired on 19 January 2021, was used as the model input (Figure 4). In this study, only the 10 m and 20 m Sentinel-2 bands that are sensitive to surface water were used. The 10 m green and VNIR bands were used to calculate the NDWI image in producing the initial surface water map, and all of the 10 Sentinel-2 bands were used for local endmember extraction in the spectral unmixing.

3.2. Comparison Methods

DE_MRF was compared with several SPM methods and several state-of-the-art spectral indexes–based, machine learning–based, and deep learning–based surface water mapping methods. DE_MRF has two main steps. First, DE_MRF uses morphological dilation and erosion operations to determine the locations of mixed pixels and uses multiple local endmembers for unmixing the mixed pixels to generate the surface water fraction image. Second, with the surface water fraction image, DE_MRF uses the MRF-based SPM to produce the 2 m resolution surface water map. To assess the effectiveness of the first step, DE_MRF was compared with previously proposed methods that use only the dilation operation in mixed-pixel detection and use the average value of the local water endmembers and local land endmembers to produce the initial surface water fraction map via spectral unmixing. To assess the effectiveness of the second step, MRF-based SPM was compared with the popular PSA. The details of the comparison methods are presented in

Table 1. Explanation of different sub-pixel mapping methods.

Name	Processes in Spectral Unmixing and SPM
	Spectral Unmixing for the Mixed Pixels	SPM to Generate the Sub-Pixel Surface Water Map
D_PSA	Using only dilation operations in mixed-pixel detection and averaging local water and local land endmembers in the unmixing	PSA
D_MRF	Using only dilation operations in mixed-pixel detection and averaging local water and local land endmembers in the unmixing	MRF
DE_PSA	Using dilation and erosion operations in mixed-pixel detection and using multiple local endmembers in the unmixing	PSA
DE_MRF (proposed)	Using dilation and erosion operations in mixed-pixel detection and using multiple local endmembers in the unmixing	MRF

. Note that all the SPM methods were applied only to the detected mixed pixels.

Furthermore, several state-of-the-art surface water mapping methods were also compared. The methods included the spectral index–based method based on thresholding methods applied to the water index. The first comparator is the edge-guided OTSU thresholding method applied to the NDWI image, that is, NDWI_OTSU [11,83]; the second comparator is the edge-guided OTSU thresholding method applied to the MNDWI image, that is, MNDWI_OTSU [14,33]. The machine learning method support vector machine (SVM), which has been extensively used in image classification and surface water mapping [84,85], was compared, and the deep learning method UNet [78], which has a simple network structure and lightweight parameters for image segmentation and classification, was also compared. SVM and UNet were applied to the 10 m Sentinel-2 multispectral image. The NDWI_OTSU, MNDWI_OTSU, SVM, and UNet all produced a pixel scale 10 m surface water map that was used for comparison.

3.3. Model Parameter and Accuracy Assessment

The model parameters of different methods were set as follows. In DE_MRF, the numbers of water and land clusters for spectral unmixing were set to 4, so that the number of endmembers in spectral unmixing was in the range of 2–8. The reason for this is that the fully constrained least squares linear spectral mixture was used, which required the number of endmembers to be no more than the number of spectral bands (10 bands in this study). Furthermore, for all SPM algorithms, the window size for the calculation of sub-pixel spatial dependence was set to 5 [86]. For the SVM classifier, the water, vegetation, impervious surface, and soil endmembers were directly extracted from the image to avoid the impact of atmosphere circumstances, and the radial basis function was selected as the kernel type used in SVM. In the UNet model, more than 2000 Sentinel-2 multispectral image patches and the corresponding label image with water and land in it, with 256 × 256 pixels in each patch, were used for training. The training images were acquired on similar dates to the images used in this study to reduce impacts such as crop phenology and satellite observation condition. Data augmentation operations such as image rotation were applied to the training image. The graphics processing unit (GPU) is NVIDIA 2060, which has 6 GB of RAM and uses cuDNN 10.0 for acceleration to run the UNet model.

All the SPM algorithms were assessed on the basis of GF-1 and GF-6 images acquired on 18 January 2021 (GF-6), 14 January 2021 (GF-1), and 11 January 2021 (GF-1), which were close to the Sentinel-2 image (19 January 2021) to avoid the impact of land cover change in accuracy assessment. The GF-1 and GF-6 images had multispectral bands at a resolution of 8 m and panchromatic bands at a resolution of 2 m. The 8 m multispectral bands were pansharpened to 2 m resolution. The GF-1 and GF-6 images were co-registered to the Sentinel-2 image and mosaicked, as presented in Figure 5. A total of 6000 sample points were adopted for assessing the accuracy of different methods. The 6000 sample points, including 3000 water pixels and 3000 land pixels, were randomly selected in the study area highlighted with yellow points in Figure 5. Many water pixels were distributed in the homogeneous land cover regions of large lakes. Different SPM algorithms were only applied to the mixed pixels determined by the morphological operations. Another 8000 sample points, including 4000 water pixels and 4000 land pixels that were selected within the mixed pixels determined by the morphological dilation and erosion operations to the NDWI image, were selected to further assess the accuracy of different SPM algorithms. The 8000 sample points were distributed in heterogeneous land cover regions highlighted with green points in Figure 5. All of the sample points were visually interpreted by experts with extensive experience in remote sensing and surface water analysis using the 2 m GF-1 and GF-6 images in Figure 5. The overall accuracy, omission error, and commission error were used for accuracy assessment. The critical success index was also used to quantify the accuracy of surface water [87]. The use of better input data (such as images with finer resolution) and the use of more advanced interpretation methods (such as visual interpretation by using expert knowledge) are effective to quantify the land cover or surface water map [37,88,89,90,91]. As the sample data were interpreted from the 2 m GF-1 and GF-6 images, the accuracy of the pixel scale classification map outputted from NDWI_OTSU, MNDWI_OTSU, SVM, and UNet at the 10 m resolution was not assessed using these 8000 samples.

4. Results

4.1. Visual Comparison of the Results

The resulting surface water maps from different algorithms are presented in Figure 6. As shown in Figure 6c, SVM generated more water bodies than other methods. Zoomed-in areas A–D in Figure 6 are presented in Figure 7. Zoomed-in area A is located at the border between Futouhu Lake and the land. As illustrated in Figure 7, NDWI_OTSU, MNDWI_OTSU, SVM, and UNet generated jagged water–land boundaries because the map was generated at the 10 m resolution. SVM incorrectly predicted the linear bridge as surface water, as highlighted with a red ellipse in zoomed-in area D in Figure 6. The main reason is that SVM is a pixel-based classifier based on pixel spectral information, whereas the water and the bridge have similar spectral values in the Sentinel-2 image.

The sub-pixel maps of D_PSA and DE_PSA contain many jagged boundary pixels. This is primarily because the PSA used for SPM does not change the surface water fractions between the input unmixed fraction image and the output sub-pixel surface water map, and the fraction image error is preserved in the resulting map. For example, if the water fraction in the unmixed fraction image is 20% higher than the actual water fraction in a Sentinel-2 pixel, then at least s² × 20% = 5 land pixels will be incorrectly labeled as a water pixel in this Sentinel-2 pixel, which may result in jagged water patches. In contrast to PSA, the MRF-based SPM algorithm does not necessarily preserve the water fraction from the unmixed fraction images in the final map. As illustrated in Figure 7, D_MRF and DE_MRF generated smoothed water–land boundaries in zoomed-in area A. The differences between the PSA and MRF maps are more evident in the zoomed-in area figure in Figure 8, which is enlarged area E from Figure 6. NDWI_OTSU, MNDWI_OTSU, SVM, and UNet generated jagged boundaries because of the relatively coarse spatial resolution (10 m) of the Sentinel-2 image. Although the four SPM methods were predicted at the sub-pixel scale with a 2 m resolution, D_PSA and DE_PSA both generated jagged boundaries because they strictly preserved the class fractions from the spectral unmixing into the resultant sub-pixel surface water map. By contrast, both D_MRF and DE_MRF, which smoothed the class boundary according to the MRF property and did not strictly preserve the unmixed class fractions to the result, generated more smoothed water–land boundary than D_PSA and DE_PSA. This finding indicates that D_PSA and DE_PSA are sensitive to the error that is propagated from spectral unmixing, whereas D_MRF and DE_MRF are more robust to the spectral unmixing error.

In Figure 7, zoomed-in areas B and C are located in paddy fields in the Jinshui Basin. It is clear that many paddy fields and the channels are not spatially connected and that the spectral reflectances of many water bodies are different. The pixel scale maps outputted from NDWI_OTSU, MNDWI_OTSU, SVM, UNet, and the sub-pixel D_PSA and DE_PSA maps contain jagged water–land boundaries. The D_MRF and DE_MRF maps contain water bodies with more smoothed boundaries. In D_MRF, many paddy fields were mapped to be spatially connected, such as those highlighted with red circles in zoomed-in areas B and C, whereas the water bodies were better mapped in the DE_MRF results. This result reveals that using both morphological dilation and erosion operations in the MRF-based SPM could reduce the over-smoothing of water pixels. Similar results were obtained for river water bodies in zoomed-in area D in Figure 7. D_MRF over-smoothed the water bodies, which are highlighted with the red circle in zoomed-in area D, whereas DE_MRF could map the small bridge over the river.

4.2. Quantitative Comparison of the Results

Table 2. Quantitative assessment results of different methods using the 6000 sample points that are randomly selected in the image as highlighted with yellow points in Figure 5. The highest overall accuracy, highest critical success index, lowest omission, and lowest commission errors are in bold.

		NDWI_OTSU	MNDWI_OTSU	SVM	UNet	D_PSA	D_MRF	DE_PSA	DE_MRF
Overall Accuracy (%)		99.0167	98.2167	98.4000	99.6167	99.5333	99.6000	99.6333	99.7167
Critical success index (Water)		0.9803	0.9644	0.9688	0.9923	0.9907	0.9920	0.9927	0.9943
Omission Error (%)	Water	1.9333	3.2667	0.5000	0.7000	0.3667	0.4000	0.2667	0.3667
	Land	0.0333	0.3000	2.7000	0.0667	0.5667	0.4000	0.4667	0.2000
Commission Error (%)	Water	0.0340	0.3092	2.6419	0.0671	0.5655	0.4000	0.4657	0.2003
	Land	1.8973	3.1725	0.5112	0.6956	0.3674	0.4000	0.2672	0.3661

presents the quantitative assessment results of different methods using the 6000 sample points that are randomly selected in the image as highlighted with yellow points in Figure 5. UNet generated higher overall accuracy and critical success index value than the other three hard classifications of NMWI_OTSU, MNDWI_OTSU, and SVM, showing the advantage of the UNet in extracting the deep semantic information for water bodies. Among all of the methods, the proposed DE_MRF generated the highest overall accuracy and the highest critical success index value for the water class. All the methods generated an overall accuracy value higher than 98% and a critical success index value higher than 0.96. The main reason for the high water mapping accuracy is mainly because many water samples were distributed in large lakes with a homogeneous land cover landscape, and the accuracies were assessed for binary maps with only water and land in them. It should be noted that, although UNet generated similar accuracy as the SPM algorithms, it cannot predict sub-pixel water distribution and, therefore, outputted a jagged water-land boundary, as shown in Figure 8.

Table 3. Quantitative assessment results of different SPM methods in the mixed pixels using the 8000 sample points that are selected near the water–land boundaries highlighted with green points in Figure 5. The highest overall accuracy, highest critical success index, lowest omission, and lowest commission errors are in bold.

		D_PSA	D_MRF	DE_PSA	DE_MRF
Overall Accuracy (%)		79.1625	80.2000	83.9375	84.4125
Critical success index (Water)		0.6907	0.7023	0.7361	0.7361
Omission Error (%)	Water	7.0000	6.6000	10.4750	13.0500
	Land	34.6750	33.0000	21.6500	18.1250
Commission Error (%)	Water	27.1588	26.1076	19.4738	17.2496
	Land	9.6473	8.9674	11.7680	13.7477

presents the quantitative assessment results of different SPM methods in the mixed pixels using the 8000 sample points that are selected near the water–land boundaries highlighted with green points in Figure 5. In general, all of the SPM methods have lower accuracy in Table 3 than in Table 2, because the samples used for validation in Table 3 were mostly distributed in heterogeneous land cover regions. The overall accuracy for the DE_PSA is higher than that of D_PSA, and the overall accuracy for DE_MRF is higher than that of D_MRF. This finding indicates that using a combination of morphological dilation and erosion operations can improve the overall accuracy more than using only the dilation operation in surface water mapping. D_PSA and D_MRF generated high omission errors in land and high commission errors in water, as presented in Table 2. This is because D_PSA and D_MRF only use the morphological dilation operation, and the mixed pixels, such as those highlighted in light green in Figure 2d, are all predicted as pure water pixels, resulting in an overestimation of water pixels and an underestimation of land pixels. In contrast, DE_PSA and DE_MRF profoundly reduced the omission error rate of land and the commission error rate of water. D_MRF generated the lowest omission error for water and the lowest commission error for land, whereas the proposed DE_MRF generated the lowest omission error for land and the lowest commission error rate for water. Both the omission and commission error rates of water and land classes are lower than 20% in the detected mixed pixels for the proposed DE_MRF, indicating the proposed method’s superior ability to map sub-pixel surface water bodies. Furthermore, both DE_PSA and DE_MRF generated the highest critical success index, and DE_MRF generated the highest overall accuracy among the comparators.

5. Discussion

To assess the proposed method for different water bodies in different parts of the world, the DE_MRF was also used in mapping sub-pixel surface water in the Dangara region of Guinea, which borders the Atlantic Ocean to the west (Figure 9), and the Terai region in Nepal in South Asia (Figure 10). The acquisition dates of the Sentinel-2 image are 21 April 2022 for the Dangara region in Guinea, and 11 November 2020 for the Terai region in Nepal. The Sentinel-2 images were processed using the same method that was adopted in the Jinshui Basin region. DE_MRF was performed in the two regions, and the produced 2 m resolution surface water maps were overlapped with the Sentinel-2 imagery as shown in Figure 9 and Figure 10. In the zoomed-in areas, the 10 m Sentinel-2 image is relatively coarse to represent the water–land boundary in Figure 9c and Figure 10c. In contrast, DE_MRF produced a 2 m resolution surface water map, and the waterlines were smoothed according to the sub-pixel mapping result in Figure 9d and Figure 10d. Considering the ability to map surface water within the Sentinel-2 pixel, the proposed DE_MRF has great potential in mapping surface waterbodies at a finer spatial resolution than the input multispectral Sentinel-2 imagery.

This study proposed a surface water mapping model to predict a 2 m surface water map from Sentinel-2 images on the basis of combined morphological dilation and erosion operations and an MRF-based SPM algorithm. The results suggested that DE_MRF improves surface water mapping when compared with previously proposed methods. Compared with the pixel-based classification that produces a 10 m resolution binary surface water map from Sentinel-2 and thus suffers from the mixed-pixel problem, the proposed method can map a 2 m resolution surface water map from Sentinel-2 to reduce the impact of the mixed pixel to a great extent. Compared with the current SPM algorithms, the improvements result from the combination of morphological dilation and erosion operations in detecting the mixed pixels, and the use of local endmembers and MRF-based SPM in determining the sub-pixel water body distribution within the mixed pixel. The advantage of the proposed DE_MRF compared with other SPM methods is evident both visually and quantitatively. The potential impact factors for the proposed method are discussed as follows.

In the proposed method, spectral unmixing and SPM are applied to the mixed pixels detected by applying the morphological operations to the initial surface water map. Therefore, the error in generating the initial surface water map may be propagated in the following mixed-pixel detection, spectral unmixing, and SPM. In this study, the initial surface water map is generated by applying edge-guided OTSU segmentation to the NDWI image. The reason for using NDWI in this study is because NDWI is calculated from the 10 m green and VNIR bands in the Sentinel-2 image. If the study area is located in a region where there are many shadows from buildings, it is helpful to incorporate other water indices, such as MNDWI and AWEI, to further improve the water segmentation in shadow and dark surface regions. In addition, the edge-guided OTSU is applied to the single NDWI water index in this study. Previous studies have indicated that using multiple water indices may produce a more accurate surface water map compared to using a single water index from Landsat images (i.e., different water indices are calculated from bands of the same 30 m spatial resolution) [92,93]. Future studies could focus on using multiple water indices calculated from Sentinel-2 bands with different spatial resolutions in producing the initial surface water map. Finally, the water index thresholding approach is adopted in producing the initial surface water map for its automation without using any prior information and simplicity. If prior knowledge about the endmembers or water label training samples is available, supervised methods such as random forest and deep learning can be adopted to enhance the accuracy in mapping the binary surface water map.

On the basis of the detected mixed pixels according to the water index thresholding method and the morphological operations, the proposed DE_MRF applied optimization algorithms in the spectral unmixing and SPM to the mixed pixels. In particular, the FCLS is used in spectral unmixing, and the SA is used in the SPM. FCLS and SA, which can predict sub-pixel information accurately, are used in this study but are relatively time-consuming. Further research could focus on using fast algorithms, such as partial unmixing algorithms [40] in spectral unmixing, and iterated conditional modes [94] in the SPM, to accelerate the process of the model in large-scale surface water mapping.

The proposed DE_MRF is mainly used to map wall-to-wall water bodies at a finer spatial resolution than the input Sentinel-2 imagery. Considering that the resultant surface water maps have more spatial details than those extracted at the pixel scale, it is promising to further extract water body information from the resulting sub-pixel water map. For example, it is favorable to extract sub-pixel river net to further calculate river width at the pixel scale [73], and it is helpful to extract the small ponds by using SPM to reduce the mixed-pixel problem in current limnology studies [3,11,40].

6. Conclusions

Sub-pixel mapping is effective in reducing the impact of the mixed-pixel problem in mapping surface water at the sub-pixel scale. This study proposed DE_MRF, a sub-pixel surface water mapping method, to map a 2 m resolution sub-pixel surface water map from Sentinel-2 images in the Jinshui Basin, China. Moreover, DE_MRF was compared with several SPM methods; the results suggested that DE_MRF generated the lowest commission error rate for water and the lowest omission error rate for land, and, therefore, had the highest overall accuracy.

Several findings were identified through the experimental results. The sub-pixel surface water mapping that applies only the morphological dilation operation to detect Sentinel-2 water pixels could result in a large commission error rate for the water class because many of the detected water pixels may be mixed. Incorporating the morphological erosion operation enabled the selection of more potential mixed pixels at the water–land boundaries used for spectral unmixing and SPM. Experimental results suggested that DE_MRF using both dilation and erosion operations generated the lowest commission error rate for water pixels. Moreover, compared with previously proposed methods that simply averaged the water (or land) pixel spectra when selecting local endmembers in the local window, the proposed method considered the intra-class spectral variability and was more suitable for mapping different water bodies that were distinctive in spectral reflectance in heterogeneous regions. Finally, the use of the MRF-based SPM algorithm could smooth the jagged boundaries that were present in maps from the PSA-based SPM algorithms and could reduce the propagation of spectral unmixing error in the final sub-pixel surface water map to a great extent. In summary, DE_MRF was found to be a simple and effective method to map sub-pixel surface water from Sentinel-2 imagery, which helps to monitor surface water resources in heterogeneous regions.