Mapping Rural Settlements from Landsat and Sentinel Time Series by Integrating Pixel- and Object-Based Methods

Abstract

Rural settlements account for 45% of the world’s population and are targeted places for poverty eradication. However, compared to urban footprints, the distribution of rural settlements is not well characterized in most existing land use and land cover maps because of their patchy and scattered organization and relative stability over time. In this study, we proposed a pixel- and object-based method to map rural settlements by employing spectral-texture-temporal information from Landsat and Sentinel time series. Spectral indices (maximum normalized difference vegetation index (NDVI) and minimum normalized difference built-up index (NDBI composite) and texture indices (vertical transmit and vertical receive (VV) polarization of mean synthetic aperture radar (SAR) composite) were calculated from all available Landsat and Sentinel-1A data from 1 January 2016 to 31 December 2018. These features were then stacked for segmentation to extract potential rural settlement objects. To better differentiate settlements from bare soil, the gradient of annual NDVI maximum (namely, gradient of change, use gradient for simplicity) from 1 January 1987 to 31 December 2018 was used. The rural training samples were selected from global urban footprint (GUF) products with a post filtering process to remove sample noise. Scatter plots between pixel- and object-based values per feature were delineated by t-distribution ellipses to determine the thresholds. Finally, pixel- and object-based thresholds were applied to four features (NDVI, NDBI, VV, gradient) in Google Earth Engine (GEE) to obtain the distribution of rural settlements in eight selected Asian regions. The derived maps of rural settlements showed consistent accuracy, with a producer’s accuracy (PA) of 0.87, user’s accuracy (UA) of 0.93 and overall accuracy (OA) reaching 90% in different landscape conditions, which are better than existing land cover products.

1. Introduction

We are living on an urbanizing planet, and the global population is now more in urban than rural [1]. Nonetheless, 45% of the world population is still living in rural areas, especially in certain countries where more than 60% of their population remains rural, such as India and some African countries [2]. Compared with urban areas, rural settlements (e.g., rural buildings and roads) have gained less attention due to their relatively smaller spatial coverage and fragmented organization. However, accurate information on rural settlement distribution is critical to the healthy development of rural undertakings and to help address the issues related to rural poverty, food insecurity, and the depletion of natural resources issues caused by environmental degradation and climate change [3,4,5].

The spatial distribution of rural settlements has usually been mapped as a part of urban areas defined by impervious surfaces or built-up areas [6,7,8]. Accordingly, maps specific for rural settlements are rarely available. Until now, numerous efforts have been made to map regional/global urban extents at different spatial resolutions (i.e., 12-m to kilometers) from remotely sensed data. Some coarse resolution urban maps include: DMSP/OLS-derived map [9], MODIS-based map [10], and Rural-Urban Mapping Project (GRUMP, [11]). However, these maps have not provided good characterizations of rural settlements because of their coarse spatial details. More recently, urban extent has been mapped in a more precise manner in terms of spatial resolution and temporal frequency, such as the 12-m resolution global urban footprint (GUF) circa 2014 [12], 30-m resolution 5-year interval Landsat-derived maps [4], 38-m resolution decadal urban extent back to 1975, namely, the global human settlement layer (GHSL, [13]), and 30-m annual maps between 1985–2018 [14]. In addition, some regional medium resolution urban maps were generated as exemplified by annual urban maps for the conterminous United States [15] and the single-year map of China [16]. Moreover, some regional and global land cover maps have classes related to human settlements such as GlobeLand30 [6], GLC30 [7], and USGS NLCD [17]. Nonetheless, most of these products suffer from a lower locational accuracy of rural settlements compared to urban [18].

There are several factors that lead to the limited mapping accuracy of rural settlements in existing urban maps. First, rural settlements (e.g., buildings and roads) are usually sparsely distributed compared to urban infrastructure. This fragmented distribution, combined with surrounding land covers (e.g., active and abandoned cropland, bare ground, forest), leads to mixed pixels in median to coarse resolution satellite images, making rural infrastructure difficult to map. Second, currently available remote sensing-based methods focus more on urban settlements instead of their rural counterparts. To calibrate classifiers that are suitable to extract urban extent, training samples for impervious surfaces are collected more from urban and lose most of the information of rural ones [16]. Therefore, the rural settlements were poorly delineated using methods designed specifically for urban land only. Third, the socioeconomic characteristics of rural settlements add to their difficulties in accurate mapping. For instance, rural settlements usually have low population density and weak economic activities compared to cities, which leads to their invisibility in nighttime light (NTL) images.

Many efforts have been made to extract human settlements (including both urban and rural areas) using supervised classification (pixel or object-based method) that heavily relies on the availability of training samples to create reliable classifier [4,19,20]. Pixel-based methods are most frequently used in mapping local to global human settlements due to their straightforward methodology and ease of transferability to other regions [14,19,20]. Object-based classification (OBC) considers each parcel as one unit which could provide a homogeneously classified result [21]. This approach delineates the boundary of land cover as a patch as opposed to a single isolated pixel [22]. There are some object-based or pixel- and object-based integrated techniques used for regional and large-scale land cover mapping [6,23,24]. Among them, GlobeLand30 products have achieved the highest average accuracy from a third-party evaluation [25]. However, the accuracy of rural human settlements in GlobeLand30 is not high enough because it first focuses on using pixel-based classification and then labels the results by the OBC method, which leads to a good delineation of settlements but loss of spatial details within land cover patches [6]. Other pixel-based products, such as GHSL and GUF, can provide spatial details of rural settlements but not good delineations of boundaries. Hence, the combined use of pixel- and OBC methods is a promising approach in mapping rural settlements. For example, extracted results from OBC can be used to incorporate other noisy pixels, such as bare land and vegetation pixels, within rural settlement patches. The pixel-based method generally provides scattered results with many spatial details. Therefore, the misclassification between rural and other land covers within each patch could be reduced by the pixel-based method.

In this study, we proposed an easily implemented approach to map rural settlements (human settlements out of urban regions) with diverse spatial organizations. The approach is applicable to areas under different socioeconomic and biophysical situations.

2. Study Area and Datasets

2.1. Study Area

In this study, we aimed to map the spatial distribution of rural settlements across Asia circa 2017. The study covered eight regions with diverse rural characteristics across varied climate zones (Figure 1), including temperate continental climates (plots 1, 3, 6, and 8), arid areas (plots 2 and 5), and subtropical rainforest (plot 4 and 7). Rural settlements present linear, irregular polygons and patchy organizations which are mixed with the background land covers, such as cropland, bare soil, forest, etc. Similar to urban areas, rural settlements also present high, medium and low-density characteristics.

2.2. Datasets

All the available Landsat surface reflectance products (with cloud masked) in the time range 1987–2018, including Thematic Mapper (TM4, TM5), Enhanced Thematic Mapper Plus (ETM+), Operational Land Imager (OLI), and Sentinel-1A SAR data, were used in this study. Six spectral bands of Landsat data (blue, green, red, near infrared (NIR), shortwave infrared 1 (SWIR1) and shortwave infrared 2 (SWIR2)) were utilized to generate spectral and temporal features. The texture features were derived from the backscattering coefficient of Sentinel-1A SAR data from 2016 to 2018. The process of preprocessing SAR, including thermal noise removal, radiometric calibration and terrain correction, was conducted using the Sentinel-1 Toolbox with additional conversations of decibels using log scaling (https://developers.google.com/earth-engine/sentinel1 (accessed on 20 January 2021)). Here, the VV polarization of Sentinel-1A was chosen to guarantee the complete coverage of our research region. Furthermore, the Visible Infrared Imaging Radiometer Suite (VIIRS) nighttime light (NTL) image was used to generate urban masks. The number of datasets used in this study is displayed in Figure 2.

The ancillary land cover products, Global Food Security Analysis-Support Data at 30 Meters (GFSAD30) and the 12-m GUF were collected as reference maps to generate samples of crop and rural settlements respectively. GFSAD30 was generated by using multisource satellite data (primarily with Landsat) for 2015 [26], and the GUF was produced by the German Aerospace Center (DLR) for 2010 using TanDEM-X and TerraSAR-X datasets with an overall accuracy of 85% [12].

3. Methods

Our method consists of 5 steps as displayed in Figure 3: (1) constructing temporal features: maximum NDVI composite (NDVI_max), NDBI minimum composites (NDBI_min), VV polarization of SAR mean composite (VV_mean), and NDVI gradient; (2) generating objects and calculating object-level features; (3) obtaining samples for different land covers; (4) calibrating thresholds and applying them to object- and pixel-based features; and (5) assessing accuracy.

3.1. Creating Feature Composites from Landsat and Sentinel Time Series

Annual spectral NDVI_max, NDBI_min, and texture composite VV_mean during 1 January 2016–31 December 2018 were composed using NDVI, NDBI and VV time series respectively. NDVI_max highlights the contrast between vegetated areas (cropland and forest) and others (i.e., bare soil and rural settlements). NDBI_min was used to discriminate against most rural settlements with other land cover classes by highlighting mid-infrared features. VV_mean was adopted to separate land cover with rough surfaces from covers with flat surfaces [16]. The NDVI and NDBI metrics are shown in Equations (1) and (2), and the composites process are shown in Equations (3)–(5).

$$NDVI=\left(NIR-Red\right)/\left(NIR+Red\right)$$

(1)

$$NDBI=\left(MIR-NIR\right)/\left(MIR+NIR\right)$$

(2)

$$NDVI=\left(NDVI\right)\frac{j}{i}$$

(3)

$$NDBI=\left(NDBI\right)\frac{j}{i}$$

(4)

$$VV=\left(VV\right)\frac{j}{i}$$

(5)

For Landsat 4, 5, and 7, the MIR and red bands correspond to Band 5 and Band 3. For Landsat 8, the MIR and red bands correspond to Band 6 and Band 4, respectively, and i and j represent the time range.

Due to the spectral similarity between bare soil and settlements, settlement products are easily mixed with bare soil information [8]. Therefore, we introduced a temporal-based feature to alleviate the misclassification. We assumed that the rural settlements remained relatively constant over a long period. Conversely, bare soil would experience fluctuations due to human activity or natural factors. Hence, we used the slope (Figure 4) of the linear fitting model that was applied to annual NDVI_max, namely, the gradient of change (using gradient for simplicity), as the indicator to capture the magnitude of change.

3.2. Generating Objects and Calculating Object-Level Features

Image segmentation is a process that separates input images into homogeneous patches, of which pixels within each patch have the same property but are different from surrounding neighborhoods [21]. The segmented objects reduce within-class variations in land covers. To date, a variety of segmentation methods have been developed such as the edge boundary method, the watershed method, the graph cuts method, the seed region growing (SRG) method or their hybrid approaches [27]. Among them, SRG is the most widely used due to its simplicity and efficiency. A complete SRG segmentation procedure commonly comprises three steps: (1) the selection of seeds, (2) the application of the SRG method and (3) the merging of segmented patches. To increase the computing efficiency of SRG methods, the selection of seeds is one of the key steps. Here, we adopted the method proposed by [27], namely, the directional gradient minima-based method (DGM), which aims to extract the seed location using the regional minima from horizontal and vertical gradient maps (Figure 5a). These selected seeds were further used for the segmentation process (Figure 5b). Then, NDVI_max, NDBI_min and VV_mean were stacked together to perform initial segmentation.

After segmentation, the pixel-based NDVI_max feature was aggregated to OBC based features by calculating the median value within each patch. The same procedure was applied to NDBI_min, VV_mean and gradient_mean, thus, four new object-based features, namely, NDVI_max_med, NDBI_min_med, VV_mean_med and gradient_med were finally generated.

3.3. Automatic Generation of Samples for Different Land Cover Class

Rural settlements, crops, forests, bare soil and water bodies are five land cover types in our research region. Since the water body could be easily masked out using the modification of the normalized difference water index (MNDWI) [28] and forests are not easily confused with rural settlements, we focused more on selecting the other two nonrural settlement land cover samples (crop and bare soil) in this study. To generate samples for each land cover type, the ancillary land cover products GUF and GFSAD30 were introduced. First, urban samples were randomly generated based on GUF products. Then, the percentage of GUF within each segmented patch greater than a predefined threshold (here set as 80%) was kept as the rural sample otherwise, it was removed from potential rural samples. This kind of filtering process helps to automatically produce samples by removing noise caused by the neighboring land covers near rural settlements. The generation of crop samples was similar to rural settlements except that the threshold for filtering was set to 90% due to the high crop density in our research region. In terms of soil samples, as there are no available soil products, we visually selected samples by taking Google Earth images as a reference. The number of samples for each land cover class per region is displayed in

Table 1. The number of samples for each land cover class per region.

Research Region	Land Cover Types
	Rural Settlements	Crop	Soil
NI	208	1338	93
CA	38	229	24
SC	58	286	0
CT	37	257	7
NX	27	91	30
SI	173	684	72
NM	23	78	0
NC	228	385	0

. We can see that the high-density rural areas have a high percentage of rural settlement samples and vice versa. For research regions, such as North China (NC) and Northern Malaysia (NM), there are no bare soil samples.

3.4. Determining Pixel- and Object-Based Thresholds

The filtered samples were applied to pixel (NDVI_max, NDBI_min, VV_mean and gradient) and object-based features (NDVI_max_obc, NDBI_min_obc, VV_mean_obc and gradient_obc). To avoid using a user-defined threshold, we propose a simple-operated method to obtain the thresholds for pixel- and OBC-based features simultaneously. Figure 6 displays the scatter plot between NDVI_obc and NDVI_pixel for three land covers. Note that the scatter plot between gradient_obc and gradient_pixel displays only the rural and bare soil samples.

We take the scatterplots for NDVI and rural settlements as an example to illustrate the process of automatic identification of pixel-and-object based thresholds. We assume that these land cover samples have a t distribution, which is suitable to delineate the central tendency of small samples. The stat_ellipse method in R language was utilized to circle their distribution with 95% confidence of all the samples (ellipse in Figure 7). We can see that the ellipse captures the most representative land cover samples with the outliers removed. Here, we selected the location of the ellipse boundary (red solid dot) as the threshold to discriminate rural settlements from photosynthetic materials. The red point is from the right end point of the major axis of the ellipse. For other features (NDBI, VV and gradient) and land cover types, the same strategy was applied to obtain corresponding thresholds.

The equation for extracting rural settlements applied in GEE is shown in Equations (6)–(9):

$$\begin{array}{l}Settlement\_obc=\\ NDVI\_obc.\mathrm{lt}(\mathrm{a}).\mathrm{and}(NDBI\_obc.\mathrm{gt}(\mathrm{b})).\mathrm{and}(VV\_obc.\mathrm{gt}(c)).\mathrm{and}(Gradient\_obc.\mathrm{lt}(\mathrm{d}))\end{array}$$

(6)

$$\begin{array}{l}Settlement\_p=\\ NDVI\_obc.\mathrm{lt}(\mathrm{a}1).\mathrm{and}(NDBI\_obc.\mathrm{gt}(\mathrm{b}1)).\mathrm{and}(VV\_obc.\mathrm{gt}(c1)).\mathrm{and}(Gradient\_obc.\mathrm{lt}(\mathrm{d}1))\end{array}$$

(7)

where a, b, c, and d are the object-based thresholds and a1, b1, c1, and d1 are the pixel-based thresholds. Settlement_p represents Settlement_pixel and the same for other features.

Then, the final rural settlements were obtained using the Equation (8):

$$\mathit{Settlement}=\mathit{Settlement}\_\mathit{obc}.\mathrm{and}(\mathit{Settlement}\_\mathit{pixel})$$

(8)

$$\mathit{Rural} \mathit{settlement}=\mathit{Settlement}.\mathrm{and}(\mathit{NTL}.\mathrm{lt}\left(\mathrm{e}\right))$$

(9)

Here, e is the empirical threshold used for urban removal and it varies with region.

3.5. Accuracy Assessment

Validation samples were randomly generated based on extracted products and a 3 × 3 rectangle buffer (90 × 90 m) was created around each validation point (Figure 8). Each research region has 200 rural and 200 nonrural samples, hence there are 1600 samples in total for eight research regions. Ground-truthed rural/nonrural areas were visually interpreted from high-resolution Google Earth images. Our verification rules were as follows: if there was at least one house located in a 3 × 3 buffer zone, this plot was set as a rural settlement; otherwise, the sample was interpreted as nonrural. The commonly used indices, producer’s accuracy (PA), user’s accuracy (UA) and overall accuracy (OA) were adopted to quantitatively validate the extracted result.

4. Results

4.1. The Extracted Results

Table 2. Comparison of the extent of rural settlements mapped in this study and existing products.

Research Region	Google Earth Image	Our Maps	GHSL	GUF	GlobeLand30
NI
CA
SC
CT
NX
SI
NM
NC

compares the extent of our rural settlements and existing products. Visually, our maps provide good representations of rural settlements across study areas, which are all superior to existing maps (except for GUF in some cases). Our map accurately captured the extent and boundary of rural settlements in study area 1 (NI). However, the GUF, GHSL and GlobeLand30 products missed most rural settlements in region 1. For study area 2 (CA) with linear low-density rural settlements, our product performed well but not as well as GUF. Similar to the first case, most rural settlements were not captured by GHSL. Although GlobeLand30 characterizes the boundary of rural settlements well in this landscape, spatial details were lost within the settlement patch. In research region 3 (SC), our extracted results are similar to that of GUF. In addition to depicting patchy settlements, some linearly distributed rural settlements (in the east and west) were also well presented and were not captured by GUF. GHSL only provides the location of rural settlements with loss of the accurate boundaries and spatial patterns. GlobeLand30 missed most rural settlements by only providing the location of the largest rural patches. In research region 4 (CT), our map is generally the same as that of GUF. However, GUF misclassified trees show textures similar to rural buildings in SAR images. Our map provides a larger extent of rural settlements than GUF with some scattered houses identified. However, sometimes our maps contain noise from bare land and nearby cropland confusions. GHSL only recognizes the central location of the rural region with a miss of 50% in rural areas and GlobeLand30 only detects larger rural patches and misses many small clusters. In research region 5 (NX), our method captured relatively accurate boundaries of rural settlement. The GUF delineates more individual houses whose size is less than 30 m (as seen in column 3, Table 2) that cannot be captured by Landsat images. GlobeLand30 only determined some regular and large-patch settlements, leaving out all the small and irregularly shaped settlements. Similarly, GHSL lost almost all rural settlements. In research region 6 (SI), our method performs better than all the other products, especially for some small-patch settlements. GlobeLand30 missed almost all rural areas, which is a common case for this product. GUF performs better than GHSL, which only provides sporadic distributions of settlements. In research region 7 (NM), our results provide similar results to GUF except that many more small-patch settlements with sizes less than 30 m were captured by GUF but not our maps. GlobeLand30 provides the location of rural settlements with some spatial details lost, and GHSL only picks 30% of rural settlements. In research region 8 (NC), our results provide a clear and accurate description of rural settlements which is similar as GUF. GlobeLand30 provides a complete description of the rural patch but with the spatial details lost. GHSL fails to delineate rural settlements in this kind of landscape. From our t-test analysis, there are statistically significant differences (p < 0.05) between our and other products (GHSL, GUF and GlobeLand30) in all eight regions, except of GHSL in research region 2 with a p value of 0.4.

4.2. The Accuracy of Extracted Rural Settlements

The commission and omission errors were balanced with each other in current settlement products as displayed in the PA and UA of GHSL, GlobeLand30 and GUF (Figure 9). The high UA and low PA of GHSL, GLS and GUF indicate that these products lost many rural settlements but seldom misclassified other land covers as rural. However, there are also some cases with both low PA and UA such as in research region 1 (north Indian), that should be paid more attention to, particularly conducting regional analysis in this region. For the eight selected regions, our results provide higher PA than others, indicating that more rural settlements were delineated within our product. However, the UA value was less satisfactory compared with PA, indicating that misclassifications from nonrural settlements were introduced by our method, but the magnitude was less than PA. Thus, in terms of the overall accuracy, our method performs better than all other products, with a 10% increase compared with GlobeLand30 and GHS and a 5% increase compared to GUF.

5. Discussion

5.1. Comparison between Different Products from the Algorithm Perspective

The algorithmic performance of different settlement products are shown in

Table 3. The example of algorithm performance for extracted rural settlements of three selected cases (a), (b) and (c). Here, the GUF and our map were overlaid to highlight their differences.

Google Earth Image	Our Map (Red) GUF (Blue)	GHSL	GlobeLand30

(a), (b) and (c). For the GUF products, noise is mainly from tree canopies near houses. For example, as GUF products rely on the TerraSAR and digital elevation model (DEM), the settlement corners and tree canopies all present strong scattering characteristics, leading to high brightness on TerraSAR data. These similar textured features lead to the misclassification of tree canopy with settlements (see second column in Table 3), even though postediting efforts have been placed on the extracted products. For the GHSL that lost almost all the settlements, the poor performance (not accurate delineation or totally lost, see third column) was from the step-by-step processing chain of producing GHSL products, of which the key concept was to remove the nonsettlements step by step. Hence, the incorrect classification of other land covers, shrubs, forests, etc. would influence the accuracy of final GHSL layers. For GlobeLand30, the settlement patch did not match well with the ground truth, and even small rural objects were lost (fourth column). Given the land cover shapes and the algorithmic description of GlobeLand30, the inaccurate performance in delineating rural settlements lies in the initial segmentation stage is not fully describe the accurate settlement boundary.

5.2. The Performance of Soil-Minimization Using Gradient Feature

The misclassification between bare soil and settlements is one of the most important unsettled issues in urban remote sensing due to the spectral similarity between them. Therefore, the commonly used spectral-based indices NDVI, NDBI, and the biophysical composition index (BCI, [29]) still have limited performance in removing bare soil without affecting the identification of settlements. Seasonal or long-lasting bare soil broadly existed in sandy land, river beaches, and fallow land which are mostly located in rural regions, leading to the overestimation of settlements being more common in rural than urban areas. In this study, we utilized the temporal-based feature gradient to suppress soil to the utmost extent. The compared results were shown in

Table 4. Examples of soil-minimization using gradient feature for three selected cases for (a), (b) and (c).

Google Earth Image	Settlement Extraction with Gradient Feature	Settlement Extraction without Gradient Feature

, with red and light blue representing settlement results with or without gradient features. We can see that large areas of bare soil in the sandy region were removed as illustrated in (a) and (b) in Table 4. In addition, the bare soil among cropland was also moderately removed as displayed in Table 4 (c).

6. Conclusions

Few studies have specifically focused on rural settlement extraction, even with the existence of large open-access satellite images and advanced classification methods. In this study, a threshold-based rural settlement extraction method was proposed using spectral-texture-temporal information by integrating a pixel- and object-based strategy. Most of the processes were performed directly in the GEE platform except segmentation, as the inherent segmentation method within GEE is not powerful enough to obtain accurate object-based results. This shortcoming hinders our progress towards contiguously large-scale rural mapping in GEE. We tested our method for eight selected Asian regions and found the following:

• Our obtained rural map achieved higher accuracy than current mainstream settlement layers/products and could provide complementary materials to the existing operational land cover maps. We also find that the current rural settlement product (even our result) has relatively poor performance (settlements lost) in the Indian area. Researchers should pay more attention when using the rural products for this region.
• Our method facilitated the removal of bare soil by using the gradient feature from annual NDVI_max information. This simple and easily obtained index effectively solved soil-impervious misclassification issues.

Our method still has some room for improvement, especially concerning the filtering process for obtaining rural samples. For example, more filtering criteria, such as spectral and texture-based thresholds, could be combined to filter the initial generated samples. Last, our method is very promising for time-series large-area mapping if the advanced segmented algorithms are fully transplanted to the GEE platform.