Next Article in Journal
Terrestrial and Martian Paleo-Hydrologic Environment Systematic Comparison with ASI PRISMA and NASA CRISM Hyperspectral Instruments
Previous Article in Journal
Data-Driven Cooperative Localization Algorithm for Deep-Sea Landing Vehicles Under Track Slippage
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 17
Issue 5
10.3390/rs17050756
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Accuracy Assessment of Four Land Cover Datasets at Urban, Rural and Metropolitan Area Level

by
Cosmina-Daniela Ursu
1,*,
Jozsef Benedek
1,2 and
Kinga Temerdek-Ivan
1
1
Faculty of Geography, Babeș-Bolyai University, 5–7 Clinicilor Street, 400006 Cluj-Napoca, Romania
2
Budapest Metropolitan University, Nagy Lajos Király Útja 1-9, 1148 Budapest, Hungary
*
Author to whom correspondence should be addressed.
Remote Sens. 2025, 17(5), 756; https://doi.org/10.3390/rs17050756
Submission received: 20 January 2025 / Revised: 18 February 2025 / Accepted: 18 February 2025 / Published: 22 February 2025
Browse Figures
Versions Notes

Abstract

:
Assessing land use/land cover changes currently represents an important avenue for achieving a better understanding of the urbanization phenomenon. Various free datasets based on satellite imagery are available, but the user should decide which one is the most suitable for their study area. The aim of the present paper is to perform an accuracy assessment of built-up areas using four datasets: Corine Land Cover Backbone (CLC Backbone), High Resolution Layers (HRL)–Imperviousness, Esri Land Cover and Dynamic World. The study case is represented by 12 major metropolitan areas (MAs) in Romania which have the most dynamic economic development and urban expansion. Confusion matrices were created, and the following metrics have been computed: overall accuracy (OA), kappa coefficient (k) and user accuracy (UA). The analysis was performed on three levels: for the entire surface of the MAs and separately for the urban and rural sides. The results at the metropolitan level show that even though CLC Backbone 2018 is the most suitable for extracting the built areas (0.85 overall accuracy), HRL and Esri Land Cover could also be used, as they share the same overall accuracy values (0.67). Significant differences exist between the urban and rural areas. CLC Backbone performed better in the rural areas (0.87) than in the urban areas (0.84). The other three datasets recorded major variations in the overall accuracy for the urban and rural areas. Esri Land Cover has the second greatest overall accuracy for the urban areas (0.81), while HRL is the second most accurate, after CLC Backbone, for assessing the rural areas (0.67). In conclusion, CLC Backbone has the best accuracy performance for all three levels of analysis. The significance of the study lies in the accuracy assessment results on the four datasets, performed at urban and rural levels. This paper aims to help researchers and decision makers choose the best dataset for assessing land use changes. Additionally, having a reliable dataset may help compute the indicators used to monitor the Sustainable Development Goals (SDGs).

1. Introduction

In the context of multiple interactions between the environmental component and anthropogenic activities, the usage of an accurate land use/land cover dataset is essential for the observation of land changes [1,2,3,4]. Moreover, the importance of land cover monitoring is noted by the United Nations, who use it for achieving the Sustainable Development Goals (SDGs) and for assessing the impact of climate changes [5]. Given the diversity of free Earth observation repositories, there are multiple ways to assess land use changes. Some researchers used raw satellite data to extract land cover by using supervised classifications or spectral indices [6,7,8,9]. Recently, the wide access to AI platforms has led to easier ways of processing satellite data. In this way, two other datasets were created: Dynamic World [10] and Esri Land Cover [3,11]. Google Earth Engine was used to extract built areas from Landsat scenes [12,13]. Also, other algorithms (e.g., random forest classification tree) were used to extract the land cover classes from Sentinel-1 and Sentinel-2 data by the European Space Agency, who created World Cover [14]. Still, it is widely admitted that the land cover surfaces differ from one source to another [1,4,15].
In terms of comparing multiple datasets in order to identify the most accurate one for a specific region, various methods have been used, using datasets with lower resolutions [2,16,17]. In the early 2000s, the main method used was pixel-by-pixel comparison, which usually imposed challenges with regard to the harmonization of the dataset [18,19]. Tchuenté, Roujean and De Jong [20] have compared four land cover products for Africa by using relative kappa values and relative overall accuracy values derived from the error matrices. This method was further used in more recent studies.
Other accuracy assessment studies were performed at regional [21], metropolitan [22] and rural levels [23]. These analysis levels were selected for their mixed urban pattern, including both highly urbanized areas and rural dispersed and fragmented settlements.
Only recently, after Sentinel-2 was released, were analyses of datasets with 10 m resolution published. Several studies compared the overall accuracy of multiple datasets, including Dynamic World and Esri Land Cover [4,10,15,24,25,26,27,28], and concluded that the obtained accuracy metrics are lower than the general values of each dataset. There are also multiple studies which insisted on different aspects regarding the accuracy assessment on land cover products [1,29,30].
Nowadays, when there is a wide range of free satellite data at high resolution [31], as well as ready-to-use data, one must be able to determine which one is more suitable for the study of land use changes. The methods of extracting land cover have improved greatly over time in the context of high-resolution data and the use of artificial intelligence (AI) for constructing automatic algorithms. Thus, when analyzing land use changes, there is no longer a need to follow the traditional workflow (downloading the satellite scenes; pre-processing corrections and calibrations; calculating indices or performing classifications; and testing the accuracy of the results). Of course, if one wants to develop a new method, this implies that each step should be completed. But for the studies that only need a dataset for assessing land use changes, datasets that have already been processed are sufficient, as they save time, and they do not require advanced knowledge for their use. Even if the accuracy of these global datasets is calculated, each user should assess the accuracy for the specific study area before using the data [27,32]. Thus, the main objective of this paper is to assess the built areas from four free datasets (CLC Backbone, HRL–Imperviousness, Dynamic World and Esri Land Cover) and to evaluate the global accuracy of each one. As demonstrated by previous studies that involved land cover/land use [33,34,35], many factors support our dataset choice. First, CLC Backbone was recently released in 2024, and we intend to see how well it classifies the built areas. Second, Dynamic World and Esri Land Cover are generated yearly based on AI processes and we want to examine their potential for providing recent data on built areas. Third, we relied on HRL as it is the oldest dataset that proved to be useful in other studies [34,35].
This study also addresses the lack of accuracy assessment studies at high resolution in Romania. Stoica et al. [36] compared Corine Land Cover (CLC) and Landsat supervised classification in the most urbanized area (the capital and its surroundings). The results show a better overall accuracy for their own Landsat classification at a 30 m resolution compared to the CLC vector polygon coverage of 25 ha MMU. Kaim et al. (2022) [23] analyzed only the rural areas of the Romanian Carpathian Mountains among other countries. Moreover, the present study examines Dynamic World and Esri Land Cover for their potential as annually updated datasets. In addition, it contributes to the existing literature by integrating the accuracy assessment of the CLC Backbone, which, to the authors’ knowledge, has only been used by Fonte et al. [4].
The aim is to help researchers, stakeholders or administrative personnel determine which dataset best suits their purpose in analyzing the evolution of built areas. The main arguments for the study’s significance lie in the choice of the analyzed datasets, the levels of analysis (metropolitan, urban and rural) and the use of the recently released CLC Backbone. This multi-level analysis addresses a significant gap in the existing literature, providing users and researchers valuable insights into the usefulness of different datasets depending on the level and study area. To date, no studies have compared freely available, long-term, publicly accessible datasets across these three different scales.
The analysis will be conducted for 2018, using 12 metropolitan areas (MAs) in Romania as the case study. We are interested not only in providing information about urban land use but also in examining the rural localities of the surrounding area. Based on the assumption that rural areas offer more available land at lower prices compared to urban centers, built areas may expand more rapidly in the suburban localities. Additionally, by considering rural localities, we can assess the accuracy of land use datasets in the context of sparsely built areas and predominantly agricultural land.
The specific objectives of the paper are:
  • To analyze the overall accuracy of four datasets: CLC Backbone, HRL–Imperviousness, Dynamic World and Esri Land Cover.
  • To assess the performance of these datasets in representing built areas at metropolitan, urban and rural levels.
  • To validate the accuracy results in four metropolitan areas.
The article is structured as follows. After a brief Introduction, the study area is presented. Next, each dataset is described, followed by the accuracy assessment. The Results section presents the accuracy values for the three levels, along with discussions in the context of existing literature. Finally, conclusions are drawn regarding the impact of the results.

2. Materials and Methods

2.1. Study Area

The 12 MAs (Figure 1) are evenly distributed across Romania’s Development Regions. They were selected based on their population in the 2021 Census [37], starting with those exceeding 200,000 inhabitants. These are the largest metropolitan areas in terms of both population and economic development, which are key drivers of built area dynamics [33,38,39]. As this study aims to assess the performance of various land cover/land use datasets in extracting built areas, it is important to select areas experiencing intense urbanization [21,22]. Bucharest MA was not considered due to its size (over 2 million inhabitants), which would overshadow the other MAs. Moreover, its built areas are denser than in any other MA, potentially introducing biases in the accuracy results.
The MAs cover a significant area, encompassing various land cover categories: built space, agricultural land, water and forests. Additionally, built space appears in different spatial configurations: the urban fabric, which is more compact in urban cores, and rural built space, which is often fragmented by small agricultural parcels. Furthermore, the location of the villages influences their structure—those in plains or valleys have a different layout from those in mountainous areas, where settlements tend to be more scattered. To ensure a comprehensive analysis, this study assesses the accuracy of the datasets in representing built space at three levels: the overall built land of each MA, the urban core and the rural area. The metropolitan level includes all built areas without distinguishing between urban and rural localities. The urban level includes only the built areas within cities, whereas the rural level refers to the rural localities that are part of the MAs. We assume that accuracy results will differ between urban and rural areas.
Previous studies have highlighted the importance of distinguishing between urban and rural areas in land use analysis [23,40,41,42]. By adopting this multi-level approach, this study offers a novel perspective, addressing gaps in the understanding of built space accuracy across different spatial configurations.

2.2. Data Source

To assess the accuracy of existing land use datasets, four products were considered (Table 1). The most recent year for which all datasets provide data is 2018, making it the selected year for analysis. These datasets were chosen because they have been used in our previous studies on built-up areas (HRL was used by [34,35]) and have also been employed in other research: Dynamic World by [43,44,45], Esri Land Cover by [32,43,46] and CLC Backbone by [4,47].
The inclusion of CLC Backbone represents a novelty, as it was only recently released (mid-2024). To our knowledge, few studies have been published using this dataset so far. Another key selection criterion was resolution—2018 was the only year for which all four datasets were available at a 10 m resolution. Additionally, we aimed to analyze two datasets that are continuously updated (Dynamic World and Esri Land Cover). Free access to near real-time data could be highly valuable for assessing the evolution of built-up areas and urban patterns.
Although each dataset has a reported accuracy level, our objective is to evaluate their accuracy at a local level, across different terrain configurations and separately for urban and rural areas.
High Resolution Layers–Imperviousness Density 2018 is a product of the European Environment Agency (EEA), developed as part of the EU Copernicus program [48]. It represents sealing density on a scale from 0% to 100% for the year 2018 and covers the EEA-38 region plus the UK. The dataset is provided as a 10 m resolution raster, produced using a calibrated normalized difference vegetation index (NDVI) and the Massive Spatial Automatic Data Analytics (MASADA) Toolbox [48]. The overall producer’s accuracy is 93.68%, while the user’s accuracy is 96.14% [48].
CLC Backbone is the enhanced high-resolution version of CORINE Land Cover, provided by the European Environment Agency (EEA) and the European Commission’s DG Joint Research Centre (JRC). The dataset covers all of Europe (EEA-38 region plus the UK) and uses the European Terrestrial Reference System 1989 (ETRS89) with the Lambert Azimuthal Equal Area (LAEA) projection. For the 2018 dataset, Copernicus Sentinel-2 satellite imagery from 2017, 2018 and 2019 (from 1 July 2017 to 31 June 2019) was used. The dataset also includes a confidence layer, indicating the reliability of the land cover class assignment for each pixel. The overall accuracy of the dataset is 90% [51].
Dynamic World is a land cover dataset produced in near real time, based on 10 m resolution Sentinel-2 imagery, and processed using a scalable cloud-based system [10]. Annotators and the Labelbox platform were used on Sentinel-2 Level 2 A Surface Reflectance imagery, as well as pre-processed, normalized Sentinel-2 Level 1 C Top of Atmosphere imagery, to create the training dataset collection. An important advantage of using Earth Engine and Cloud AI Platform is the ability to process large quantities of satellite data. Data validation was carried out using a confusion matrix and comparisons with other datasets.
Esri Land Cover is a global map of land use/land cover (LULC) derived from ESA Sentinel-2 imagery at a 10 m resolution. The dataset is generated using Impact Observatory’s deep learning AI land classification model [11]. To facilitate better visualization of changes over time, ESRI created a web-based mapping application called “Living Atlas” [52]. The platform displays a land cover classification in nine classes for the period 2017–2023. Additionally, the dynamic interface allows for visual change analysis and provides statistics for selected areas. The imagery is available for comparison between the classification and real land cover.
The Results section begins with “General Observations” regarding the visual differences among the four datasets. This comparison serves as a first step in identifying the most accurate dataset. Various urban patterns are analyzed for both urban and rural areas. The raw satellite imagery is expected to provide visual clarity, enabling a more effective comparison. Following this preliminary visual assessment, the accuracy results will determine the most reliable dataset based on accuracy metrics.

2.3. Data Processing

The current study employs a visual assessment approach, which has also been used in other research [4,10,15,24,26]. The authors emphasize the importance of having the same number of classes for comparison, even if this requires combining multiple classes. In our case, the only class analyzed was the sealed surface. For CLC Backbone, Dynamic World and Esri Land Cover, a single class defined the built areas. However, for Imperviousness, degrees of imperviousness ranging from 1% to 100% were available.
The first accuracy assessment was conducted on the metropolitan area, without distinguishing between urban and rural areas. The sampling design included 400 random points (Figure 2), which were generated and validated in ArcGIS, based on the urban area boundary. This number of points allowed for a high level of statistical confidence, striking a balance between too few points, which would not have captured all the variability, and too many, which could have unnecessarily increased the workload. Starting with 100 points, we gradually increased the sample size to 400, as this number was sufficient to ensure an adequate representation of land cover across the entire metropolitan area, while remaining manageable in terms of data processing and validation.
The classification between urban and rural areas was then performed (Figure 3). All cities were included in the urban category, while the rural category comprised the remaining localities. This differentiation was made using the territorial boundaries. Of the 400 points, 200 were randomly generated for the urban areas and another 200 for the rural areas within the selected MAs.
To validate the accuracy assessment performed at the metropolitan level, four MAs were selected: Cluj, Constanţa, Iaşi and Timişoara. The aim of this validation was to determine whether the dataset with the highest overall accuracy for the entire metropolitan area would remain the most accurate when tested on individual metropolitan areas. For each MA, 50 validation points were generated for the urban surface.
The pixel values from each of the four raster datasets were then extracted and reclassified into two classes: 1 (built areas) and 2 (other areas). In the case of HRL, imperviousness is classified in degrees ranging from 1% to 100%. Therefore, all values above 1% were considered “built”. The results were compared with ground truth data using Google Earth Pro. Each point was located in satellite imagery from 2018, and if it fell on a built surface, it was assigned a value of 1; otherwise, it was assigned a value of 2. Some observations were made regarding values assignment. If a point was located near a major road, it was classified as built space due to the 10 m resolution. Additionally, cemeteries and industrial parks were also considered built areas.
After assigning the values, the reclassified tables for the four datasets, along with the Google Earth validation data, were exported to Excel for accuracy calculation.

2.4. Accuracy Assessment

To evaluate the accuracy of built-up area classifications, a set of 400 random points was generated from the ground truth, categorized into urban, rural and metropolitan built-up areas. These points were then compared with the built-up areas classified according to the Dynamic World, Esri Land Cover, High Resolution Layers–Imperviousness and CLC Backbone datasets. The ground truth was derived from the interpretation of Google Earth Pro satellite imagery for the year 2018.
To validate the built-up area classifications from the four datasets against the ground truth, a confusion matrix was created, from which the following metrics were calculated: overall accuracy (OA), kappa coefficient (k) and user accuracy (UA). OA assesses the overall performance of each dataset in classifying built-up areas, representing the proportion of correctly classified pixels out of the total reference pixels. The kappa coefficient measures the agreement between each dataset’s classifications and the actual classifications. UA evaluates the probability that areas classified as “built-up” were correctly identified.
The kappa coefficient was calculated using the following formula [53]:
k = N ∑ i = 1 r x i i − ∑ i = 1 r ( x i + x + i ) N 2 − ∑ i = 1 r ( x i + x + i )
where: N represents the total number of pixels; x i i represents the observation in row i and column i;   x i is the marginal total of row i; x + i   is the marginal total of column i.
The kappa (k) values range from 0 to 1, with 1 indicating almost perfect agreement between the observed and predicted classes, while values closer to 0 signify poor or slight agreement, as shown in Table 2 [53,54]. In our analysis, all calculated kappa values were positive, with no negative values encountered.
Table 3 presents the confusion matrix at the metropolitan level, based on 400 points. The same methodology was applied to urban and rural areas, with 200 points assigned to each. The results shown are specific to CLC Backbone, being further explained in the Results section.

3. Results

3.1. General Observations

The visual comparison revealed differences between the four datasets (Figure 4). Overall, CLC Backbone and HRL–Imperviousness appear to delineate built-up areas more accurately than the AI-generated datasets (Dynamic World and Esri Land Cover), despite having the same resolution. CLC Backbone even captures small vegetated areas between buildings.
In contrast, Dynamic World and Esri Land Cover tend to overclassify green spaces as built-up areas. However, there is a distinction between the two: Esri Land Cover is more permissive in its classification of built areas. In the urban centers (Figure 4a), where building density is higher and small parks or other vegetated spaces exist, it is crucial for land cover datasets to clearly distinguish between green areas and built-up surfaces. Moreover, higher accuracy is expected in the urban centers compared to rural areas, where buildings are more sparsely distributed and interspersed with small agricultural parcels (Figure 4b–d).
Another key difference between the four datasets is their ability to delineate main roads. CLC Backbone and HRL–Imperviousness appear to capture major roads more effectively than Dynamic World or Esri Land Cover. However, a limitation is that only wide, tree-free roads—such as highways, European roads, and bypasses—are classified as built-up areas. CLC Backbone even includes small roads in the built-up category, whereas HRL does not delineate them. Airport runways are better recognized by CLC Backbone, while the other datasets classify them only partially as built-up areas. Additionally, Dynamic World and Esri Land Cover categorize cemeteries as built areas, whereas HRL includes them only partially, and CLC excludes them entirely.

3.2. Accuracy Results for the Entire Urban Surface of the 12 MAs

Among the four datasets analyzed (Table 4), CLC Backbone achieved the highest overall accuracy (0.85) for the entire urban surface, demonstrating its strong performance in extracting built-up areas across the study area. This is further supported by its high user accuracy (0.95), indicating that areas classified as built-up in this dataset are highly likely to be correctly identified. At the other end of the spectrum, Dynamic World exhibited the weakest performance in accurately identifying built-up areas, as reflected in its lower overall accuracy and user accuracy scores. The kappa coefficient showed values close to 0.5 for all four datasets, suggesting a moderate agreement between the classifications and the ground truth.
At the level of urban localities (Table 5), CLC Backbone again demonstrated the best performance, achieving the highest overall accuracy of 0.84. Both Esri Land Cover (0.81) and Dynamic World (0.80) showed reasonable accuracy. These results indicate that over 80% of classified areas were correctly identified in these three datasets. The kappa coefficient remained around 0.5, suggesting a moderate agreement between the classified areas and the ground truth (Table 2). CLC Backbone also recorded the highest user accuracy (0.90) for urban areas, meaning that 90% of the areas classified as built-up were correctly identified. In contrast, the overall accuracy for Dynamic World (0.75) and Esri Land Cover (0.74) was notably lower than that of CLC Backbone.
When comparing the accuracy of the four datasets for rural localities (Table 6), CLC Backbone demonstrates the best performance in detecting built-up areas. This is reflected in its highest accuracy metrics among the datasets, with an overall accuracy of 0.87 and a kappa index of 0.63. In contrast, the overall accuracy for the other datasets is significantly lower, ranging from 0.56 to 0.67. The kappa index is also lower, with values of 0.47 for Dynamic World, 0.43 for Esri Land Cover and 0.59 for HRL, which is closer to CLC Backbone. Additionally, user accuracy for the other datasets is substantially lower (0.36 and 0.4) compared to 0.84 for CLC Backbone, further emphasizing its higher reliability in identifying built-up areas in rural regions.
Examining the overall accuracy values for the four datasets (Figure 5) across the entire MA, as well as for urban and rural areas separately, several preliminary conclusions can be drawn. For all three levels, CLC Backbone achieves the highest accuracy, as initially anticipated. However, it performs better in rural areas (0.87) than in urban areas (0.84), while the overall metropolitan area accuracy falls somewhere in between. In contrast, HRL follows the opposite pattern, performing better in urban areas (0.79) than in rural areas (0.67). Across the entire MA, it achieves the same overall accuracy (0.67) as Esri Land Cover, which also has a higher OA in urban areas (0.81) than in rural areas (0.59). Similarly, Dynamic World follows the same trend, with a higher accuracy in urban areas (0.8) compared to rural areas (0.56).
With the exception of CLC, all other datasets performed better in urban areas than in rural areas, largely due to the higher construction density and more compact urban patterns. Among the datasets, HRL achieved the highest OA in rural areas, while Esri Land Cover performed best in urban areas. However, the differences between the three datasets are minor. Across the entire metropolitan area surface, HRL and Esri Land Cover share the same accuracy value (0.67), followed closely by Dynamic World with a slightly lower value.

3.3. Accuracy Results for Four MAs

The results for the four metropolitan areas (Figure 6) reveal significant differences in accuracy, not only within each MA, but also when comparing them.
CLC Backbone consistently achieved the highest overall accuracy across all four metropolitan areas, reinforcing its strong performance at other levels of analysis. However, variations between MAs can still be observed. HRL recorded the highest OA values in Cluj and Constanţa MAs (0.72) but had lower values in Iaşi (0.62) and Timişoara (0.6). Esri Land Cover achieved its highest accuracy in Constanţa MA (0.8), while in the other MAs, its values did not exceed 0.68. Dynamic World also performed best in Constanţa MA (0.78), with lower accuracy in the remaining MAs. Across all four datasets, the highest accuracy values were recorded in Constanţa MA. Esri Land Cover showed the best OA results for Iaşi and Timişoara MAs, while HRL had the highest value in Cluj MA.

4. Discussion

4.1. Explaining the Differences Between the Analyzed Datasets

The results are consistent with previous studies. Bie et al. [25] obtained an overall accuracy of 62.54% for Esri Land Cover and highlighted its strong performance in identifying built-up areas, even when applied to an arid area. Venter et al. [24] also compared Esri Land Cover and Dynamic World, along with other datasets, and concluded that Esri Land Cover had a higher OA (75%) than Dynamic World (72%). Wang and Mountrakis [26] analyzed multiple datasets for the United States and concluded that Esri Land Cover performed well only in the cropland and grass/shrub classes, while its accuracy for built-up areas was lower. However, they encouraged the use of Dynamic World due to its near real-time dataset generation. Fonte et al. [4] suggested that differences in overall accuracy values could also be influenced by the classification methods used in each dataset. Still, in their national-level analysis for Portugal, CLC Backbone achieved the highest overall accuracy compared to the other five analyzed datasets.
The novelty of this study lies in several aspects. First, it analyzes the CLC Backbone dataset, which was released in 2024 and has rarely been used in research to date [4]. Second, the study’s innovation comes from the selection of the datasets for comparison, as they are based on different classification methods. Third, the analysis is conducted at three levels: the overall metropolitan area, as well as urban and rural areas separately. Compared to existing studies, which have focused on global [16,17,20,24,26,27], national [4,19] and regional levels [15,18,25,28], this research employs a more detailed scale and different datasets than previous studies at a local level [21,22,23]. The distinction between analysis levels is based on the assumption that datasets should perform differently in urban and rural areas due to the variations in urban density. Moreover, to the authors’ knowledge, there are no prior studies assessing the overall accuracy of high-resolution layers in Romania.
The scientific literature presents several reasons for accuracy differences between various datasets. Olofsson et al. [1] notice that map classification error must be considered when analyzing land cover changes, and datasets should be used with caution. Brown et al. [10] attribute the lack of uniform accuracy across all scenes to variations in image quality. Other factors influencing accuracy include the quality of Sentinel-2 cloud masking and variability in land cover and condition. Moreover, lower accuracy is often observed in mixed areas containing both built-up surfaces and vegetation [10]. The CLC Backbone product manual also acknowledges confusion between sealed and sparsely vegetated areas. Although post-processing operations have been applied to address this issue, some misclassification remains in the final product [52]. Examining the accuracy values at urban and rural levels (Table 5 and Table 6), CLC Backbone performs best in rural areas, while the other three datasets exhibit lower overall accuracy. This confirms that misclassifications between built-up and vegetated areas are more prevalent in rural sites, whereas urban areas, with their denser constructions patterns, are classified more accurately.
To achieve accurate classifications, CLC Backbone uses a comprehensive sample dataset for training and validation. This dataset includes: “adjusted and filtered LUCAS from EUROSTAT 2018, stratified automated LC class annotations based on existing land use/land cover maps, visual sample point photo-interpretation from VHR imagery, NDVI time series, national LC datasets, aerial imagery, or LiDAR data collected” [51] (p. 18). Moreover, the production and validation processes include an iterative approach to sampling and regional calibration, which helps reduce regional differences [51]. The quality control for CLC Backbone is more rigorous than that of other datasets, as it is based on ground truth pixel comparisons. However, for built-up areas, some limitations remain, as confusion may rise between sealed and sparsely vegetated areas due to high-reflectance surfaces [51].
For Dynamic World, it is acknowledged that built-up areas and classes such shrub and scrub, bare ground, crop, grass and flooded vegetation tend to exhibit lower accuracies [10]. Bie et al. [25] suggest that differences between datasets stem from variations in the classification scheme, which can lead to inconsistent results. Venter et al. [24] observed that the scale used for validation can also contribute to discrepancies among datasets.
Issues related to built-up areas mixed with vegetation, bare ground, crops, grass and flooded vegetation have also been noted in the context of temporal dynamics [10]. This presents a potential direction for further research—analyzing overall accuracy for a different year (available for all datasets) and comparing it with the present results. Such an approach would allow for an evaluation of each dataset’s performance over time.
Another key consideration when selecting a land cover product is that the choice of dataset should align with the study’s purpose. Venter et al. [24] observed that each dataset exhibits varying levels of global and local accuracy, as well as class-specific biases. The authors also highlighted differences across urban settlement types, a finding that is reinforced by the present study. Similarly, Xu et al. [27] reached the same conclusion, emphasizing that users should assess both the accuracy of the dataset within their area of interest and its suitability in terms of spatial detail.
As the results indicate, there is a significant discrepancy between the reported accuracy of the original datasets (which is generally considered to be above 85%) and the accuracy values calculated at a local level in Romania. Zheng et al. [15] also found that regional accuracy tends to be lower than the published accuracy of each dataset. Similarly, Fonte et al. [4] reached the same conclusion for Portugal, observing that accuracy values at the national level were lower than the global accuracy reported for each dataset.
As previously mentioned, the United Nations highlights the importance of land cover monitoring for achieving the Sustainable Development Goals (SDGs) and assessing the impact of climate changes [5,55]. The improved performance of CLC in rural areas may be particularly relevant for monitoring suburbanization, which tends to progress more rapidly in localities surrounding major urban centers [33,34,35]. In addition, the impact of land cover/land use changes could be studied more effectively with a reliable dataset. However, it is crucial to make an informed choice based on the specific objectives of the study. The results presented here apply solely to built-up areas; further research should explore other land cover categories.

4.2. Limitations and Future Research Directions

While this study provides valuable insights into the comparison of overall accuracy across the four datasets, certain limitations should be acknowledged to ensure a balanced perspective. The first limitation concerns the frequency of dataset updates: Esri Land Cover is updated annually (between 2017 and 2023), while Dynamic World operates in near real time. In contrast, HRL and CLC Backbone are updated only once every three years.
The second limitation relates to sample size. Although the number of random points was selected based on the extent of the metropolitan areas, a larger study area would require an increased sample size to ensure a more precise analysis.
Another limitation is the focus on a single land cover/land use class—built-up areas. Future studies could address this by including multiple classes and comparing the performance of several datasets. Additionally, analyzing overall accuracy for a different year and comparing the results with the present findings could provide insights on the consistency of these datasets over time.

5. Conclusions

For the entire urban surface of the 12 metropolitan areas analyzed, the CLC Backbone dataset proved to be the most suitable for extracting built-up areas, achieving high overall accuracy with minimal omission and commission errors. This result was expected, as CLC is continuously validated and corrected through ground truth comparisons. The other datasets (HRL, Dynamic World and Esri Land Cover) exhibited variations in overall accuracy depending on the analysis level. At the urban level, Esri Land Cover demonstrated the best performance, although HRL and Dynamic World showed no significant differences, making them equally suitable for built-up area extraction in urban environments. For rural localities, the HRL dataset performed best in classifying built-up areas due to its high accuracy and low classification errors. At the metropolitan area level (both rural and urban areas combined), HRL and Esri Land Cover had the same overall accuracy, and Dynamic World performed slightly lower but was still comparable to the other two datasets.
These findings may help guide researchers and decision makers in selecting the most appropriate dataset for assessing built-up area expansion at the local level. As previous studies have shown, it is advisable to test the accuracy of a dataset before use, as results can vary depending on the study area. The overall accuracy results highlight the ability of each dataset to distinguish built-up areas from other land cover categories. Having a reliable and regularly updated built-up dataset can support the analysis of suburbanization processes and help assess the impact of land cover changes on the environment, ultimately promoting sustainable development.

Author Contributions

Conceptualization, C.-D.U., K.T.-I. and J.B.; methodology, C.-D.U., K.T.-I. and J.B.; software, C.-D.U. and K.T.-I.; validation, C.-D.U. and K.T.-I.; formal analysis, C.-D.U. and K.T.-I.; writing—original draft preparation, C.-D.U. and K.T.-I.; writing—review and editing, C.-D.U., K.T.-I. and J.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Babeş-Bolyai University, through PATJ SB project, contract number 17540.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Acknowledgments

The authors acknowledge the funding support for this study.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Olofsson, P.; Foody, G.M.; Stehman, S.V.; Woodcock, C.E. Making better use of accuracy data in land change studies: Estimating accuracy and area and quantifying uncertainty using stratified estimation. Remote Sens. Environ. 2013, 129, 122–131. [Google Scholar] [CrossRef]
  2. Pesaresi, M.; Corbane, C.; Julea, A.; Florczyk, A.J.; Syrris, V.; Soille, P. Assessment of the added-value of Sentinel-2 for detecting built-up areas. Remote Sens. 2016, 8, 299. [Google Scholar] [CrossRef]
  3. Karra, K.; Kontgis, C.; Statman-Weil, Z.; Mazzariello, J.C.; Mathis, M.; Brumby, S.P. Global land use/land cover with Sentinel 2 and deep learning. In Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 11–16 July 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 4704–4707. [Google Scholar]
  4. Fonte, C.C.; Duarte, D.; Jesus, I.; Costa, H.; Benevides, P.; Moreira, F.; Caetano, M. Accuracy Assessment and Comparison of National, European and Global Land Use Land Cover Maps at the National Scale—Case Study: Portugal. Remote Sens. 2024, 16, 1504. [Google Scholar] [CrossRef]
  5. Szantoi, Z.; Geller, G.N.; Tsendbazar, N.E.; See, L.; Griffiths, P.; Fritz, S.; Gong, P.; Herold, M.; Mora, B.; Obregón, A. Addressing the need for improved land cover map products for policy support. Environ. Sci. Policy 2020, 112, 28–35. [Google Scholar] [CrossRef]
  6. Rozenstein, O.; Karnieli, A. Comparison of methods for land-use classification incorporating remote sensing and GIS inputs. Appl. Geogr. 2011, 31, 533–544. [Google Scholar] [CrossRef]
  7. Zhang, Q.; Li, B.; Thau, D.; Moore, R. Building a better urban picture: Combining day and night remote sensing imagery. Remote Sens. 2015, 7, 11887–11913. [Google Scholar] [CrossRef]
  8. Dolean, B.E.; Bilașco, Ș.; Petrea, D.; Moldovan, C.; Vescan, I.; Roșca, S.; Fodorean, I. Evaluation of the built-up area dynamics in the first ring of Cluj-Napoca metropolitan area, Romania by semi-automatic GIS analysis of Landsat satellite images. Appl. Sci. 2020, 10, 7722. [Google Scholar] [CrossRef]
  9. Liu, H.; Gong, P.; Wang, J.; Wang, X.; Ning, G.; Xu, B. Production of global daily seamless data cubes and quantification of global land cover change from 1985 to 2020-iMap World 1.0. Remote Sens. Environ. 2021, 258, 112364. [Google Scholar] [CrossRef]
  10. Brown, C.F.; Brumby, S.P.; Guzder-Williams, B.; Birch, T.; Hyde, S.B.; Mazzariello, J.; Czerwinski, W.; Pasquarella, V.J.; Haertel, R.; Ilyushchenko, S.; et al. Dynamic World, Near real-time global 10 m land use land cover mapping. Sci. Data 2022, 9, 251. [Google Scholar] [CrossRef]
  11. ESRI. Sentinel-2 10-Meter Land Use/Land Cover. Available online: https://livingatlas.arcgis.com/landcover/ (accessed on 12 July 2024).
  12. Patel, N.N.; Angiuli, E.; Gamba, P.; Gaughan, A.; Lisini, G.; Stevens, F.R.; Tatem, A.J.; Trianni, G. Multitemporal settlement and population mapping from Landsat using Google Earth Engine. Int. J. Appl. Earth Obs. Geoinf. 2015, 35, 199–208. [Google Scholar] [CrossRef]
  13. Gorelick, N.; Hancher, M.; Dixon, M.; Ilyushchenko, S.; Thau, D.; Moore, R. Google Earth Engine: Planetary-scale geospatial analysis for everyone. Remote Sens. Environ. 2017, 202, 18–27. [Google Scholar] [CrossRef]
  14. Zanaga, D.; Van De Kerchove, R.; De Keersmaecker, W.; Souverijns, N.; Brockmann, C.; Quast, R.; Wevers, J.; Grosu, A.; Paccini, A.; Vergnaud, S.; et al. ESA WorldCover 10 m 2020 V100. OpenAIRE 2021. Available online: https://www.openaire.eu/ (accessed on 22 September 2024).
  15. Zheng, K.; He, G.; Yin, R.; Wang, G.; Long, T. A Comparison of Seven Medium Resolution Impervious Surface Products on the Qinghai–Tibet Plateau, China from a User’s Perspective. Remote Sens. 2023, 15, 2366. [Google Scholar] [CrossRef]
  16. Pérez-Hoyos, A.; García-Haro, F.J.; San-Miguel-Ayanz, J. Conventional and fuzzy comparisons of large scale land cover products: Application to CORINE, GLC2000, MODIS and GlobCover in Europe. ISPRS J. Photogramm. Remote Sens. 2012, 74, 185–201. [Google Scholar] [CrossRef]
  17. Hua, T.; Zhao, W.; Liu, Y.; Wang, S.; Yang, S. Spatial consistency assessments for global land-cover datasets: A comparison among GLC2000, CCI LC, MCD12, GLOBCOVER and GLCNMO. Remote Sens. 2018, 10, 1846. [Google Scholar] [CrossRef]
  18. Pietroniro, A.; Soulis, E.D. Comparison of global land-cover databases in the Mackenzie basin, Canada. In Remote Sensing and Hydrology 2000; IAHS: Wallingford, UK, 2001; pp. 552–557. [Google Scholar]
  19. Caetano, M.; Araújo, A. Comparing land cover products CLC2000 and MOD12Q1 for Portugal. In Global Developments in Environmental Earth Observation from Space; IOS Press: Amsterdam, The Netherlands, 2006; pp. 469–477. [Google Scholar]
  20. Tchuenté, A.T.K.; Roujean, J.L.; De Jong, S.M. Comparison and relative quality assessment of the GLC2000, GLOBCOVER, MODIS and ECOCLIMAP land cover data sets at the African continental scale. Int. J. Appl. Earth Obs. Geoinf. 2011, 13, 207–219. [Google Scholar] [CrossRef]
  21. Klotz, M.; Kemper, T.; Geiß, C.; Esch, T.; Taubenböck, H. How good is the map? A multi-scale cross-comparison framework for global settlement layers: Evidence from Central Europe. Remote Sens. Environ. 2016, 178, 191–212. [Google Scholar] [CrossRef]
  22. Diaz-Pacheco, J.; Gutiérrez, J. Exploring the limitations of CORINE Land Cover for monitoring urban land-use dynamics in metropolitan areas. J. Land Use Sci. 2013, 9, 243–259. [Google Scholar] [CrossRef]
  23. Kaim, D.; Ziółkowska, E.; Grădinaru, S.R.; Pazúr, R. Assessing the suitability of urban-oriented land cover products for mapping rural settlements. Int. J. Geogr. Inf. Sci. 2022, 36, 2412–2426. [Google Scholar] [CrossRef]
  24. Venter, Z.S.; Barton, D.N.; Chakraborty, T.; Simensen, T.; Singh, G. Global 10 m land use land cover datasets: A comparison of Dynamic World, World Cover and ESRI land cover. Remote Sens. 2022, 14, 4101. [Google Scholar] [CrossRef]
  25. Bie, Q.; Luo, J.; Lu, G. Accuracy Performance of Three 10-m Global Land Cover Products Around 2020 in an Arid Region of Northwestern China. IEEE Access 2023, 11, 133215–133228. [Google Scholar] [CrossRef]
  26. Wang, Z.; Mountrakis, G. Accuracy Assessment of Eleven Medium Resolution Global and Regional Land Cover Land Use Products: A Case Study over the Conterminous United States. Remote Sens. 2023, 15, 3186. [Google Scholar] [CrossRef]
  27. Xu, P.; Tsendbazar, N.E.; Herold, M.; de Bruin, S.; Koopmans, M.; Birch, T.; Carter, S.; Fritz, S.; Lesiv, M.; Mazur, E.; et al. Comparative validation of recent 10 m-resolution global land cover maps. Remote Sens. Environ. 2024, 311, 114316. [Google Scholar] [CrossRef]
  28. Kang, J.; Yang, X.; Wang, Z.; Cheng, H.; Wang, J.; Tang, H.; Bai, Z. Comparison of three ten meter land cover products in a Drought region: A case study in Northwestern China. Land 2022, 11, 427. [Google Scholar] [CrossRef]
  29. Olofsson, P.; Foody, G.M.; Herold, M.; Stehman, S.V.; Woodcock, C.E.; Wulder, M.A. Good practices for estimating area and assessing accuracy of land change. Remote Sens. Environ. 2014, 148, 42–57. [Google Scholar] [CrossRef]
  30. Stehman, S.V.; Foody, G.M. Key issues in rigorous accuracy assessment of land cover products. Remote Sens. Environ. 2019, 231, 111199. [Google Scholar] [CrossRef]
  31. Phiri, D.; Simwanda, M.; Salekin, S.; Nyirenda, V.R.; Murayama, Y.; Ranagalage, M. Sentinel-2 data for land cover/use mapping: A review. Remote Sens. 2020, 12, 2291. [Google Scholar] [CrossRef]
  32. Chaaban, F.; El Khattabi, J.; Darwishe, H. Accuracy assessment of ESA WorldCover 2020 and ESRI 2020 land cover maps for a Region in Syria. J. Geovisualization Spat. Anal. 2022, 6, 31. [Google Scholar] [CrossRef]
  33. Benedek, J.; Ursu, C.-D.; Varvari, Ş. Growth pole policy’s induced development and spatial inequalities in the metropolitan areas of Romania-a critical assessment. Tér és Társadalom. 2022, 36, 47–67. [Google Scholar] [CrossRef]
  34. Ursu, C.D.; Benedek, J. Evolution of Built Surfaces Based on Copernicus High Resolution Layers. The Case of Growth Poles-Based Metropolitan Areas, Romania. J. Settl. Spat. Plan. 2022, 13, 45–59. [Google Scholar]
  35. Holobâcă, I.H.; Benedek, J.; Ursu, C.D.; Alexe, M.; Temerdek-Ivan, K. Ratio of Land Consumption Rate to Population Growth Rate in the Major Metropolitan Areas of Romania. Remote Sens. 2022, 14, 6016. [Google Scholar] [CrossRef]
  36. Stoica, I.V.; Vîrghileanu, M.; Zamfir, D.; Mihai, B.A.; Săvulescu, I. Comparative assessment of the built-up area expansion based on Corine land cover and landsat datasets: A case study of a post-socialist city. Remote Sens. 2020, 12, 2137. [Google Scholar] [CrossRef]
  37. National Institute of Statistics. 2021 Census. 2021. Available online: https://www.recensamantromania.ro/rezultate-rpl-2021/rezultate-definitive/ (accessed on 5 May 2024).
  38. Ionescu-Heroiu, M.; Cristea, M.; China, A.-M.; Butacu, B.-C.; Dolean, B.E.; Vinţan, A.D.; Frant, O.; Mihăilescu, G.M.; Moldovan, S.C.; Sfârlea, E.V.; et al. Romania Catching-Up Regions: Metropolitan Romania (English); World Bank Group: Washington, DC, USA, 2019; Available online: https://documents.worldbank.org/en/publication/documents-reports/documentdetail/255131580296079611/romania-catching-up-regions-metropolitan-romania (accessed on 12 January 2025).
  39. Kriss, P.; Ivanov, I.; Taralunga, N.; Racoviceanu, S.C.; Stadler, R.-L.; Ghintuiala, D.-L.; Ionescu-Heroiu, M.; Silimela, Y.T.; Vinţan, A.D.; Moldoveanu, G.A.; et al. Romania Catching-Up Regions: Interjurisdictional Cooperation Models (English); World Bank Group: Washington, DC, USA, 2021; Available online: http://documents.worldbank.org/curated/en/193121580281956431/Romania-Catching-Up-Regions-Interjurisdictional-Cooperation-Models (accessed on 12 January 2025).
  40. Lu, D.; Hetrick, S.; Moran, E. Land cover classification in a complex urban-rural landscape with QuickBird imagery. Photogramm. Eng. Remote Sens. 2010, 76, 1159–1168. [Google Scholar] [CrossRef] [PubMed]
  41. Wei, X.; Zhang, W.; Zhang, Z.; Huang, H.; Meng, L. Urban land use land cover classification based on GF-6 satellite imagery and multi-feature optimization. Geocarto Int. 2023, 38, 2236579. [Google Scholar] [CrossRef]
  42. Uhl, J.H.; Leyk, S. Towards localized accuracy assessment of remote-sensing derived built-up land layers across the rural-urban continuum. arXiv 2024, arXiv:2403.00166. [Google Scholar]
  43. Paris, C.; Martinez-Sanchez, L.; van der Velde, M.; Sharma, S.; Sedona, R.; Cavallaro, G. Accuracy assessment of land-use-land-cover maps: The semantic gap between in situ and satellite data. In Image and Signal Processing for Remote Sensing XXIX; SPIE: Bellingham, WA, USA, 2023; Volume 12733, pp. 187–200. [Google Scholar]
  44. Ahmed, R.; Zafor, M.A.; Trachte, K. Land-use and land-cover changes in Cottbus city and Spree-Neisse district, Germany, in the last two decades: A study using remote sensing data and Google Earth Engine. Remote Sens. 2024, 16, 2773. [Google Scholar] [CrossRef]
  45. Panek-Chwastyk, E.; Dąbrowska-Zielińska, K.; Markowska, A.; Kluczek, M.; Pieniążek, M. Advanced utilization of satellite and governmental data for determining the coverage and condition of green areas in Poland: An experimental statistics supporting the Statistics Poland. Int. J. Appl. Earth Obs. Geoinf. 2024, 130, 103883. [Google Scholar] [CrossRef]
  46. Bathe, K.D.; Patil, N.S. Assessment of land use-land cover dynamics and its future projection through Google Earth Engine, machine learning and QGIS-MOLUSCE: A case study in Jagatsinghpur district, Odisha, India. J. Earth Syst. Sci. 2024, 133, 111. [Google Scholar] [CrossRef]
  47. Sannier, C.; Ivits, E.; Maucha, G.; Maes, J.; Dijkstra, L. Harmonized Pan-European Time Series for Monitoring Soil Sealing. Land 2024, 13, 1087. [Google Scholar] [CrossRef]
  48. European Environment Agency. Imperviousness Density 2018 (raster 10 m and 100 m), Europe, 3-Yearly. 2018. Available online: https://land.copernicus.eu/en/products/high-resolution-layer-imperviousness/imperviousness-density-2018 (accessed on 28 June 2024).
  49. European Environment Agency. CLC+ Backbone. 2024. Available online: https://land.copernicus.eu/en/products/clc-backbone (accessed on 10 July 2024).
  50. Dynamic World App. Available online: https://dynamicworld.app/ (accessed on 12 July 2024).
  51. European Environment Agency. CLC+ Backbone Product Specification and User Manual, Issue 3. 2022. Available online: https://land.copernicus.eu/en/technical-library/clc-backbone-product-user-manual/@@download/file (accessed on 9 June 2024).
  52. Waterman, R. Global Land Cover Revealed. 2023. Available online: https://www.esri.com/arcgis-blog/products/arcgis-living-atlas/imagery/global-land-cover-revealed/ (accessed on 28 July 2024).
  53. Rwanga, S.; Ndambuki, J. Accuracy Assessment of Land Use/Land Cover Classification Using Remote Sensing and GIS. Int. J. Geosci. 2017, 8, 611–622. [Google Scholar] [CrossRef]
  54. Landis, J.R.; Koch, G.G. A One-Way Components of Variance Model for Categorical Data. Biometrics 1977, 33, 671–679. [Google Scholar] [CrossRef]
  55. The United Nations. The 17 Goals. Available online: https://sdgs.un.org/goals (accessed on 12 July 2024).
Remotesensing 17 00756 g001
Figure 1. Study area—(a). Location of Romania in Eastern Europe; (b). Location of the 12 Metropolitan Areas.
Figure 1. Study area—(a). Location of Romania in Eastern Europe; (b). Location of the 12 Metropolitan Areas.
Remotesensing 17 00756 g001
Remotesensing 17 00756 g002
Figure 2. Flowchart of the accuracy assessment.
Figure 2. Flowchart of the accuracy assessment.
Remotesensing 17 00756 g002
Remotesensing 17 00756 g003
Figure 3. Localization of random validation points for urban and rural areas.
Figure 3. Localization of random validation points for urban and rural areas.
Remotesensing 17 00756 g003
Remotesensing 17 00756 g004
Figure 4. Visual comparison of the four datasets: (a) Cluj MA—urban area; (b) Braşov MA—rural areal; (c) Iaşi MA—rural area; (d) Timişoara—urban and rural area.
Figure 4. Visual comparison of the four datasets: (a) Cluj MA—urban area; (b) Braşov MA—rural areal; (c) Iaşi MA—rural area; (d) Timişoara—urban and rural area.
Remotesensing 17 00756 g004
Remotesensing 17 00756 g005
Figure 5. Overall accuracy values for the four datasets.
Figure 5. Overall accuracy values for the four datasets.
Remotesensing 17 00756 g005
Remotesensing 17 00756 g006
Figure 6. Overall accuracy of the four MAs.
Figure 6. Overall accuracy of the four MAs.
Remotesensing 17 00756 g006
Table 1. Land cover datasets.
Table 1. Land cover datasets.
DatasetResolution for Each YearLevel of UpdatingProvider
High Resolution Layers–Imperviousness20, 100 m (2006, 2009, 2012, 2015)
10, 100 m–2018		3-yearlyEuropean Environment Agency [48]
CLC Backbone10 m (2018, 2021)3-yearlyEuropean Environment Agency [49]
Dynamic World10 m (2015–up to present) Near real-time dataYearlyGoogle and the World Resources Institute [50]
Esri Land Cover10 m (2017–2023)YearlyESRI [11]
Table 2. Interpretation of kappa values.
Table 2. Interpretation of kappa values.
Kappa StatisticsStrength of Agreement
<0.00Poor
0.00–0.20Slight
0.21–0.40Fair
0.41–0.60Moderate
0.61–0.80Substantial
0.81–1.00Almost perfect
Table 3. Confusion matrix at the metropolitan level for CLC Backbone.
Table 3. Confusion matrix at the metropolitan level for CLC Backbone.
CLC BackboneBuilt_UpOtherTotal
Built_up1398147
Other54199253
Total 193207400
Table 4. Accuracy results for the entire urban surface of the 12 MAs.
Table 4. Accuracy results for the entire urban surface of the 12 MAs.
12 MAsDynamic WorldCLC BackboneEsri Land CoverHRL
Overall accuracy0.620.850.670.67
Kappa0.490.50.490.5
User accuracy0.570.950.60.68
Table 5. Accuracy results for the urban localities.
Table 5. Accuracy results for the urban localities.
UrbanDynamic WorldCLC BackboneEsri Land CoverHRL
Overall accuracy0.80.840.810.79
Kappa0.510.50.510.5
User accuracy0.750.90.740.78
Table 6. Accuracy results for the rural localities.
Table 6. Accuracy results for the rural localities.
RuralDynamic WorldCLC BackboneEsri Land CoverHRL
Overall accuracy0.560.870.590.67
Kappa0.470.630.430.59
User accuracy0.360.840.40.4
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ursu, C.-D.; Benedek, J.; Temerdek-Ivan, K. Accuracy Assessment of Four Land Cover Datasets at Urban, Rural and Metropolitan Area Level. Remote Sens. 2025, 17, 756. https://doi.org/10.3390/rs17050756

AMA Style

Ursu C-D, Benedek J, Temerdek-Ivan K. Accuracy Assessment of Four Land Cover Datasets at Urban, Rural and Metropolitan Area Level. Remote Sensing. 2025; 17(5):756. https://doi.org/10.3390/rs17050756

Chicago/Turabian Style

Ursu, Cosmina-Daniela, Jozsef Benedek, and Kinga Temerdek-Ivan. 2025. "Accuracy Assessment of Four Land Cover Datasets at Urban, Rural and Metropolitan Area Level" Remote Sensing 17, no. 5: 756. https://doi.org/10.3390/rs17050756

APA Style

Ursu, C.-D., Benedek, J., & Temerdek-Ivan, K. (2025). Accuracy Assessment of Four Land Cover Datasets at Urban, Rural and Metropolitan Area Level. Remote Sensing, 17(5), 756. https://doi.org/10.3390/rs17050756

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop