Uncertainties in Classification System Conversion and an Analysis of Inconsistencies in Global Land Cover Products

In this study, using the common classification systems of IGBP-17, IGBP-9, IPCC-5 and TC (vegetation, wetlands and others only), we studied spatial and areal inconsistencies in the three most recent multi-resource land cover products in a complex mountain-oasis-desert system and quantitatively discussed the uncertainties in classification system conversion. This is the first study to compare these products based on terrain and to quantitatively study the uncertainties in classification system conversion. The inconsistencies and uncertainties decreased from high to low levels of aggregation (IGBP-17 to TC) and from mountain to desert areas, indicating that the inconsistencies are not only influenced by the level of thematic detail and landscape complexity but also related to the conversion uncertainties. The overall areal inconsistency in the comparison of the FROM-GLC and GlobCover 2009 datasets is the smallest among the three pairs, but the smallest overall spatial inconsistency was observed between the FROM-GLC and MODISLC. The GlobCover 2009 had the largest conversion uncertainties due to mosaic land cover definition, with values up to 23.9%, 9.68% and 0.11% in mountainous, oasis and desert areas, respectively. The FROM-GLC had the smallest inconsistency, with values less than 4.58%, 1.89% and 1.2% in corresponding areas. Because the FROM-GLC dataset uses a hierarchical classification scheme with explicit attribution from the second level to the first, this system is suggested for producers of map land cover products in the future.


Introduction
Land use/cover products are essential input datasets in land surface modelling or climate modelling [1,2].Using high accuracy land cover datasets provides reliable information on carbon, water, and nitrogen processes for further ecology, climate, and hydrology studies [3,4].With the advent of high-resolution imagery and more robust techniques, moderate-resolution remote sensing data sources have emerged in recent years, and the scientific community has witnessed a significant ISPRS Int.J. Geo-Inf.2017, 6, 112 2 of 17 increase in the availability of land cover maps.Land cover products include the International Geosphere-Biosphere Program (IGBP) DIScover (IGBP-DIS) (IGBP-DIS) [5], the University of Maryland (UMD) Land Cover [6], Global Land Cover 2000 (GLC 2000) [7], the Ecosystem Classification and Land Surface Parameters Database (ECOCLIMAP) [8], the Moderate Resolution Imaging Spectroradiometer (MODIS) Land Cover Product (MODISLC) [9], the Global Land Cover Map for 2009 (GlobCover 2009) [10], and the newest Finer Resolution Observation and Monitoring of Global Land Cover (FROM-GLC) [11].The IGBP-DIS and UMD datasets belong to the first generation of 1 km global land cover maps, and are derived from 1981 to 1993 Advanced Very High Resolution Radiometer (AVHRR) data using the IGBP (17 land use types in total) and simplified IGBP (14 land use types in total) classification schemes, respectively [6].The GLC 2000 dataset is the first 1 km global map derived from the Satellite Pour l'Observation de la Terre (SPOT)-4 satellite using the Food and Agriculture Organization of the United Nations (FAO) classification scheme (23 land use types in total) [12].The ECOCLIMAP classification scheme comes from the combination of the IGBP-DIS and UMD land cover types and has a spatial scale of 1 km [13].MODIS provides global land cover with a spatial resolution of 500 m using five types of classification schemes [14].The GlobCover dataset is the first 300 m global land cover map, along with United Nations Land Cover Classification Schemes (UN LCCS) [10], and the open source FROM-GLC dataset is the first 30 m global land cover using Landsat data developed with unique classification scheme based on land cover types from the FAO and IGBP.Despite the diversity of land cover products available, both data producers and users are frustrated with lack of adequate comparison between such products.Because these land cover products use different classification schemes and spatial resolutions, there is difficulty in selecting and comparing these products for a given application.A specific way to compare datasets is to perform a relative comparison of various land cover maps, first reconciling their thematic classification systems into more aggregated categories after resampling the datasets to be into the same spatial resolution [15].Using common classification systems based on the definition of each class in the original land cover products [16,17] or standards in reference to FAO [18], IGBP [12], or other dataset, some previous studies have highlighted general patterns of agreement, inconsistencies and accuracy among different land cover products at global [19,20], continental [18], national [16], and provincial scales [21].Other studies not only demonstrated the compatibility and discrepancies between different datasets, but also qualitatively discussed the impacts of landscape inhomogeneity, thematic resolution, spatial resolution and mis-registration errors on product accuracy [20,22].However, few studies have focused on quantitatively examining the uncertainties of classification system conversion, and examined the inconsistencies in the complex land surface areas or approached the subject from the perspective of the complex landscape features, including mountains, oases and deserts, where the landscape is influenced mainly by natural or artificial factors.
Northwest China, located in an arid region of the central Eurasia, has large and complex mountain-basin landscapes.In relying on limited water resources from mountainous areas, artificial oasis systems with relatively high primary productivity have developed between mountains and basins [23].Thus, a Mountain-Oasis-Desert System (MODS) has formed in this region.Generally, the elevation increases dramatically from a few hundred meters in the basin to over 5000 m above sea level in the mountainous areas over a horizontal distance of less than 200 km.This topography generally develops an extremely heterogeneous vertical zone spectrum of land cover types [24], including snow/ice, alpine meadows, mid-mountain forest/meadows, low-mountain dry grasslands, alluvial basin oases and basin deserts.High accuracy land cover data may be obtained more easily for regions with a single land cover type, such as desert areas.However, low accuracy land cover data may be obtained for mountainous areas in the MODS.Assessing land cover products in different mountainous, oasis and desert areas reveals more detailed drawbacks and benefits and provides a promising perspective for various applications in different fields which study on mountainous or basin areas.
To quantitatively study the uncertainties in classification system conversion and gain an understanding of the discrepancies in the recent land cover products for different areas in a complex MODS, the Heihe River Basin (HRB) was selected as our study area, and the three most recent global land cover products MODISLC (500 m), GlobCover 2009 (300 m), and FROM-GLC (30 m) were compared with each other.Each of them used different classification schemes, supporting the present investigation.To assess the effects of diverse thematic details on the uncertainties and discrepancies in the datasets, we selected a 17-class IGBP classification system (IGBP-17), a 9-class IGBP classification system (IGBP-9), a 5-class IPCC classification system (IPCC-5) and finally at the highest level of aggregation, vegetation, wetlands and others only (TC) as common classification systems.The detailed research steps include: (1) quantitatively analyzing the uncertainty caused by classification system conversion of the three land cover products among IGBP-17, IGBP-9, IPCC-5 and TC; and (2) showing the spatial and areal inconsistencies of the three land cover products in different MODS areas based on the uncertainties.

Study Area
Northwest China contains most of the world's mountain-basin terrain.Due to this special topography, snow and glacier melt and precipitation [25] in mountainous areas are the primary sources of water for lakes and inland basins in mountain-basin systems.This limited water promotes the development of relatively high-productivity oasis systems between the mountains and basins.The HRB is the second-largest inland river basin in China.It ranges between longitude 97.02 • -102 • E, and latitude 37.12 • -42.12 • N. Its approximate area is 143,000 km 2 .It was selected as our study area because it contains the special MODS and is a microcosm of the terrain and climate of mountain-basin landscapes in Northwest China.Based on the elevation differences within these regions and the importance of oases, the HRB's mountain-basin landscapes can be divided into three regions: an upper mountainous area, a middle oasis area, and a lower terminal arid desert area around Ejin Banner (see Figure 1a) [26], with the elevation in these three regions ranging from 5380-2000 m, 1700-1300 m, and 1450-871 m above sea level, respectively (see Figure 1a) [27].There are obvious climatic differences between the upper and lower regions, resulting in diverse landscapes that include glaciers, forests, irrigated crops, and the Gobi desert [28].

Study Area
Northwest China contains most of the world's mountain-basin terrain.Due to this special topography, snow and glacier melt and precipitation [25] in mountainous areas are the primary sources of water for lakes and inland basins in mountain-basin systems.This limited water promotes the development of relatively high-productivity oasis systems between the mountains and basins.The HRB is the second-largest inland river basin in China.It ranges between longitude 97.02°-102° E, and latitude 37.12°-42.12°N. Its approximate area is 143,000 km 2 .It was selected as our study area because it contains the special MODS and is a microcosm of the terrain and climate of mountainbasin landscapes in Northwest China.Based on the elevation differences within these regions and the importance of oases, the HRB's mountain-basin landscapes can be divided into three regions: an upper mountainous area, a middle oasis area, and a lower terminal arid desert area around Ejin Banner (see Figure 1a) [26], with the elevation in these three regions ranging from 5380-2000 m, 1700-1300 m, and 1450-871 m above sea level, respectively (see Figure 1a) [27].There are obvious climatic differences between the upper and lower regions, resulting in diverse landscapes that include glaciers, forests, irrigated crops, and the Gobi desert [28].

Land Cover Products
Three recent global land cover products were selected as datasets to be evaluated in this study: the MODISLC 5.1, GlobCover 2009, and the FROM-GLC datasets.These products are derived from newer satellite images and are validated or assessed by the producers on worldwide scales in reference to Google Earth, Virtual Earth, Yahoo Satellite, and others.The usefulness of these three datasets to different regional investigators has rarely been reported.
The MODISLC is derived from the Moderate Resolution Imaging Spectrora diometer (MODIS) and is produced by Boston University.Collection 5.1 has changed substantially relative to Collection 4, with updated input data, algorithms and ancillary datasets and a spatial resolution of 500 m [9].GlobCover 2009 is the second version of GLOBCOVER project dataset.This dataset has a spatial resolution of 300 m and is derived from the MEdium Resolution Imaging Spectrometer (MERIS) instrument aboard ENVISAT.The FROM-GLC [11] is the first fine scale global map product extracted from Landsat Thematic Mapper (TM) and Enhanced Thematic Mapper Plus (ETM+) data with a spatial resolution of 30 m, covering the years between 2000 and 2010.In this study, the MODISLC dataset was downloaded from Earth Observing System Data and Information System [29], the GlobCover 2009 was downloaded from European Space Agency GlobCover Portal [30], and the FROM-GLC was downloaded from the Center for Earth System Science, Tsinghua University [31]..Detailed characteristics of these three datasets are summarized in Table 1, and the detailed classification schemes are individually listed in Tables 2 and 3.

Methods for Classification System Conversion
Different classification schemes are used by the FROM-GLC, GlobCover 2009 and MODISLC, including a unique classification scheme for FROM-GLC that combines the systems of the FAO and IGBP [11], UN LCCS for GlobCover 2009 and IGBP for MODISLC.However, differences in the classification system are prominent (see Tables 2 and 3).For example, the FROM-GLC is the only product in which no distinction is made between evergreen and deciduous forest classes, while the MODISLC is the only product in which no distinction is made between rained and irrigated crops, and GlobCover 2009 is the only product having a type definition of more than four vegetation type mosaics.To overcome the problem of conflicting classification systems, the thematic classification system of the FROM-GLC, the GlobCover 2009 and the MODISLC were converted into four common classification systems based on the original definition of classes in each land cover product, which defaults to selecting the dominant category [6,12,14,16,[18][19][20].When some categories from the original classification systems could not completely be attributed into any category in the common classification systems due to conflicting definitions, we classified them into corresponding types based on knowledge or by referring to data like the DEM, and labeled them as ambiguous types in this study resulting in uncertainties during classification system conversion (see Section 3.2).The four common classification systems used were IGBP-17, IGBP-9, IPCC-5 and TC.The detailed categories of common classification systems and corresponding relationships can be seen in Table 4.

Uncertainty during the Classification System Conversion
When the classification schemes of the FROM-GLC, GlobCover 2009 and MODISLC datasets were converted into the four common classification systems, some categories from the original classification systems could not completely be attributed into any category in the common classification systems due to differences in class definitions.We labeled these categories as ambiguous types in this study, resulting in uncertainties during classification system conversion (* terms in Table 4).We summarized four main ambiguous types during classification system conversion, including (1) no dominant type; (2) different percentage of the dominant type; (3) the type definition broader than the corresponding type in the common classification system; and (4) labeling errors.Details can be seen below.
Ambiguous type 1: there is no dominant type according to the definition of the land cover type.For example, the class 110 (mosaic forest or shrubland (50%-70%) and grassland (20%-50%)) dominant type cannot be distinguished between forest and shrubland, resulting in attribution difficulty in the IGBP-17 common classification system.Ambiguous type 2: although it contains the dominant class according to the definition of the original classification system, the percentage of this type is different from that of the corresponding class in common classification system.For example, class 20 in the GlobCover 2009 (mosaic cropland (50%-70%) and vegetation (grassland/shrubland/forest) (20%-50%)) is dominated by cropland, which can only be converted into class 15 (cropland/natural vegetation mosaic) in the IGBP-17 common classification system according to the definition of each class, but the percentage of the land cover types between the original and the common classification system is different.The crop type accounts for 50%-70% and the percentage of other types are 20%-50% for GlobCover 2009 class 20, but the percentages of both crop and vegetation of the class 15 in the IGBP-17 common classification system do not exceed 60%.
Ambiguous type 3: the type definition in the original classification system is coarser than the corresponding type in common classification system.For example, the class 21 (broadleaf) and ISPRS Int.J. Geo-Inf.2017, 6, 112 9 of 17 22 (needleleaf) in the original classification system of the FROM-GLC are also difficult to place into either class 2 (evergreen needleleaf forest), 3 (evergreen broadleaf forest), 4 (deciduous needleleaf forest) or 5 (deciduous broadleaf forest) in the IGBP-17 common classification system, since there is no distinction between evergreen and deciduous forest types in the FROM-GLC system.
Ambiguous type 4: the type in the original classification system is labeled incorrectly due to a classification error.For example, classes 9 and 10 (Woody Savannas and Savannas) in the original classification system of MODISLC are classification errors because these two types are rare in the HRB and they could be either mosaic grassland/forest or shrubland.Classification system conversion must be performed on the basis of local experts' experience and knowledge when ambiguous types appear.
Uncertainties of classification system conversion caused by ambiguous types in the four common classification systems were quantitatively calculated with the following formula: where U = the uncertainty ratio caused by classification system conversion due to ambiguous types, N j = the total number of pixels of ambiguous types, n = the number of land cover types in the common classification system, N i = the total number of pixels of one type in the common classification system, and ∑ n i N i = the total number of pixels of all land cover types.

The Method for Assessing Areal and Spatial Inconsistency
Areal and spatial inconsistencies were explored using pixel-by-pixel comparisons between the FROM-GLC, GlobCover 2009, and MODISLC in the common classification systems.
Areal Inconsistency of each Class (AIC) and Overall Areal Inconsistency (OAI) in four common classification systems were computed with the following formulas [12]: where n = the total number of land use types in the common classification systems, X i = total area percentage of land use type i in one of the FROM-GLC, the GlobCover 2009 and the MODIS_IGBP, Y i = total area percentage of class i in one of other three land cover products, and OAI = overall areal inconsistency in the common classification systems.The first step for obtaining the pairwise spatial inconsistencies between the FROM-GLC (30 m), the GlobCover 2009 (300 m), and MODISLC (500 m) datasets using the four common classification systems level involved up-scaling higher spatial resolution land cover into the corresponding dataset's lower spatial resolution.A pixel in a low spatial resolution usually represents only one type of land use type, whereas the corresponding high spatial resolution pixel includes more than one land use type.In this study, a low spatial resolution pixel was considered to be 100% correct when it agreed with the dominant type of the corresponding high spatial resolution pixels and was considered to be 0% correct when it disagreed.Majority filtering technology was used to upscale the high spatial resolution land cover into lower resolutions.The Overall Spatial Inconsistency (OSI) between a given pair of these three land cover products was calculated according to the formula below [12]: where N (i =j) = the number of pixels for which the type is different from another one at the same location when compared to different datasets (either FROM-GLC, GlobCover 2009, or MODISLC), and N = the total number of pixels.

Areal Inconsistency
The areal inconsistencies of each land use type from pairwise comparisons of the FROM-GLC, GlobCover 2009 and MODISLC datasets using four common classification systems are shown in Table 5.During the classification system conversion to IGBP-17, the areal inconsistencies were mainly present in grassland (up to 38.31%), cropland (up to 13.58%) and barren or sparsely vegetated (up to 43%) classes.During the classification system conversion to both IGBP-9 and IPCC-5, the areal inconsistencies were mainly caused by grassland (up to 30.98% and 39.61%, respectively) and others (up to 22.88% and 41.88%, respectively) (see italicized and bold words in Table 5).During the classification system conversion to TC system, the areal inconsistencies were mainly caused by vegetation (up to 40.21%) and others (up to 41.88%).These land cover types were the dominant types in our study area, illustrating that the different percentages of dominant types among land cover products can greatly influence areal inconsistencies.In addition, the areal inconsistency of croplands (17.89% in mountainous area) between the FROM-GLC and the GlobCover 2009 using IGBP-9 is slightly higher than the inconsistent result (15.36%) from paper [21], but the areal inconsistencies for croplands (4.13% and 0.23% in oasis areas and desert areas, respectively) are far less than the result in the paper mentioned above.Overall areal inconsistencies between pairs of the FROM-GLC, GlobCover 2009 and MODISLC in four common classification systems are shown in the Figure 2. Areal inconsistencies decreased with the increasing level of aggregation of the classification system, from IGBP-17 to TC.
The largest areal inconsistency appeared in mountainous areas, and the smallest areal inconsistency was found in the desert area, indicating that landscape complexity is an important factor influencing areal inconsistency.Assessing or comparing land cover products from the perspective of the terrain provides a better reference than assessments performed from other viewpoints for different applications, such as administrative division considerations.For example, the accuracy of land cover products in mountainous areas is more important for hydrology modeling in MODS in arid areas because snow and glacier melt and precipitation in mountainous areas are the primary sources of water for lakes and inland basins in mountain-basin systems.
ISPRS Int.J. Geo-Inf.2017, 6, x FOR PEER REVIEW 11 of 17 Overall areal inconsistencies between pairs of the FROM-GLC, GlobCover 2009 and MODISLC in four common classification systems are shown in the Figure 2. Areal inconsistencies decreased with the increasing level of aggregation of the classification system, from IGBP-17 to TC.
The largest areal inconsistency appeared in mountainous areas, and the smallest areal inconsistency was found in the desert area, indicating that landscape complexity is an important factor influencing areal inconsistency.Assessing or comparing land cover products from the perspective of the terrain provides a better reference than assessments performed from other viewpoints for different applications, such as administrative division considerations.For example, the accuracy of land cover products in mountainous areas is more important for hydrology modeling in MODS in arid areas because snow and glacier melt and precipitation in mountainous areas are the primary sources of water for lakes and inland basins in mountain-basin systems.

Spatial Inconsistencies
The distribution and overall spatial inconsistencies in the pairwise comparisons of the FROM-GLC, Globcover 2009 and MODISLC datasets are shown in Figure 3.The overall spatial inconsistencies between the FROM-GLC and the MODISLC datasets are 18.57%, 18.05%, 17.44%, and 14.95% using the aggregation of common classification systems IGBP-17, IGBP-9, IPCC-5 and TC, respectively.The spatial inconsistencies between the FROM-GLC and GlobCover 2009 are 22.16%, 18.46%, 18.31%, and 12.52% using the four common classification systems, and those between the GlobCover 2009 and MODISLC datasets are 23.13%,18.13%, 18.03%, and 11.93%, respectively.The spatial inconsistencies decreased with the increasing level of aggregation of common classification systems from IGBP-17 to TC.

Spatial Inconsistencies
The distribution and overall spatial inconsistencies in the pairwise comparisons of the FROM-GLC, Globcover 2009 and MODISLC datasets are shown in Figure 3.The overall spatial inconsistencies between the FROM-GLC and the MODISLC datasets are 18.57%, 18.05%, 17.44%, and 14.95% using the aggregation of common classification systems IGBP-17, IGBP-9, IPCC-5 and TC, respectively.The spatial inconsistencies between the FROM-GLC and GlobCover 2009 are 22.16%, 18.46%, 18.31%, and 12.52% using the four common classification systems, and those between the GlobCover 2009 and MODISLC datasets are 23.13%,18.13%, 18.03%, and 11.93%, respectively.The spatial inconsistencies decreased with the increasing level of aggregation of common classification systems from IGBP-17 to TC.
According to Figure 3, the spatial inconsistencies in the pairwise comparison of the FROM-GLC, Globcover 2009 and the MODISCLC decreased from mountainous to desert areas.The spatial inconsistencies among the three land cover products mainly appear at the northwest in mountainous areas and mainly occur to the southeast in oasis areas because there was substantial land cover type between crop and grass in these regions.According to Figure 3, the spatial inconsistencies in the pairwise comparison of the FROM-GLC, Globcover 2009 and the MODISCLC decreased from mountainous to desert areas.The spatial inconsistencies among the three land cover products mainly appear at the northwest in mountainous areas and mainly occur to the southeast in oasis areas because there was substantial land cover type between crop and grass in these regions.

Uncertainties in Classification System Conversion
Since the original classification system of MODISLC is the IGBP classification scheme, during classification system conversion from the original classification scheme into the IGBP-17 common classification system, the classification system conversion uncertainty was zero.However, classes 9 and 10 (see Table 2) in the original classification system (mentioned in Section 3.2) were ambiguous

Uncertainties in Classification System Conversion
Since the original classification system of MODISLC is the IGBP classification scheme, during classification system conversion from the original classification scheme into the IGBP-17 common classification system, the classification system conversion uncertainty was zero.However, classes 9 and 10 (see Table 2) in the original classification system (mentioned in Section 3.2) were ambiguous type 4 due to labeling errors, and class 15 (see Table 2) was ambiguous type 1 because there was no dominant type according to the definition, resulting in classification system conversion uncertainties using the IGBP-9 and IPCC-5 common classification systems.There were no classification system conversion uncertainties using TC common classification system.During the classification system conversion process for the GlobCover 2009 from the original classification scheme into the four common classification systems, class 20 (see Table 2), mentioned in Section 3.2, was ambiguous type 2 due to the percentage difference between the class in the original classification system and that in the IGBP-17 common classification system; classes 30 and 110 (see Table 2) were ambiguous type 1 because there was no dominant type; and class 130 (see Table 2) was also ambiguous type 3 due to a more coarse class definition.The uncertainties caused by ambiguous types of classes 20 and 130 were eliminated using IGBP-9 common classification system, and that of class 30 was also eliminated using the IPCC-5 common classification system; however, that of class 110 was not eliminated until the TC common classification system was used.
During the classification system conversion process for the FROM-GLC, classes 21, 22 and 40 (see Table 3) were ambiguous type 3 due to a coarser type definition in the IGBP-17 common classification system.Since the FROM-GLC uses a hierarchical classification scheme and had explicit relationships between the first level and the second, the uncertainties caused by ambiguous types were all eliminated when using the IGBP-9, IPCC-5 and TC common classification systems.All ambiguous types are denoted by * words in Table 4.The quantitative uncertainties in the classification system conversion of the FROM-GLC, the GlobCover 2009 and MODISLC to the four common classification systems were shown in Figure 4.
ISPRS Int.J. Geo-Inf.2017, 6, x FOR PEER REVIEW 13 of 17 type 4 due to labeling errors, and class 15 (see Table 2) was ambiguous type 1 because there was no dominant type according to the definition, resulting in classification system conversion uncertainties using the IGBP-9 and IPCC-5 common classification systems.There were no classification system conversion uncertainties using TC common classification system.During the classification system conversion process for the GlobCover 2009 from the original classification scheme into the four common classification systems, class 20 (see Table 2), mentioned in Section 3.2, was ambiguous type 2 due to the percentage difference between the class in the original classification system and that in the IGBP-17 common classification system; classes 30 and 110 (see Table 2) were ambiguous type 1 because there was no dominant type; and class 130 (see Table 2) was also ambiguous type 3 due to a more coarse class definition.The uncertainties caused by ambiguous types of classes 20 and 130 were eliminated using IGBP-9 common classification system, and that of class 30 was also eliminated using the IPCC-5 common classification system; however, that of class 110 was not eliminated until the TC common classification system was used.
During the classification system conversion process for the FROM-GLC, classes 21, 22 and 40 (see Table 3) were ambiguous type 3 due to a coarser type definition in the IGBP-17 common classification system.Since the FROM-GLC uses a hierarchical classification scheme and had explicit relationships between the first level and the second, the uncertainties caused by ambiguous types were all eliminated when using the IGBP-9, IPCC-5 and TC common classification systems.All ambiguous types are denoted by * words in Table 4.The quantitative uncertainties in the classification system conversion of the FROM-GLC, the GlobCover 2009 and MODISLC to the four common classification systems were shown in Figure 4.During the classification system conversion to IGBP-17, IGBP-9 and IPCC-5, the GlobCover 2009 dataset had the relative largest uncertainties due to too much class definition using mosaic, and the FROM-GLC dataset had the smallest inconsistency of the three systems due to its clear classification scheme (see Figure 4).In mountainous areas, the classification system conversion uncertainties of the GlobCover 2009 were 23.9%, 16.47% and 16.2% using the three common classification systems, those of the MODISLC were 0%, 3.08% and 3.08%, and the uncertainties in the FROM_GLC dataset conversion were 4.58%, 0% and 0%.In oasis areas, the classification system conversion uncertainties of the GlobCover 2009 dataset using the three common classification systems were 9.68%, 5.57% and 5.56%, those of the MODISLC dataset were 0%, 5.17% and 5.17%, and the uncertainties of FROM_GLC dataset were 1.89%, 0% and 0%.In desert areas, the classification system conversion uncertainties of the GlobCover 2009 dataset using the three common classification systems were 0.11%, 0.05% and 0.05%, those of the MODISLC dataset were 0%, 0.34% and 0.34%, and the uncertainties of FROM_GLC were 1.2%, 0% and 0%.The reason why the FROM-GLC had the smallest classification system conversion uncertainties among the three datasets is that it uses the hierarchical classification scheme, in which explicit attribution of land cover types is implied from the second level to the first, reducing the uncertainties.Therefore, we suggest that hierarchical classification schemes should be used in the production of landcover mapping.During the classification system conversion to IGBP-17, IGBP-9 and IPCC-5, the GlobCover 2009 dataset had the relative largest uncertainties due to too much class definition using mosaic, and the FROM-GLC dataset had the smallest inconsistency of the three systems due to its clear classification scheme (see Figure 4).In mountainous areas, the classification system conversion uncertainties of the GlobCover 2009 were 23.9%, 16.47% and 16.2% using the three common classification systems, those of the MODISLC were 0%, 3.08% and 3.08%, and the uncertainties in the FROM_GLC dataset conversion were 4.58%, 0% and 0%.In oasis areas, the classification system conversion uncertainties of the GlobCover 2009 dataset using the three common classification systems were 9.68%, 5.57% and 5.56%, those of the MODISLC dataset were 0%, 5.17% and 5.17%, and the uncertainties of FROM_GLC dataset were 1.89%, 0% and 0%.In desert areas, the classification system conversion uncertainties of the GlobCover 2009 dataset using the three common classification systems were 0.11%, 0.05% and 0.05%, those of the MODISLC dataset were 0%, 0.34% and 0.34%, and the uncertainties of FROM_GLC were 1.2%, 0% and 0%.The reason why the FROM-GLC had the smallest classification system conversion uncertainties among the three datasets is that it uses the hierarchical classification scheme, in which explicit attribution of land cover types is implied from the second level to the first, reducing the uncertainties.Therefore, we suggest that hierarchical classification schemes should be used in the production of landcover mapping.
The uncertainties in classification system conversion decreased with the level of the thematic detail, from the IGBP-17 to TC common classification systems.There were no uncertainties caused by classification system conversion using the TC common classification system for the three products (see Figure 4).Those common classification systems using both too fine a class definition and too detailed thematic classes are not suitable as common classification schemes for comparing land cover products with different classification systems.There are great uncertainties in classification system conversion (up to 23.9% in this study) caused by ambiguous types, which result in unreliable indirect validation accuracy and inconsistency.A uniform international classification scheme could be a solution for the high classification system conversion uncertainties caused by ambiguous types.
The uncertainties in classification system conversion also decreased from mountainous to desert areas.The largest uncertainties were up to 23.9% in mountainous areas, 9.7% in oasis areas and 1.2% in the desert areas.This result was because mountainous areas had more land cover types than desert and more complex vegetation distribution patterns than artificial oasis areas.

Discussion and Importance of the Study
Previous comparisons of land cover products have been made only at global [19,20], continental [18], national [16], or provincial scales [21], since they focused on general patterns of inconsistencies or indirect validation accuracy of the products, which is meaningful to large scale studies.To the knowledge of the authors, this is the first study to compare the inconsistency of three recent land cover products (the MODISLC, GlobCover 2009, and FROM-GLC) from a terrain perspective in a complex mountain-oasis-desert area.We found that the overall areal inconsistency of the FROM-GLC and GlobCover 2009 is relatively small among the pairs of the three products, but the overall spatial inconsistency between the FROM-GLC and MODISLC is relatively small.The result is consistent with the conclusions in similar studies [12,[16][17][18].The areal inconsistencies between the MODISLC and the GlobCover 2009 in this study were up to 52% and 25% in mountainous areas and oasis areas, respectively.The result in oasis areas is lower than the value as reported in Ref. [16] (45.27%), but the result in mountainous areas is higher than the value reported in that paper.The location of these differences and the relative quality of the maps are helpful information for end users of these products and provide more information than non-terrain based approaches like administrative division for different applications.For example, regional hydrological modeling researchers in arid areas may wish to select MODISLC or GlobCover 2009 as their base dataset, between which there are fewer spatial inconsistence in mountainous areas because snow and glacier melt and precipitation in mountainous areas are the primary sources of water for lakes and inland basins in the mountain-basin systems.However, ecologists researching oasis evolution may use the FROM-GLC or Globcover 2009 dataset as base data due to the relatively good quality of oasis areas in these products.This research may also be helpful in providing training areas for the producer of land cover products.
Existing comparisons have qualitatively discussed the impacts of landscape inhomogeneity, thematic resolution, spatial resolution and mis-registration errors on indirect validation accuracy [20,22].However, we quantitatively highlighted the uncertainties in classification system conversion in this study.Although we also recognized that a number of external factors (like map projections, resolution unifications and mis-registration) are also the sources of the uncertainties and discrepancies among the three products, they are not the focus of this paper and may be further explored in the future study.We found that the uncertainties in classification system conversion for the MODISLC were mainly from label error, those for the Globcover 2009 were the largest and mainly from class definition using mosaics, like (1) no dominant type; (2) different percentage of the dominant type; and (3) the class definition being coarser than the corresponding type in the common classification system.The FROM-GLC had the smallest uncertainties due to the explicit relationships between different classification levels.For studies at scales of 300 m, 500 m, 1 km, and even larger, the class definitions of the Globcover 2009 may be acceptable and suitable as the basis data for ecological modeling [32] aimed at calculating carbon and water based on the sub-pixel ratio of various vegetation functional types in order to save computational consumption, but mixed classes lack clear definitions and their more or less arbitrary percentage thresholds pose significant challenges for users.

Conclusions
Using the widely used classification systems of IGBP-17, IGBP-9, IPCC-5 and TC, we studied the spatial and areal inconsistencies in the most recent global land cover products including the FROM-GLC, GlobCover 2009 and MODISLC datasets, and quantitatively discussed the uncertainties in classification system conversion.
The areal and spatial inconsistencies and uncertainties in classification system conversion decreased with the decreasing of thematic detail in the classification scheme, from IGBP-17 to TC, and with elevation from mountainous to desert areas.This indicates that the assessment of areal and spatial inconsistencies is influenced not only by the thematic detail of the common classification systems and landscape complexity but also by uncertainties in classification system conversion.It is also worth noting that, for the users, a given land cover type occurring in mountainous areas might show a substantially different physiological response from that located in oasis or desert areas.Therefore, comparing the areal and spatial inconsistencies of land cover products from the perspective of the terrain provides more value than analysis based on other viewpoints for different applications, such as administrative division considerations.Uncertainties in classification system conversion using the common classification systems are inevitable and unsolvable when comparing land cover products using different classification schemes.During a classification system conversion process using four common classification systems, we summarized four ambiguous types resulting in uncertainties.The FROM-GLC dataset had the fewest uncertainties of the three products during the classification system conversion to IGBP-17, IGBP-9 and IPCC-5 classification schemes because this dataset uses a hierarchical classification scheme in which explicit attribution is implied from the second level to the first, reducing the uncertainties.Therefore, we suggest that hierarchical classification schemes be used by the producers of land cover mapping.Only by using hierarchical classification schemes can the uncertainties of classification system conversion be reduced as much as possible.
ISPRS Int.J. Geo-Inf.2016, 6, 112 3 of 17 land cover products MODISLC (500 m), GlobCover 2009 (300 m), and FROM-GLC (30 m) were compared with each other.Each of them used different classification schemes, supporting the present investigation.To assess the effects of diverse thematic details on the uncertainties and discrepancies in the datasets, we selected a 17-class IGBP classification system (IGBP-17), a 9-class IGBP classification system (IGBP-9), a 5-class IPCC classification system (IPCC-5) and finally at the highest level of aggregation, vegetation, wetlands and others only (TC) as common classification systems.The detailed research steps include: (1) quantitatively analyzing the uncertainty caused by classification system conversion of the three land cover products among IGBP-17, IGBP-9, IPCC-5 and TC; and (2) showing the spatial and areal inconsistencies of the three land cover products in different MODS areas based on the uncertainties.

Figure 1 .
Figure 1.(a) DEM; and (b) land cover maps of the study area (GlobCover 2009 using the IGBP-17 classification system as an example).

Figure 1 .
Figure 1.(a) DEM; and (b) land cover maps of the study area (GlobCover 2009 using the IGBP-17 classification system as an example).

Figure 2 .
Figure 2. Overall areal inconsistencies between pairwise of the FROM-GLC, GlobCover 2009 and MODISLC datasets in four common classification systems (F-M: areal inconsistencies between the FROM-GLC and MODISLC, F-G: areal inconsistencies between the FROM-GLC and GlobCover 2009; G-M: areal inconsistencies between the GlobCover 2009 and MODISLC).

Figure 2 .
Figure 2. Overall areal inconsistencies between pairwise of the FROM-GLC, GlobCover 2009 and MODISLC datasets in four common classification systems (F-M: areal inconsistencies between the FROM-GLC and MODISLC, F-G: areal inconsistencies between the FROM-GLC and GlobCover 2009; G-M: areal inconsistencies between the GlobCover 2009 and MODISLC).

Figure 3 .
Figure 3. Overall spatial inconsistencies (OSI) and the distribution thereof in mountain-oasis-desert system in the pairwise comparison of the FROM-GLC, GlobCover 2009 and MODISLC datasets in the four common classification systems (F-M: areal inconsistencies in the comparison between the FROM-GLC and MODISLC datasets; F-G: areal inconsistencies between the FROM-GLC and GlobCover 2009 datasets; G-M: areal inconsistencies in the comparison between the GlobCover 2009 and MODISLC datasets).

Figure 3 .
Figure 3. Overall spatial inconsistencies (OSI) and the distribution thereof in mountain-oasis-desert system in the pairwise comparison of the FROM-GLC, GlobCover 2009 and MODISLC datasets in the four common classification systems (F-M: areal inconsistencies in the comparison between the FROM-GLC and MODISLC datasets; F-G: areal inconsistencies between the FROM-GLC and GlobCover 2009 datasets; G-M: areal inconsistencies in the comparison between the GlobCover 2009 and MODISLC datasets).

Figure 4 .
Figure 4.The quantitative uncertainties of classification system conversion from the FROM-GLC, GlobCover 2009 and MODISLC datasets using four common classification systems.

Figure 4 .
Figure 4.The quantitative uncertainties of classification system conversion from the FROM-GLC, GlobCover 2009 and MODISLC datasets using four common classification systems.

Table 1 .
Characteristics of the land cover products used in this study.

Table 2 .
Classification schemes of the GlobCover 2009 and the MODISLC.
* Land cover categories not present in the case study area.

Table 3 .
Classification scheme from the FROM-GLC.

Table 4 .
Lookup table for converting classification schemes of the GlobCover 2009, MODISLC, and FROM-GLC datasets into the four common classification systems.
* Uncertain types when conversion was performed.

Table 5 .
Areal inconsistencies for each land use type in four common classification systems in pairwise comparisons of the FROM-GLC, GlobCover 2009, and MODISLC datasets.