Comparison of Land Cover Categorical Data Stored in OSM and Authoritative Topographic Data

Abstract

This study aims at a comparative analysis of quantitative data, namely, OSM and BDOT10k. Analyses were conducted in a 1 km² hexagonal grid, in seven test counties located in different regions of Poland, differing in the degree of urbanization, land cover and natural environment. It is assumed that the authors’ consolidated regional classification of the Compound Correspondence Index CCI_Rn is attributed to the geometric mapping unit based on TOPSIS values, and their statistical measure of dispersion enables the comparison of datasets for individual geographically disjointed areas according to uniform criteria, e.g., the number of topographic features stored in analyzed datasets, both polygonal (buildings, forests, surface water) and linear (roads, watercourses, railroads). The final results of the regional assessment outperform the local classification giving a higher level of data compliance. Overestimation of regional concordance ranges from 9 to 20% of the county area, with an average of 3% reduction in the area where the two datasets (BDOT10k and OSM) have comparable information ranges. Areas of medium and high nonconformity are decreased by an average of 2.4%.

1. Introduction

Information is a key asset in the 21st century, referred to as the information age. The information age is, in turn, inextricably linked to the information society, a society based on information and knowledge. Furthermore, geospatial (geographical) information (GI) plays, as noted by [1,2], a crucial role among available information, as it is the basis for 60–80% of decisions made by public administrations [3]. In the third decade of the 21st century, geospatial information forms a foundation of the information-based society. Geographic information and GI technology are present in almost every sector of science, the economy and industry. Nowadays, in addition to traditional maps and databases, they offer a plethora of web applications that can solve real-world problems. Furthermore, these analyses are presented in a form understandable to the end user and the consumer of information, and add value to the economy, science and society.

The large amount and variety of data not only create new challenges in effective management and analysis, but also create opportunities to explore the potential value of data. Data, including geospatial data, are not error-free, and the results of their analysis are burdened with uncertainty. Data entry errors lead to inaccurate geospatial data, which in turn has wide implications and, as discussed by Bielecka and Burek [4], significantly affects the results of the final analysis, and can lead to wrong decisions and financial losses. Therefore, the GI society is tirelessly striving to improve the reliability of geospatial data, proposing methods for assessing quality and reliability and testing the suitability of data for a specific task (their fitness for purpose). Data quality assessment is particularly important in the context of selecting a dataset that best meets the user’s needs. The selection of fit-for-purpose data is usually supported by an analysis of their quality or completeness, often understood as information capacity. The analysis is carried out mainly by comparison with reference data considered to be very reliable. The motivation for this study is therefore to develop a composite measure that allows a comparative assessment of the completeness of two datasets. Contrary to studies conducted so far, described in Section 2, the assessment of completeness refers to the basic analytical unit, facilitating a detailed assessment of completeness in particular geographical locations. The comprehensive methodology developed is based on a compensatory comparative analysis (TOPSIS), not previously used in categorical data evaluation, using the linear ranking of the authoritative Compound Correspondence Index (CCI) and statistical measures of dispersion for its in explicite spatial visualization. Therefore, this study aims to compare the volume of information (capacity) of two open spatial datasets: (1) Database of Topographic Objects (BDOT10k), made available on an open basis by the Head Office of Geodesy and Cartography, and (2) OpenStreetMap, created by volunteers from all over the world. This methodology contributes to both academics and practitioners by helping a user select the best (fit-for-purpose) spatial dataset available for the task at hand. The decision behind the choice of topographic data, both authoritative and voluntary, lies in their wide use in a plethora of applications, e.g., environmental analysis [5,6], protecting and validating landscapes [7,8], and sustainable rural and urban planning [9,10]. The elaborated methodological framework is intended to facilitate cartographic modeling in the analysis of selecting the best geospatial dataset, namely, the data that are most fit for purpose.

2. Related Works

2.1. Categorical Data Comparison—Literature Review

The literature on geospatial data quality is extremely rich, and those on the implementation of machine learning solutions as part of data quality improvement strategies are gaining popularity. This brief review will thus cover only one aspect, namely, the comparative analysis of maps and datasets containing categorical data. Categorical data typically result from mapping, classification, or modeling; hence most of the literature in the field of qualitative data comparison deals with land cover/land use or landscape data [11]. Differences between categorical maps can be characterized and measured in a variety of ways [12], from descriptive or inferential statistics to advanced data mining. However, two approaches dominate among comparative studies of categorical data, the first based on a cross-tabulation matrix to summarize the degree of data association, and the second using spatial and statistical analysis to determine data comparisons in locational and qualitative dimensions.

The Cohen kappa coefficient of agreement derived from a cross-tabulation matrix, i.e., the relative rating of two or more classifications based on the proportion of correctly allocated cases, dominates most of the literature [13,14]. The Kappa coefficient can also be used with missing data, as pointed out by De Raadt et al. [15]. The authors analyzed three kappa coefficients: Gwet’s kappa, regular kappa, and listwise deletion kappa. They found that both Gwet’s kappa and listwise deletion kappa outperform regular category kappa in terms of bias, and generally have a very small mean squared error. It is even possible to use weighted kappa, introduced by Cohen in 1968 [16], which is of the utmost importance when disagreements between datasets are not equally important. Vanbelle and Albert [17] remarked that, under certain conditions, “the weighted kappa coefficient is equivalent to the product-moment correlation coefficient”. In 2000, Pontius [18] introduced some extension of kappa, namely, kappa with random change agreement and kappa for location (Klocation).

The kappa coefficient, however, has some disadvantages. Foody [13] pointed out that the sample used to evaluate maps should be non-dependent. However, in practice, this assumption is almost impossible, since the same sample of ground data sites is often used for each case of map elaboration. In conclusion, in [13], the author also suggests using a measure of the proportion of correctly assigned cases. A similar opinion is shared by Pontius and Millones [18], who summed up more than a decade of research on the kappa coefficient and concluded that the simple measures of quantitative and allocation disagreement are even more useful in showing differences in categorical data. Moreover, Pontius [19] also suggests that in data agreement analysis, statistical measures of dispersion such as mean deviation, mean absolute deviation and correlation and slope are even more helpful in interpreting data dissimilarity than indices of agreement.

In view of the aforementioned limitations of kappa, researchers use some qualitative–quantitative approaches that deserve attention due to their combination of evaluation and analysis. Multi-criteria analytical techniques predominantly rely on statistical methods or a combination with data mining techniques. Li and Reynolds [20] quantified spatial heterogeneity in categorical maps using ANOVA statistics. Among scientists, a very popular method for comparing categorical data is clustering. However, as noted by Lex et al. [21], the clustering of multi-dimension data can conceal some important relations between object classes, and the final results strongly depend on the algorithm used. Promising results were obtained by Hagen [22], using the fuzzy set theory, especially for ambiguities in determining the “location of the category (fuzziness of location) and in the definition of the category (fuzziness of category)”. The simultaneous analysis of location and quantity was also the subject of research by Pontius and Suedmeyer [23], who developed a new technique of budgeting agreement and disagreement between two categorical maps. Their methodology also includes stratification, hard and soft classification and multiple resolutions to compare maps by quantity and location. Wabiński et al. [24] adapted the structural information measure, introduced by the Russian cartographer Salistchev [25], to compare tactile thematic maps, which extended the use of comparative measures to maps developed for the needs of visually impaired people. Comber et al. [26] recall that different methods of data comparison require many disparate processing steps, and may lead to different results and conclusions. This assessment was also supported by Boots and Csillag [27], based on the results of an expert workshop conducted in March 2004.

2.2. OpenStreetMap Data Quality

With the advent of OSM, due to the lack of control over the VGI data that characterize most user-provided web resources, many questions have been raised about the quality and reliability of the data. However, as summarized in Arsjani et al.’s [28] study on OpenStreetMap, in some regions, OSM geodata are more complete and geometrically and semantically more accurate than the corresponding proprietary datasets. Haklay [29] was the first who compared the quality of OpenStreetMap data with the Meridian2 data maintained by the Ordnance Survey (OS). The research results published in the fourth year of the project indicate the comparable quality of OS and OSM in terms of accuracy and completeness. The most comprehensive assessment of OSM quality was presented by Girres and Touya [30], providing such elements of spatial data quality as geometric, attribute, semantic and temporal accuracy, logical consistency, completeness, lineage and usage. The outcome of their research shows heterogeneous quality across regions and land use elements, indicating relatively high positioning accuracy and very different completeness depending on the density of OSM volunteers. Mondzech and Sester [31] analyzed OpenStreetMap, focusing on determining the optimal routes for pedestrian traffic. Nevertheless, as demonstrated by Ciepluch et al. [32] and Zielstra and Zipf [33], other topographic data with which OSM data are compared are not always more accurate.

Contrary to the extensive analysis of buildings and roads in OSM [31,34], the first attempt at analyzing land use features was carried out by Hagenauer and Helbich [35]. The authors compared the land use polygons and land use patterns of OSM and Urban Atlas data, and found that the location agreements expressed by kappa were 91, 79 and 76% across the three classification levels, while the attributes of both datasets matched at 81, 67 and 65%. Significantly worse completeness and accuracy results were observed by Zhou et al. [36] when comparing OSM land cover with global CCI-LC (Climate Change Initiative Land Cover) data in 168 countries. In 129 countries, completeness did not exceed 40%, and in only 17 European countries was it higher than 60%. Much better results were obtained for accuracy, which was higher than 60% in 149 countries.

3. Materials and Methods

3.1. Study Area and Data Used

The study covers the territory of seven Polish counties (an administrative unit corresponding to the European Territorial Units for Statistics NUTS 4), which are located in different physical–geographical mesoregions and reflect the diversity of both the natural and anthropogenic environment, making them representative (

Table 1. General characteristics of the counties under consideration.

Description	Piaseczno	Sokólski	Sanocki	Słupski	Ostrowski	Otwocki	Międzyrzecki
Geographical Subprovinces 1	Central Polish Lowlands	Podlasie- Bialystok Upland	Eastern Beskids	South Baltic Coast	Central Polish Lowlands	Central Polish Lowlands	Greater Poland Lake District
Area (km2) 2	621.12	2054.34	1223.62	2347.59	1159.92	616.46	1387.61
People	190,607	64,902	92,900	98,761	161,581	124,283	57,100
Population density	311	32	81	43	139	202	41.5
Number of cities	4	4	2	2	2	3	3
Urbanization level (%)	47.8	41.7	47.2	20.7	53.7	61.8	52.3
Land use 2 (km2): Built-up and artificial	82.51	72.93	43.65	85.15	55.62	58.91	24.08
Forest	132.88	547.91	586.67	864.13	347.59	250.15	735.43
Agriculture	387.37	1426.28	512.14	1234.97	728.53	276.51	513.42
Water bodies	16.44	6.71	13.60	110.65	13.31	11.14	38.39
Protected area	Chojnów Landscape Park, protected landscape areas	Knyszyn Forest, Biebrza National Park	Słonne Mountains Landscape Park, protected landscape areas	Słowiński National Park	Landscape Park Dolina Baryczy, protected landscape area	Masovian Landscape Park, Landscape Park Dolina Środkowego Świdra	Notecka Forest, Pszczewska Landscape Park

¹ Solon et al. [39]; ² Based on data from the National Register of Boundaries (PRG); Based on Cadastral data 2021 from geoportal.gov.pl.

). Due to various threats, including the violation of state borders’ integrity, they are of strategic importance for security. The area of interest (see Figure 1) covers 3.1% of Poland. Piaseczno and Otwocki counties are located in the central part of Poland, adjacent to the capital city of Warsaw, Ostrowski, in Great Poland. The next two study areas are situated along the eastern border of Poland, namely, Sokólski (in the north) along the border with Belarus, and Sanocki (in the south) on the Polish–Ukrainian and Polish–Slovak border. The sixth county Słupski is located in the Baltic Sea coastal zone. Last but not least, Międzyrzecki county is located in western Poland. Five of the regions of interest have previously been the subject of OSM data quality assessments [37,38].

Two topographic datasets were investigated, namely, the National Database of Topographic Objects (BDOT10k), managed by the Head Office of Geodesy and Cartography, and OpenStreetMap, created on a voluntary basis. Six thematic layers were analyzed in detail: three polygon layers, such as buildings, forests and water bodies, and three linear ones—streams and canals, paved roads and railway lines. Topographic data were chosen because they reflect the complex relationships between components of the geographical environment, related to morphology, geology, hydrology, vegetation, and microclimate.

3.2. Method Applied

3.2.1. Main Methodological Assumptions and Research Question

The basic research problem to which this study refers concerns the definition of a Compound Correspondence Index (CCI), at local and regional scales, and the determination of the number and optimal class ranges that explicitly indicate the spatial location of differences in the information capacity of two investigated datasets. The WLC (Weighted Linear Combination) method for comparative, multi-criteria analysis was used on the basis of such criteria as differences in the area covered by buildings, forest and water bodies, and the lengths of roads, railways and rivers. The minimum difference indicates a very similar information volume, while maximum implies large differences between the two sets. It is assumed that the consolidated regional classification of the Compound Correspondence Index (regional CCI, hereinafter referred to as CCI_Rn), attributed to the 1 km² hexagonal grid, and their statistical measure of dispersion, enables the comparison of datasets for individual geographically disjoint areas according to uniform criteria. The decision to use hexagons has some advantages, as it is closer in shape to circles than squares, potentially reducing bias due to edge effects [40].

Therefore, the answering of two research questions is the priority of this study, and the questions are as follows:

Q1—Is there the difference between the CCI value calculated for grid cells of all research areas together (CCI_Rn) and the CCI value calculated in grid cells separately for each area (local CCI; hereinafter referred to as CCI_Ln), and if so, how big?

Q2—How does the value of regional CCI (CCI_Rn) change if we include another research area in the analysis, i.e., how sensitive is the CCI_Rn?

The answers to the questions allow us to verify the hypothesis that CCI_Rn underestimates the dissimilarity between analyzed datasets, indicating slightly higher compliance than the local CCI.

This approach is innovative as it allows the comparison of two sets containing qualitative data for geographically disjoint areas, thus enabling the user to choose one of them consciously and responsibly. It also shows the differences between CCI classifications at the local and regional levels.

3.2.2. Research Schema

The research was conducted in four consecutive phases: (1) preparatory; (2) computational; (3) sensitivity analysis; and (4) visualization. The preparatory phase relied on data acquisition, checking and preprocessing, including coordinate transformation, hexagonal grid creation, as well as assigning appropriate attributes to grid cells, e.g., the area covered by buildings, forests and water reservoirs and the length of roads, streams and railways in both datasets BDOT10k and OSM. This stage is effectively described in a former publication of Borkowska and Pokonieczny [37]. Phase 2—calculation CCI based on TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution). Phase 3—sensitivity assessment, which aimed to analyze the variability of regional CCI values when the research area is extended to other regions (disjoint or adjacent), based on descriptive statistics measures of dispersion and centrality and spatial inferential statistics, Global Moran’s I. Phase 4—visualization, which is based on the portrayal of the CCI value in the form of a choropleth map. The five class intervals reflect the conformity of BDOT10k and OSM data. Contrary to the classic Likert scale [41], reverse ordering was used, consistent with the CCI values, and so class 1, starting with the value 0, meant full compliance, and class 5 meant maximum non-compliance. Class ranges are based on one standard deviation interval. All research phases and stages are shown in Figure 2.

3.2.3. Compound Correspondence Index Calculation

TOPSIS is perceived as the most widely used approach among multi-criteria decision analysis (MCDA) [42,43]. This classical MCDA method, formerly developed by Hwang and Yoon in 1981 [44], addresses complex decision problems involving conflicting goals, uncertainty and different data formats. TOPSIS has gained popularity in a plethora applications because it evaluates real-world problems based on a variety of criteria, both quantitative and qualitative. Simultaneously, it takes into account the mutual distances to positive and negative ideal solutions, and finally orders the rank of preferences based on their relative proximity (a combination of these two distance measures). This study uses the classical TOPSIS method to portray differences in the thematic scope (understood as the area covered by buildings, forests and water reservoirs and the lengths of roads, rivers and railways) of two topographic datasets (BDOT10k and OSM) in the form of a composite index to compare and rank the obtained alternatives.

The seven standardized steps of the TOPSIS method described in detail by many papers, e.g., Pavic and Novoselac [45], Zavadskas et al. [46], are in general presented below.

1.

Problem description

The decision relies on the selection of BDOT10k or OSM sets (two alternatives; m = 2) based on the minimal value of the k criteria (k = 6), as shown in Equation (1):

$$\begin{gathered} x_1=|{BT}_B-{OSM}_B|,x_2=\left|{BT}_F-{OSM}_F\right|,x_3=\left|{BT}_W-{OSM}_W\right|, \\ {x}_4=\left|{BT}_{Ro}-{OSM}_{Ro}\right|,x_5=\left|{BT}_S-{OSM}_S\right|,x_6=\left|{BT}_{Ra}-{OSM}_{Ra}\right|, \end{gathered}$$

(1)

where BT denotes the BDOT10k dataset; OSM—OpenStreetMap data; subscripts _{B, F, W}—total area covered by buildings, forest and water bodies in the grid cell, and _{Ro, S, Ra}—total length of paved roads, streams, and railways, respectively.

2.

Calculation of the normalized decision matrix (n_ij) using the quotient method (Equation (2)):

$$n_{ij}=\frac{x_{ij}}{\sqrt{{\sum }_{i=1}^m{x}_{ij}^2}},i=1\dots m;j=1\dots n.$$

(2)

where m is the number of alternatives, and n is the number of hexagonal grid cells.

3.

Calculation of the weighted normalized decision matrix—Equation (3):

$$r_{ij}=w_j\times n_{ij}=1, i=1\dots m;j=1\dots n.$$

(3)

The criterion for weighting objects was their recognizability on satellite and aerial imagery, which is the main source for their acquisition [34,47,48]. Thus, buildings and forests were given a weight of $w_j=$ 0.25, paved roads and railways $w_j=$ 0.15, while water bodies and streams just $w_j=$ 0.10. Futhermore, these weighting rules are also used in accessibility and passability analyses and, as noted by Pokonieczny [49], are extremely important in crisis management.

4.

Determine the positive (PIS) and negative (NIS) ideal solutions, as shown by Equations (4) and (5).

$$PIS=\left\{r_1^{-},\dots r_j^{-}\dots r_n^{-}\right\},where v_j^{+}=\left\{\mathrm{m}\mathrm{a}\mathrm{x}(r_j^i\right\} if j\in B;\min r_j^i if j\in J^{\prime }$$

(4)

$$NIS=\left\{r_1^{-},\dots r_j^{-}\dots r_n^{-}\right\},where v_j^{+}=\left\{\min (r_j^i\right\} if j\in B;\max r_j^i if j\in J^{\prime }$$

(5)

As all criteria used are destimulants, the positive ideal solution was computed as $r_j^i$ min, while the negative ideal was $r_j^i$ max.

5.

Calculate the separation measure S_i of each alternative (relative closeness to the positive ideal solution) as (Equations (6) and (7)):

$$S_i^{+}=\sqrt{{\displaystyle \sum _i^n}{\left({r_{ij}-r}_j^{+}\right)}^2};S_i^{-}=\sqrt{{\displaystyle \sum _i^n}{\left({r_{ij}-r}_j^{-}\right)}^2}i=1,\dots ,m$$

(6)

6.

Calculate the closeness coefficient of the alternatives (CC_i) as:

$$C{C}_i=\frac{S_i^{-}}{S_i^{+}+S_i^{-}}$$

(7)

7.

Sort alternatives in descending order, whereby the highest CC_i indicates the best performance in relation to the evaluation criteria.

4. Results

4.1. Local CCI Diversity

The CCI_L differs between regions considered, taking the highest values in Ostrowski and Międzyrzecki (see

Table 2. Descriptive CCI_L statistics.

Statistics	Piaseczno	Sokólski	Sanocki	Słupski	Ostrowski	Otwocki	Międzyrzecki
Mean	0.0915	0.0390	0.0678	0.0414	0.0427	0.0989	0.0353
Median	0.0754	0.0304	0.0533	0.0313	0.0314	0.0832	0.0271
Minimum	0	0	0	0	0	0	0
Maximum	0.4828	0.5329	0.4995	0.4764	0.5765	0.5069	0.5899
Q1	0.0420	0.0159	0.0315	0.0165	0.0181	0.0542	0.0129
Q3	0.1196	0.0499	0.0790	0.0507	0.0538	0.1246	0.0427
Variance (σ2)	0.0052	0.0015	0.0039	0.0018	0.0017	0.0046	0.0016
Std. Dev. (σ)	0.0723	0.0386	0.0627	0.0426	0.0406	0.0680	0.0405
Coeff. of variation	78.9755	98.8908	92.3641	103.0427	95.1072	68.7311	114.7669
Interquartile range (IQR)	0.0775	0.0340	0.0475	0.0342	0.0357	0.0704	0.0298

), which are both located in Greater Poland. The counties are characterized by a similar urbanization level (53.7 and 52.3, respectively) and land cover structure (see Table 1). Very high statistical dispersion of CCI_L, as measured by the coefficient of variation, is observed in Słupski and Międzyrzecki counties, where population density, afforestation and percentage of area covered by agricultural land are similar. However, both counties differ significantly in terms of area, with Słupski county being twice as large.

The values of the local Compound Correspondence Index in Otwocki and Piaseczno counties are characterized by the highest value of IQR, as well as standard deviation, and thus indicate a high dispersion of local CCI values. Furthermore, mean CCI_L takes a value greater than σ, which indicates that for most of the hexagonal cells, the CCI_L value is lower than the mean value, i.e., the data consistency is relatively high here. The standard deviation of 0.068 and 0.072 in Otwocki and Piaseczno counties is almost twice as high as the lowest value recorded in Sokólski county (0.039). Furthermore, the relatively high diversity of analyzed data is proven by variance, which takes the value of 0.0052, 0.0046 and 0.0039 in Piaseczno, Otwocki, and Sanocki counties, respectively.

The CCI_L values in all analyzed counties reveal clustering, as indicated by the spatial autocorrelation Global Moran’s I statistics, with z-score ranges of 21.80 to 14.62 (p-values < 0.001) in Piaseczno and Słupski counties, respectively.

A predominance of areas with low and very low differentiation between BDOT10k and OSM (CCI_L first and second class) ranging from 83.5% to 85.3% was observed in Słupski and Międzyrzecki counties. The relative lack of congruence, defined as semi-compliance, with the highest values of 13.1% to 13.3% characterizes Otwocki, Piaseczno and Ostrowski regions. A great diversity in the analyzed datasets (CCI_L fourth and fifth class) is noted in Otwocki (9.3%) and Piaseczno (9%) counties. The remaining districts stand out with relatively small noncompliance, amounting in Ostrowski to 7.1%, in Sanocki to 6.6% and in Słupski to 6.3%, with the least in Sokólski (5.6%) and Międzyrzecki (5.1%) (see

Table 3. The percentage of a county’s area in CCI_L classes.

Class	Description	Range	Percentage of the County’s Area (%)
			Piaseczno	Sokólski	Sanocki	Słupski	Ostrowski	Otwocki	Międzyrzecki
1	maximum compliance	−0.50 σ < CCIL	35.5	32.1	30.6	30.6	33.2	34.1	29.1
2	moderate compliance	−0.5 σ < CCIL ≤ 0.5 σ	42.3	49.6	52.2	52.9	46.5	43.5	56.2
3	semi-compliance	0.5 σ ≤ CCIL ≤ 1.5 σ	13.1	12.7	10.6	10.3	13.3	13.1	9.6
4	moderate noncompliance	1.5 σ ≤ CCIL ≤ 2.5 σ	5.6	2.7	3.1	3.1	4.6	9.3	2.5
5	maximum noncompliance	CCIL > 2.5 σ	3.4	2.9	3.5	3.2	2.5	-	2.6

). The level of compliance between the BDOT10k and OSM datasets is shown in Figure 3.

4.2. Regional CCI Diversity and Sensitivity Analysis

As the region’s area expands, the range of regional CCI values increases, and the local differences between the data become blurred. The variance, standard deviation and interquartile range decrease, which proves that CCI_R7 values are less diverse than CCI_R6, CCI_R5 and CCI_R4 (

Table 4. Regional CCI descriptive statistics.

Statistics	CCIR4 1	CCIR5	CCIR6	CCIR7
Mean	0.0327	0.0272	0.0281	0.0263
Median	0.0241	0.0201	0.0208	0.0193
Minimum	0	0	0	0
Maximum	0.4971	0.5289	0.5299	0.5389
Q1	0.0123	0.0104	0.0108	0.01
Q3	0.0410	0.0340	0.0352	0.0327
Variance (σ2)	0.0012	0.0008	0.0008	0.0008
Std. Dev. (σ)	0.0343	0.0287	0.0289	0.0278
Coefficient of variation (cv)	104.9797	105.3271	102.7925	105.6453
Interquartile range (IQR)	0.0287	0.0236	0.0244	0.0227

¹ CCI_R4—denotes the regional CCI computed for four counties, namely, Piaseczno, Sokólski, Sanocki and Słupski; CCI_R5—five counties, Ostrowski was added; CCI_R6—six counties, additionally Ostrowski; and CCI_R7—seven regions, Międzyrzecki added.

). Regardless of the region’s extent, the CCI_Rn mean value is greater than the median and slightly lower than the standard deviation, indicating a predominance of values smaller than the mean, i.e., maximum and moderate compliance between BDOT10k and OSM. Nevertheless, CCI_Rn is characterized by a large disparity, based on cv, which takes values above 100%.

The standard deviation of CCI_R4 (0.0343) is smaller than that of CCI_L in Sokólski county (CCI_L σ = 0.0386), with the lowest CCI_L value, which value is less than half that of the highest value in Piaseczno county (CCI_L σ = 0.0723). Additionally, in terms of variance, the value of CCI_R4 (0.0012) is noticeably smaller than that of CCL_L, whereas, in Piaseczno county, the variance is four times higher (CCI_L σ² = 0.0052) (see also Table 2). The maximum compliance between BDOT10k and OSM oscillates around 32% of the entire region area. In contrast, moderate compliance, with CCI_Rn values in the range of −0.5 σ to 0.5 σ>, represents just over 50% (

Table 5. The percentage of a county’s area belonging to regional CCI class of compliance.

Class	Description	Interval Size	Percentage of the Counties Area (%)
			Regional CCIR4	Regional CCIR5	Regional CCIR6	Regional CCIR7
1	maximum compliance	−0.50 σ < CCIRn	31.9	31.5	32.3	32.1
2	moderate compliance	−0.5 σ < CCIRn ≤ 0.5 σ	50.9	51.3	50.1	51.2
3	semi-compliance	0.5 σ ≤ CCIRn ≤ 1.5 σ	11.1	10.9	11.0	10.4
4	moderate noncompliance	1.5 σ ≤ CCIRn ≤ 2.5 σ	6.1	3.6	3.9	6.3
5	maximum noncompliance	CCIRn > 2.5 σ	-	2.7	2.7	-

). The maximum disagreement between the data is basically negligible. The area where both BDOT10k and OSM data have the lowest correspondence between geospatial objects varies between 3.6 and 6.3%. Figure 4 presents a cartographic visualization of regional CCI₇.

The highest IQR indicates that the spread of the CCI_Rn values declines when the region’s area expands, with the highest value in Piaseczno (CCI_R4–IQR = 0.045; CCI_R5–7–IQR 0.035) and the lowest in Sokólski (CCI_R4–IQR = 0.020; CCI_R5–7–IQR 0.017). In the remaining counties, the IQR amounted to about 0.025 (see

Table 6. CCI_Rn descriptive statistics in each county.

County	CCIRn	Mean	Median	Min	Max	Q1	Q3	IQR	σ2	σ
	CCIR4	0.0500	0.0327	0.0000	0.4356	0.0170	0.0620	0.0450	0.0028	0.0527
	CCIR5	0.0403	0.0270	0.0000	0.3778	0.0141	0.0495	0.0354	0.0018	0.0425
Piaseczno	CCIR6	0.0403	0.0271	0.0000	0.3778	0.0142	0.0495	0.0353	0.0018	0.0425
	CCIR7	0.0398	0.0261	0.0000	0.3845	0.0137	0.0491	0.0354	0.0018	0.0429
	CCIR4	0.0249	0.0174	0.0000	0.4971	0.0091	0.0300	0.0209	0.0009	0.0304
Sokólski	CCIR5	0.0208	0.0147	0.0000	0.3949	0.0078	0.0249	0.0172	0.0006	0.0254
	CCIR6	0.0209	0.0147	0.0000	0.3912	0.0077	0.0250	0.0172	0.0007	0.0256
	CCIR7	0.0201	0.0138	0.0000	0.3909	0.0072	0.0238	0.0166	0.0006	0.0255
	CCIR4	0.0290	0.0263	0.0000	0.1741	0.0148	0.0382	0.0234	0.0004	0.0204
Sanocki	CCIR5	0.0348	0.0315	0.0000	0.1975	0.0176	0.0458	0.0282	0.0006	0.0243
	CCIR6	0.0291	0.0264	0.0000	0.1731	0.0149	0.0384	0.0235	0.0004	0.0204
	CCIR7	0.0279	0.0249	0.0000	0.1619	0.0141	0.0372	0.0231	0.0004	0.0196
	CCIR4	0.0336	0.0257	0.0000	0.3280	0.0133	0.0422	0.0290	0.0011	0.0331
Słupski	CCIR5	0.0283	0.0215	0.0000	0.2699	0.0110	0.0355	0.0245	0.0008	0.0280
	CCIR6	0.0283	0.0215	0.0000	0.2684	0.0111	0.0354	0.0243	0.0008	0.0279
	CCIR7	0.0269	0.0203	0.0000	0.2639	0.0105	0.0336	0.0232	0.0007	0.0269
	CCIR5	0.0272	0.0191	0.0000	0.5289	0.0111	0.0328	0.0217	0.0009	0.0298
Ostrowski	CCIR6	0.0272	0.0193	0.0000	0.5299	0.0111	0.0329	0.0218	0.0009	0.0298
	CCIR7	0.0268	0.0188	0.0000	0.5389	0.0106	0.0320	0.0213	0.0009	0.0300
	CCIR6	0.0384	0.0310	0.0000	0.2429	0.0193	0.0479	0.0286	0.0009	0.0292
Otwocki	CCIR7	0.0369	0.0295	0.0000	0.2472	0.0183	0.0460	0.0277	0.0008	0.0289
Międzyrzecki	CCIR7	0.0217	0.0169	0.0000	0.2373	0.0084	0.0276	0.0192	0.0005	0.0223

). In Sanocki, Słupski and Otwocki counties, the standard deviation CCI_Rn takes values lower than the mean CCI_Rn in more than 50% of the grid cells (µ > σ), indicating the predominance of high agreement between BDOT10k and OSM data. In each county, regardless of the spatial extent of the region, the variance does not change, and the standard deviation decreases slightly, as does the achieved maximum value of CCI_Rn.

Figure 5 presents a cartographic visualization of CCI_Rn in two counties: Piaseczno, with the highest spread of CCI, and Sokólski with the lowest.

4.3. Local vs. Regional CCI

Each of the counties analyzed is characterized by an overestimation of the area of maximum compliance by regional CCI compared to local CCI. The largest growth of maximum data compliance takes place in Otwocki and Piaseczno counties, amounting to 50.9% and 40.1%, respectively, and the lowest in Międzyrzecki (16%) and Słupski counties (14.7%). The overestimation of a very good match between BDOT10k and OSM does not depend on the spatial extent of the region, i.e., it does not change when counties are added and the extent of the region increases. The upsurge in maximum data compliance was at the expense of a moderate compliance class, showing an average of 17.2% (the highest, 29.8%, in Otwocki and the lowest, 6.8%, in Międzyrzecki counties), and that in comparable data agreement (semi-conformance class) showed an average of 8.2% (the highest, 12.2%, in Otwocki and the lowest, 4.1%, in Słupski); low and very low compliance classes (moderate and maximum non-compliance) increased by 3.5% and 1.9%, respectively. The maximum decline in the area considered non-compliant occurred in Piaseczno and Otwocki counties. However, their extents do not exceed 9%.

5. Discussion

5.1. Semantic Uncertainty

The comparative analysis of data requires semantic similarity in the investigated concepts. As emphasized by many researchers [34,50,51], this assumption is a prerequisite for proper comparative data quality assessment. Semantic similarity in GIScience enables not only data comparison, but also the integration of data from different sources and their further analysis. If the semantic similarity between concepts is ignored, the evaluation or data comparison will be inaccurate [52]. The semantic consistency of concepts in IG can be analyzed at the general level, i.e., only the definition of the object and its geometrical representation are given [53], or at the detailed level, which also comprises definitions of attributes [34,54]. In general, semantic inconsistency results from the classification used, as well as scaling [53]. Scientists use numerous methods to quantify and visualize geodata uncertainties, many of which rely on mathematical models. Semantic uncertainty results from incomplete knowledge of spatial features and phenomena. Traditionally, the boundaries of geographical features are treated as discrete and mutually exclusive, and often, the true location of the boundary is unknown and subject to positional and semantic errors. Zhao et al. [52] introduced a conceptual framework of ontology that includes the definition, semantic relationship, nature, and attributes of geographic features (concept). The authors defined semantic similarity as “calculated on the basis of the feature similarity of ontology concept and the semantic distance of concept to measure the semantic similarity of ontology concept. The smaller the semantic distance between concepts, the higher the similarity between concepts”.

Based on OSMwiki [55] and Ministry Regulation [56], it is seen that in both datasets, a “building” is a man-made structure with a roof, standing (more or less) permanently in one place and geometrically represented by a polygon. Differences in definition only arise when “tag:building use” is considered. During forest surveys, two tags, “natural = wood” and “landuse = forest” are used to map. They represent forests or other areas of trees. Both tags together are compliant with the forest definition in BDOT10k. Water bodies are represented in OSM by the following tags: “natural = water”, “landuse = reservoir” and “water = reservoir”; these are equivalent to “surface water”, i.e., areas occupied by rivers, canals and reservoirs. “Highway = {motorway, trunk, primary, secondary, tertiary, unclassified, residential}” is the principal tag for the road network and corresponds to “road”, while “tag:railway = rail” matches “track or set of tracks” in BDOT10k. Finally, streams, rivers and other watercourses in OSM have the tag “waterway = {river, stream, tidal_channel}”, and in BDOT10k are named “river and stream/channel” [37].

5.2. Validity and Applicability of Data Comparison

Decisions to use fit-for-purpose geospatial datasets are heavily based on data quality, and in particular, information volume, which is understood as the number of geographical features captured. OSM data are rich and heterogeneous, and their quality strongly depends on the degree of urbanization, as mentioned by [34,57]. On the contrary, BDOT10k is perceived as very reliable as authoritative data [58]. It is updated every 2–3 years [56]. Both datasets are used in many applications in Poland by commercial companies and administrations, and in science and education.

Risk assessment and risk management, spatial planning, and environmental protection and monitoring applications often require detailed data when every object is important. In loss assessment (e.g., due to flood or fire), indicating access roads and planning the locations of investments and many other applications, every building, road, railway and object that constitute some kind of barrier, such as water or forest, is important. When the analyzed sets are characterized by high data agreement, the choice of the set is definitely less important than when the two sets vary from each other. It is then necessary to analyze the quality of the datasets based on the indicators described, inter alia, by Borkowska and Pokonieczny [37], and to make a choice supported by the cartographic visualization of data quality [38].

It is worth mentioning that OSM data are made available alongside official data in the form of the WMS (Web Map Service) in the national web portals (Geoportals), providing access to geographic information and spatial data services, e.g., in Poland, Germany, France and Greece [59,60,61,62].

6. Conclusions

It was assumed that the selection of a set of geospatial data for a specific application (fit-for-purpose data) could be completed on the basis of the local or regional CCI. Geospatial datasets are assessed by clustering hexagonal grid cells based on ordered CCI similarity. Like any data classification, it is an exploratory procedure because it leads to an understanding of objects and processes. The research is theoretical and methodological in nature, but the close connection to a specific problem situation also gives it a cognitive aspect.

Assuming that the study area consists of several spatially disconnected areas (e.g., counties, cities), the regional CCI makes it possible to assess the suitability of the sets according to a common scale based on the standard deviation. However, the results of the regional assessment outperform the local classification, giving better results, i.e., a higher level of data compliance. The overestimation of regional compliance ranges from 9 to 20% of the county’s area, with an average of 3% reduction in the area over which the two datasets (BDOT10k and OSM) have comparable information scope. Areas of medium and large incompatibility are reduced by an average of 2.4%. Sensitivity analysis shows that neither the size of the region nor the spatial location of the counties had a significant impact on the values of the regional CCI.

The CCI values in all analyzed districts revealed clustering. A greatest variation between BDOT10k and OSM data was observed in areas with a high degree of urbanization (e.g., Piaseczno city, Otwock city) and near the course of major transportation routes. The analyses carried out did not prove statistically significant correlations between the CCI coefficients and the land cover elements studied (buildings, roads, rivers, railways, forests and water bodies).

OSM is a valuable source of up-to-date geographic data in emergency mapping, with capacities including, but not limited to, identifying infrastructure at risk of destruction, collapsed buildings, fires and accessibility, which can be important inputs for the orientation of rescuers on the ground. The use of these data in areas with varying degrees of completeness and timeliness of the other official spatial data is of particular importance. Nevertheless, the volunteer type of data may raise issues about quality. The analysis performed of the compliance of the OSM dataset in comparison with official data allows the selection of a set with appropriate characteristics suitable for the intended purpose.

Conducting and comparing several counties using a common analytical framework allows for synthesizing how complete the analyzed geodatasets are, and identifying potential commonalities and differences across places.

The method proposed in this paper has several limitations. Among them are TOPSIS constrains, such as the way that variables are weighted, correlations between variables, and the possibility of an alternative that is close to the ideal point and the nadir point simultaneously. An additional limitation is the CCI designation that is based solely on criteria (variable) such as differences in the area and length of the geographic feature analyzed in the OSM and BDOT10k datasets. In future work, our research will primarily address at least some of the limitations mentioned above. Thus, we will use a different WLC approach to assess the correspondence between OSM and authoritative topographic data. The Hellwig’s information capacity method, a the method of optimal predictor selection, is considered for the selection of explanatory variables to be used in a model to evaluate geodatasets. Another research question to deliberate regards the impact of the scope and methods of weighting variables.