European Digital Elevation Model Validation against Extensive Global Navigation Satellite Systems Data and Comparison with SRTM DEM and ASTER GDEM in Central Macedonia (Greece)

: Digital elevation models (DEMs) are a widely used form of topographic information, with some of the most popular being the Shuttle Radar Topography Mission (SRTM) DEM and the Advanced Spaceborne Thermal Emission and Reﬂection Radiometer Global Digital Elevation Model (ASTER GDEM). These two sources of topographical information are the main constituents of the European Union Digital Elevation Model (EU-DEM), which is a relatively new dataset of the EU’s Copernicus Land Monitoring Service. In this context, the purpose of this study was to validate EU-DEM for its vertical accuracy and to compare it with SRTM DEM and ASTER GDEM data. This was achieved in a Geographic Information System (GIS) environment, using extensive—in the order of tens of thousands of points—geodetic Global Navigation Satellite System (GNSS) measurements and appropriate pre-processing steps. The absolute elevation errors results had a Root Mean Square Error (RMSE) of 2.7 m at a 90% conﬁdence level and characterize the performance of EU-DEM from local to regional scale, generally conﬁrming that it is an enhanced source of elevation information when compared with its predecessors.


Introduction
Digital elevation models (DEMs) are the most popular form of topographic data. The positive impact of (almost) global and publicly available DEMs from space-based measurements, such as the Shuttle Radar Topography Mission (SRTM) DEM data [1] and the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model (ASTER GDEM) [2][3][4], has been verified by the plethora of their applications. SRTM DEM has been described as the most dramatic advance in cartography since Mercator [5], whereas current products, like ALOS Global Digital Surface Model "ALOS World 3D-30 m (AW3D30)" [6] or WorldDEM™ [7], and near-future mission concepts such as the Lidar Surface Topography (LIST) [8] emphasize the quest for increasingly accurate descriptions of the global topography and underline the topography's time dependence, turning its measurement into a continuous task [5]. on ASTER GDEM data alone or the combination of ASTER GDEM and Russian topographic maps [10]. The RMSE of the nominal vertical accuracy of the upgraded version (EU-DEM v1.1) has been estimated to be in the order of 7 m [35,36]. The quality of the data has been verified by several quality control process, including a statistical comparison and visual checks [36].
A local scale (area of about 25 × 15 km) and relatively small elevation range (139-682 m) of EU-DEM v1 was more recently reported in Hungary [37]. EU-DEM v1 was compared against ASTER GDEM, SRTM DEM, and contour-based DEMs from 1:10,000 scale topographic maps. The latter DEM was interpolated with a 10-m cell size and then downsampled to 30 m to represent the ground truth values, ultimately estimating the RMSE of the vertical accuracy of EU-DEM to be between 6.0 m and 7.8 m.
Based on this background, the objective of this study was to validate the vertical accuracy of EU-DEM (both v1 and v1.1) on a local to regional scale against primary elevation data measured in the field with high reliability, as well as to compare it with SRTM DEM and ASTER GDEM data. This was essentially achieved in a Geographic Information System (GIS) environment using extensive-in the order of tens of thousands of points-geodetic Global Navigation Satellite Systems (GNSS) measurements preceded by appropriate pre-processing steps. The main purpose was to examine the hypothesis that EU-DEM synthesizes the advantages of its sources, i.e., is characterized by a higher vertical accuracy than SRTM DEM and ASTER GDEM.
As EU-DEM is produced through a fusion processing chain including both horizontal and vertical shifting as well as smoothing of the input layers [36], this independent study involved a blind experiment in that the exact algorithm that produced EU-DEM from SRTM DEM and ASTER GDEM data was not known a priori.

Study Area
The study area extends over an area of about 100 × 70 km in Central Macedonia, Northern Greece ( Figure 1a). Land cover consists mainly of agricultural areas or pastures (~60%), shrubs or low vegetation (~20%), and forests (~10%) (Figure 1b). Its topography varies from extensive lowlands and (locally) below Mean Sea Level (MSL) elevation areas, owing to subsidence, to hilly and mountainous regions (Figure 1c). Since 2008, the area has been serving as a test site for the GNSS-assisted validation of various DEMs derived from Earth observation data, such as the four available versions of SRTM DEM [26], the first version of ASTER GDEM (released in 2009), photogrammetrically produced DEMs from stereoscopic SPOT images, Interferometric Synthetic Aperture Radar (InSAR) DEMs from ERS tandem data, as well as DEMs originating from the digitization of topographical maps [34].

Digital Elevation Models
The DEMs simultaneously evaluated in this study included:

Digital Elevation Models
The DEMs simultaneously evaluated in this study included: (  [38]. A thorough validation of SRTM DEM for the area of interest can be found in Mouratidis et al. [26]. Note that according to Mouratidis et al. [26], for the area of interest, the CGIAR-CSI SRTM v4 is identical to SRTM v2 from NASA. The reason why v4 is used here is because it is provided in larger tiles than v2; thus, for a reasonable study area extent, the process of having to mosaic different SRTM DEM tiles can usually be avoided. The data are provided in the World Geodetic System of 1984 (WGS 84) latitude and longitude and orthometric height with respect to the Earth Geopotential Model of 1996 (EGM96) [39]. (4) ASTER GDEM version two (v2): This stereoscopic product is an improved version of the first release in 2009, with a resolution of 1 arcsecond (about 23 × 30 m in the study area) [4]. It includes 260,000 additional scenes to improve coverage, a smaller kernel-to-yield correlation, higher spatial resolution, and improved water masking. The negative 5 m overall bias observed in ASTER GDEM v1 was eliminated in the newer version. ASTER GDEM v2 is distributed in 1 • × 1 • tiles. The data are given in WGS 84 latitude and longitude and orthometric height with respect to the EGM96 geoid.

GNSS Data
The original reference dataset used for the validation of DEMs consists of a total of about 70,000 GNSS (GPS and GLONASS) points (Figure 1c), collected during three campaigns between 2008 and 2011. More specifically, the dataset consists of about 60,000 kinematic and static GPS observations from 2008 [26], about 10,000 kinematic and static GNSS (using GPS and GLONASS) points from Mouratidis and Albanakis [40], as well as a few additional GNSS points (again, using both GPS and GLONASS signals) collected in static mode in 2011. The kinematic data were surveyed on roads, whereas the static observations involve the various positions of the reference receivers (bases) used for the kinematic surveys.
Both static and kinematic observations were collected with dual-frequency GNSS receivers and were post-processed. The RMSE of the vertical accuracy of all data collected during the period 2008-2011 ranged between 1 cm and 1.5 m at 95% confidence. The orthometric elevations of the data collected range from −2 m to 890 m.
The raw GNSS data are referenced to the International Terrestrial Frame of 2005 (ITRF2005) [41] and provide geometric (ellipsoidal) heights with reference to the World Geodetic System 1984 (WGS84) ellipsoid.

Other Data and Software
Additional data used for the validation of the various DEM included Corine Land Cover information of 2012 (CLC 2012) [42].
Most of the processing took place in a GIS software, namely ArcGIS™ (ESRI, Redlands, CA, USA).

Horizontal and Vertical Datum Transformations
As EU-DEM only concerns Europe and adjacent areas, the reference systems used for the validation procedures in this study were chosen to be those of EU-DEM, i.e., ETRS89 (horizontal) and EVRS2000 (vertical). Therefore, GNSS, SRTM DEM, and ASTER GDEM data were all initially converted to ETRS89/EVRS2000.
In order to be consistent with ETRS89/EVRS2000, the following steps were followed: (1) WGS84 refers to the current ITRF solution with an accuracy of 10 cm [43]. Using the Boucher-Altamimi memo [44], the ETRS89 coordinates were calculated. For this purpose, MATLAB (MathWorks, Natick, MA, USA) scripts were developed and then tested against the officially released tools [45], with the accuracy of the transformation being better than 20 cm.
With respect to the calculation of orthometric elevations, although EGG08 exists for Europe, it has not been validated in Greece [46]. The vertical data for Greece are not connected to EVRS and there is no real evidence of EGG08 accuracy in Greece because data is not publically available. A satisfactory Earth Gravitational Model 2008 (EGM2008) assessment is available for Greece [47]. Thus, instead of EGG08, the EGM 2008 geoid was used for ellipsoidal to orthometric elevation transformations of GNSS data.
Concerning the GNSS point dataset, the RMSE of the accuracy of ellipsoidal height (h) determination with differential GPS (DGPS) is in the order of 0.5-1 m [48]. Orthometric heights (H) are then derived from ellipsoidal heights, by subtracting the geoidal undulation (N), i.e., H = h RMSE N [49]. Thus, for the present case study, N refers to the EGM2008 [50], from which the geoidal undulation accuracy in Greece is estimated with a standard deviation of approximately 15 cm [51][52][53]. With the above considerations and considering the error propagation, the RMSE of the overall original accuracy of the orthometric elevations for the GNSS points collected is in the order of 1 m.
All datasets used in the validation process (apart from the CLC 2012 layer, which was already georeferenced in ETRS89-LAEA) were also projected to ETRS89-LAEA coordinates, in order to (1) evaluate the DEM vertical accuracy by deriving and considering slope information and (2) validate EU-DEM v1.1, which is only provided in projected coordinates.

GNSS Data Pre-Processing
The original 70,000 GNSS points, equivalent to a density of about 10 points/km 2 in the study area, were filtered in several steps: (1) points with vertical RMSE > 1 m were eliminated; (2) based on a priori knowledge of the study area, points under overpasses, bridges, etc. were eliminated; (3) points whose elevation had a standard deviation of more than 1 m within a 30-m distance or more than 5 m for 90-m distance were removed using respective thresholds [15,26]; and (4) to ensure equally distributed weight among the evaluated EU-DEM pixels and increased vertical accuracy, multiple GNSS points that fell within a single EU-DEM one arcsecond pixel (approx. 23 m × 30 m at the latitude of the area of interest) were averaged.
After completing the above procedures, the final GNSS dataset for validating the available DEMs consisted of 12,943 points, each one corresponding to a discrete EU-DEM pixel. These GNSS points were then saved as a separate GIS vector layer, with the orthometric elevation value of each point registered in the layer's attribute table.

Pre-Processing of DEMs
As a first pre-processing step, the 3 arcsecond SRTM data were resampled to a 1 arcsecond grid, using nearest neighbor interpolation to match the grids of EU-DEM and ASTER GDEM without altering the original pixel (elevation) values.
Next, the actual area of interest, corresponding to the spatial extent of GNSS points available, was defined in each dataset and the relevant subsets were created, each containing 4542 × 2384 pixels (approx. 100 × 70 km).
The raster DEM data were compared against vector GNSS points in a GIS environment by converting the former to vector format [23,26]. More specifically, all DEMs were converted to vector GIS layers (shapefiles), as polygons (rectangles) with same size as the former raster cells, while the elevation value of each cell was retained in the attribute table of the respective polygon. Subsequently, the vector GIS layer of the final GNSS points was overlain on each of the DEM shapefiles. Finally, the DEM elevation value at the location of each GNSS point was transferred in the attribute table of the GNSS in a separate column (one for every DEM). With this, we validated DEM elevation via GNSS elevations by comparing two columns of corresponding data values in a spreadsheet.

Direct Comparison of EU-DEM with SRTM DEM and ASTER GDEM
Considering EU-DEM is practically a mathematical product mainly stemming from ASTER GDEM and SRTM DEM data, prior to assessing the accuracy of the individual DEMs, we acquired an overview of the differences between the three datasets. To this end, we completed a relative DEM comparison by executing the following subtractions: (

Absolute Elevations
Absolute vertical accuracy was estimated for all available DEMs with reference to the 12,943 GNSS points. The computed differences were calculated following a "DEM-GNSS" subtraction approach in all three cases, i.e., ASTER GDEM-GNSS, SRTM DEM-GNSS, and (EU-DEM)-GNSS. For EU-DEM, two separate evaluations were completed: for the original EU-DEM and for the EU-DEM upgrade (v1.1).
Histograms (Figures 4 and 5) of the results show that the elevation values of EU-DEM are closer to the "real" values measured by GNSS. This was quantified in the corresponding descriptive statistics (Table 1), where mean, standard deviation, and range (min-max) of the values of EU-DEM elevation errors are considerably better than those of ASTER GDEM and SRTM DEM. Additionally, given the known non-normality of the differences (residuals) between ground data and DEMs (kurtosis and skewness), some more robust statistical measures have been included according to Höhle and Höhle [54], so that the absolute elevation validation is also based on median and Normalized Median Absolute Deviation (NMAD) values. and SD of 7.3 m) at both low and high elevations, but the deviations over rugged terrain and steep slopes are generally smaller than those with respect to SRTM DEM. The differences between SRTM DEM and ASTER GDEM (Figure 2c) presented a mean value of 4.4 m and a SD of 8.3 m. The result of this evaluation is both visually and statistically similar with the EU-DEM-ASTER GDEM comparison, verifying that EU-DEM and SRTM data values are overall not too far apart. The upgraded version of EU-DEM (EU-DEM v1.1) presented significant elevation differences from the original version, especially over hilly and mountainous areas, with an overall SD of 2.1 m (Figure 3).

Absolute Elevations
Absolute vertical accuracy was estimated for all available DEMs with reference to the 12943 GNSS points. The computed differences were calculated following a "DEM-GNSS" subtraction approach in all three cases, i.e., ASTER GDEM-GNSS, SRTM DEM-GNSS, and (EU-DEM)-GNSS. For EU-DEM, two separate evaluations were completed: for the original EU-DEM and for the EU-DEM upgrade (v1.1).
Histograms (Figures 4 and 5) of the results show that the elevation values of EU-DEM are closer to the "real" values measured by GNSS. This was quantified in the corresponding descriptive statistics (Table 1), where mean, standard deviation, and range (min-max) of the values of EU-DEM elevation errors are considerably better than those of ASTER GDEM and SRTM DEM. Additionally, given the known non-normality of the differences (residuals) between ground data and DEMs (kurtosis and skewness), some more robust statistical measures have been included according to Höhle and Höhle [54], so that the absolute elevation validation is also based on median and Normalized Median Absolute Deviation (NMAD) values. Table 1 shows that the EU-DEM upgrade (v1.1) has succeeded in considerably reducing the outliers in v1 (reduction in kurtosis and range); however, it is overall somewhat less accurate than

Relative Elevation
To estimate the relative vertical accuracy, a random selection of 1000 GNSS points from the total of 12,943 took place. Relative accuracy was then calculated (Figures 6 and 7, Table 2) on the basis of almost 500,000 computed elevation differences, i.e., of all possible combinations between the selected points. The results indicate that, with respect to relative errors, EU-DEM is the most consistent dataset, but with SRTM DEM being very close in all statistics. ASTER GDEM has significantly larger error range and standard deviation values compared with the other two datasets ( Table 2).   Table 1 shows that the EU-DEM upgrade (v1.1) has succeeded in considerably reducing the outliers in v1 (reduction in kurtosis and range); however, it is overall somewhat less accurate than the original EU-DEM version.

Relative Elevation
To estimate the relative vertical accuracy, a random selection of 1000 GNSS points from the total of 12,943 took place. Relative accuracy was then calculated (Figures 6 and 7, Table 2) on the basis of almost 500,000 computed elevation differences, i.e., of all possible combinations between the selected points. The results indicate that, with respect to relative errors, EU-DEM is the most consistent dataset, but with SRTM DEM being very close in all statistics. ASTER GDEM has significantly larger error range and standard deviation values compared with the other two datasets ( Table 2).

Slope
Irrespective of the elevation at which it occurs, slope is a crucial factor that controls DEM vertical accuracy [19,23,26,27,55]. Efforts to decompose and model elevation differences on the basis of aspect and slope terrain classes, to overcome the systematic errors in datasets such as SRTM, have already been reported [56][57][58] .
To investigate this aspect, slope was calculated from the most accurate and higher spatial resolution available dataset, i.e., from the original version of EU-DEM, which was subsequently plotted against elevation errors for all other DEMs, together with the modelled DEM errors presented by Becek [57] (Figure 8).
The results show the consistency of SRTM DEM and EU-DEM: elevation errors are small and increase gradually with slope, as expected. This primarily occurs over flat terrain up to slope values of about 12°, whereas EU-DEM continues to be very consistent up to 14°. ASTER GDEM does not

Slope
Irrespective of the elevation at which it occurs, slope is a crucial factor that controls DEM vertical accuracy [19,23,26,27,55]. Efforts to decompose and model elevation differences on the basis of aspect and slope terrain classes, to overcome the systematic errors in datasets such as SRTM, have already been reported [56][57][58].
To investigate this aspect, slope was calculated from the most accurate and higher spatial resolution available dataset, i.e., from the original version of EU-DEM, which was subsequently plotted against elevation errors for all other DEMs, together with the modelled DEM errors presented by Becek [57] (Figure 8).
The results show the consistency of SRTM DEM and EU-DEM: elevation errors are small and increase gradually with slope, as expected. This primarily occurs over flat terrain up to slope values of about 12 • , whereas EU-DEM continues to be very consistent up to 14 • . ASTER GDEM does not exhibit this behavior; starting at considerably higher values, errors tend to decrease or remain at the same level up to slopes of 16 • . The overall bias of ASTER GDEM is also evident through the comparison against the modeled errors for a 30 m DEM. deteriorated accuracy with respect to the original EU-DEM version, as expected from the previous results of absolute elevation evaluation (Table 1). However, for maximum slope values (21°-39°), the error of EU-DEM v1.1 increases by almost 5 m compared to the respective error of the original EU-DEM version.
Overall, the errors of the original version of EU-DEM are closer to the expected (modelled) errors for a 30 m DEM.

Land Cover
SRTM DEM, ASTER GDEM, and EU-DEM are Digital Surface Models (DSMs), so thus include the elevation of buildings, trees, and other objects, and do not necessarily refer to the ground surface. Consequently, depending on the type of land cover, the actual terrain elevation could be overestimated by DEMs, even if the measurement itself is accurate.
Therefore, as a last step, robust statistics, i.e., median ( Figure 9, Table 3) and NMAD ( Figure 10, Table 4) of the absolute elevation errors were used in order to evaluate the impact of different land At higher slope values (>13 • ), owing to its higher spatial resolution, ASTER GDEM begins to exceed the accuracy of SRTM DEM in many cases. Progressively, for slope values greater than 20 • , ASTER GDEM permanently exhibits higher accuracy than SRTM DEM. The same does not occur between ASTER GDEM and EU-DEM, as their elevation values generally remain equally reliable in steep slope areas.
The upgraded version (v1.1) of EU-DEM exhibits a rather constant (by about 0.5-1m) deteriorated accuracy with respect to the original EU-DEM version, as expected from the previous results of absolute elevation evaluation (Table 1)

Land Cover
SRTM DEM, ASTER GDEM, and EU-DEM are Digital Surface Models (DSMs), so thus include the elevation of buildings, trees, and other objects, and do not necessarily refer to the ground surface. Consequently, depending on the type of land cover, the actual terrain elevation could be overestimated by DEMs, even if the measurement itself is accurate.
Therefore, as a last step, robust statistics, i.e., median ( Figure 9, Table 3) and NMAD ( Figure 10, Table 4) of the absolute elevation errors were used in order to evaluate the impact of different land cover classes on elevation accuracy. Corine Land Cover 2012 datasets were employed for this purpose.
better. The upgraded version of EU-DEM (v1.1) is consistently characterized by lower accuracy than the original version for all land cover classes, but is also of considerably lower accuracy when compared to SRTM data for almost all land cover classes (apart from wetlands). In forested areas as well as open spaces with little or no vegetation, EU-DEM v1.1. performs worse than ASTER-GDEM. The variability in errors in EU-DEM v1.1 was always considerably lower than both SRTM DEM and ASTER-GDEM and occasionally even lower than that of the original version of EU-DEM.        The results demonstrate that the elevation accuracy of the original version of EU-DEM is higher than in any other DEM for all land cover classes except for forests, where SRTM performs slightly better. The upgraded version of EU-DEM (v1.1) is consistently characterized by lower accuracy than the original version for all land cover classes, but is also of considerably lower accuracy when compared to SRTM data for almost all land cover classes (apart from wetlands). In forested areas as well as open spaces with little or no vegetation, EU-DEM v1.1. performs worse than ASTER-GDEM. The variability in errors in EU-DEM v1.1 was always considerably lower than both SRTM DEM and ASTER-GDEM and occasionally even lower than that of the original version of EU-DEM.

Discussion
The results of this study show that EU-DEM is a product with generally more similarities to SRTM DEM in terms of elevation values, but closer to ASTER GDEM for slopes greater than 20 • . Thus, if restricted to a mountainous area, or steep slopes to be exact, EU-DEM would more approximate ASTER GDEM rather than SRTM data.
The RMSE of the absolute and relative vertical accuracy of EU-DEM were estimated at 4.0 and 5.3 m, respectively, which are better than SRTM DEM (4.8 m and 6.1 m, respectively) and ASTER GDEM (10.2 m and 11.0 m, respectively). Additionally, more robust statistics (median and NMAD) indicate the consistently better performance of EU-DEM. The calculated accuracy applies to the specific area of interest and is thus of local value, but may diverge considerably in other areas depending on the type of terrain and slope. To this end, the whole original EU-DEM dataset (i.e., of regional or even continental value) performs better at both low and high elevations when compared with either SRTM DEM or ASTER GDEM. With respect to the findings of the previous evaluation of the original version of EU-DEM in Hungary [37], the RMSE values were found to be considerably lower (4.0 m in this study versus 6.0 m to 7.8 m in Józsa et al. [37]). When compared to the official validation results [10], the RMSE values (95%) of this study are somewhat higher than those calculated for Greece (3. Concerning the validation of elevation accuracy against different types of land cover, forests present the most problematic areas in all DEMs, with the exception of ASTER-GDEM. ASTER-GDEM has considerably lower accuracy over agricultural areas, which can be linked to its low accuracy over relatively flat terrain (where most agricultural activity is performed).
The EU-DEM upgrade (v1.1) reduces the overall variability in the elevation errors and suppresses outliers. However, this has been accompanied by some loss of elevation accuracy with respect to the original EU-DEM, as indicated by their respective RMSEs of 3.4 m and 3.0 m at 95% confidence. Therefore, as both versions are the result of the implementation of mathematical formulas and do not represent original measurements, they ought to be handled with extra care, especially when the existence of artefacts may have a severe impact (e.g., in hydrological modelling or geomorphological investigations).

Conclusions
In conclusion, EU-DEM is an advancement regarding its combination of spatial resolution and vertical accuracy, achieved by maintaining the higher resolution of ASTER GDEM and the vertical accuracy of both SRTM DEM and ASTER GDEM in low and high slope areas, respectively. While benefiting from the 1 arcsecond spatial resolution of ASTER GDEM, EU-DEM generally follows the consistency of SRTM DEM for relatively low slope values, whereas it exhibits the accuracy of ASTER GDEM in steep terrain. Generally, regarding vertical accuracy and spatial resolution, EU-DEM is an enhanced product, benefiting from both SRTM DEM and ASTER GDEM datasets and meeting the objectives of its implementation.
Overall, owing to the ever growing need for an accurate description of the global topography and its changes, the production of modern and (almost) global or regional DEMs is becoming more frequent, in the form of either primary or secondary products. As these new elevation datasets are typically becoming more accurate, the validation procedure is also becoming more demanding in terms of availability, volume, sampling, and accuracy of reliable reference topographic information. Such information can be relatively easily retrieved with the use of kinematic GNSS surveys, using dual frequency receivers with hundreds of channels and taking advantage of the GNSS constellations currently available (GPS, GLONASS, GALILEO, and BEIDOU).
As a consequence, part of future research on DEM validation should focus both on the accuracy assessment of more DEMs as they are being made available to the scientific community, but also on providing appropriate reference datasets for assessing the reliability and limitations of new and popular topographic products.