Evaluation of ASTER GDEM2 in Comparison with GDEM1, SRTM DEM and Topographic-Map-Derived DEM Using Inundation Area Analysis and RTK-dGPS Data

Abstract

This study evaluates the quality of the Advanced Spaceborne Thermal Emission Radiometer-Global Digital Elevation Model version 2 (ASTER GDEM2) in comparison with the previous version (GDEM1) as well as the Shuttle Radar Topographic Mission (SRTM) DEM and topographic-map-derived DEM (Topo-DEM) using inundation area analysis for the projected location of the Karian dam, Indonesia. In addition, the vertical accuracy of each DEM is evaluated using the Real-Time Kinematic differential Global Positioning Systems (RTK-dGPS) data obtained from an intensive geodetic survey. The results of the inundation area analysis show that GDEM2 produced a higher maximum contour level (MCL) (64 m) than did GDEM1 (55 m), and thus, GDME2 has a better quality. In addition, the GDEM2-derived MCL is similar to those produced by SRTM DEM (69 m) and Topo-DEM (62 m). The improvement in the contour level in GDEM2 is believed to be related to the successful removal of voids (artifacts) and anomalies present in GDEM1. However, our RTK-dGPS results show that the vertical accuracy of GDEM2 is much lower than that of GDEM1 and the other DEMs, which is contradictory to the accuracy stated in the GDEM2 validation document. The vertical profiles of all DEMs show that GDEM2 contains a comparatively large number of undulation effects, thereby resulting in higher root mean square error (RMSE) values. These undulation effects may have been introduced during the GDEM2 validation process. Although the results of this study may be site-specific, it is important that they be considered for the improvement of the next GDEM version.

1. Introduction

The release of the Advanced Spaceborne Thermal Emission Radiometer-Global Digital Elevation Model version 2 (ASTER GDEM2) has enriched the availability of free-of-charge DEM sources, which are especially useful for developing countries, and prompted users to assess its quality and accuracy. In addition, the GDEM2 version is expected to increase the accuracy of its previous version (GDEM1). In this study, we evaluated the quality of GDEM2 relative to GDEM1, the Shuttle Radar Topographic Mission (SRTM) DEM [1] and a topographic-map-derived DEM (Topo-DEM) using inundation area analysis. The vertical accuracy of each DEM was evaluated using Real-Time Kinematic differential Global Positioning Systems (RTK-dGPS) data collected from a geodetic survey.

Since the initial release in 2003, the SRTM DEM (hereafter called SRTM) has been improved several times, culminating most recently with Version 4. SRTM Version 4, which has a 3 arcsec (approximately 90 m × 90 m) ground resolution, is reported to have a vertical error of less than 16 m at a 90% confidence level [2,3]. In addition, ASTER GDEM data have been improved with the GDEM2 version in October of 2011. It has been reported that the overall accuracy of GDEM2, which has a 30 m ground resolution, is approximately 17 m at a 90% confidence level [4], which is three meters more accurate than GDEM1. Assessments of the accuracy in many different locations throughout the world are critical for improving the next generation of GDEMs.

2. Study Area

To analyze the quality of the newly released GDEM2 in comparison to other DEMs, we focused on the projected location of the Karian dam in the Ciujung watershed, Banten Province, Indonesia. Karian is one of several locations proposed by the local government for the construction of a new dam in the anticipation of population growth and an increased need for water supply by 2025. The coordinates of the dam axis are 106°16′56.10″E; 6°24′45.40″S and 106°17′14.70″E; 6°24′57.50″S.

3. Materials and Methods

3.1. DEM Datasets

GDEM1 and GDEM2 data were obtained from http://asterweb.jpl.nasa.gov [4], and SRTM v4.1 data were acquired from http://srtm.csi.cgiar.org [5]. The Topo-DEM data were derived from digital topographic maps (scale 1:25,000) after being transformed into a DEM using the Triangulated Irregular Network tool in ArcGIS 10 (ESRI, Redlands, CA, USA). The topographic map itself was originally derived from aerial photos (acquired in 1993/1994) using analytical photogrammetric methods, and their accuracy was determined by a field survey by the National Mapping Coordination Agency of Indonesia (the history of the map is written in the legend of the map). The grid size of the Topo-DEM was set to 12.5 m based on both the map accuracy specifications and the scale [6]. Consequently, the grid size of the other DEMs was also resampled to 12.5 m using nearest neighbor interpolation, and the raster pixel type/depth was set to use a 16-bit signed integer format for map algebra operations.

All four of the DEMs (GDEM1, GDEM2, SRTM and Topo-DEM) were referenced to a World Geodetic System (WGS84) horizontal datum and to an Earth Gravitational Model 1996 (EGM96) vertical (geoid) datum. To avoid horizontal offsets, a simple “shifting” method was applied following the method of Hirt et al. [5], where one dataset was systematically shifted by small increments (0.5 arcsec) in all directions and compared against an unshifted dataset. The occurrence of offsets was judged by the root mean square error (RMSE). The results revealed that horizontal offsets did not occur any of the DEMs.

For the purpose of the analysis, we considered Topo-DEM to be the most accurate of the compared DEM datasets because Topo-DEM was produced from high-resolution aerial photos followed by photogrammetry and was verified using a field survey. Therefore, the elevation values contained in the Topo-DEM are actually the bare ground elevations, and thus, this dataset is termed a Digital Terrain Model (DTM). In addition, the elevation values in SRTM and ASTER GDEM constitute the height of the tree canopies and man-made features and thus are termed Digital Surface Models (DSMs) [7].

3.2. Inundation Area Analysis

The basic idea of inundation area analysis is to delineate the impoundment area in a watershed that will be covered by water for a specific purpose, such as for flood and contingency planning analysis [8–10], sedimentation in urban drainages [11], irrigation systems [12] and tsunami run-off areas [13]. Because the main source of information for this analysis is elevation data, inundation area analysis is therefore suitable for evaluating the quality of DEMs. The quality of a DEM itself is mainly dependent on the accuracy of the elevation values, the number of voids (artifacts) and the number of anomalies. In addition to inundation area analysis, other methods are often used for assessing quality of a DEM, including watershed delineation analysis [14] and stream networking analysis [15,16].

In this study, inundation area analysis was applied to determine the maximum contour level (MCL) for the proposed Karian dam. The MCL represents the widest possible inundation area (i.e., the impoundment area) in a watershed that can be covered by water, and the quality of a DEM is evaluated based on the MCL values. The location of the dam axis was used as the basis for the analysis. After applying a fill-and-sink removal procedure [17], all DEM sources were analyzed in ArcGIS 10.

3.3. Vertical Accuracy Assessment

In addition to the inundation area analysis, by which the MCL (impoundment boundary) is evaluated, an assessment of the vertical accuracy of each DEM was also investigated using RTK-dGPS data obtained from a geodetic survey according to a procedure that was previously used in a study in Greece [18]. To examine the accuracy of the DEMs at the watershed scale, the geodetic survey was implemented in several locations of the Ciujung watershed (Figure 1).

Two dGPS Promark3 handsets (Magellan, Smyrna, TN, USA) equipped with 110454-type antennas and 111359-type radio modems were employed during a 3-day field observation study. At each station, the fixed GPS unit was set to a known position (above a national geodetic control point) [19], and the mobile GPS unit, the so-called rover, was used to record the x-y-z positions along a trajectory. An initialization-bar occupation process was required before data collection at each station to resolve the integer ambiguity between the satellites and the rover [20]. Both GPS devices were set to record the positional data at 5-s intervals.

Due to the presence of unpaved roads along certain trajectories, especially in the upstream areas where a car could not travel smoothly at a constant speed, we installed the rover GPS on a motorbike. The use of a motorbike, of course, has an advantage and disadvantage. The advantage was that the motorbike could travel over unpaved roads without difficulty, and the disadvantage was that the motorbike induced a small amount of vibration on the equipment.

The Global Navigation Satellite Systems (GNSS) software version 3.10.11 (Ashtech, Westminster, CO, USA) was used in the post-processing analysis, and we used the RMSE to assess the accuracy due its capacity to encompass both the random and systematic errors in the data [21]. The RMSE has become a standard statistical tool for analyzing DEM accuracy and has been used by the U.S. Geological Survey (USGS) and in many other studies [5,7,18].

4. Results and Discussion

4.1. Inundation Area Analysis

The inundation area analysis applied using the available DEMs produced four impoundment boundaries for the proposed Karian dam. The produced boundaries differ in their MCL, size and volume (

Table 1. Maximum contour level, inundation area and water volume derived from various Digital Elevation Models (DEMs) in the projected location of the Karian Dam.

Topo-DEM			SRTM DEM			ASTER GDEM1			ASTER GDEM2
CL	IA	WV	CL	IA	WV	CL	IA	WV	CL	IA	WV
63 (over)			70 (over)			56 (over)			65 (over)
62 (max)	10.56	154.33	69 (max)	14.81	177.62	55 (max)	6.51	73.61	64 (max)	8.01	99.75

Notes: CL = Contour Level (m); IA = Inundation Area (km²); WV = Water Volume (million·m³).

). However, in this study, we only focused on the MCL because the size and volume are secondary products that require a separate investigation.

Of the evaluated models, the MCL resulting from GDEM2 was the most similar to the MCL obtained using Topo-DEM, which was considered the most accurate dataset among the comparable DEMs. The MCLs obtained using Topo-DEM and GDEM2 were 62 m and 64 m, respectively. The SRTM data produced a slightly higher MCL of 69 m. In addition, the GDEM1 data yielded the lowest MCL value of only 55 m. The shapes of the impoundment boundaries obtained using GDEM2 and SRTM were highly similar to that of the Topo-DEM. Meanwhile, a small difference in the boundary was observed for GDEM1 (Figure 2).

The more accurate contouring level of GDEM2 relative to SRTM and GDEM1 demonstrated that GDEM2 has been improved regarding voids, anomalies and flat lake surface problems. These artifacts have been substantially reduced from GDEM1 by the National Geospatial-Intelligence Agency (NGA) using an extensive visual identification method [4]. The remaining difference in the MCL of Topo-DEM and the MCLs of SRTM/GDEM2 could be related to conceptual differences in DTM/DSM. The effect of the canopy and man-made features on the elevation value is an interesting subject for future studies. The factors influencing the large differences in the size and volume of the impoundment boundaries also require further investigation.

Although the observations reported here are based on a simple analysis, the results imply that GDEM1 users should be cautious when using these data, especially when GDEM1 is used for hydrologic studies that require precise results, such as flood disaster analysis. Up to this point, the use of GDEM2 and SRTM is strongly recommended because these datasets provided better impoundment boundaries.

The results of this analysis, however, could not provide information regarding the accuracy of the vertical elevation of the DEMs. Although an MCL is based on elevation values, the created boundary is strongly dependent on the conditions surrounding the dam axis and the projected dam, such as the terrain, slope and land cover. The accuracy of a DEM in a large watershed can only be evaluated by conducting an intensive geodetic survey, in which a huge number of geographic positions (x, y and z) of the earth are densely recorded over a trajectory in several different types of terrain and land cover. After performing a geodetic survey, the vertical profile of each DEM can be compared with the elevation values that were measured in the field.

4.2. Vertical Accuracy Assessment

Although the analysis presented above demonstrated an improvement of GDEM2 over GDEM1, the elevation values of the DEMs must also be assessed. In this section, the vertical accuracy of each DEM is evaluated based on the height values derived from a geodetic RTK-dGPS dataset.

4.2.1. The Quality of RTK-dGPS Data

Within the limitation of the survey design, as described in the Materials and Methods section, the RTK-dGPS data produced satisfactory Positional Dilution of Precision (PDOP) values with an average of 2.26 ± 0.54 (determined from 3,661 survey points); 36.5% of the values were within the range of 1–2, whereas the remaining 63.5% were within the range of 2–5. Based on the work of Kaya and Saritaş [22], the PDOP ranges of 1–2 and 2–5 are classified as “excellent” and “good”, respectively. The number of satellites in view (SV), with an average of 8.21 ± 1.30, was also adequate for typical hydrological studies.

The measures of PDOP and SV are useful for assessing GPS data quality [23,24]. An additional important measure of GPS quality is the estimated accuracy of GPS data. It was difficult to estimate the overall horizontal and vertical accuracies of our data because the rover GPS traveled across different types of land cover. The results showed excellent accuracy when the rover GPS traveled across open roads; however, the accuracy decreased when certain high objects (e.g., trees and man-made features) were present along the roadsides or when the distance between two GPS unit was greater than 1 km.

An example of the vertical accuracy assessment is presented using data from Station 6 (Figure 3), where two peaks of biases appeared in the result due to the aforementioned influences. At this station, the estimated horizontal and vertical accuracies were 0.625 m and 0.605 m, respectively, or 0.427 m and 0.402 m, respectively, when the influencing factors are filtered out. The biases from the other stations were higher and dependent on the land cover types along the roads and the distance between the pair of GPS units. Here, we note that the survey design requires certain improvements to eliminate the effects of the land cover types and decrease the distance between the GPS units.

4.2.2. Vertical Accuracy of the DEMs

After filtering out the large outliers influenced by high objects, the RTK-dGPS data were used to measure the vertical accuracy of all DEMs compared in this study, including the Topo-DEM. Although the Topo-DEM was considered to have the most accurate data, it may contain certain elevation biases relative to the RTK-dGPS data. The vertical accuracy assessment was applied not only to the DEMs at their original resolutions but also to the DEMs after being resampled to the smallest resolution, which was equal to the 12.5 m resolution of the Topo-DEM.

Our results showed that when comparing the DEMs at their original resolutions, the Topo-DEM demonstrated the most accurate data with an average RMSE of 3.204 m at the 95% confidence level (

Table 2. Results of vertical accuracy assessment between various DEMs with Real-Time Kinematic differential Global Positioning Systems (RTK-dGPS) data using a linear regression model.

Station	RMSE (m) (α = 0.95) (Original Resolution)				RMSE (m) (α = 0.95) (Resampled to 12.5 m Resolution)				Slope
	Topo-DEM	SRTM	GDEM1	GDEM2	Topo-DEM	SRTM	GDEM1	GDEM2	(%)
B01	2.927	3.571	3.512	5.319	2.927	3.269	3.526	4.912	2.66
B02	4.020	2.348	3.801	4.771	4.020	1.963	3.835	4.330	2.92
B03	3.110	3.924	5.393	5.745	3.110	2.742	5.169	5.194	3.67
B04	3.676	3.820	4.159	4.690	3.676	2.935	4.101	4.140	4.04
B05	3.044	4.171	3.985	6.230	3.044	3.146	4.036	6.003	4.82
B06	1.414	2.635	2.470	4.914	1.414	1.725	2.426	4.523	4.49
B07	3.175	2.898	4.459	5.821	3.175	2.725	4.430	5.412	2.56
B08	4.033	3.505	6.233	7.759	4.033	3.185	6.117	7.158	2.99
B09	2.598	2.214	2.629	6.203	2.598	1.510	2.523	5.890	3.86
B10	4.535	4.006	4.046	6.513	4.535	3.495	4.056	4.911	3.23
B11	2.712	2.661	3.807	4.543	2.712	2.388	3.728	4.232	2.46
Average	3.204	3.250	4.045	5.683	3.204	2.644	3.995	5.155	3.43

). Among the satellite-derived DEMs, the SRTM data showed a better vertical accuracy than both GDEM1 and GDEM2. More surprisingly, the vertical accuracy of GDEM2 was less than the accuracy of GDEM1, as it achieved a much higher RMSE compared to GDEM1. This result was surprising because it was contradictory to the accuracy reported by the GDEM2 validation team [4].

After resampling to 12.5 m × 12.5 m resolution, the comparison of the accuracy of all DEMs was repeated. The resampling procedure greatly increased the accuracy of all DEMs, as demonstrated by their average RMSEs. Moreover, the vertical accuracy of SRTM was now better than that of Topo-DEM, which indicates that the Topo-DEM data may need certain corrections or revalidation. Nevertheless, this procedure demonstrated that SRTM continued to yield a better accuracy than both GDEM versions, and GDEM2 data yielded the lowest vertical accuracy of all the tested DEM datasets. The greater accuracy of SRTM over GDEM1 has been previously reported in many studies [5,14,25,26]. However, this study is the first investigation to report the lower accuracy of GDEM2 data.

To determine the cause of the relatively low accuracy of GDEM2, we plotted the vertical profiles of the DEMs along the dGPS trajectories. Three examples of these profiles are shown in Figure 4. We chose these trajectories because for these data, the rover travelled along good-quality paved roads. These results demonstrated that the GDEM2 data contain a large extent of undulation effects, thereby causing high RMSE values.

Given that the rover GPS for these stations was moving along good-quality paved roads, it is impossible that the trajectories reflect a large degree of unevenness of the roads. Instead, we believe that the undulation effects in GDEM2 may have been introduced during the validation process. The GDEM2 validation team has explained that the validation process included cloud effects removal, land cover reclassification, canopy effects reduction, the addition of a number of scenes and other factors [4] through which the undulation effects may have been inadvertently introduced.

4.2.3. Undulation Effects

In addition to the elevation difference in the DTM/DSM, which was previously addressed, it is always important to remember that elevation information contained in a dataset is different depending on the baseline reference. GPS delivers ellipsoidal heights, whereas SRTM and GDEM provide mean sea level (MSL) heights. The difference between these data, which is called the geoid height, varies for any location of in world because the MSL heights are affected by gravitational forces.

We analyzed Station 7 an example of the dataset (Figure 5). In our data, the elevation of the Topo-DEM was mostly (but not in all cases) lower than the SRTM and GDEM because the later datasets contained the bare ground elevation. In addition, the SRTM and GDEM elevations were higher than in the Topo-DEM because they included the canopy height information. At this station, the average geoid heights are 3.835 m, 4.478 m and 5.372 m for SRTM, GDEM1 and GDEM2, respectively. Although disparities between the DEMs are best judged according to their RMSE values [5,7,18], the average geoid heights and the undulation pattern, as displayed in Figure 5, can help in understanding the disparities between datasets.

5. Conclusions

This study provides additional information for public users and the GDEM2 validation team regarding the quality of GDEM2 data. Inundation area analysis of the projected Karian dam (Indonesia) and the RTK-dGPS data collected from Ciujung watershed were used to assess the quality of GDEM2 data. The results of inundation area analysis showed that the GDEM2 data was highly improved by the removal of voids and anomalies and thereby produced a better MCL (impoundment boundary). The MCL produced from GDEM2 was 64 m, which was a much better value than the MCL produced from GDEM1 (55 m) and closer to the MCL produced from Topo-DEM (62 m) and SRTM-DEM (69 m). However, the vertical accuracy of GDEM2 was found to be lower than that of GDEM1 and the other DEMs, as indicated by the RTK-dGPS data and the RMSE values. The average RMSE values for the Topo-DEM, SRTM, GDEM1 and GDEM2 were 3.204, 3.250, 4.045 and 5.683, respectively. The lower accuracy of GDEM1 could be caused by undulation effects, which were found throughout the observed stations. Although our initial findings cannot be generalized to all GDEM2 data, we are pleased to report that there is evidence of the undulation effects in GDEM2 present in the study area. We believe that the undulation effects may have been introduced during the validation process, perhaps due to cloud effect removal, land cover reclassification or other factors that are unknown to the authors. To obtain a clear understanding of these effects, an intensive study involving additional sampling stations over a wider area is necessary in the future.