Accuracy Assessment of Different Digital Surface Models

Digital elevation models (DEMs), which can occur in the form of digital surface models (DSMs) or digital terrain models (DTMs), are widely used as important geospatial information sources for various remote sensing applications, including the precise orthorectification of high-resolution satellite images, 3D spatial analyses, multi-criteria decision support systems, and deformation monitoring. The accuracy of DEMs has direct impacts on specific calculations and process chains; therefore, it is important to select the most appropriate DEM by considering the aim, accuracy requirement, and scale of each study. In this research, DSMs obtained from a variety of satellite sensors were compared to analyze their accuracy and performance. For this purpose, freely available Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) 30 m, Shuttle Radar Topography Mission (SRTM) 30 m, and Advanced Land Observing Satellite (ALOS) 30 m resolution DSM data were obtained. Additionally, 3 m and 1 m resolution DSMs were produced from tri-stereo images from the SPOT 6 and Pleiades high-resolution (PHR) 1A satellites, respectively. Elevation reference data provided by the General Command of Mapping, the national mapping agency of Turkey—produced from 30 cm spatial resolution stereo aerial photos, with a 5 m grid spacing and ±3 m or better overall vertical accuracy at the 90% confidence interval (CI)—were used to perform accuracy assessments. Gross errors and water surfaces were removed from the reference DSM. The relative accuracies of the different DSMs were tested using a different number of checkpoints determined by different methods. In the first method, 25 checkpoints were selected from bare lands to evaluate the accuracies of the DSMs on terrain surfaces. In the second method, 1000 randomly selected checkpoints were used to evaluate the methods’ accuracies for the whole study area. In addition to the control point approach, vertical cross-sections were extracted from the DSMs to evaluate the accuracies related to land cover. The PHR and SPOT DSMs had the highest accuracies of all of the testing methods, followed by the ALOS DSM, which had very promising results. Comparatively, the SRTM and ASTER DSMs had the worst accuracies. Additionally, the PHR and SPOT DSMs captured man-made objects and above-terrain structures, which indicated the need for post-processing to attain better representations.


Introduction
Digital elevation models (DEMs) are important data sources for several applications that require surface height information [1].A DEM is a 3D projection of the Earth that can be categorized into two groups: digital terrain models (DTMs), which are free of trees, buildings, and all types of objects, and digital surface models (DSMs), which reflect the Earth's surface, including all man-made and natural objects [2].A DEM can be found in a raster data format, which is an array of square cells DSM data can be categorized into two groups based on coverage extension, resolution, and delivery options.The first group is produced from medium-resolution spatial sensors, which are available worldwide and are mainly distributed free of charge.The Shuttle Radar Topography Mission (SRTM), the Advanced Spaceborne Thermal Emission and Reflection Radiometer Global DEM (ASTER GDEM), and the Advanced Land Observing Satellite (ALOS) World 3D (AW3D) are categorized in this group and capture nearly all of Earth's landmass free of charge.The second group is composed of local DEMs that are produced from medium-to very high-resolution optical or SAR satellite image data for a limited area of interest.In this study, the vertical accuracies of DSMs belonging to these two groups were evaluated together.
Although global accuracy metrics are available for medium-resolution DSM data, to our knowledge, there has not been a detailed comparison of local DSMs that are produced from high-resolution stereo/tri-stereo satellite images that considers the effects of different land cover characteristics on vertical accuracy.
The main objective of this research is to evaluate the relative vertical agreement of different DEMs that are in the DSM form compared to a reference 5 m grid spaced DSM.For this purpose, the ASTER 30 m, SRTM 30 m, and ALOS 30 m DSMs were obtained for the research area using a 3 m resolution DSM produced from tri-stereo images of the SPOT 6 satellite and a 1 m resolution DSM produced from tri-stereo images of the Pleiades high-resolution (PHR) 1A satellite.The reference DSM was produced from 30 cm spatial resolution aerial photos and provides ±1 m and ±3 m vertical accuracy at the 90% confidence interval (CI) in flat and hilly areas, respectively.The relative vertical agreement of the DSMs was tested with different accuracy assessment approaches in order to provide information about the following aspects: 1.
The comparative and quantitative vertical accuracy of the DSMs in the study region.

2.
The ranking of the comparative accuracy of the DSMs for specific land cover classes.

3.
The performance of the DSMs in bare lands (i.e., terrain representation).

Study Area and Data
The study area selected for this research is inside the Istanbul metropolitan area in Turkey.The study area was selected according to data availability, and it includes forested, residential, and industrial areas of Istanbul that have experienced minimal change in recent decades.Specifically, the residential areas that consisted of high-rises and different types of buildings were good candidates for evaluation of extreme conditions (Figure 1).In this research, freely available ASTER 30 m, SRTM 30 m, and ALOS 30 m DEMs were used as the first dataset group.These data were in DSM form, and a general description of each dataset is provided below.
NASA released the latest version of the SRTM DEM (v.3.0) in 2015.SAR images with a very small base=to-height ratio were processed to obtain DSMs by using interferometric SAR principles.The cartographic products, which were generated from the SRTM data, were sampled numerically on a grid of 1 arcsec (approximately 30 m) [26].Several studies were performed in order to verify the accuracy of the SRTM DEMs by comparing the results with various reference data and other DEM products [27][28][29][30].The validation report provided a 6.6 m absolute vertical error, according to kinematic GPS-based ground control point (GCP) comparison, and an 8.5 m absolute vertical error, according to the land GCP comparison for Eurasia [31].More recent studies reported 12.4 m and 11.9 m vertical root mean square errors (RMSEs) for Europe and Eurasia, respectively [32].
The first version of the ASTER GDEM data was introduced to the global user community in July 2009, and an enhanced version (v.2), which was produced with additional data, improved water masking, and improved vertical accuracy, was distributed on 17 October 2011, by NASA and the METI.The ASTER GDEM v.2 vertical and horizontal RMSEs were calculated as approximately 12 m and 6 m, respectively, which showed an important quality improvement over the previous version [27].In particular, the ASTER and SRTM DEMs can have faulty elevation values due to radar shadows, clouds, or low contrast [30].
A new global DSM dataset was produced from the 2.5 m spatial resolution data acquired by the Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM) onboard the ALOS.The AW3D project provided the DSM with a decent resolution of 0.15 arcsec (approximate 5 m), which is currently the most precise global-scale elevation.The first version of the AW3D DSM was distributed to commercial bases by the NTT DATA and RESTEC in March 2016 [33].In 2015, the Japan Aerospace Exploration Agency (JAXA) released a free-of-charge DSM named the AW3D-30 m, which was a global DSM dataset with a horizontal resolution of approximately 30 m (1 arcsec).In fact, these data were a resampled version of the 5 m mesh version of the AW3D [34].A recent study performed by Takuku et al. reported a 3.28 m vertical RMSE worldwide and a 3.69 m vertical RMSE for Turkey [35].
The second group of DSM datasets in this research was produced from tri-stereo images acquired by the PHR 1A and 1B and the SPOT 6 and 7 optical satellites.The PHR 1A and PHR 1B satellites were successfully launched into orbit in 2011 and 2012, respectively.The PHR satellites have In this research, freely available ASTER 30 m, SRTM 30 m, and ALOS 30 m DEMs were used as the first dataset group.These data were in DSM form, and a general description of each dataset is provided below.
NASA released the latest version of the SRTM DEM (v.3.0) in 2015.SAR images with a very small base=to-height ratio were processed to obtain DSMs by using interferometric SAR principles.The cartographic products, which were generated from the SRTM data, were sampled numerically on a grid of 1 arcsec (approximately 30 m) [26].Several studies were performed in order to verify the accuracy of the SRTM DEMs by comparing the results with various reference data and other DEM products [27][28][29][30].The validation report provided a 6.6 m absolute vertical error, according to kinematic GPS-based ground control point (GCP) comparison, and an 8.5 m absolute vertical error, according to the land GCP comparison for Eurasia [31].More recent studies reported 12.4 m and 11.9 m vertical root mean square errors (RMSEs) for Europe and Eurasia, respectively [32].
The first version of the ASTER GDEM data was introduced to the global user community in July 2009, and an enhanced version (v.2), which was produced with additional data, improved water masking, and improved vertical accuracy, was distributed on 17 October 2011, by NASA and the METI.The ASTER GDEM v.2 vertical and horizontal RMSEs were calculated as approximately 12 m and 6 m, respectively, which showed an important quality improvement over the previous version [27].In particular, the ASTER and SRTM DEMs can have faulty elevation values due to radar shadows, clouds, or low contrast [30].
A new global DSM dataset was produced from the 2.5 m spatial resolution data acquired by the Panchromatic Remote-sensing Instrument for Stereo Mapping (PRISM) onboard the ALOS.The AW3D project provided the DSM with a decent resolution of 0.15 arcsec (approximate 5 m), which is currently the most precise global-scale elevation.The first version of the AW3D DSM was distributed to commercial bases by the NTT DATA and RESTEC in March 2016 [33].In 2015, the Japan Aerospace Exploration Agency (JAXA) released a free-of-charge DSM named the AW3D-30 m, which was a global DSM dataset with a horizontal resolution of approximately 30 m (1 arcsec).In fact, these data were a resampled version of the 5 m mesh version of the AW3D [34].A recent study performed by Takuku et al. reported a 3.28 m vertical RMSE worldwide and a 3.69 m vertical RMSE for Turkey [35].
The second group of DSM datasets in this research was produced from tri-stereo images acquired by the PHR 1A and 1B and the SPOT 6 and 7 optical satellites.The PHR 1A and PHR 1B satellites were successfully launched into orbit in 2011 and 2012, respectively.The PHR satellites have agile sensors that enable tri-stereo along-track acquisitions, which result in three images over the same area with different acquisition angles during a single pass, one of which is almost in nadir.This configuration provides a promising dataset for modeling topography within the observed scene [36].The spatial resolutions of the panchromatic (PAN) and multispectral (MS) images are 0.70 m and 2.80 m, respectively, with a theoretical swath of 21 km.The SPOT 6 satellite was launched in 2012 and was followed by the SPOT 7 satellite in 2014.These twin satellites also have the capability of tri-stereo along-track imaging and provide 1.5 m PAN spatial resolution and 6 m MS spatial resolution products with a 60 km swath width [37].
The acquisition parameters of the tri-stereo satellite images used in this research are given in Table 2. To perform the tests, elevation reference DSM data were provided by the General Command of Mapping (Harita Genel Komutanlı gı-HGK).The HGK DSM data were produced from 30 cm spatial resolution stereo aerial photographs acquired with a Microsoft UltraCam Eagle multispectral camera onboard a Beechcraft Super King Air B-200 aircraft [38].The reference DSM was produced using automated image matching and was edited manually to mask gross errors and water surfaces (i.e., seas, lakes, and wide streambeds).The 5 m grid spaced DSM provides ±1 m and ±3 m vertical accuracy at the 90% (CI) in flat and hilly areas, respectively [39].As the HGK DSM was produced from the highest spatial resolution input data and underwent a manual editing process, it was selected as the reference data when compared to the other datasets used in this research.All DSM data used in this study are presented in Figure 2.
ISPRS Int.J. Geo-Inf.2018, 7, x FOR PEER REVIEW 5 of 16 agile sensors that enable tri-stereo along-track acquisitions, which result in three images over the same area with different acquisition angles during a single pass, one of which is almost in nadir.This configuration provides a promising dataset for modeling topography within the observed scene [36].
The spatial resolutions of the panchromatic (PAN) and multispectral (MS) images are 0.70 m and 2.80 m, respectively, with a theoretical swath of 21 km.The SPOT 6 satellite was launched in 2012 and was followed by the SPOT 7 satellite in 2014.These twin satellites also have the capability of tri-stereo along-track imaging and provide 1.5 m PAN spatial resolution and 6 m MS spatial resolution products with a 60 km swath width [37].
The acquisition parameters of the tri-stereo satellite images used in this research are given in Table 2. To perform the tests, elevation reference DSM data were provided by the General Command of Mapping (Harita Genel Komutanlığı-HGK).The HGK DSM data were produced from 30 cm spatial resolution stereo aerial photographs acquired with a Microsoft UltraCam Eagle multispectral camera onboard a Beechcraft Super King Air B-200 aircraft [38].The reference DSM was produced using automated image matching and was edited manually to mask gross errors and water surfaces (i.e., seas, lakes, and wide streambeds).The 5 m grid spaced DSM provides ±1 m and ±3 m vertical accuracy at the 90% (CI) in flat and hilly areas, respectively [39].As the HGK DSM was produced from the highest spatial resolution input data and underwent a manual editing process, it was selected as the reference data when compared to the other datasets used in this research.All DSM data used in this study are presented in Figure 2.

DSM Generation from PHR 1A and 1B and SPOT 6 and 7 Satellite Images
There are different types of commercial and scientific software that process high-resolution satellite optical imagery for DSM generation.The tri-stereo satellite images used in this research were processed in the OrthoEngine module of the PCI Geomatica software for DSM extraction.The DSM was generated by image correlation of the epipolar geometry and application of the acquisition geometry with a rational function model.The Wallis filter was applied to the DSM generation in order to improve the contrast and matching ratio.The automated image matching algorithm was applied to produce matching points from the pixels in the image pairs.The algorithm used a mean-normalized cross-correlation with a multiscale strategy.In the next step, three epipolar images-fore-nadir, nadir-after, and fore-after-were produced at the spatial resolution of the source images.In the last step, the DSM was produced according to the mean sea level datum, with a sampling factor of 2 that was equivalent to two times that of the spatial resolution [20].

Production of Land Cover Map and Independent Checkpoints
The land cover classification of the study area was performed by the visual interpretation of on-screen digitization, using the 30 cm orthophotos, that was produced from the same aerial photographs used in the reference HGK DSM data.These HGK orthophotos provide ±2.5 m horizontal positional accuracy according to unpublished reports from the producer.Generated classes were generalized according to the density of the land cover inside the patch (Figure 3a).A resulting map was used to perform a comparative accuracy analysis of the DSMs related to the land cover classes.Figure 3b shows the elevation map generated from the reference HGK DSM data.The tested area was between 0 m (sea level) and 190 m, according to the reference data.Checkpoints were generated randomly with a stratified sampling strategy based on the elevation intervals.

DSM Generation from PHR 1A and 1B and SPOT 6 and 7 Satellite Images
There are different types of commercial and scientific software that process high-resolution satellite optical imagery for DSM generation.The tri-stereo satellite images used in this research were processed in the OrthoEngine module of the PCI Geomatica software for DSM extraction.The DSM was generated by image correlation of the epipolar geometry and application of the acquisition geometry with a rational function model.The Wallis filter was applied to the DSM generation in order to improve the contrast and matching ratio.The automated image matching algorithm was applied to produce matching points from the pixels in the image pairs.The algorithm used a meannormalized cross-correlation with a multiscale strategy.In the next step, three epipolar images-forenadir, nadir-after, and fore-after-were produced at the spatial resolution of the source images.In the last step, the DSM was produced according to the mean sea level datum, with a sampling factor of 2 that was equivalent to two times that of the spatial resolution [20].

Production of Land Cover Map and Independent Checkpoints
The land cover classification of the study area was performed by the visual interpretation of onscreen digitization, using the 30 cm orthophotos, that was produced from the same aerial photographs used in the reference HGK DSM data.These HGK orthophotos provide ±2.5 m horizontal positional accuracy according to unpublished reports from the producer.Generated classes were generalized according to the density of the land cover inside the patch (Figure 3a).A resulting map was used to perform a comparative accuracy analysis of the DSMs related to the land cover classes.Figure 3b shows the elevation map generated from the reference HGK DSM data.The tested area was between 0 m (sea level) and 190 m, according to the reference data.Checkpoints were generated randomly with a stratified sampling strategy based on the elevation intervals.

Geometric Registration
For the vertical accuracy assessment of several DSMs, it is important to ensure horizontal location matching.The ASTER, SRTM, and ALOS data have acceptable horizontal accuracies

Geometric Registration
For the vertical accuracy assessment of several DSMs, it is important to ensure horizontal location matching.The ASTER, SRTM, and ALOS data have acceptable horizontal accuracies regarding their spatial resolution.During the production of the PHR and SPOT DSMs, the geometric accuracy was improved by using GCPs.The co-registration of the DSMs was controlled visually by checking the locational fit of streambeds and roads that were observable in the DSM data.As a result, all of the DSM data obtained from different sources were horizontally matched with each other.

Accuracy Assessment
After the preprocessing steps were completed, the vertical accuracy assessment of the DSMs was performed in three different aspects.
In the first approach, a point-based assessment was performed on bare lands in order to evaluate the accuracy of the DSMs for terrain representation.For this purpose, 25 checkpoints that were located in bare lands were collected from 30 cm resolution HGK orthophotos.Then, height values belonging to these GCPs were derived for each DSM datum, including the reference DSM, using an overlay analysis.After deriving the height information, RMSE, accuracy (at the 95% CI), and standard deviation were calculated separately for all of the DSM data (Equations ( 1) and ( 2)).

RMSE
where Z i corresponds to the height value measured from the reference DSM and Z' i corresponds to the height value measured from the test DSMs.
In the second approach, 1030 randomly selected checkpoints were selected from the whole study region to assess the vertical accuracy of the DSMs.Stratified sampling was performed according to the different elevation intervals derived from the reference DSM.The spatial distribution of checkpoints is provided in Figure 3b.The accuracy metrics provided in the first approach were also calculated for these points, and their land cover labels were created by using an intersection analysis with the land cover map provided in Figure 3a.The accuracy metric given in Equation ( 2) is based on the assumption that vertical errors are normally distributed.To test this condition, frequency histograms of height differences (theoretically, the errors) were produced for each DSM (Figure 4).According to the results, the ALOS, ASTER, and SPOT DSMs provide normal distribution characteristics, and the PHR DSM is very close to a normal distribution with a very slight positive skew, while the same situation is observed for the SRTM DSM with a very slight negative skew.These results indicate that the accuracy metric can be used for this research.In the last approach, vertical profiles were produced for different geographic directions that corresponded to 45° angular intervals for the whole DSM dataset.In the next step, elevation change characteristics for each DSM in 1 km length portions of the profiles under several topographic conditions and land cover scenarios were determined.Figure 5 presents the locations of the profiles over the study region.In the last approach, vertical profiles were produced for different geographic directions that corresponded to 45 • angular intervals for the whole DSM dataset.In the next step, elevation change characteristics for each DSM in 1 km length portions of the profiles under several topographic conditions and land cover scenarios were determined.Figure 5 presents the locations of the profiles over the study region.

Results of the Point-Based Assessment
After the preprocessing steps, the vertical accuracy of the DSMs was evaluated.For the first approach, elevation information extracted from 25 checkpoints was compared with that corresponding to the reference DSM.According to the comparison results given in Table 3, the DSM produced from the tri-stereo PHR images provided the lowest overall RMSE and the highest accuracy in bare terrain conditions.The ALOS and SPOT DSMs also provided acceptable RMSEs at approximately 2 m, while the SRTM DSM ranked fourth, and the ASTER DSM ranked last with the highest RMSE and the lowest accuracy.When the standard deviation of the errors was examined, the PHR, ALOS, and SPOT DSMs provided lower values, which implies that the error magnitudes for the different checkpoints were similar, while the other DSMs showed variations among each other.In the second approach, 1030 randomly selected points were used in the accuracy assessment, and the same metrics used in the first approach were extracted.Because these points were distributed homogenously over the study region, the results provided a generalized quantification of accuracy for different land cover and topography types.It is important to note that the PHR and SPOT DSMs were not post-processed and included defects due to improper image matching.Some of these defects were observable in the Northwest side of the PHR DSM in Figure 2. Considering the defects mentioned above, a secondary analysis was performed by removing 30 points, which provided

Results of the Point-Based Assessment
After the preprocessing steps, the vertical accuracy of the DSMs was evaluated.For the first approach, elevation information extracted from 25 checkpoints was compared with that corresponding to the reference DSM.According to the comparison results given in Table 3, the DSM produced from the tri-stereo PHR images provided the lowest overall RMSE and the highest accuracy in bare terrain conditions.The ALOS and SPOT DSMs also provided acceptable RMSEs at approximately 2 m, while the SRTM DSM ranked fourth, and the ASTER DSM ranked last with the highest RMSE and the lowest accuracy.When the standard deviation of the errors was examined, the PHR, ALOS, and SPOT DSMs provided lower values, which implies that the error magnitudes for the different checkpoints were similar, while the other DSMs showed variations among each other.In the second approach, 1030 randomly selected points were used in the accuracy assessment, and the same metrics used in the first approach were extracted.Because these points were distributed homogenously over the study region, the results provided a generalized quantification of accuracy for different land cover and topography types.It is important to note that the PHR and SPOT DSMs were not post-processed and included defects due to improper image matching.Some of these defects were observable in the Northwest side of the PHR DSM in Figure 2. Considering the defects mentioned above, a secondary analysis was performed by removing 30 points, which provided extreme absolute errors (±30 to 80 m) in the PHR and SPOT DSMs.After removal of these points, the RMSEs decreased and the accuracy increased in all of the DSMs, which produced a significant improvement in the PHR DSM (Table 4).These results indicate a need for the post-processing of local DSMs from high-resolution images to remove artifacts.After the removal of points with extreme error, the PHR and SPOT DSMs provided the highest accuracy and the lowest RMSEs.The PHR DSM had some elevation defects due to high-rise buildings, although the detection of these objects was successful.This DSM was followed by the ALOS, SRTM, and ASTER DSMs, respectively.Standard deviation metrics showed behavior to the RMSE and accuracy, indicating that the variations in error magnitude were directly proportional to the RMSE and accuracy.The accuracies of the DSMs for different land cover types were evaluated by grouping the checkpoints according to their land cover source, then calculating the overall RMSE for each land cover type for all of the DSMs (Table 5).The evaluation results showed that the SPOT DSM provided closer RMSE values for all classes and the best accuracies on average.The PHR DSM provided similar results to the SPOT DSM (excluding the high-rise buildings) and ranked second in terms of average accuracy.The ALOS DSM provided very promising results by achieving similar accuracies to the local DSMs for four different land cover types; however, the accuracies in forests and over high-rise buildings were significantly lower, resulting in a third-place ranking on average.The SRTM and ASTER DSMs provided similar accuracies, with a slightly lower RMSE in the SRTM DSM on average.A table is produced according to the RMSE values, which ranks the DSMs based on land cover type (Table 6).

Accuracy Assessment by Profile
For the second part of the accuracy assessment, five different 1000 m length profile portions were evaluated, the directions of which are defined in Figure 4.For the evaluation, profiles from each DSM were overlaid on their respective reference profiles to detect the differences visually.Land cover type and distance are presented along the horizontal axis, and elevation is presented along the vertical axis.
According to Profile 3a, from the residential area review, although the ALOS, ASTER, and SRTM DSMs were close to the reference DSM, their spatial resolution was not enough to detect the individual buildings.The SPOT and PHR DSMs successfully captured the buildings; however, the main problem with those DSMs was the changing elevation values that were observed along the same buildings, specifically towards the edges and corners.Therefore, further post-processing is needed for high-resolution DSMs, specifically for man-made object elevations (Figure A1).
According to Profile 3b, there was an 80 m elevation change in the second profile route.The ASTER DSM provided the worst result for this profile, with a 10 m shift compared to the reference DSM.There was an important height increase with respect to the reference in the 2630th meter of the SPOT DSM profile that most likely corresponded to new building construction.This change was not available in the PHR DSM (Figure A2).
According to the route in Profile 3c, the PHR and SPOT DSMs were able to detect high buildings.In addition, the SPOT and PHR DSMs provided similar results as compared to the reference data in forest patches.The ALOS, ASTER, and SRTM DSMs failed to represent buildings in the 3200th and 3650th meters, which was most likely due to differences in image acquisition and building construction dates (Figure A3).
In the Profile 4a route, a sudden and similar altitude change was detected in all of the DSMs.The height change between 2340 m and 2440 m could not be detected with the ASTER DSM or the SRTM DSM.Moreover, buildings that were observable in the 2750 m to 3000 m interval of the profile could only be determined by the PHR and SPOT DSMs (Figure A4).
In Profile 4b, the accuracy differences of the DSMs in the mountainous, dense forest area were obvious.Errors in this area were relatively high for the SRTM and ASTER DSMs.It turns out that the SPOT and PHR DSMs were extremely accurate in this area, followed by the ALOS DSM.Differences in the ASTER DSM reached 20 m, and errors were observable in a majority of the route.In the SRTM DSM, there was a soft transition between heights, which caused faults to increase in the top and pit areas (Figure A5).
As a general observation, the DSMs produced from the high-resolution satellite images provided comparatively higher accuracies according to several evaluation methods applied in this research.While the SPOT DSM ranked first, followed by the PHR DSM in complex land cover, the PHR DSM provided the highest accuracy in bare terrain conditions.Local DSMs, such as the SPOT and PHR DSMs mentioned in this research, provided up-to-date elevation information for the region of interest and were more successful in the detection of above-terrain objects by means of their high spatial resolutions.On the other hand, the production of these local DSMs required GCPs to ensure horizontal and vertical accuracies, and post-processing was needed to remove artifacts for better quality.
Concerning the freely available global DSMs, the ALOS DSM provided more than satisfactory results, with acceptable accuracy on bare lands, producing results that were similar to the local, high-resolution DSMs in four out of the seven land cover classes.It was ranked third in the average evaluation.The higher accuracy of the ALOS DSM over the SRTM and ASTER DSMs can be explained by the original 5 m spatial resolution that was down-sampled to 30 m for free distribution.Another factor influencing the higher accuracy of the ALOS DSM was that it was produced from the most recent dataset of these three DSMs.The SRTM and ASTER DSMs provided similar accuracies in most situations, while the SRTM DSM had slightly better accuracy in five out of the seven land cover classes when compared to the ASTER DSM.It should be noted that the freely available DSMs were produced from datasets that were acquired in a single period, and their accuracy is limited for areas that have been subject to significant land cover changes in recent years.

Conclusions
DSMs are very important data sources for several remote sensing and geospatial applications; therefore, it is important to analyze the accuracy of DSMs.This research provided comparative evaluation of DSMs in terms of relative vertical agreement using accuracy metrics.The results of this study illustrated higher accuracy values for the PHR and SPOT DSMs, which was coherent with the spatial resolution of the input dataset.In most cases, a high-resolution DSM improved vertical accuracy; however, there were several noisy effects in these DSMs, specifically at the borders and corners of man-made structures, which require further processing of high-resolution DSMs.The ALOS DSM produced very good results, specifically compared to other freely available DSMs.Although the ALOS DSM had a 30 m grid spacing, it could be deduced that this was due to the acquisition of strong signals from the original 5 m DSM, which was produced from the 2.5 m images.The accuracies of the DSMs varied with respect to different land cover categories.The DSMs produced better accuracy values for rare residential and road classes when the elevation differences were not considerable.While comparing the different DSMs, the source of the data became more important, especially for dynamic regions.It was not always possible to find different datasets that were obtained at similar times; therefore, it is important to find stable regions within the study region to obtain reliable evaluations from the different DSMs.Absolute accuracy metrics can be derived with the presence of highly accurate reference data such as LIDAR-based point clouds.

Figure 1 .
Figure 1.Geographic representation of the study area with a Pleiades high-resolution (PHR) image and a general overview of the region from Google Earth©.

Figure 1 .
Figure 1.Geographic representation of the study area with a Pleiades high-resolution (PHR) image and a general overview of the region from Google Earth©.

Figure 2 .
Figure 2. The gray levels represent the DSM dataset and the reference DSM data.

Figure 2 .
Figure 2. The gray levels represent the DSM dataset and the reference DSM data.

Figure 3 .
Figure 3. (a) Land cover map of the study area; (b) reference elevation map from HGK data and the locations of checkpoints.

Figure 3 .
Figure 3. (a) Land cover map of the study area; (b) reference elevation map from HGK data and the locations of checkpoints.

Figure 4 .
Figure 4.The height difference distribution of the DSMs derived from 1000 checkpoints.

Figure 4 .
Figure 4.The height difference distribution of the DSMs derived from 1000 checkpoints.

Figure A3 .
Figure A3.Visuals and cross-sections of Profile 3c.Figure A3.Visuals and cross-sections of Profile 3c.

Figure A3 .
Figure A3.Visuals and cross-sections of Profile 3c.Figure A3.Visuals and cross-sections of Profile 3c.

Figure A5 .
Figure A5.Visuals and cross-sections of Profile 4b.Figure A5.Visuals and cross-sections of Profile 4b.

Figure A5 .
Figure A5.Visuals and cross-sections of Profile 4b.Figure A5.Visuals and cross-sections of Profile 4b.

Table 1 .
Summary of previous studies on the production/accuracy assessment of digital surface models (DSMs).GeoEye-1 and TerraSAR-X

Table 2 .
Acquisition properties of the PHR 1A and SPOT 6 satellites' tri-stereo images.

Table 2 .
Acquisition properties of the PHR 1A and SPOT 6 satellites' tri-stereo images.

Table 3 .
Accuracy metrics derived from the 25 checkpoints under bare terrain conditions (m).

Table 3 .
Accuracy metrics derived from the 25 checkpoints under bare terrain conditions (m).

Table 4 .
Accuracy metrics derived from 1000 randomly selected points after extreme removal (m).

Table 5 .
RMSE of the DSMs with respect to land cover classes (m).

Table 6 .
Accuracy ranking of the DSMs for each land cover class (sorted by increasing RMSE values).