Evaluating Elevation Change Thresholds between Structure-from-Motion DEMs Derived from Historical Aerial Photos and 3DEP LiDAR Data

: This study created digital terrain models (DTMs) from historical aerial images using Structure from Motion (SfM) for a variety of image dates, resolutions, and photo scales. Accuracy assessments were performed on the SfM DTMs, and they were compared to the United States Geological Survey’s three-dimensional digital elevation program (3DEP) light detection and ranging (LiDAR) DTMs to evaluate geomorphic change thresholds based on vertical accuracy assessments and elevation change methodologies. The results of this study document a relationship between historical aerial photo scales and predicted vertical accuracy of the resultant DTMs. The results may be used to assess geomorphic change thresholds over multi-decadal timescales depending on spatial scale, resolution, and accuracy requirements. This study shows that if elevation changes of approximately ± 1 m are to be mapped, historical aerial photography collected at 1:20,000 scale or larger would be required for comparison to contemporary LiDAR derived DTMs.


Introduction
There is growing recognition of the importance of geomorphological mapping of three-dimensional (3D) change, particularly in the context of anthropogenic landscapes in urban environments [1][2][3][4][5][6][7]. One of the most significant advancements in geomorphological mapping over the past decade has been the increased availability of high-resolution digital elevation models (DEMs) produced from light detection and ranging (LiDAR) and other sources. Programs such as the United States Geological Survey's (USGS) three-dimensional digital elevation program (3DEP) aim to achieve nation-wide LiDAR coverage through data acquisition coordinated between the USGS National Geospatial Program (NGP) and partner-funded LiDAR collections [8]. The USGS 3DEP also strives to create consistency amongst the multitude of LiDAR collects through the development of specific data quality guidelines [9]. However, since most topographic LiDAR acquisition has occurred only relatively recently (within the past 10-15 years), there are few historical elevation datasets with the spatial resolution and accuracy suitable for change detection over multi-decadal to century time-scales. Coarser-resolution DEMs are available that represent previous eras (e.g., the Shuttle Radar Topography Mission (SRTM), the Advanced Spaceborne Thermal Emission and Reflection (ASTER) Global DEM, etc.); however, there are limitations to comparing coarse-resolution DEM data with fine-resolution data to map geomorphic change [10][11][12]. Even amongst the available 3DEP LiDAR datasets there is variability in the collection parameters and thereby the quality and accuracy as well as the temporal resolution of the data.

Data
Data used in this study consisted of reference DEMs developed from 3DEP LiDAR point cloud source data, high-resolution orthoimagery, seven sets of historical aerial stereoscopic photographs, and the most recent, 2018, 3DEP LiDAR dataset for long-term temporal comparison. Details about each of these datasets are described in the following sections. In this paper, the term DEM refers broadly to multiple types of elevation data. The term digital surface model (DSM) is used when referring to DEMs that include vegetation, structures, and other non-ground features. Digital terrain model (DTM) is used to describe DEMs where vegetation and non-ground features have been removed either through filtering or LiDAR point-cloud classification routines.

Reference Elevation from LiDAR
At the time of this research, four different LiDAR datasets were available in the study area: 2008, 2012, 2014, and 2018. Data from 2008, 2012, and 2018 were used in this study. Table 1 provides

Data
Data used in this study consisted of reference DEMs developed from 3DEP LiDAR point cloud source data, high-resolution orthoimagery, seven sets of historical aerial stereoscopic photographs, and the most recent, 2018, 3DEP LiDAR dataset for long-term temporal comparison. Details about each of these datasets are described in the following sections. In this paper, the term DEM refers broadly to multiple types of elevation data. The term digital surface model (DSM) is used when referring to DEMs that include vegetation, structures, and other non-ground features. Digital terrain model (DTM) is used to describe DEMs where vegetation and non-ground features have been removed either through filtering or LiDAR point-cloud classification routines.

Reference Elevation from LiDAR
At the time of this research, four different LiDAR datasets were available in the study area: 2008, 2012, 2014, and 2018. Data from 2008, 2012, and 2018 were used in this study. Table 1 provides additional details about the horizontal and vertical resolution and an explanation of how each dataset was used in this project [9]. The 2008 USGS 3DEP LiDAR DTM, with a vertical accuracy of ±0.30 m and a horizontal resolution of 3 m, was used to produce a set of ground control points and independent check points (GCPs and ICPs respectively). The 11 GCPs (Table 2) were selected in locations that have experienced minimal topographic change (as interpreted from high-resolution orthoimagery, historical analysis, and fieldwork). These GCPs were used as control in SfM model development for each set of historical aerial images. Additionally, a set of 32 ICPs were selected in locations that were free of vegetation or structures as check points from which the vertical accuracy of the output SfM DEMs were evaluated. The locations of the GCPs and ICPs are shown on the map in Figure 2. GCPs and ICPs were derived primarily from manmade features that appeared to have not changed over the 68-year period. Features selected for control points and check points were primarily corners of paved walking paths and road intersections on the grounds of cemeteries, churches, and schools. Natural features were avoided since natural landscape change could not be ruled out. Features with any significant height, such as buildings, fences, or large trees, were not used to avoid positions that may be represented by parallax in image sets or otherwise may not represent ground elevations. The 2018 LiDAR point cloud was used to produce a 3 m DTM, which was ultimately compared to the 1950 SfM DTM to determine the total topographic change in the study area.

Historical Aerial Stereo Image Processing
The USGS maintains an archive of over 4.3 million frames of aerial photography collected by different federal agencies from the mid-1930s to the present. Referred to as the 'Single Aerial Photo Frames' collection, it contains aerial imagery ranging in scale, size, film type, quality, and coverage. In addition, a second archive contains the collection of an additional 1.3 million frames of National Aerial Photography Program (NAPP) imagery collected from 1987 to 2007 at 1:40,000 scale. For both archives, imagery was acquired with stereo coverage on cloud-free days, and most sets have a minimum forward overlap of 60% and sidelap of 30%. From these archives, individual photo frames were scanned by the USGS Earth Resources Observation and Science (EROS) Data Center using a digital scanning back system at a resolution of 25 microns, or 400 dpc (1016 dpi). After image scanning, a geometric calibration was applied to each image to correct for distortions caused by the scanning process (not a lens distortion correction) [16].
Stereo photo sets of multiple scales were acquired from the USGS Single Aerial Frame Photo and NAPP archives for seven different dates ranging from 1950 to 2002. Table 3 shows the scales, acquisition dates, and extent of stereo coverage with respect to the Piney Branch watershed boundary for each set of stereo aerial imagery used in the study.

Historical Aerial Stereo Image Processing
The USGS maintains an archive of over 4.3 million frames of aerial photography collected by different federal agencies from the mid-1930s to the present. Referred to as the 'Single Aerial Photo Frames' collection, it contains aerial imagery ranging in scale, size, film type, quality, and coverage. In addition, a second archive contains the collection of an additional 1.3 million frames of National Aerial Photography Program (NAPP) imagery collected from 1987 to 2007 at 1:40,000 scale. For both archives, imagery was acquired with stereo coverage on cloud-free days, and most sets have a minimum forward overlap of 60% and sidelap of 30%. From these archives, individual photo frames were scanned by the USGS Earth Resources Observation and Science (EROS) Data Center using a digital scanning back system at a resolution of 25 microns, or 400 dpc (1016 dpi). After image scanning, a geometric calibration was applied to each image to correct for distortions caused by the scanning process (not a lens distortion correction) [16].
Stereo photo sets of multiple scales were acquired from the USGS Single Aerial Frame Photo and NAPP archives for seven different dates ranging from 1950 to 2002. Table 3 shows the scales, acquisition dates, and extent of stereo coverage with respect to the Piney Branch watershed boundary for each set of stereo aerial imagery used in the study.   Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] of matched image positions. Multi-View Stereo algorithms (MVS) then perform dense cloud extraction on the matched areas.    Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] of matched image positions. Multi-View Stereo algorithms (MVS) then perform dense cloud extraction on the matched areas.    Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] of matched image positions. Multi-View Stereo algorithms (MVS) then perform dense cloud extraction on the matched areas. KA000 12   Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] of matched image positions. Multi-View Stereo algorithms (MVS) then perform dense cloud extraction on the matched areas.    Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] of matched image positions. Multi-View Stereo algorithms (MVS) then perform dense cloud extraction on the matched areas. VDGQ0 12,000 152.58 B&W 9

DSM Development and Accuracy Assessment
1:40,000 1988 APR 20 Remote Sens. 2020, 12, x FOR PEER REVIEW 6 of 17  Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] Figure 3 shows a workflow diagram that graphically illustrates the general processing steps in the methodology. Prior to processing, all aerial images were cropped using Adobe Photoshop to remove the frame and fiducial information. SfM processing was done using Agisoft Metashape Pro to produce a DSM and orthophoto from each set of images. SfM begins with a photo alignment process, wherein all the images in a set are matched for coincident points using the scale invariant feature transform algorithm (SIFT) [17]. SIFT processing results in three-dimensional orientation of the photo positions in space. Next, bundle adjustment of these images generates a sparse point cloud [18] of matched image positions. Multi-View Stereo algorithms (MVS) then perform dense cloud extraction on the matched areas.

DSM Development and Accuracy Assessment
From the dense point cloud, an ultra-fine resolution orthophoto and DSM were generated for each set of historical aerial photographs. The resolution of each DSM and orthophoto was automatically calculated by the Agisoft Metashape software based on input image resolution. The GCPs derived from the 2008 LiDAR-based high-resolution DEM and 2014 high-resolution orthoimagery (HRO) were used to rectify and adjust the output model.
DEMs have unique error characteristics that depend substantially on the source data and methods used to develop them. Since this error is compounded when DEMs are directly compared, it is necessary to document and account for the error in both the input datasets and the difference dataset [19]. Robust accuracy assessment is essential to ensure that actual physical changes in landform elevation can be distinguished from noise and errors in the individual DEMs [20]. From the dense point cloud, an ultra-fine resolution orthophoto and DSM were generated for each set of historical aerial photographs. The resolution of each DSM and orthophoto was automatically calculated by the Agisoft Metashape software based on input image resolution. The GCPs derived from the 2008 LiDAR-based high-resolution DEM and 2014 high-resolution orthoimagery (HRO) were used to rectify and adjust the output model.
DEMs have unique error characteristics that depend substantially on the source data and methods used to develop them. Since this error is compounded when DEMs are directly compared, it is necessary to document and account for the error in both the input datasets and the difference Because the elevation output product of SfM procedures is a DSM and the reference elevation surface is a DTM, the inclusion of elevation values from surficial features such as trees and structures could greatly affect the accuracy assessment of the ground surface modeled by the DSM. For this reason, the SfM DSMs were processed to remove, or greatly reduce, the above ground-surface features present in the elevation model following the methods of [21]. These methods utilized GlobalMapper's (v.17) LiDAR automatic non-ground classification routines to filter the DSMs to DTMs. The routine conducts an initial point classification to remove outlier point values, followed by classification of the remaining points as either ground or non-ground. A point was classified as non-ground if the plane between it and the nearest two ground points exceeded user-derived slope parameters. This "planarity"-based evaluation was performed for a local window throughout the study area for each dataset. After the planarity filtering, remaining areas of vegetation that persisted in the DSM were noted, and a negative vertical offset, determined from the average canopy height, was applied to remove these patches. While automated filtering of the DSM using these tools cannot completely eliminate the effect of vegetation, structures, etc. on the elevation model, filtering procedures create a close approximation of a DTM for each historical DSM that can be compared to a LiDAR-based DTM.
Accuracy was assessed at a set of points independent of the GCPs used in SfM modeling. Referred to as ICPs, these accuracy assessment locations were selected in open areas that exhibited little or no evidence of topographic change over time. The vertical accuracy of the DSMs developed from historic aerial imagery was assessed using the root-mean-square error (RMSE) metric, calculated as where x i is the elevation value from the reference DEM,x i is the elevation value from the SfM DSM, and n is the number of observations

Topographic Change Detection: Thresholding the DEM of Difference
Topographic change was quantified by direct comparison of two DTMs by subtracting the older DTM from the more recent DTM resulting in a DEM of Difference (DoD) where the values indicate the amount of vertical change in the elevation surface between two datasets [22]. Due to differing vertical accuracies of each DTM, it is necessary to specify a threshold value for the DoD that characterizes this combined vertical inaccuracy. The thresholding of the DoD accounts for the uncertainty in each of the input datasets and allows for the interpretation of changes with an understanding of the margin of error. While many methods of estimating uncertainty in the DoD have been suggested (for a review see [23]), this study used two common methods of establishing an error threshold for the DoD: combined RMSE and the Root of the Sum in Quadrature (RSiQ).
The first method for establishing the amount of error in DoD values is to add the RMSE values of the two input DEMs. Elevation values within the range of the RMSE sum are interpreted as not able to be distinguished from error in the DoD. Therefore, this method provides a relatively conservative estimate of the potential error in the DoD, as it assumes the maximum potential for error in the combined DoD. A less conservative estimate of the possible error inherent in DoDs utilizes the RSiQ method [23][24][25][26], which is conceptually similar to the RMSE accuracy metric but assumes that, in two datasets with errors that are normally distributed, there is low probability of comparing data in the extreme tails of the distribution against one another. Assuming the vertical errors of individual DEM data are random errors and independent of one another, the RSiQ method can be used to calculate the values in the DoD that cannot be distinguished from combined error values. This RSiQ value for each dataset is calculated as where δq is the Root of Sum in Quadrature, δx 2 is the vertical RMSE of the more recent DEM, and δx 2 is the vertical RMSE of the older DEM Building upon this equation, some research has pointed to the use of a probabilistic thresholding technique using the RSiQ, which takes the form of a statistical z-score equation to assess the significance of different ranges of DoD error [27] and can provide useful information regarding the significance of difference values, which include both error and elevation change. Using this method, the user may map elevation change at the 68% confidence interval (t ≥ 1) and at the 95% confidence interval (t ≥ 1.96) depending on the mapping requirements for significance of topographic change: where t is the threshold value, x is the more recent DEM value,x is the older DEM vale, δx 2 is the vertical RMSE of the more recent DEM, and δx 2 is the vertical RMSE of the older DEM.

Resolution and Accuracy of DSMs Produced from Historical Aerial Imagery
A representative portion of DTMs produced by SfM analysis and subsequent filtering are shown in Figure 4, and the corresponding orthoimages produced are shown in Figure 5. Elevation data for the seven dates of historical aerial photography used in this study are available as supplementary materials. The stream channel interpreted from each date of orthoimage is overlain on top of the DTM and orthoimage for reference. where is the Root of Sum in Quadrature, is the vertical RMSE of the more recent DEM, and is the vertical RMSE of the older DEM Building upon this equation, some research has pointed to the use of a probabilistic thresholding technique using the RSiQ, which takes the form of a statistical z-score equation to assess the significance of different ranges of DoD error [27] and can provide useful information regarding the significance of difference values, which include both error and elevation change. Using this method, the user may map elevation change at the 68% confidence interval (t ≥ 1) and at the 95% confidence interval (t ≥ 1.96) depending on the mapping requirements for significance of topographic change: where is the threshold value, is the more recent DEM value, is the older DEM vale, is the vertical RMSE of the more recent DEM, and is the vertical RMSE of the older DEM.

Resolution and Accuracy of DSMs Produced from Historical Aerial Imagery
A representative portion of DTMs produced by SfM analysis and subsequent filtering are shown in Figure 4, and the corresponding orthoimages produced are shown in Figure 5. Elevation data for the seven dates of historical aerial photography used in this study are available as supplementary materials. The stream channel interpreted from each date of orthoimage is overlain on top of the DTM and orthoimage for reference.  The results of DTM accuracy assessments by date and scale of imagery are shown in Table 4   The results of DTM accuracy assessments by date and scale of imagery are shown in Table 4 Table 5   The results of topographic change between 2018 and 1950 are shown in Figures 7 and 8. These two figures represent topographic change employing the RSiQ method. Figure 7 utilizes the ± 1.43 threshold for the 2018-1950 interval. Areas of negative (−) change depict loss of elevation and are symbolized in shades of blue, whereas positive (+) changes depict accretion and are symbolized in shades of red. Within the extent of the study area, the most substantial elevation loss was -9.79 m, and the most substantial elevation gain was 8.18 m. Figure 8

Conclusion
SfM is a viable and low-cost/low barrier-to-entry method for producing DSM/DTMs from historical aerial photography. DTMs produced from SfM and aerial photography are comparable to LiDAR elevation data for assessing geomorphic change over multi-decadal timescales. USGS aerial photo archives and 3DEP LiDAR data are world-class sources for historical geomorphic change analysis and are also highly accessible to researchers since each dataset is publicly available at no cost.
The quality of resultant SfM DTMs was found to be dependent upon the scale of input aerial photography. Larger-scale photography (1:24,000 or larger) generally had lower vertical error (lower RMSE) values and, thus, is considered more closely comparable to 3DEP LiDAR data. However, depending on the scale of analysis, smaller-scale photography (for example, 1:40,000) may be adequate for detecting elevation change of a certain magnitude.
This study found variability in vertical RMSE values of DTM data produced from imagery of the same photo scale. Values are shown in Table 4 for both the calculated RMSE from the Agisoft software as well as the assessment done using ICPs. The results show lower error calculated in Agisoft than is calculated by the ICP assessment. This likely is due to several factors, but one important one is that the Agisoft RMSE values measure the accuracy of the overall modeled parameters, and the RMSE values developed from the ICPs measure vertical differences of the filtered DTM data. Filtering of elevation data changes the values inherent in the output DTM. Based on the ICP analysis, DTM RMSE values derived from 1:24,000 scale aerial photography ranged from ±0.92 to ±1.81 m. Using the linear function formula from Figure 6 (y = 4.2 × 10-6x + 0.2461), calculation of the y intercept for 1:24,000 scale photography predicts a vertical RMSE of ±1.25. This value is roughly equivalent to the output DTM horizontal resolution. The linear function formula provides a tool for researchers assessing the magnitude of elevation changes that one might expect based solely upon photo scale. For example, if elevation changes of approximately ±1 m are required to be mapped, then calculation of the y intercept indicates that historical aerial photography of 1:20,000 scale or larger would be required. Similarly, 1:10,000 scale photography would be predicted to yield DTM vertical RMSE values of ±0.68. However, this study also shows that aerial photography at 1:24,000 scale

Conclusions
SfM is a viable and low-cost/low barrier-to-entry method for producing DSM/DTMs from historical aerial photography. DTMs produced from SfM and aerial photography are comparable to LiDAR elevation data for assessing geomorphic change over multi-decadal timescales. USGS aerial photo archives and 3DEP LiDAR data are world-class sources for historical geomorphic change analysis and are also highly accessible to researchers since each dataset is publicly available at no cost.
The quality of resultant SfM DTMs was found to be dependent upon the scale of input aerial photography. Larger-scale photography (1:24,000 or larger) generally had lower vertical error (lower RMSE) values and, thus, is considered more closely comparable to 3DEP LiDAR data. However, depending on the scale of analysis, smaller-scale photography (for example, 1:40,000) may be adequate for detecting elevation change of a certain magnitude.
This study found variability in vertical RMSE values of DTM data produced from imagery of the same photo scale. Values are shown in Table 4 for both the calculated RMSE from the Agisoft software as well as the assessment done using ICPs. The results show lower error calculated in Agisoft than is calculated by the ICP assessment. This likely is due to several factors, but one important one is that the Agisoft RMSE values measure the accuracy of the overall modeled parameters, and the RMSE values developed from the ICPs measure vertical differences of the filtered DTM data. Filtering of elevation data changes the values inherent in the output DTM. Based on the ICP analysis, DTM RMSE values derived from 1:24,000 scale aerial photography ranged from ±0.92 to ±1.81 m. Using the linear function formula from Figure 6 (y = 4.2 × 10-6x + 0.2461), calculation of the y intercept for 1:24,000 scale photography predicts a vertical RMSE of ±1.25. This value is roughly equivalent to the output DTM horizontal resolution. The linear function formula provides a tool for researchers assessing the magnitude of elevation changes that one might expect based solely upon photo scale. For example, if elevation changes of approximately ±1 m are required to be mapped, then calculation of the y intercept indicates that historical aerial photography of 1:20,000 scale or larger would be required. Similarly, 1:10,000 scale photography would be predicted to yield DTM vertical RMSE values of ±0.68.
However, this study also shows that aerial photography at 1:24,000 scale produced DTM data of < 1m vertical RMSE (±0.92 m for the 1973 dataset). This result indicates that the quality of output DTMs is reliant upon other factors beyond the photo scale; thus, additional observations and tests deriving DTMs from historical photography in different settings will enhance the understanding of their quality and accuracy.
Other factors to consider for assessing quality of SfM DTMs from historical aerial photography are the original image quality of photo and the quality of the image scanning. The photo geometry (exterior orientation), amount of stereo overlap, and the number of photo frames all contribute to the ability of the SfM software to align photos in the stereo model. The study area itself is also a factor, and its topographic relief and landcover characteristics (type and amount of vegetation or manmade features) may contribute to the quality of output data. With regards to quantifying topographic change, the filtering mechanisms used to remove vegetation or anthropogenic features are vitally important because this process can introduce error into the DEM and may cause areas to appear as having changed when little or no change has occurred. Finally, the quality and location of control points is an important consideration, especially in historical research, since most features and ground area has experienced some change over multiple decades. These changes may not always be adequately documented or otherwise known; thus, error may be introduced if these areas are selected for control points.
The values of the DoD indicate changes in elevation detectable from the source data and must be evaluated together with the uncertainty of the input DTMs. It can be difficult, depending the accuracy of the DTMs used in the analysis, to differentiate between 'real' elevation gain or loss and 'noise' caused by the propagation of error to the DoD. Thresholding the DoD provides a means of establishing the minimum amount of change that can be measured, given the accuracies of the input DTMs. When particularly conservative thresholding methods are used, the amount of error may almost entirely overwhelm the amount of elevation change. Thus, it is important to try to reduce the error inherent in the historical DEMs.
There are also challenges in assessing the distribution of errors in resultant DEMs as a result of significant topographic changes. It can be extremely difficult to find control point locations that have not changed over multiple decades. Even seemingly insignificant changes caused by subsidence, erosion, or vegetation succession can substantially affect the analysis of change over time, notwithstanding larger changes associated with urban development such as road or infrastructure construction. This severely limits the ability to acquire large sets of randomly assigned locations from which to evaluate elevation error and impacts the resultant RMSE values in each of the historical datasets.
This study specifically investigated two different methods of distinguishing topographic change from error in the two input DEMs, namely the combined RMSE and the RSiQ methods. While the summed RMSE method is commonly used, possibly due to its simplicity to implement and explain, the RSiQ method is useful because it assumes that the error in a DoD pixel is not the combined extremes of errors from the input DEMs. Moreover, the RSiQ value can be used to evaluate the comparability of available DEMs in historic topographic change analysis, as it represents the detectable level of change in DEM differencing. While changes beneath this RSiQ value theoretically cannot be distinguished from noise, the DoD values may still be useful in establishing the pattern of elevation changes, and thus useful for visual interpretation of patterns of change.
The orthoimages produced by the SfM process have additional utility for visual interpretation of land cover changes as well. In particular, changes in stream channel patterns were apparent through comparison of the channel visible in each subsequent SfM orthoimage. These changes are evident when the streams interpreted from each year are overlaid (Figures 4 and 5). Mapping stream channel changes and other feature-based changes may enable better discrimination of DoD error differences if change can be associated with specific land-use land cover transitions.
Though other factors beyond photo scale contribute to expected error and data quality, this research produced a valuable tool that may be used to assess anthropogenic changes for planning and research purposes [7]. Areas of further study include investigations into improvements of SfM modeling using historical aerial photographs. Pre-processing improvements including better methods for cropping fiducial information from photo frames (for example, those demonstrated by [28]), adding camera calibration information in SfM software, and other techniques to optimize photo alignment may improve the interior orientation modeling parameters. Mean filtering of the original photo frames may also be helpful in improving the quality of image matching and photo alignment to assist with correlation and extraction of elevation values from pixels. Improvements could also be made in identifying an appropriate number and location of control points, and testing and analyzing the distribution of error would aid in understanding how that error propagates through to the output DTM datasets.
In summary, programs such as USGS 3DEP respond to growing mapping needs by providing recent, highly accurate, and high-spatial resolution elevation data using LiDAR technology. The applicability and potential use of 3DEP LiDAR data for mapping geomorphic and anthropogenic change are increased by the integration of SfM DEM data using historical aerial photography.