Digital surface models (DSMs) are traditionally delivered using terrestrial or aerial surveys (e.g., aerial photogrammetry and laser scanning), which are often time-consuming, difficult to organize, and costly [1
]. Aerial photogrammetry has significantly advanced DSMs with the introduction of the new global position system (GPS) and digital camera technology that have led to reduced costs and increased efficiency [2
Technological miniaturization has led to further reductions in the cost of aerial photogrammetry with the introduction of Unmanned Aerial Systems (UASs), which are becoming increasingly popular for landscape mapping [6
]. UASs equipped with GPS and optical cameras are low-cost alternatives to the classical manned aerial photogrammetry in the short- and close-range domain applications [7
The introduction of a user-friendly photogrammetric technique, called Structure-from-Motion (SfM), has produced a significant revolution in the field, where any researcher or technician can afford high-resolution topographic reconstruction for even low-budget research and applications [8
]. SfM produces orthoimagery and digital surface/elevation models (DSM/DEMs) with very high spatial resolution in the order of centimeters [9
], which is crucial for many applications, especially for change detection studies [11
SfM and multi-view stereo (MVS) algorithms allow the creation of DSMs and orthomosaics without prior information on camera parameters, such as focal length or radial distortion, and provide a flexible and low-cost alternative, enabling high temporal frequency and optimal timing of the missions [3
]. However, unlike with laser scanning, SfM-MVS methods are not capable of penetrating vegetation cover. Therefore, the choice between these two options depends on the purpose of the mapping and available sensors.
The accuracy of SfM-derived DSMs is highly variable, and the causes are still not fully understood, as explained in the review by Smith and Vericat [13
]. A number of factors may affect the precision of UAS-derived orthoimagery and digital elevation data, such as flight parameters (e.g., elevation above ground level; AGL), flight speed, direction, orientation of the camera, and the camera’s focal length), image quality, processing software, the morphology of the studied area, and the type of vehicle (fixed or rotary wing). For instance, short focal length lenses used for low altitude flights introduce considerable geometric distortion into UAS-derived imagery, compromising its overall accuracy.
Most available SfM software packages operate like a black-box with several default parameter settings. It has been shown that appropriate settings can reduce the positioning error of SfM-MVS products [14
], but processing workflow and accuracy assessment methodologies need to be optimized and standardized [15
In this context, ground control points (GCPs) are commonly used to increase the precision of SfM-MVS products, even though their collection is a laborious and time-intensive part of UAS campaigns. Generally, at least three GCPs are necessary to allow the SfM-MVS algorithms to take advantage of such information, but the minimum number of GCPs needed to produce a specific quality is still uncertain. James et al. [14
] recommended a minimum of four to five GCPs and emphasized accurate camera calibration. They showed that a high Root Mean Square Error (RMSE) for three GCPs decreases markedly for six GCPs, especially when considering the vertical component. These results may be influenced by the characteristics of the study case, the equipment (e.g., camera, drone and software), and the workflow adopted.
Singh and Frazier [17
], when exploring this issue, analyzed approximately 66 studies and did not identify a discernible relationship between the number of GCPs and the size of the study area. They identified a weak negative relationship between the number of GCPs collected per hectare and the RMSE with significant scattering.
The literature offers a wide range of choices for the number and spatial distribution of GCPs used to support SfM-MVS algorithms. Such experiences, taken individually, do not provide clear guidance for the identification of the appropriate number of GCPs, but together, these studies provide a valuable source of information for the definition of some recommendations for reducing planar and vertical errors. A selection of the most recent publications dealing with the impact of GCPs, in terms of configurations and numbers, on DSM quality is reported in Table 1
Having extracted all the available data contained in the mentioned references, we were able to depict the relationship between the measured planar and the vertical RMSE as a function of the GCP density (Figure 1
). This allowed us to compare the outcomes of different studies and identify the relative dependence between DSM accuracy and GCP density. The graph in Figure 1
presents a mixture of studies completed at different sites and under different configurations, but all studies highlighted a clear trend—DSM accuracy tends to increase with the number of GCPs, and asymptotic behavior is rapidly reached.
This result was confirmed by Rock et al. [18
] and Tonkin and Midgley [29
], who observed similar behavior. Gindraux et al. [30
] suggested that the optimal number of GCPs can be determined at that point at which there is no significant decrease in error. Based on this concept, Figure 2
provides some indications as to the optimal number of GCPs necessary to produce good DSM quality. For all the experiments, the errors observed in the vertical precision are systematically higher compared with the horizontal precision and decrease more slowly with an increase in GCPs. The planar error tends to stabilize after reaching 5 GCP/ha, whereas 10 GCPs/ha are needed to reach the same condition for vertical precision. This emphasizes the need to find new strategies to improve DSM accuracy, especially regarding the vertical accuracy.
The literature review offered useful indications about the optimization of the number of GCPs. However, the synergic effects of their number and spatial arrangement, as well as flight characteristics, are not yet fully understood [12
]. Optimizing UAS campaigns would, therefore, be an important step toward improving the effectiveness and reliability of UAS-derived products.
In this study, we explored the impact of both UAS flight characteristics (e.g., altitude, camera tilt, and flight plan) and GCP density on the accuracy of a three-dimensional (3D) model of a small earthen dam. Analyses helped with understanding the procedure to increase the reliability of digital surface models, which provide critical information in environmental and hydrological science.
2. Materials and Methods
2.1. Study Area
The survey experiment was executed on an earthen dam next to a village called Pișchia, 20 km northwest from Timisoara in Western Romania. The Pișchia dam, managed by the National Water Administration, has a volume of approximately 500,000 m3
, which is used to supply drinking water and for recreational activities (e.g., fishing). It has a trapezoidal cross-section with a side slope of 1:3 and a maximum elevation of about 10 m. The surrounding area is characterized by agricultural land with gentle slopes (Figure 2
2.2. Primary Data Collection
All flights were performed with DJI Phantom 4 Pro quadcopter (DJI, Shenzhen, China), featuring a gimbaled 1-inch 20-megapixel CMOS sensor with a mechanical shutter. The focal length of the lens was 24 mm (full-frame equivalent). The data were stored in 24 bit JPG format, and the pixel size was 2.41 μm. The camera sensitivity was set to ISO100 for all images, with the aperture ranging from 4 to 5.6 and the shutter times ranging between 1/120 and 1/500 s. All images were georeferenced with the on-board GPS. The WGS84 coordinates were stored in JPG EXIF. Mission planning was executed in Pix4Dcapture, which enabled control of the camera tilt. All six flights were performed on 4 April 2018, between 10:00 a.m. and 12:50 p.m. UTC. Flight missions were planned with a side overlap of 60% and a front overlap of 80%.
In order to explore the impact of mission planning on the overall accuracy of UAS-derived DSM, different flight plans were created, with changing flight trajectories, camera tilt, and the elevation of the flight. Several characteristics of the six flights are summarized in Figure 3
, whereas other parameters, such as camera settings and additional mission flight settings (e.g., overlap), were kept constant. The reference elevation refers to the take-off location, which was about 14 m above the average elevation of the surface. Some examples of the images obtained by different configurations are shown in Figure 4
, where an area from the central part of the dam is reproduced. From these images, the outlet tower and the spillway of the dam can be recognized.
Flights were planned to cover an area of approximately 100 × 270 m (about 2.7 ha) and the extent is highlighted in Figure 2
B (see red line boundaries). In the same figure, we report the UAS-derived DSM of the area.
For the aim of the present study, only a portion of the dam was studied using about 16 GCPs distributed along the main structure of the dam and in the adjacent agricultural area. They were placed along five longitudinal alignments, trying to measure the full range of elevation changes. The maximum vertical variation of the GCP positions was about 10.6 m.
The GCP positions were determined using the Leica 1200 system of RTK (Real-Time Kinematic) GNSS (Global Navigation Satellite System) rover (Leica Geosystems AG, Heerbrugg, Switzerland) and a precise Leica 1201 Total Station to achieve a precision better than 3 mm for all GCPs. To determine a geodetic base, corrections were acquired from the Romanian Position Determination System (ROMPOS) network of GNSS permanent stations. The study area where determinations were realized is located at a distance of 18.5 km from the TIM1 reference station (http://rompos.ro/index.php/en/
2.3. Data Processing
Images retrieved by each flight were processed using Agisoft PhotoScan v.1.4.3 to derive a 3D model of the area. The same workflow was repeated each time while keeping the software settings constant, and following a sequence of commands: (1) photo alignment with high accuracy, (2) optimizing alignment, (3) dense cloud building with high quality aggressive depth filtering, (4) mesh building using a dense cloud, (5) texture building with the default blending mode, (6) tiled model building, (7) DSM building using the default settings, and (8) orthomosaic generation.
In the preliminary phase, we focused on the use of the geotagged images alone, excluding the use of GCPs. Measured GCPs were adopted as check points only to validate the results. Elaboration without the GCPs allowed us to better understand the role of the flight mode and the combination of different flights on the resulting DSM. This was completed by exploring the accuracies of DSMs obtained using imagery extracted from a single flight and from the combinations of two flights. The resulting combinations displayed wide variability in the precision of planar coordinates and elevation. This preliminary analysis allowed the identification of the best performing flight configuration and the benefits due to the use of combined flights.
To increase the quality of the 3D model, GCPs should be included in the Bundle Block Adjustment (BBA). The number and distribution of GCPs per unit area are not univocally identified in the literature, as highlighted in the introduction [28
]. The number of GCPs necessary for the survey is influenced by the extent of the study area and its morphology, camera deployed, internal GPS precision, and the type of survey.
In the second phase, we analyzed the two previously identified imagery datasets in order to explore the role played by GCP density and distribution. For the second phase of the analysis, a variable number of GCPs ranging from 3 to 9 were randomly selected from the 16 control points available, while the remaining GCPs were employed as check points. This second analysis was useful for understanding the mutual benefit of flight combinations and well-designed GCP distribution. Proper use of the two settings enhances the potential of SfM-MVS algorithms in providing good quality DSMs. The comparison between single and multiple flights combined with the use of GCPs was stimulated by the need to better understand the benefits of combining multiple flights.
UAS-derived 3D models provide a new strategy for monitoring land surfaces with an extremely high level of detail. The literature offers a wide range of applications for the operational use of UASs for 3D model reconstruction based on SfM-MVS algorithms, offering different strategies aimed at minimizing the errors of these 3D models. A review of these studies enabled the identification of the control exerted by GCP density on the planar and vertical accuracy of UAS-derived DSMs. The planar accuracy of SfM-MVS outputs is generally higher than the vertical accuracy. Therefore, the number of GCPs needed to produce stable results is generally lower for planar coordinates. On average, five GCPs/ha are enough to produce a good performance on the plane, but doubled density is needed for elevation (Figure 1
). The final result is highly influenced by other factors such as flight pattern and configuration, camera quality, and local morphological complexity.
In our analysis, we focused on 3D model optimization exploiting the combination of different flight configurations and the optimal GCP design. For this reason, we explored the impact of: (1) the combined use of images obtained from different flight patterns and configurations, and (2) the use of a variable number of GCPs. Both approaches are already used in a practical application without clear identification of the benefits associated with combining images acquired using different flight plans and camera settings, or the quantification of the impact of such choices. The optimum density and distribution of GCPs remain poorly understood.
According to our results, mission planning is a critical preliminary step that may significantly affect the final results. In certain circumstances, a well-defined single flight may be sufficient to produce adequate quality for the overall survey. The combination of flights with differing configurations can retrieve information from different viewpoints and angles that can increase the resulting accuracy. Given the optical nature of SfM-MVS algorithms, the challenge is then to maximize the number of observations of each individual point retrieved across the area of interest. The use of a tilted camera may be beneficial in order to improve the robustness of the geometrical model, thereby increasing the number of tie points describing inclined surfaces. The tilt of the camera should be defined according to both the local morphology and the resolution required. For the studied case, characterized by a trapezoidal earthen dam with an elevation of about 10 m and gentle slopes with elevation changes of 15 m, a tilt of 20° combined with a 0° flight provided the best results. This result can be justified by examining the results reported on the diagonal of Table 2
. The accuracy of planar coordinates was generally higher when using a nadir camera setting with lower flight altitude, but the vertical error was always lower for the flight with a 20° tilted camera. Therefore, a combination of two flights tends to optimize both characteristics of DSMs.
The flights operated on two orthogonal routes provided additional benefits to the description of the area, allowing a relevant reduction of error (for comparison, see the results of the combination 1–2). Comparing the accuracy of different DSMs obtained with a single flight or a combination of flights, the flight combination and tilted camera significantly increased the vertical accuracy, clearly benefiting to the process of DSM construction.
The error magnitude is also influenced by the flight altitude that controls the image resolution. The use of multiple flights at different flight altitudes is a common practice to improve survey accuracy in aerial photogrammetry [32
]. We observed a beneficial effect on the relative elevation accuracy of DSM, but such improvement is probably also influenced by the lower resolution of the images and is less effective in general than the use of a 20° tilted camera. In the literature, contrasting results on the relative impact of the flight altitude have been reported. For example, Gómez-Candón et al. [23
] showed a weak relationship of RMSE with flight height.
The orthorectification of images is traditionally performed using GCPs. Other options should also be considered, such as investing in carrier-phase GPS receivers and processing workflow that can reduce the amount of fieldwork needed in terms of GCP collection [33
], which is especially useful if monitoring larger areas.
Our results show that, with an increasing number of GCPs, the quality of the 3D model increases. With five GCPs, a planar RMSE of less than one centimeter (about 0.2 cm) was reached. High vertical accuracy requires significant additional effort. Our dataset confirms the need to increase the number of GCPs to achieve a stable result in terms of the vertical RMSE. Even if the maximum number of nine GCPs was adopted, the obtained vertical accuracy was still one order of magnitude larger than the planar accuracy. Only adopting a combination of flights significantly in the vertical accuracy of the DSM, reaching a precision of about 4 cm after more than seven GCPs were adopted.
High planar accuracy can be obtained with a relatively small number of GCPs well spread in space, whereas vertical accuracy requires a larger number of GCPs and is less sensitive to their relative distribution in space. As shown in our study, the vertical accuracy, a critical variable in several studies, can be significantly improved using a combination of flights, including the use of a tilted camera, thereby reducing the laborious and complicated collection of GCPs. Notably, this method is the most efficient approach to reduce vertical error among the many methods explored herein.
The proposed study is based on low-cost technology that is commonly used, but there is a tendency toward the inclusion of more accurate positioning systems in UASs that will dramatically reduce the need for ground operations. These new systems are still expensive, but will certainly become more affordable shortly. For instance, Gerke and Przybilla [34
] explored the impact of onboard RTK-GNSS on DSM accuracy and observed a significant enhancement in absolute image orientation accuracy with this option.
In this paper, we provided recommendations for UAS-surveys aimed at the derivation of 3D surface models. Exploiting the available literature on this topic and our field experiences, a number of suggestions are put forward:
UAS-derived orthomosaics can produce a planar accuracy of a few centimeters, whereas the vertical accuracy of DSMs is always lower. This is likely due to the fact that most UASs adopt a camera in a zenithal position that provides a more accurate description of planar features. Vertical measurements are generally more complex, but also critical for studies of change detection.
The flight plan and camera configuration may significantly impact the overall quality of the resulting DSM. Therefore, it should be planned thoroughly to produce the best depiction of the entire area. For instance, a transversal survey with respect to a given structure provides better description and quality of the resulting 3D surface.
The use of a tilted camera can improve the amount of information (retrieved number of points) for inclined surfaces, providing higher DSM elevation accuracy. The tilted camera images increase the robustness of the geometrical model, providing also a possible strategy to reduce the total number of GCPs adopted over a given area. This can be beneficial especially in inaccessible areas.
The combination of several flights may be extremely beneficial for DSM accuracy. This improves the overall quality of the results, exploiting information redundancy derived by different flight plans and camera configurations.
The planar and vertical accuracies can be improved by increasing the number of GCPs and their relative distances. It is therefore convenient to evenly spread GCPs in space. In many cases, such ideal settings are not possible and a combination of flights, that include the use of a tilted camera, can be used to reduce sensitivity to this parameter in the final vertical accuracy of the DSMs.
Our described experiment cannot be considered exhaustive; however, it provides insights into the problems discussed and can serve as a guideline for future applications. Extending the analysis to new case studies and landscape morphologies is highly desirable to provide clear and more detailed guidelines for UAS applications to considering other factors influencing DSM accuracy, such as site area and morphology, sensor, and internal GPS precision. The outcomes of the presented research lead to a number of useful results applicable to UAS applications in different conditions. There is no simple solution for the optimal number of GCPs, but our analysis helps with understanding some general concepts. To conclude, the flight should be planned carefully in order to optimize the amount of information retrieved by the camera deployed, and combined flight settings can significantly improve the overall quality of the 3D models, even for the most critical dimension—the vertical accuracy.
The present manuscript represents a preliminary step for the definition of guidelines for UAS-derived DSMs. Nevertheless, the procedure is influenced by several additional factors that have not been taken into consideration in the present study. Therefore, we plan to extend our study considering different morphologies, SfM-algorithms and devices in future activities.