Accuracy Assessment of a UAV Direct Georeferencing Method and Impact of the Configuration of Ground Control Points
Abstract
:1. Introduction
2. Materials and Methods
2.1. The Study Area
2.2. Data Collection
2.3. Data Processing
- Image feature extraction and matching. The software automatically identifies many conspicuous points in each image, regardless of image scale or perspective, and similar feature points are recognized in multiple images. After locating the feature points in each image, similar feature points are recognized in multiple images. The quality of feature matching depends on the texture and overlap success of the image [33,34];
- Iterative bundle adjustment. The purpose of BA is to determine internal and external orientation elements of the images by minimizing the reprojection errors between predicted and observed points, which can be converted into a nonlinear least-squares problem [35]. By applying the BA, the three-dimensional structure of the scene, the internal and external orientation elements of the camera are estimated at the same time;
- Model optimization based on control points. GCPs provide additional external information about reconstructed scene geometry. The optimization process in Photoscan refines the camera position and reduces non-linear project deformations by incorporating GCPs [36];
- Point cloud density matching. The MVS image matching algorithm operates on a single-pixel scale of the image to build dense clouds and increases the point density by several orders of magnitude;
- Generate DSM and DOM. Using the dense point cloud as input, other results, such as DSM and DOM, can be produced. The outliers in the dense point cloud are removed before the dense point cloud is interpolated to generate DSM, and then the DOM is generated by digital differential correction based on DSM.
2.4. Quality Assessment
3. Results
3.1. Model Evaluation Based on RMSE
3.2. Point Cloud Evaluation Based on M3C2 Distance
3.3. Influence of GCPs Distribution
4. Discussion
- Error analysis based on the RMSE shows that adding one GCP can help to reduce the deviation, but there may still be dome error, as shown in the study of Rosnell and Javernick [27,38]. When two GCPs were used, the mean vertical difference was reduced to 0.003 m, and the horizontal RMSE was 0.0198 m, approximately 1.1 GSD, when 3 GCPs were used. From the work in some natural environments to the investigation of infrastructure, such as modelling water runoff during rain, different projects have different requirements for the fineness of ground features, so the required accuracy depends on the purpose of generating DSM. Therefore, using only one GCP may not meet the high accuracy standards; two to three GCPs are recommended for a trade-off between accuracy and work efficiency. The influence of each error on RMSE is directly proportional to the size of the square error, and therefore RMSE is sensitive to large differences and does not reflect terrain changes. Because of scale differences, errors that do not occur in flat areas may also occur in sloped areas [15]. The natural environment presents a series of complexities, including changing vegetation cover, strong topographic relief, and changes in texture. Future studies will need to assess the impact of these complexities on the accuracy of the results. Calculating RMSE is a common error assessment method when the actual dataset of the ground surface is a set of distribution points rather than a continuous, real surface. Error evaluation benefits from a larger number and more evenly distributed checkpoints. Gomes et al. arranged 270 vertical checkpoints in an area of about 0.22 km2, with a density of 1227 checkpoints per km2 [39]. In the study of Tomaštík, the density of checkpoints at the three sites was approximately 11363, 3674 and 2749 per km2 [8]. Thus, even if the number of GCP measurements on the ground is minimized, the role of checkpoints in the error assessment is critical. In the future, the plan is to deploy as many checkpoints as possible in the study area, build an accurate error surface, analyze the spatial distribution characteristics of errors, and better verify the results; these tasks will help to understand and reduce the potential error sources in the UAV SfM workflow [36];
- The M3C2 algorithm eliminates the error introduced by the interpolation process. Lower error measurements with M3C2 are comparable to point-to-point or point-to-mesh; the method has been widely used in research based on point cloud change detection [40,41,42,43]. Due to the high point density of UAV matching point clouds, an intercomparison can actually be regarded as continuous [44]. At zero GCP, the M3C2 distance error shows randomness in the study area. The standard deviation range of the comparison between the reference point cloud and the point cloud of different projects is 0.021–0.028 m. The change is relatively small, and the point cloud only deviates in the vertical direction, which is similar to the study of Tomaštík and Štroner et al. [6,44]. Standard deviation is an indicator of precision. In some applications of UAV SfM, the accuracy of geolocation is not as important as the repeatability (precision) of data. For example, comparing multi temporal measurement data to study the change in terrain with time, more attention is paid to the relative change between data, and the quality of multi temporal data can be improved through cooperative registration;
- GCP distribution experiments show that the uniform distribution of GCPs is crucial when using more than one GCP. Figure 10b uses nonlinear curve fitting. In Gindraux’s study, linear fitting was used to determine that, on average, the vertical accuracy decreased by 0.09 m when the distance from the nearest GCP increased by 100 m [45]. In the experiment with four GCPs, the accuracy was improved after moving the GCP slightly towards the center of the study area compared with placing the GCP at the edge of the study area, which is similar to the study of Martínez. Martínez’s study concluded that the best horizontal accuracies are achieved by placing GCPs around the edges of the study area, but it is also essential to place GCPs inside the area with a stratified distribution to optimize vertical accuracy [28];
- The DSM vertical RMSE and DOM horizontal RMSE obtained by direct georeferencing with the GSD set to 1.7 cm/pixel were 0.087 and 0.041 m, respectively. Without GCPs, the accuracy of the results was highly dependent on the accuracy of the image position data. The following measures can be taken to improve the accuracy of results obtained without GCPs. UAV cross flights and imagery with large overlap can provide redundant data and improve the reliability of image matching, which requires high computing power [1,39]. The addition of oblique images helps to accurately estimate the internal and external orientation elements in the process of bundle adjustment, extract vertical features such as building sidewalls, and obtain the best vertical accuracy [13]. A more accurate GCP measurement method can be used, rather than simply increasing the number of GCPs. Another recommendation is to use a tripod-mounted prism instead of a pole-mounted prism, as well as the RTK static measurement method when time permits. If inaccurate coordinates are introduced when measuring GCPs, a more complex error surface will be introduced, as opposed to reducing the initial deformation [14].
5. Conclusions
- UAV SfM is a flexible and efficient method to obtain high-resolution topographic data. The direct georeferencing method based on the RTK/PPK fusion difference to obtain high-accuracy image positions has potential for improving the accuracy of the products, especially when GPS measurements are difficult, as well as reducing the dependence on the GCPs in the bundle adjustment, and decreasing the field work time and cost.
- The research results show that the vertical RMSE of the DSM obtained by direct georeferencing was 0.087 m, approximately equal to 5.12 GSD. The horizontal RMSE of the DOM was 0.041 m, approximately equal to 2.41 GSD. Both values reached the centimeter positioning accuracy and achieve the application research of decimeter-error scale. The accuracy of UAV direct georeferencing could be guaranteed through careful flight planning, an appropriate survey, and accurate data post-processing. In the study of terrain change detection, we suggest evenly deploying two to three GCPs to achieve a good compromise between appropriate accuracy, repeatability and efficiency.
- GCPs should be uniformly distributed in the study area and contain at least one GCP near the center of the domain to reduce the dome effect. With an increase in the number of GCPs in the bundle adjustment, both the horizontal error and vertical error decreased, and the horizontal error was always lower than the vertical error. When the density of the GCPs was greater than 12 GCP/km2 and 10 GCP/km2, respectively, the decrease in the vertical and horizontal errors was not obvious. The minimum vertical and horizontal RMSE were 0.032 (~1.88 GSD) and 0.015 m (~0.88 GSD), respectively.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Acknowledgments
Conflicts of Interest
References
- Zhang, H.; Aldana-Jague, E.; Clapuyt, F.; Wilken, F.; Vanacker, V.; Van Oost, K. Evaluating the potential of post-processing kinematic (PPK) georeferencing for UAV-based structure-from-motion (SfM) photogrammetry and surface change detection. Earth Surf. Dyn. 2019, 7, 807–827. [Google Scholar] [CrossRef] [Green Version]
- Ullman, S. The interpretation of structure from motion. Proc. R. Soc. Lond. Ser. B Biol. Sci. 1979, 203, 405–426. [Google Scholar]
- Eltner, A.; Kaiser, A.; Castillo, C.; Rock, G.; Neugirg, F.; Abellán, A. Image-based surface reconstruction in geomorphometry–merits, limits and developments. Earth Surf. Dyn. 2016, 4, 359–389. [Google Scholar] [CrossRef] [Green Version]
- James, M.R.; Robson, S. Straightforward reconstruction of 3D surfaces and topography with a camera: Accuracy and geoscience application. J. Geophys. Res. Earth Surf. 2012, 117, F3. [Google Scholar] [CrossRef] [Green Version]
- Kim, D.-W.; Yun, H.S.; Jeong, S.-J.; Kwon, Y.-S.; Kim, S.-G.; Lee, W.S.; Kim, H.-J. Modeling and testing of growth status for Chinese cabbage and white radish with UAV-based RGB imagery. Remote Sens. 2018, 10, 563. [Google Scholar] [CrossRef] [Green Version]
- Tomaštík, J.; Mokroš, M.; Saloň, Š.; Chudý, F.; Tunák, D. Accuracy of photogrammetric UAV-based point clouds under conditions of partially-open forest canopy. Forests 2017, 8, 151. [Google Scholar] [CrossRef] [Green Version]
- Lian, X.; Li, Z.; Yuan, H.; Hu, H.; Cai, Y.; Liu, X. Determination of the Stability of High-Steep Slopes by Global Navigation Satellite System (GNSS) Real-Time Monitoring in Long Wall Mining. Appl. Sci. 2020, 10, 1952. [Google Scholar] [CrossRef] [Green Version]
- Godone, D.; Allasia, P.; Borrelli, L.; Gullà, G. UAV and Structure from Motion Approach to Monitor the Maierato Landslide Evolution. Remote Sens. 2020, 12, 1039. [Google Scholar] [CrossRef] [Green Version]
- Long, N.; Millescamps, B.; Guillot, B.; Pouget, F.; Bertin, X. Monitoring the topography of a dynamic tidal inlet using UAV imagery. Remote Sens. 2016, 8, 387. [Google Scholar] [CrossRef] [Green Version]
- Long, N.; Millescamps, B.; Pouget, F.; Dumon, A.; Lachaussée, N.; Bertin, X. Accuracy assessment of coastal topography derived from UAV images. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2016, 41, B1. [Google Scholar]
- Immerzeel, W.W.; Kraaijenbrink, P.D.A.; Shea, J.; Shrestha, A.; Pellicciotti, F.; Bierkens, M.F.P.; De Jong, S.M. High-resolution monitoring of Himalayan glacier dynamics using unmanned aerial vehicles. Remote Sens. Environ. 2014, 150, 93–103. [Google Scholar] [CrossRef]
- Mallalieu, J.; Carrivick, J.L.; Quincey, D.J.; Smith, M.W.; James, W.H. An integrated Structure-from-Motion and time-lapse technique for quantifying ice-margin dynamics. J. Glaciol. 2017, 63, 937–949. [Google Scholar] [CrossRef] [Green Version]
- Zeybek, M. Accuracy assessment of direct georeferencing UAV images with onboard global navigation satellite system and comparison of CORS/RTK surveying methods. Meas. Sci. Technol. 2021, 32, 065402. [Google Scholar] [CrossRef]
- Sanz-Ablanedo, E.; Chandler, J.H.; Rodríguez-Pérez, J.R.; Ordóñez, C. Accuracy of unmanned aerial vehicle (UAV) and SfM photogrammetry survey as a function of the number and location of ground control points used. Remote Sens. 2018, 10, 1606. [Google Scholar] [CrossRef] [Green Version]
- Cucchiaro, S.; Fallu, D.J.; Zhang, H.; Walsh, K.; Van Oost, K.; Brown, A.G.; Tarolli, P. Multiplatform-SfM and TLS data fusion for monitoring agricultural terraces in complex topographic and landcover conditions. Remote Sens. 2020, 12, 1946. [Google Scholar] [CrossRef]
- McMahon, C.; Mora, O.E.; Starek, M.J. Evaluating the Performance of sUAS Photogrammetry with PPK Positioning for Infrastructure Mapping. Drones 2021, 5, 50. [Google Scholar] [CrossRef]
- Padró, J.C.; Muñoz, F.J.; Planas, J.; Pons, X. Comparison of four UAV georeferencing methods for environmental monitoring purposes focusing on the combined use with airborne and satellite remote sensing platforms. Int. J. Appl. Earth Obs. Geoinf. 2019, 75, 130–140. [Google Scholar] [CrossRef]
- Nolan, M.; Larsen, C.; Sturm, M. Mapping snow depth from manned aircraft on landscape scales at centimeter resolution using structure-from-motion photogrammetry. Cryosphere 2015, 9, 1445–1463. [Google Scholar] [CrossRef] [Green Version]
- Tomaštík, J.; Mokroš, M.; Surový, P.; Grznárová, A.; Merganič, J. UAV RTK/PPK method—An optimal solution for mapping inaccessible forested areas? Remote Sens. 2019, 11, 721. [Google Scholar] [CrossRef] [Green Version]
- Erenoglu, R.C.; Erenoglu, O. A case study on the comparison of terrestrial methods and unmanned aerial vehicle technique in landslide surveys: Sarıcaeli landslide, Çanakkale, NW Turkey. Int. J. Environ. Geoinf. 2018, 5, 325–336. [Google Scholar] [CrossRef]
- Turner, D.; Lucieer, A.; Watson, C. An automated technique for generating georectified mosaics from ultra-high resolution unmanned aerial vehicle (UAV) imagery, based on structure from motion (SfM) point clouds. Remote Sens. 2012, 4, 1392–1410. [Google Scholar] [CrossRef] [Green Version]
- Shahbazi, M.; Sohn, G.; Théau, J.; Menard, P. Development and evaluation of a UAV-photogrammetry system for precise 3D environmental modeling. Sensors 2015, 15, 27493–27524. [Google Scholar] [CrossRef] [Green Version]
- Mian, O.; Lutes, J.; Lipa, G.; Hutton, J.J.; Gavelle, E.; Borghini, S. Accuracy assessment of direct georeferencing for photogrammetric applications on small unmanned aerial platforms. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2016, 40, 77. [Google Scholar] [CrossRef] [Green Version]
- Hugenholtz, C.; Brown, O.; Walker, J.; Barchyn, T.E.; Nesbit, P.; Kucharczyk, M.; Myshak, S. Spatial accuracy of UAV-derived orthoimagery and topography: Comparing photogrammetric models processed with direct geo-referencing and ground control points. Geomatica 2016, 70, 21–30. [Google Scholar] [CrossRef]
- Agüera-Vega, F.; Carvajal-Ramírez, F.; Martínez-Carricondo, P. Assessment of photogrammetric mapping accuracy based on variation ground control points number using unmanned aerial vehicle. Measurement 2017, 98, 221–227. [Google Scholar] [CrossRef]
- Tahar, K.N. An evaluation on different number of ground control points in unmanned aerial vehicle photogrammetric block. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2013, 40, 93–98. [Google Scholar] [CrossRef] [Green Version]
- Rosnell, T.; Honkavaara, E. Point cloud generation from aerial image data acquired by a quadrocopter type micro unmanned aerial vehicle and a digital still camera. Sensors 2012, 12, 453–480. [Google Scholar] [CrossRef] [Green Version]
- Martínez-Carricondo, P.; Agüera-Vega, F.; Carvajal-Ramírez, F.; Mesas-Carrascosa, F.J.; García-Ferrer, A.; Pérez-Porras, F.-J. Assessment of UAV-photogrammetric mapping accuracy based on variation of ground control points. Int. J. Appl. Earth Obs. Geoinf. 2018, 72, 1–10. [Google Scholar] [CrossRef]
- Reshetyuk, Y.; Mårtensson, S.G. Generation of highly accurate digital elevation models with unmanned aerial vehicles. Photogramm. Rec. 2016, 31, 143–165. [Google Scholar] [CrossRef]
- Stott, E.; Williams, R.D.; Hoey, T.B. Ground control point distribution for accurate kilometre-scale topographic mapping using an RTK-GNSS unmanned aerial vehicle and SfM photogrammetry. Drones 2020, 4, 55. [Google Scholar] [CrossRef]
- Lague, D.; Brodu, N.; Leroux, J. Accurate 3D comparison of complex topography with terrestrial laser scanner: Application to the Rangitikei canyon (NZ). ISPRS J. Photogram. Remote Sens. 2013, 82, 10–26. [Google Scholar] [CrossRef] [Green Version]
- Agisoft LCC. Agisoft PhotoScan. Available online: http://www.agisoft.com (accessed on 20 February 2017).
- Yu, Z.; Zhou, H.; Li, C. Fast non-rigid image feature matching for agricultural UAV via probabilistic inference with regularization techniques. Comput. Electron. Agric. 2017, 143, 79–89. [Google Scholar] [CrossRef]
- Jiang, S.; Jiang, W. On-board GNSS/IMU assisted feature extraction and matching for oblique UAV images. Remote Sens. 2017, 9, 813. [Google Scholar] [CrossRef] [Green Version]
- Snavely, N.; Seitz, S.M.; Szeliski, R. Modeling the world from internet photo collections. Int. J. Comput. Vis. 2008, 80, 189–210. [Google Scholar] [CrossRef] [Green Version]
- James, M.R.; Robson, S.; d’Oleire-Oltmanns, S.; Niethammer, U. Optimising UAV topographic surveys processed with structure-from-motion: Ground control quality, quantity and bundle adjustment. Geomorphology 2017, 280, 51–66. [Google Scholar] [CrossRef] [Green Version]
- CloudCompare v2.10.2. Available online: https://www.danielgm.net/cc/ (accessed on 18 July 2020).
- Javernick, L.; Brasington, J.; Caruso, B. Modeling the topography of shallow braided rivers using Structure-from-Motion photogrammetry. Geomorphology 2014, 213, 166–182. [Google Scholar] [CrossRef]
- Pessoa, G.G.; Carrilho, A.C.; Miyoshi, G.T.; Amorim, A.; Galo, M. Assessment of UAV-based digital surface model and the effects of quantity and distribution of ground control points. Int. J. Remote Sens. 2021, 42, 65–83. [Google Scholar] [CrossRef]
- Stumpf, A.; Malet, J.P.; Allemand, P.; Deseilligny, M.P.; Skupinski, G. Ground-based multi-view photogrammetry for the monitoring of landslide deformation and erosion. Geomorphology 2015, 231, 130–145. [Google Scholar] [CrossRef]
- Cucchiaro, S.; Cavalli, M.; Vericat, D.; Crema, S.; Llena, M.; Beinat, A.; Marchi, L.; Cazorzi, F. Monitoring topographic changes through 4D-structure-from-motion photogrammetry: Application to a debris-flow channel. Environ. Earth Sci. 2018, 77, 632. [Google Scholar] [CrossRef]
- Milan, D.J.; Heritage, G.L.; Large, A.R.G.; Fuller, I.C. Filtering spatial error from DEMs: Implications for morphological change estimation. Geomorphology 2011, 125, 160–171. [Google Scholar] [CrossRef]
- Cook, K.L. An evaluation of the effectiveness of low-cost UAVs and structure from motion for geomorphic change detection. Geomorphology 2017, 278, 195–208. [Google Scholar] [CrossRef]
- Štroner, M.; Urban, R.; Reindl, T.; Seidl, J.; Brouček, J. Evaluation of the georeferencing accuracy of a photogrammetric model using a quadrocopter with onboard GNSS RTK. Sensors 2020, 20, 2318. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Gindraux, S.; Boesch, R.; Farinotti, D. Accuracy assessment of digital surface models from unmanned aerial vehicles’ imagery on glaciers. Remote Sens. 2017, 9, 186. [Google Scholar] [CrossRef] [Green Version]
UAV Body | D-CAM2000 Aerial Module | ||
---|---|---|---|
System standard takeoff weight | 2.8 kg | Camera | SONY a6000 |
Standard load | 200 g | Effective pixels | 24.3 million |
Endurance | 74 min | Sensor | 23.5 × 15.6 mm (aps-c) |
Remote control distance | 20 km (max) | Focal length | 25 mm |
Number of GCPs | t | Degree of Freedom | Significance | Mean Difference Value | 95% Confidence Interval | |
---|---|---|---|---|---|---|
Lower Limit | Upper Limit | |||||
1 | −8.175 | 119 | 3.64 × 10−13 | −0.025 | −0.031 | −0.019 |
2 | 0.760 | 119 | 0.449 | 0.003 | −0.004 | 0.009 |
Project | Mean (m) | std (m) | Project | Mean (m) | std (m) |
---|---|---|---|---|---|
16-0 | 0.062 | 0.025 | 16-3 | 0.027 | 0.022 |
16-1/k1 | 0.046 | 0.023 | 16-4/k2,k5,k9,k12 | −0.015 | 0.028 |
16-1/k8 | 0.023 | 0.022 | 16-4/k1,k6,k8,k15 | 0.023 | 0.021 |
16-1/k15 | 0.053 | 0.025 | 16-5 | 0.021 | 0.021 |
16-1/k10 | 0.041 | 0.023 | 16-6 | 0.013 | 0.021 |
16-1/k6 | 0.031 | 0.022 | 16-7 | 0.016 | 0.021 |
16-2/k1, k10 | 0.032 | 0.023 | 16-8 | 0.01 | 0.023 |
16-2/k6, k15 | 0.024 | 0.028 | 16-9 | 0.01 | 0.021 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, X.; Lian, X.; Yang, W.; Wang, F.; Han, Y.; Zhang, Y. Accuracy Assessment of a UAV Direct Georeferencing Method and Impact of the Configuration of Ground Control Points. Drones 2022, 6, 30. https://doi.org/10.3390/drones6020030
Liu X, Lian X, Yang W, Wang F, Han Y, Zhang Y. Accuracy Assessment of a UAV Direct Georeferencing Method and Impact of the Configuration of Ground Control Points. Drones. 2022; 6(2):30. https://doi.org/10.3390/drones6020030
Chicago/Turabian StyleLiu, Xiaoyu, Xugang Lian, Wenfu Yang, Fan Wang, Yu Han, and Yafei Zhang. 2022. "Accuracy Assessment of a UAV Direct Georeferencing Method and Impact of the Configuration of Ground Control Points" Drones 6, no. 2: 30. https://doi.org/10.3390/drones6020030
APA StyleLiu, X., Lian, X., Yang, W., Wang, F., Han, Y., & Zhang, Y. (2022). Accuracy Assessment of a UAV Direct Georeferencing Method and Impact of the Configuration of Ground Control Points. Drones, 6(2), 30. https://doi.org/10.3390/drones6020030