Determining the Optimal Number of Ground Control Points for Varying Study Sites through Accuracy Evaluation of Unmanned Aerial System-Based 3D Point Clouds and Digital Surface Models

: The rapid development of drone technologies, such as unmanned aerial systems (UASs) and unmanned aerial vehicles (UAVs), has led to the widespread application of three-dimensional (3D) point clouds and digital surface models (DSMs). Due to the number of UAS technology applications across many ﬁelds, studies on the veriﬁcation of the accuracy of image processing results have increased. In previous studies, the optimal number of ground control points (GCPs) was determined for a speciﬁc area of a study site by increasing or decreasing the amount of GCPs. However, these studies were mainly conducted in a single study site, and the results were not compared with those from various study sites. In this study, to determine the optimal number of GCPs for modeling multiple areas, the accuracy of 3D point clouds and DSMs were analyzed in three study sites with di ﬀ erent areas according to the number of GCPs. The results showed that the optimal number of GCPs was 12 for small and medium sites (7 and 39 ha) and 18 for the large sites (342 ha) based on the overall accuracy. If these results are used for UAV image processing in the future, accurate modeling will be possible with minimal e ﬀ ort in GCPs.


Introduction
Various studies have been conducted to investigate the utility of unmanned aerial systems (UASs) and unmanned aerial vehicles (UAVs) as the technologies have developed and popularized. These studies mainly acquire data through UAVs equipped with optical (RGB), multi-spectral, or infrared sensors [1][2][3], and produce 3D point clouds and digital surface models (DSMs) using image processing programs. Before the popularization of UAV technology, satellites, manned aircrafts, and professional surveillance cameras were the prominent methods of data acquisition for research [4,5].
The 3D point cloud is a set of data in which each point has a 3D coordinate value. The initial 3D point cloud was acquired through an expensive laser scanner [6,7]. However, recently, it could be constructed from images taken with a camera using the Structure from Motion-Multi View Stereo (SfM-MVS) algorithm [8][9][10]. DSM is a 2.5D raster format data generated with stereo images [5,11] and is also produced by interpolating point clouds built using the SfM-MVS algorithm [12,13]. Image processing results, such as 3D point clouds and DSMs built using UAV images, have been applied to various fields, such as geography, environment, administration, and industry [14][15][16][17].
Due to the increase in the number of studies that produce and utilize results of UAV image processing, studies that verify their accuracy are also increasing. Parameters that affect the accuracy of results are flight parameters [18][19][20], such as the front and side overlap and flight altitude, interior

Data Collection and Photogrammetry Process
The target sites were classified as SS, MS, and LS; then, 3, 6, 9, 12, 15 (14 in the case of the SS, where the increase in the number of GCPs was halted owing to space restrictions), and 18 GCPs were placed in each target site to investigate the effects of the number of GCPs on the accuracy of 3D point clouds and DSMs for these sites. The 3D point clouds and DSMs were produced using each GCP as an exterior orientation parameter in SfM-based image processing. The accuracy of the generated results was analyzed through CPs. In the SS, no vegetation growth was observed on the ground surface because aggregate was continuously carried in and out. The MS was composed of tidal flats, where flood and ebb tides were repeated, with artificial structures behind them; thus, no vegetation was in the tidal flats. The LS was also covered with buildings, pavements, and sidewalk blocks; therefore, there was no vegetation except street trees.

Data Collection and Photogrammetry Process
The target sites were classified as SS, MS, and LS; then, 3, 6, 9, 12, 15 (14 in the case of the SS, where the increase in the number of GCPs was halted owing to space restrictions), and 18 GCPs were placed in each target site to investigate the effects of the number of GCPs on the accuracy of 3D point clouds and DSMs for these sites. The 3D point clouds and DSMs were produced using each GCP as an exterior orientation parameter in SfM-based image processing. The accuracy of the generated results was analyzed through CPs.
The research method can be divided into in situ operations, including UAV flights, image acquisition, and land survey; the photogrammetry process, where results are produced according to the number of GCPs; and accuracy analysis ( Figure 2).
Drones 2020, 4, x FOR PEER REVIEW 4 of 19 The research method can be divided into in situ operations, including UAV flights, image acquisition, and land survey; the photogrammetry process, where results are produced according to the number of GCPs; and accuracy analysis ( Figure 2).

In Situ Operations
The field survey was divided into UAV flight, image acquisition, and land survey using a global navigation satellite system (GNSS). A field survey for the SS was conducted on 10 October 2018. In the SS, the Inspire 1 Pro model was used, and images were captured using a Zenmuse X5 camera. As the area of the SS was small, imaging could be completed during a single flight.
A field survey for the MS was conducted on 2 December 2016. In the MS, the Phantom 3 Advanced model was used, and the camera mounted on the UAV (FC300S) was used. As the area of the MS exceeded the one-time flight coverage of the rotary-wing UAV, four missions were executed.
A field survey for the LS was conducted on 13 April 2017. In the LS, the QuestUAV DataHawk, a fixed-wing UAV, and ILCE-QX1 imaging camera were used. As the area of the LS exceeded the coverage of the fixed-wing UAV, which had a longer flight time than the rotary-wing UAV, three flight missions of the fixed-wing UAV were executed.
For land surveys, the Trimble R8s model was used for the SS and LS, and the Huace X90 model was used for the MS. This is summarized in Table 1.

In Situ Operations
The field survey was divided into UAV flight, image acquisition, and land survey using a global navigation satellite system (GNSS). A field survey for the SS was conducted on 10 October 2018. In the SS, the Inspire 1 Pro model was used, and images were captured using a Zenmuse X5 camera. As the area of the SS was small, imaging could be completed during a single flight.
A field survey for the MS was conducted on 2 December 2016. In the MS, the Phantom 3 Advanced model was used, and the camera mounted on the UAV (FC300S) was used. As the area of the MS exceeded the one-time flight coverage of the rotary-wing UAV, four missions were executed.
A field survey for the LS was conducted on 13 April 2017. In the LS, the QuestUAV DataHawk, a fixed-wing UAV, and ILCE-QX1 imaging camera were used. As the area of the LS exceeded the coverage of the fixed-wing UAV, which had a longer flight time than the rotary-wing UAV, three flight missions of the fixed-wing UAV were executed.
For land surveys, the Trimble R8s model was used for the SS and LS, and the Huace X90 model was used for the MS. This is summarized in Table 1.
Automatic path flight measurements for uniform image acquisition, SfM-based image processing, accuracy analysis, and mapping were processed using the same software for each site. Pix4d Capture was used for automatic path flight measurements. Agisoft Metashape (v. 1.5.3) was used for SfM-based image processing, CloudCompare (v. 2.10.2) for accuracy analysis, and ArcMap (v. 10.1) for mapping, as summarized in Table 2.
The coordinate system used for both flight and imaging was WGS 84 (EPSG: 4326), and the TM Korea 2000 or Central Belt 2010 (EPSG: 5186) coordinate system was used for GCP measurement. Although different coordinate systems for flight and measurement were used, absolute orientation was performed following the land survey performance during the image processing process.

Construction of 3D Point Clouds and DSMs and Their Accuracy Evaluation
SfM is a technology for reconstructing the camera's position and direction from multi-shot two-dimensional (2D) images and restoring the subject and scene in 3D. This technology, based on computer vision, was developed in the 1990s and became widely used in the 2000s [31,51].
In this study, imaging (acquisition of 2D images) was performed in each study site using the UAVs. Based on this, 3D point clouds and DSMs were constructed as final results using Metashape, a software program based on the SfM algorithm. Metashape goes through the following: Camera calibration-align-absolute orientation-camera align optimization-3D point cloud generation-DSM generation (to produce the final results from 2D images).
This imaging process is a minimal procedure required to produce a 3D point cloud and DSM. As this study aims to identify the accuracy of 3D point clouds and DSMs according to the area of each study site and the number of GCPs, image processing was performed by applying multiple GCP sets for each study site.
The accuracy of the produced 3D point clouds and DSMs can be evaluated using various methods. In this study, the accuracy of x, y, z, xy, and xyz was evaluated for verification using the errors and RMSE between the constructed results and the measured CPs. The errors represent individual errors between the created point clouds and CPs. RMSE indicates the overall accuracy of the results by combining individual errors and is one of the generally used criteria for position accuracy [28,52].
The error between the 3D point cloud and the CP was calculated using the "Cloud/Cloud distance" tool in CloudCompare. Similarly, the error between the DSM and the CP was calculated using the "Extract Values to Point" tool in ArcMap. The mean distance and standard deviation were calculated using the calculated error with the programs, and finally RMSE x, y, and z were produced (Equations (1)-(5)): where ∆x i is the difference between the CP coordinates and coordinates determined from the 3D point cloud and DSM, and n is the number of points. The same equation applies to RMSE Y and RMSE Z mutatis mutandis: where (RMSE) x represents the x-direction error in the plane between the CPs and the produced 3D point cloud (Equation (1)) and (RMSE) y represents the y-direction error in the plane between the CPs and the generated 3D point cloud (Equation (2)). The individual xand y-direction error is calculated as (RMSE) xy , a radius error, which corresponds to the horizontal error of the 3D point cloud (Equation (3)): Further, (RMSE) z represents the z-direction (vertical) error. For vertical accuracy testing, different methods are used in non-vegetated terrain (where errors typically follow a normal distribution suitable for RMSE statistical analyses) and vegetated terrain (where errors do not necessarily follow a normal distribution) [28]. In this study, (RMSE) z was applied for the evaluation of the vertical accuracy because all of the study sites were non-vegetated terrains (Equation (4)). (RMSE) xyz represents the error of the 3D point cloud in the overall direction (easting, northing, and elevation) (Equation (5)).

In Situ Operation
In the SS, a total of 52 aerial images were acquired through a flight at an altitude of 120 m. The front and side overlap were 85% and 65%, respectively. In the MS, imaging was performed at an altitude of 70 m, and both the front and side overlap were 80%. As the site exceeded the flight radius of the rotary-wing UAV, four flights were performed, and a total of 1022 aerial images were acquired. In the LS, imaging was performed at an altitude of 150 m, and both the front and side overlap were 80%. As the site exceeded the flight radius of the fixed-wing UAV, three flights were performed, and 1163 aerial images obtained. The result of UAV and field survey is summarized in Table 3. The arrangement adopted for the placement and the number of the GCPs is as follows: The number of GCPs was increased by a multiple of 3, starting with the minimum number of GCPs required to obtain an absolute orientation in Metashape. The GCPs were arranged to form a central polygon that covered the study area while keeping the gaps between the GCPs as uniform as possible.
Although the SS area was small, there was a difference in altitude between the aggregate stacked at the center of the site and the surroundings. Considering this, GCPs were evenly placed at the top of the aggregate and in the surrounding areas. A total of 14 GCPs were placed. Among them, 3, 6, 9, 12, and 14 GCPs were present in the absolute orientation process of image processing. The MS was long in the N-S direction due to the coastal area's nature, and GCPs were placed accordingly. A total of 18 GCPs were placed, and divided into six cases (3,6,9,12,15,and 18 GCPs). In the LS, there were 18 GCPs as in the MS.
At least 20 CPs are required to evaluate the accuracy of the produced image processing results [28]. Surveying CPs across the study area can ensure higher reliability. However, the limitations of time and labor restrict the number of CPs surveyed. Therefore, only the CPs that were around the main targets, such as aggregate mounds, sand beaches, and main streets in each study area, were measured. In the SS, 311 points were acquired in the upper and surrounding areas of the aggregate yard; the MS, 79 points were acquired by setting two survey lines across the sand beach; and in the LS, 436 points were acquired by planning one survey line in the long direction of the site and two lateral lines in the short direction, as shown in Figure 3. The arrangement adopted for the placement and the number of the GCPs is as follows: The number of GCPs was increased by a multiple of 3, starting with the minimum number of GCPs required to obtain an absolute orientation in Metashape. The GCPs were arranged to form a central polygon that covered the study area while keeping the gaps between the GCPs as uniform as possible.
Although the SS area was small, there was a difference in altitude between the aggregate stacked at the center of the site and the surroundings. Considering this, GCPs were evenly placed at the top of the aggregate and in the surrounding areas. A total of 14 GCPs were placed. Among them, 3, 6, 9, 12, and 14 GCPs were present in the absolute orientation process of image processing. The MS was long in the N-S direction due to the coastal area's nature, and GCPs were placed accordingly. A total of 18 GCPs were placed, and divided into six cases (3,6,9,12,15,and 18 GCPs). In the LS, there were 18 GCPs as in the MS.
At least 20 CPs are required to evaluate the accuracy of the produced image processing results [28]. Surveying CPs across the study area can ensure higher reliability. However, the limitations of time and labor restrict the number of CPs surveyed. Therefore, only the CPs that were around the main targets, such as aggregate mounds, sand beaches, and main streets in each study area, were measured. In the SS, 311 points were acquired in the upper and surrounding areas of the aggregate yard; the MS, 79 points were acquired by setting two survey lines across the sand beach; and in the LS, 436 points were acquired by planning one survey line in the long direction of the site and two lateral lines in the short direction, as shown in

Constructing 3D Point Clouds and DSMs
The SS had five GCP sets generating five 3D point clouds and DSMs. For the MS and LS, they produced six 3D point clouds and DSMs because they had six GCP sets ( Figure 4). There was a small difference in the number of points, point density, DSM, and orthomosaic of each 3D point cloud depending on the number of GCPs. Since the average number of points in a 3D point cloud is proportional to the area, a large difference in the average number of points is seen corresponding to the study area. Table 4 shows the average values of the produced results. There were differences in the resolutions of DSM and orthomosaic because 3D point cloud processing was set to "Medium" in Metashape. "Medium" downscales the original scale at a ratio of 1:4. When a 3D point cloud and DSM are produced at the original scale in Metashape, the average resolution of the DSM (shown in Table 4) shows a 4-time increase, thereby increasing the value of the average resolution of DSM to become the same as the resolution of orthomosaic. The average point density and the average number of points also show an increase by 4 times. The image processing results were not produced on the original scale due to their capacity. The restriction on capacity is less for the SS and MS because the area is not large. In the case of the LS, however, the area is excessively large, and capacity can also become excessive if 3D point clouds and DSMs are produced in a 1:1 scale, making the post-processing of the results difficult.

Constructing 3D Point Clouds and DSMs
The SS had five GCP sets generating five 3D point clouds and DSMs. For the MS and LS, they produced six 3D point clouds and DSMs because they had six GCP sets ( Figure 4). There was a small difference in the number of points, point density, DSM, and orthomosaic of each 3D point cloud depending on the number of GCPs. Since the average number of points in a 3D point cloud is proportional to the area, a large difference in the average number of points is seen corresponding to the study area. Table 4 shows the average values of the produced results. There were differences in the resolutions of DSM and orthomosaic because 3D point cloud processing was set to "Medium" in Metashape. "Medium" downscales the original scale at a ratio of 1:4. When a 3D point cloud and DSM are produced at the original scale in Metashape, the average resolution of the DSM (shown in Table 4) shows a 4-time increase, thereby increasing the value of the average resolution of DSM to become the same as the resolution of orthomosaic. The average point density and the average number of points also show an increase by 4 times. The image processing results were not produced on the original scale due to their capacity. The restriction on capacity is less for the SS and MS because the area is not large. In the case of the LS, however, the area is excessively large, and capacity can also become excessive if 3D point clouds and DSMs are produced in a 1:1 scale, making the post-processing of the results difficult.

Analysis of the Accuracy of 3D Point Clouds
The accuracy of 3D point clouds was evaluated using the horizontal (xy), vertical (z), and total (xyz) errors between the 3D point clouds and CPs. Figure 5 shows the average horizontal error between the 3D point cloud and CPs according to the number of GCPs. Figure 5a shows the average horizontal error distribution of the SS, and the error ranged from 0.046 to 0.050 m. Figure 5b shows the average horizontal error distribution of the MS. The average horizontal error was 0.103 m when the number of GCPs was three. This error ranged from 0.041 to 0.045 m when the number was six or larger. Figure 5c shows the average horizontal error distribution of the LS. The average horizontal error was 0.453 m when the number of GCPs was three, with a range of 0.061 to 0.063 m for other cases.

Analysis of the Accuracy of 3D Point Clouds
The accuracy of 3D point clouds was evaluated using the horizontal (xy), vertical (z), and total (xyz) errors between the 3D point clouds and CPs. Figure 5 shows the average horizontal error between the 3D point cloud and CPs according to the number of GCPs. Figure 5a shows the average horizontal error distribution of the SS, and the error ranged from 0.046 to 0.050 m. Figure 5b shows the average horizontal error distribution of the MS. The average horizontal error was 0.103 m when the number of GCPs was three. This error ranged from 0.041 to 0.045 m when the number was six or larger. Figure 5c shows the average horizontal error distribution of the LS. The average horizontal error was 0.453 m when the number of GCPs was three, with a range of 0.061 to 0.063 m for other cases.     Figure 7 shows the average total errors between the 3D point clouds and CPs in each site. Figure  7a shows the average total error distribution of the SS, and the error ranged from 0.063 to 0.111 m. The average total error of the MS was 0.972 m for three GCPs, as shown in Figure 7b, and it ranged from 0.055 to 0.07 m. Figure 7c shows the total error distribution of the LS. The average total error was 3.951 m for three GCPs, and it ranged from 0.088 to 0.174 m in other cases.
The 3D point clouds produced for each GCP set were analyzed by using the average error range. In the case of the SS area, there was no significant difference observed in the average horizontal error corresponding to the increase or decrease in the GCP number. However, in the cases of the MS and LS areas, the average horizontal error showed a decrease when the number of GCPs was increased. Additionally, the average vertical error decreased significantly when six GCPs were used. The average total error showed a similar pattern to the average vertical error; however, when more than 12 GCPs were used, there was no significant decrease observed in the average total error.  Figure 7 shows the average total errors between the 3D point clouds and CPs in each site. Figure 7a shows the average total error distribution of the SS, and the error ranged from 0.063 to 0.111 m. The average total error of the MS was 0.972 m for three GCPs, as shown in Figure 7b, and it ranged from 0.055 to 0.07 m. Figure 7c shows the total error distribution of the LS. The average total error was 3.951 m for three GCPs, and it ranged from 0.088 to 0.174 m in other cases.  Figure 7 shows the average total errors between the 3D point clouds and CPs in each site. Figure  7a shows the average total error distribution of the SS, and the error ranged from 0.063 to 0.111 m. The average total error of the MS was 0.972 m for three GCPs, as shown in Figure 7b, and it ranged from 0.055 to 0.07 m. Figure 7c shows the total error distribution of the LS. The average total error was 3.951 m for three GCPs, and it ranged from 0.088 to 0.174 m in other cases.
The 3D point clouds produced for each GCP set were analyzed by using the average error range. In the case of the SS area, there was no significant difference observed in the average horizontal error corresponding to the increase or decrease in the GCP number. However, in the cases of the MS and LS areas, the average horizontal error showed a decrease when the number of GCPs was increased. Additionally, the average vertical error decreased significantly when six GCPs were used. The average total error showed a similar pattern to the average vertical error; however, when more than 12 GCPs were used, there was no significant decrease observed in the average total error.

Analysis of the Accuracy of DSMs
DSMs are produced by interpolating 3D point clouds. Their vertical errors were analyzed as the elevation of the surface is considered important. Figure 8 shows the average vertical errors between The 3D point clouds produced for each GCP set were analyzed by using the average error range. In the case of the SS area, there was no significant difference observed in the average horizontal error corresponding to the increase or decrease in the GCP number. However, in the cases of the MS and LS areas, the average horizontal error showed a decrease when the number of GCPs was increased. Additionally, the average vertical error decreased significantly when six GCPs were used. The average total error showed a similar pattern to the average vertical error; however, when more than 12 GCPs were used, there was no significant decrease observed in the average total error.

Analysis of the Accuracy of DSMs
DSMs are produced by interpolating 3D point clouds. Their vertical errors were analyzed as the elevation of the surface is considered important. Figure 8 shows the average vertical errors between the DSMs, produced by varying the number of GCPs and CPs in each site.

Analysis of the Accuracy of DSMs
DSMs are produced by interpolating 3D point clouds. Their vertical errors were analyzed as the elevation of the surface is considered important. Figure 8 shows the average vertical errors between the DSMs, produced by varying the number of GCPs and CPs in each site. Figure 8a shows the average vertical error distribution of the SS, which ranged from −0.011 to 0.074 m. Figure 8b shows the average vertical error distribution of the MS. The average error was −0.939 m for three GCPs, with a −0.019 to −0.061 m range in other cases. Figure 8c shows the average vertical error distribution of the LS. The average error was 3.986 m when three GCPs were used, and it ranged from −0.01 to 0.11 m in other cases. The vertical error observed in the DSMs was similar to the average vertical error observed in the 3D point clouds.    Figure 8c shows the average vertical error distribution of the LS. The average error was 3.986 m when three GCPs were used, and it ranged from −0.01 to 0.11 m in other cases. The vertical error observed in the DSMs was similar to the average vertical error observed in the 3D point clouds.

Comprehensive Comparison
The errors between the constructed results and CPs were analyzed in the above sections to verify the 3D point clouds and DSMs constructed for each site. RMSE by region and the number of GCPs were analyzed to evaluate the overall accuracy of the constructed results ( Figure 9; Table 5). The RMSE analysis showed similarity to the error analysis. Excluding the case of the SS area, the RMSE showed a significant increase when 6 GCPs were used. The total RMSE seemed to be optimized when the 12 GCPs were used, although there were differences depending on the area of the study area.  The RMSE analysis showed similarity to the error analysis. Excluding the case of the SS area, the RMSE showed a significant increase when 6 GCPs were used. The total RMSE seemed to be optimized when the 12 GCPs were used, although there were differences depending on the area of the study area.

Discussion
As mentioned in the introduction, the number of GCPs used in the study is important for performing absolute orientation in UAV surveys [24,53] because installing GCPs is not only labor intensive but also time consuming [35,36]. In addition, a large number of GCPs may increase the time required to mark drone images during image processing. Therefore, determining the optimal number of GCPs to construct 3D point clouds and DSMs is significant as it enables time-efficient work without wasting labor.
To find the optimal number of GCPs according to each target sites area, the accuracy of the 3D point clouds and DSMs was calculated according to the number of GCPs. GCPs were evenly distributed in each target site so that the GCP network could form a central point polygon capable of covering the site [26,43,54].
Drone modeling has various end-users, and each end-user requires different types of accuracy (vertical, horizontal, or total accuracy) depending on their needs. Therefore, the horizontal, vertical, and total accuracy were calculated individually. Whether this accuracy met the horizontal/vertical accuracy criteria of the American Society for Photogrammetry and Remote Sensing (ASPRS) was assessed [25]. In the Accuracy Quality criteria of ASPRS, the ground sample distance (GSD) of the original image becomes the judgment criterion of accuracy for horizontal accuracy, but the vertical accuracy only distinguishes the absolute accuracy. However, all of the three study sites were non-vegetated terrains. Therefore, non-vegetated in the vertical accuracy class was considered for the judgment of the accuracy (Table 6).

Horizontal Errors and RMSEs of 3D Point Clouds
In the SS, the average horizontal errors were 0.05 m for three GCPs and 0.047 m for 14 GCPs. The smallest average horizontal error was 0.046 m when using six GCPs. The standard deviation ranged from 0.019 to 0.020 m, indicating no significant difference between whether the number of GCPs was the largest and the smallest. RMSE also ranged from 0.050 to 0.054 m, corresponding to the range of less than 2 * GSD (5.94 cm) in all cases. In summary, using only three GCPs in the small area of the SS met the horizontal position accuracy criterion of ASPRS. Therefore, using only three GCPs is sufficient when only considering the horizontal accuracy.
The average horizontal error of the MS exceeded 0.1 m when the number of GCPs was three and became smaller when it was six. The error was the smallest (0.041 m) when the number of GCPs was nine. The standard deviation was 0.055 m for three GCPs and sharply decreased from six GCPs with a range of 0.014 to 0.016 m when the number of GCPs was between 6 and 18. RMSE was also inaccurate (approximately 5 * GSD) when three GCPs were used but exhibited an accuracy of less than 2 * GSD (5.38 cm) from six GCPs. As the horizontal accuracy criterion of ASPRS could be met when the number of GCPs was six or larger for the area of the MS, it is necessary to secure at least six GCPs when only the horizontal accuracy is considered.
The accuracy trend of the LS, according to the number of GCPs was very similar to that of the MS. When the number of GCPs was three, the average horizontal error was 0.453 m. However, the error sharply decreased and ranged from 0.061 to 0.067 m when the number of GCPs was between 6 and 18. The standard deviation also decreased from 0.78 m to 0.023-0.028 m when the number of GCPs increased from three. RMSE was also excessively large (close to 1 m) when the number of GCPs was three but decreased to less than 2 * GSD with six GCPs. This indicates that using at least six GCPs can meet the horizontal accuracy criterion of ASPRS for sites whose area is similar to or larger than that of the MS.

Vertical Errors and RMSEs of 3D Point Clouds
In the SS, the average vertical error was the largest (−0.063 m) when the number of GCPs was three. As the number increased, the error slowly decreased and showed a tendency to converge to zero. The difference between the two cases was only 1 mm when using 12 and 14 GCPs. The standard deviation was also the largest for three GCPs. However, it slowly decreased and there was a small difference when 12 and 14 GCPs were used. RMSE also exceeded 0.1 m for three GCPs but was less than 0.1 m for six and nine GCPs. It was close to 0.04 m from 12 GCPs. These results indicate that at least six GCPs must be installed in small areas, such as SS, considering the vertical accuracy and that at least 12 GCPs must be used to achieve the 5 cm class of ASPRS.
The average vertical error of the MS was very inaccurate (approximately −0.9 m) with three GCPs. As the number increased from 6 to 12, the average vertical error tended to decrease, reaching −0.027 m when it was 12. The error was −0.018 m when 15 and 18 GCPs were used. The standard deviation amounted to 0.7 m when the number of GCPs was three. It was 0.036 m for six GCPs and approximately 0.03 m from nine GCPs. RMSE was 1.148 m for three GCPs and slightly exceeded 0.005 m for six and nine GCPs. It was close to 0.03 m from 12 GCPs. In summary, excellent accuracy could be secured for MS with six GCPs as in SS when considering the vertical position accuracy, the accuracy within 5 cm class was observed when 12 or more GCPs were used.
In the LS, the error was close to 3.8 m with three GCPs, and it exceeded 0.1 m even when the number increased to six. The error was 0.06 m for nine GCPs, and it recorded a stable error range only when the number of GCPs reached 12. The standard deviation also recorded a vast difference when the number of GCPs increased from three to nine but became less than 0.08 m when 12 GCPs were used. The RMSE of the LS became less than 0.1 m from 12 GCPs, unlike the SS and MS, which exhibited excellent accuracy from six GCPs, and the LS exhibiting the highest accuracy using 18 GCPs. In the LS with a large area, the accuracy showed a tendency to improve as the number of GCPs increased. Therefore, higher accuracy could have been achieved through the use of more GCPs. In many cases, however, UAV modeling is not performed in large areas, such as in LS [45].

Total Errors and RMSEs of 3D Point Clouds
While the horizontal and vertical errors represent xy-and z-direction errors, the total error determines the 3D error in the xyz direction. The mean total errors of the SS were approximately 1 m with three, six, and nine GCPs were used. They significantly decreased to 0.064 and 0.063 m when 12 and 14 GCPs were used. The standard deviation also exhibited the smallest difference when 12 or more GCPs were used. RMSE exceeded 0.1 m for three and six GCPs and presented the best results (0.068 and 0.067 m) when 12 and 14 GCPs were used. Therefore, for the area of the SS, 12 or more GCPs must be used to obtain high accuracy when considering the total accuracy.
In the case of the MS, the average error slightly changed by millimeters from 12 GCPs, which is also true of the standard deviation and RMSE. As RMSE converged to less than 0.06 m from 12 GCPs, it is necessary to install 12 or more GCPs to obtain high accuracy.
In the LS, the average error, standard deviation, and RMSE were significantly reduced when 12 or more GCPs were used as in MS. RMSE was approximately 0.08 m from 12 GCPs. However, the difference in the MS is that the error and standard deviation were somewhat reduced when 18 GCPs were used compared to 12 and 15 GCPs. Thus, 12 or more GCPs must be used to secure high accuracy in large areas, such as LS. If more than 18 GCPs are used, this may increase the accuracy slightly.

Vertical Errors and RMSEs of DSMs
As DSMs are produced by interpolating 3D point clouds, their accuracy is similar to that of 3D point clouds. In the SS, the average vertical error of DSM was the largest when the number of GCPs was three. It decreased as the number of GCPs increased, and there was almost no difference when 12 and 14 GCPs were used. The standard deviation was also the smallest when using 12 and 14 GCPs similarly to 3D point clouds. Further, RMSE was approximately 0.04 m from 12 GCPs.
In both MS and LS, there was a small difference from the vertical errors of the corresponding 3D point clouds. In summary, in the case of DSM, at least 12 GCPs must be secured to ensure the accuracy of the result in a situation where vertical errors are considered.

Comprehensive Discussion
For each study site, the number of GCPs to be used to derive highly accurate horizontal, vertical, and total accuracy was different (Table 7). In this table, "minimum" represents the minimum number of GCPs to achieve the accuracy of approximately 0.1 m, which is "excellent" accuracy. Whereas, "optimal" is the number of GCPs required to meet the accuracy of ASPRS. The criteria for vertical and total errors are relative. This study's results showed that using only three GCPs could result in the optimal accuracy when considering the horizontal accuracy in a small area. Regarding the vertical and total accuracy in typical drone survey areas, such as SS and MS, 6 GCPs are required for excellent accuracy and 12 GCPs for optimal accuracy. Therefore, installing more than 12 GCPs in an area of approximately 39 ha or less is inefficient.
In the LS, the optimal accuracy was observed from six GCPs when considering the horizontal accuracy. The vertical and total accuracy shown excellent results when at least 12 GCPs were used, and the optimal accuracy was observed using 18 GCPs. In previous studies, the accuracy improved as the number of GCPs increased for very large areas, but the accuracy improvement alongside the increase in the number of GCPs was unknown [19,24,26,43]. In this study, the degree of improvement was not evident, either.
Thus, in typical areas (such as SS and MS), using 12 GCPs will be able to produce highly accurate 3D point clouds and DSMs. In larger areas, the installation of more than 12 GCPs will be required depending on the needs. These results indicate that using too many GCPs may not be effective compared to the labor and cost [55,56], even if the area of the target site is diverse, unlike calculating the optimal number of GCPs per area in previous studies [26,29,41,43].
In order to find methods that can improve UAV modeling accuracy, research has been conducted on solutions, such as camera angle, altitude, overlap, interior orientation parameters of the camera, and calibration, rather than on exterior orientation parameters, such as GCPs. In addition, the need for GCP installation was significantly reduced through the RTK and PPK techniques [25,27,37]. Therefore, with the development of technology, the dependence on the number of GCPs is expected to decrease gradually. Nevertheless, discussion on the number of GCPs will continue, as the method of using GCPs is the most widely used UAV modeling method and the most accurate method. There are extremely diverse target sites for UAV modeling, as almost all the surface of the earth can be targeted [57][58][59][60][61]. Thus, the elevation and roughness of the ground surface are different. In our study, the three study areas were similar because there was little or no vegetation. However, the main study areas showed variation as follows: (i) Aggregation and bare land (SS), (ii) sand and gravels of beach (MS), and (iii) artificial/man-made areas (LS). In addition, the study on the number of GCPs mentioned in the introduction [38][39][40][41][42][43] also had a difference in elevation of less than 100 m in the study area except for the study of Enoc Sanz-Ablanedo et al. [43]. Generally, the distance between GCPs is referred to as the plane distance, but if the height difference in the study area is large, the height of the study area should be considered because the slant distance increases in proportion to the height difference. If the characteristics of the ground surface are considered in future studies, this may produce interesting results.
Herein, different UAVs and cameras were used for the three study sites. Various models and cameras were used as research materials because UAVs from hobby-type UAVs to industrial UAVs are routinely considered in UAV modeling. Therefore, it will be possible to apply the results of this study more universally. However, in future research, it will be essential to conduct research by reinforcing variable control and changing only the relative area for the same UAV, camera, and site.
Up to 18 GCPs were used for each site. LS exhibited the smallest RMSE with 18 GCPs. In future research, if the set interval for the number of GCPs is reduced to less than 3 and 18 GCPs or more are used, the results can be used to reinforce the results of this study.

Conclusions
In this study, the accuracy of 3D point clouds and DSMs were analyzed in three study sites with different areas according to the number of GCPs to propose the optimal number of GCPs for the 3D UAV modeling of various areas.
When the horizontal accuracy was considered, three or more GCPs had to be used in the SS, while six or more GCPs had to be used in the MS and LS. In terms of the vertical accuracy, using only 12 GCPs reached the optimal accuracy in SS and MS, and 18 GCPs in the LS. When considering the total accuracy that covers both the horizontal and vertical accuracy, using only 12 GCPs exhibited the optimal accuracy in SS and MS, and again 18 GCPs in the LS as with the vertical accuracy.
When 3D point clouds, DSMs, and orthomosaics are produced using UAVS, the installation of GCPs requires a lot of time and labor, both indoors and outdoors. Most of the previous studies discussed the number of GCPs in only one study area. On the other hand, our study selected various UAV modelling sites (encompassing natural environment, large industrial complexes, and environmental monitoring areas) based on different areas. We expect that if the results of this study are applied to actual UAV modeling, it may be possible to reduce the time and labor required for GCP installation.